How to help reduce the pain of transitioning to JavaScript

There are many complaints about JavaScript 

Two major ones coming from a .NET background are; 

1) dynamic language increases errors and reduces the availability of productivity tools like refactoring and autocompletion 

2) duplication of code between the front-end and back-end 

These are valid complaints, and do have an impact on productivity.  But. 

I see the benefits of rich and responsive UI in the browser as an important part of producing great client outcomes.  I'm not arguing for JavaScript everywhere, static content (and the vast majority of content managed content) can be perfectly functional, responsive and pretty without complex javascript development, but for interactive systems such as LOB applications, traditional server side solutions (the traditional loop: HTTP-{GET|POST}->Process->HTTP-RESPONSE->client render HTML->repeat) are not sufficient. 

Even in low-latency on-premises intranet environments this model is outdated for all but the simplest of solutions, and with cloud and distributed workforces increasing, the inefficiencies of this approach are becoming more significant.   

I am not advocating for Single Page Applications, where a whole raft of additional complexities arise for often minimal realized benefit, but a middle ground is necessary. 

So what are some options to increase productivity and produce great web applications? 

Rich Server-Side Frameworks  

Vaadin and the now-defunct Lightswitch remove the front-end from the development process altogether.  The server-side object model or designer tool is used to provide the screen definition, and the HTML is generated based on the server-side definition.  The frameworks are usually smart enough to support rich client-side processing as well (without such features, you are really better off with MVC-style solutions with server-side HTML templating engine like ASP.NET MVC/Razor, DJango and Ruby on Rails). 

These solutions attempt to reduce the amount of front-end code (HTML or JavaScript) by making the framework that generates the HTML also generate a lot of helper JavaScript code for you.  Some of these do things well (MVC unobtrusive validation isn't bad) while others do it poorly (ASP.NET Content Panels) but as soon as you need to do something slightly different, you either need to modify the framework (if you can) or write a buch of exceptional front-end code to get it the way you want. 

Server-side Frameworks with UI Controls 

Unlike frameworks like Vaadin however, the developer is responsible for defining the general website HTML, and any JavaScript required to support the controls used, which re-introduces the original productivity drains mentioned.  The benefit is that the developer still doesn't need to build the controls, and often the controls have a well-defined integration pattern with different back-end technologies. 

There are half-assed measures like ASP.NET Content Panels (and most of the ASP.NET Ajax Extensions) and better solutions like ASP.Net MVC unobtrusive validation that allow the server-rendered HTML to also dynamically generate client-side code to provide client-side processing that would otherwise require a postback.   

More complete UI frameworks like KendoUI alleviate some of the consistency issues, but require considerable JavaScript boilerplate to implement and don't resolve many of the dynamic language and code duplication issues except to generally reduce the amount of code needed to use the controls vs writing your own control implementations. 

JavaScript turtles (all the way down) 

NodeJS proponents tout that a single codebase across front-end and back-end code will improve productivity, and why limit your front-end functionality when you can use that code in the back-end. 

A counter argument is that there will always be 'client specific' code and 'server specific' code – yes using a common language means reduced context switching fatigue, and you can share functionality between the front and back-end, but at the end of the day you are still writing code for two separate execution paths – your back-end won't have code that calls the back-end services, but you will certainly need that code in your front-end. 

This also emphasises the issues of untyped languages (unless you use something like TypeScript), and you are also removing all the years of .NET expertise in one fell swoop. 

.NET turtles 

I include this as an aside, as it is experimental and very early days, but with the growth of WebAssembly and tools like Blazor, it is possible to build .NET websites that run natively in the client browser.   

This doesn't quite work in the way one might expect, rather than incorporating something like ASP.NET MVC in the browser, it uses stand-alone razor-syntax pages defining layout and functionality within the page.   In many respects it is a step away from "good architectural separation of concerns" but to some extend does align to component-based web design models like React and Vue, so with a better application state management engine this could work well. 

Embracing full-stack development 

Treating the client-side as a first-class citizen in your solution doesn't directly address the productivity issues raised, but can indirectly make a dramatic difference. 

Familiarity with the language and strong design patterns (SOLID principles, modularistion, and dependency management), modern tooling (modern VS, and VS Code, ESLint) goes a long way to making JavaScript work well.   

The more familiar you are with something, the less likely you are to make basic mistakes.  Untyped languages will always risk spelling, capitalisation or type inconsistencies (a = 1, b = "1", a == b) so it takes time to be productive for people used to compilers picking these basic things up, but if you have a solid foundation (compared to the mess of global namespace objects and spaghetti events of early JavaScript and JQuery development) then these sorts of errors will be reduced.   

TypeScript 

The use of TypeScript can also address the type safety issues, and is a big draw for those who are used to relying on the compiler to catch simple errors.  The cost is learning a new language, and interoperability issues between TypeScript and JavaScript libraries, but for large and complex sites this could provide a significant improvement in productivity.   

The use of TypeScript or better familiarity with JavaScript will not address the fact that you will need to duplicate code between the front and back end when doing full stack development. 

Elmish and Fable are similar solutions using a functional paradigm (probably a step too far for most developers) 

Code Generation 

As noted in JavaScript turtles, the issue of code duplication isn't always relevant.  You will be required to write client specific code in multi-tier architecture regardless of whether it is one language or two.  

Synchronisation of identical code like DTOs (although as a dynamic language JavaScript really doesn't need this), static reference information (things like enums that are used to drive application logic), and front-end business logic (e.g. validation that you also want to ensure is applied server side) is something that does require tedious duplication when using different languages, so that is a good point to focus on.  

It is possible to use Roslyn inspectors to read metadata about your back-end code and emit basic javascript modules.   

You may take the DTO object and validation rules and generate JavaScript object validation code for each field, any Enums tagged for 'front end use' may be picked up and defined in a JavaScript module, or even building API modules for each of your MCV controller methods. 

Boilerplate reduction 

Rather than using code-generation, you can also use and define libraries to consume the back-end services as required.  You may use swagger-js to inspect and call your service libraries instead of defining (or generating) a list of URLs and 'fetch()' requests and promises for example. 

For validation you might actually create service endpoints that validate an object or field on the server side, and have simple library which performs that validation on the server.   

Standardisation and Experience 

Obviously the use Code Generation or Boilerplate libraries/functions comes at a development cost, either writing those libraries, or finding good ones and learning how to use them.  In the absence of that time, understanding where those solutions apply and appropriately abstracting them is a really good starting point.   

For example, annotate all types you WANT to expose to the front-end (if you aren't using a framework that does so automatically).  It takes ~10 lines of code to define a custom attribute, and ~20 characters to apply one. 

Similarly, abstract your front-end code into specific-purpose libraries with independent configuration (e.g. API libraries with a separate 'root url' defined as a depdency, and separate validation rules from validation execution and field application).  This will allow you to start to incorporate functionality later, even if you know you don't have the time or skills to solve those problems to begin with.   

Conclusions 

There are many valid reasons why .NET developers eschew JavaScript, but there are also a lot of opportunities and areas in which large improvements can be made to productivity without having to become an expert in all things JavaScript.   

The crux however is having someone to drive those efficiencies, and sufficient team engagement to take them on board.  The more engagement, the further you can help your team gain those efficiencies.  Even without the time to work on innovated frameworks and solutions, there are basics you can instill into your team such as modularisation, dependency management, and core library consistency to help reduce the sticker shock of picking up JavaScript.   

With enough team engagement you can bring in things like TypeScript, or UI frameworks like Vue, React or Angular, and beyond that, with dedicated support from your teams you can start to introduce tools and frameworks that can automate a lot of what is required.   

Service Mesh and API Management

A colleague of mine has been looking at a complete development platform transformation to better meet the needs of the organisation.  There are some very lofty goals he wants to achieve, and we've had some very interesting (and exciting) discussions on where he wants to head. 

One of the discussions we have had is in relation to the management of services and the difference between Service Mesh solutions and API Management.  On the surface there are a lot of overlaps between the two concerns, and it isn't immediately obvious what API Management solutions offer that Service Mesh solutions don't.  

Breaking it down, the Service Mesh should be responsible for ensuring your services are available.  Capabilities such as rich security models, automatic scaling, monitoring and control dashboards, and infrastructure abstraction are also core features of Service Mesh platforms.  Solutions like Istio can also provide advanced cross-cutting concerns such as service redirection, logging and caching without having to incorporate those capabilities in the underlying services.

API Management on the other hand is about how those services are exposed to consumers.  This is aimed at ensuring the people who need to use your services can access them, and have the necessary tools to use them.  Like a Service Mesh, API Management is concerned about access security, service failover and monitoring, but are not responsible for controlling the services themselves.  Instead the API Management layer is responsible for managing which APIs are accessible to which consumers, transforming the services to meet consumer constraints, setting service access limits, and defining billing and utilisation policies.

API management wasn't something my colleague had looked into, but it is an important part of defining how the services would be accessed.  The industry has a tendency to equate "API Management" with "Monetisation" and it is easy to discount API Management when looking at the capabilities of Service Mesh solutions, but when you consider the key differentiation between managing services and managing consumers, there's definitely value in looking at both.

Risk vs Progress

Last week I had to get my hands dirty setting up proprietary physical servers using a serial port connection and terminal interface.  It was an fun diversion and an important step in a long-running project.  I also thought it was interesting that on a 150k+ piece of hardware I was forced to trim plastic from the included serial port connector to fit the server.

It was not particularly difficult to complete, but it took a lot of work and effort to get that far; the subsequent data center installation this week will be an important milestone in the overall project.  It will also highlight the dichotomy between risk and progress, with a further 4 week wait until the equipment can be configured by the vendor.  Despite being arguably greater risk, the lack of alternatives meant that approval to physically configure the device could be granted (including hacking away at the connector), but because the software configuration elements can be performed by the vendor remotely it was considered prudent to ensure the vendor performed these activities despite the extended timeframe involved.

There's obviously a fine line between risk and progress, and I'm not one to advocate not following appropriate procedures, but when a process is well documented and really quite straight forward, it's hard not to get frustrated by overly risk averse engagements.

Working with Tuples - dereferencing tuples in function declarations

I've been trying to do more little code tidbits to ensure I keep my skills up since I'm rarely coding any more, and F# is a good way to stretch my skills in unfamiliar ways.

I've done a few "useful" things with it now and I'm really enjoying it.  One was purely for fun, another for personal use but not exactly fun, and another for actual work.

One thing I found difficult was working effectively with Tuples, which even though Records are so easy to use in F#, tuples are even easier to use and I have tended to use them perhaps more than I should.
Take this code, which basically just sums the "second value from each tuple in the list"

let list = [(a, 10);(b, 5)]
let aggregateValues = list |> List.sumBy(fun tuple -> snd tuple)

The readability of that is horrible as you start adding more complex types or multi-value tuples.


A really simple tip was shown to me from someone who has actually delivered F# projects

let aggregateValues = list |> List.sumBy(fun (category,value) -> value)

by simply dereferencing the tuple and using appropriate field names, it is much clearer what is happening.  Now I knew this could be done on things like match clauses but I didn't realise you could do this in a method declaration.