Consistency, Coupling, and Complexity at the Edge

[ad_1]

Key Takeaways

  • RESTful API design is great for developing API first platforms but not so great for developing GUI applications.
  • GraphQL API Design is great for Backends for Frontends (BfFs) but not so great for data services.
  • Accidental complexity is a bad thing, and should be avoided if possible. It can be managed if architects enforce a clear separation of concerns in the various layers and components.
  • Coupling is a bad thing too; however, it also is inevitable. While architects should strive to keep the coupling loose between GUI and data services, it is understood that there will be tight coupling between frontend applications and BfFs.
  • In addition to putting a BfF between each GUI application and the data services, adopting an API gateway is also useful for maintaining a clear SoC.

     

It’s been over a decade since the current incarnation of microservice architecture has been embraced by technology companies, with mixed results. The benefits of microservices are limited when the dependencies between them are such that changes to one necessitate changes to the others. This is what is known as tight coupling. Dependencies between components are a fact of life.

What good is a microservice if it cannot deliver what the user wants? This article focuses on how to maintain useful microservices and still benefit from the process improvements promised by the migration of monoliths to microservices. The key is to maintain a clear separation of concerns in the various layers and employ design principles best suited to each layer.

RESTful API Design

In the year 2000, a co-founder of the Apache HTTP server project submitted a Ph.D. dissertation called “Architectural Styles and the Design of Network-Based Software Architectures.” Chapter 5 covers REST, or Representational State Transfer, as an architectural style for distributed hypermedia systems. A good resource for learning about REST as it serves in API design is this blog on the Richardson Maturity Model.

There is a close and consistent alignment between the design of APIs and the HTTP standard. With RESTful API design, the path part of the URI identifies a specific entity (also known as a resource) in a hierarchical taxonomy. The HTTP verb identifies the type of action to be made to the entity. Entities can link to other entities via the path part of the other entity’s URI. There is also consistency in how to interpret the meaning of the resulting HTTP status code.

There are many reasons why companies that provide online applications are strongly motivated to design their APIs as if they were a platform. Perhaps they wish to open up additional revenue streams from 3rd parties or upsell to power users. They may wish to make it easier for different teams to call each other’s APIs. They may wish to organize their APIs in such a way that they are easily reused by similar or related products in the same portfolio. Finally, they should design APIs that are easy to develop test automation for.

Just as companies develop GUI applications that are easy for users to understand, platform companies should develop APIs that are easy for developers to understand. RESTful APIs are well suited for this requirement.

In the days before RESTful API design, APIs were based on what is known as the Remote Procedure Call (RPC). There was no consistency in the design of these APIs. This made those APIs hard to reason about or understand what they do. REST introduced consistency in API design. When you combine REST with OpenAPI, you have a very easy way for developers to quickly learn how to use your APIs.

Swagger specification for a RESTful API that supports a rudimentary news feed.

Although RESTful APIs are easy for backend services to call, they are not so easy for frontend applications to call. That is because an emotionally satisfying user experience is not very RESTful. Users don’t want a GUI where entities are nicely segmented. They want to see everything all at once unless progressive disclosure is called for. For example, I don’t want to navigate through multiple screens to review my travel itinerary; I want to see the summary (including flights, car rental, and hotel reservation) all on one screen before I commit to making the purchase.

When a user navigates to a page on a web app or deep links into a Single Page Application (SPA) or a particular view in a mobile app, the frontend application needs to call the backend service to fetch the data needed to render the view. With RESTful APIs, it is unlikely that a single call will be able to get all the data. Typically, one call is made, then the frontend code iterates through the results of that call and makes more API calls per result item to get all the data needed. Not only does this complicate frontend development, but it also increases page load time for the application. More on this later.

Here is another issue where RESTful APIs are not a good fit for frontend GUI concerns. There is no support for push notifications in RESTful API design, per se, but there is support for callbacks (implemented by webhooks) in the open API spec. Webhooks don’t help as much for push notifications as WebSockets. Webhooks and WebSockets are different in that a web browser does not support webhooks but does support WebSockets. That is because the WebSocket is initiated by the frontend and stays connected while the backend sends updates to the frontend. A webhook is initiated by the backend but the browser doesn’t have a permanent IP address to receive those requests. Because the path identifies a specific entity in RESTful API design, the request and response shouldn’t change shape dramatically. To save on connections, an SPA may open a single WebSocket for all types of push notifications. The shape of each message could be very different.

New user requirements, such as additional data fields, may mandate change on both the frontend and backend. This is the fundamental cause for tight coupling. Tight coupling across teams slows the pace of development for reasons best understood via Conway’s law. 

Communication costs across teams are higher than within a single team. Teams that have a mix of both the frontend and backend developers can also be lacking effectiveness. While notionally on the same team, this structure often still has a divide between the frontend and backend developers. Almost as an unwritten addendum to Conway’s law, the hidden sub-teams hide some of the complexity of the software.  

Release management also gets complicated when backend services and frontend apps are tightly coupled. One of the biggest advantages to microservices is that you don’t have to release everything all at once. Tightly coupled components often do have to be released almost at the same time and both have to be rolled back if one has to be rolled back.

GraphQL API Design

In the year 2015, Facebook introduced a different approach to API design known as Graph Query Language (GQL) or GraphQL. It includes the specification of a schema of hierarchically organized types, including up to three special types: query, mutation, and subscription. The caller sends a command (consistent with the schema) that provides the criteria and specifies the shape of the data that it is expecting in the response.  

The most popular, full featured, and mature implementation of the framework for GraphQL servers is shepherded by a small San Francisco startup known as Apollo. Their approach makes it very easy to compose new functionality from the clients without a lot of changes to the server.

The backend developer has to code the schema and the resolvers. The framework calls the resolvers identified in the request then stitches together each resolver’s response in a way that matches what the caller specified.

GraphQL schema of a similar rudimentary news feed.

Because resolvers are at the attribute level and because the mechanism for fetching the underlying data may actually retrieve more than one attribute at a time, there is the possibility of wastefully fetching the same data repeatedly. This is what is known as the N+1 problem. Backend code should compensate for that with some kind of per-request cache.

Time to Live (TTL)-based, Least Recently…

[ad_2]

Read More:Consistency, Coupling, and Complexity at the Edge