Architecture of API-Management In An API-First World — Part III

A detailed tour of the world of API Management and systems based on the API-First methodology.

Swagger was the standard for API definition until version 2. Version 3 is known as OpenAPI and is the right standard to follow.

This is the third chapter of a 3-piece article, describing API Management in API-First Architecture.

Chapter 1: Introduction and explanation of the different terms and concepts used in these posts.

Chapter 2: The 4 (some say 5) roles of an API Management system

Chapter 3: Thoughts about designing API-First systems in large organizations.

But First, the statement:
While the thoughts here are based on conversations and experience with real customers and systems, these are my theories, and do not reflect any project or job that I have been involved in.

A deeper dive into the term “API First”

On the surface, the concept of API-First means that we now build a system that allows external entities to interact with our system through many individual, stand-alone endpoints, enabling flexibility and the ability to grow in ways we weren’t able before.

But that gives us the understanding of API Driven application. I would like to discuss the “firstness” of this concept. Here’s how I explain the term from two different perspectives:

1. The API is first in the sense that it’s “front and center” of the solution being provided. The API is the product that you’re selling — you commit to your customer ( your back-end developer/the team from Finance/external partners who are using whatever it is you’re providing) that given a certain input, you will provide some output in response. A collection of those APIs is a module, which grows to an app, and that might become a system. But at the end of the day — you are communicating on the level of the single API.
2. The API is the first thing that gets defined. Gone are the days where the back-end developer finished their phase, ran a build and sent out an automatically generated OpenAPI document. The owner of the OpenAPI contract is no longer the developer, but the product or project manager. The output of the technical specification is (at least) an OpenAPI document.

Back to API Management.

We’ve gone through a quick tour of the API world and the purpose of jobs of an API Management system. I would like to focus on 3 points that concern me as an API architect, giving more of a bone to chew on rather than definitive guideline of “this is the way it should be done”:

1. How does a new service come to the world?
   New services are the off-springs of logical concepts that aim to provide an answer to a business problem. I’ve received many types of requests for services in the many years I’ve been a developer. They come in all shapes and forms — sometimes they are a bunch of WhatsApp text messages, scattered emails and a couple of phone calls, other customers have produced 60–100 page Word documents specifying what and how to build, and then there are those who provide the OpenAPI document and ask to fill in the code for the services described.
   While I’m clearly an advocate of the last method, I want to take a step back, and discuss the creation of that OpenAPI document that is sent out.
   In an API-First world, once we define the business goal we want to achieve, we break it down to independent stand-alone endpoints, each doing one task that is intuitively understood from it’s name.
   (We would also like these calls to be stateless, meaning that the call contains all the data and variables needed in order to perform the task at hand. However, that is not always possible, and trying to force this rule leads to complicated code or mediocre services. Try to follow this guideline, but delivering the best service is the main goal, and having maintainable code is second only to that priority. After those two come all our desires, methodologies, and best practices).
   These calls are defined and documented into an OpenAPI document which is agreed upon by all actors (architect, UI, Security, UX, Back-end, etc), and that document is given to the API Management team to load and enable everyone to interact with. At first, calling the services mentioned will just validate that the request is consistent with requirements, and return a stub representing a valid response. Gradually, as the providers behind the services develop the necessary business-logic, the stubs are removed and replaced with real code. We would like to have a Postman (or any other similar application) collection with all the calls defined, and that would act as a layer in our integration tests.
2. Everything Is A Service:
   The foundation of this concept, which is the core of API-First and API Economy was laid by Jeff Bazos in a very famous email he sent back in 2002 (Please read it). I’ve been giving a lot of thought as to how this should be best implemented, and a dedicated article will be written covering that. But for the scope of this conversation — modules and features should be developed in a mindset that whatever it is that is being written — it will be used by the person in the next cubicle and the team on the other side of the planet in the same way.
   API Management system and team should not only facilitate this direction, but be the ambassador and the enabler of all development which sees API as front and center of the operation. API Management is where security and accessibility meet (and clash). Pushing and facilitating an API-First mindset in an organization is not an easy task, and things that work for one organization might be a disaster for the other, but this is where this team shines and brings its most important value to the development pipeline.
3. Does API Management assume responsibility for business-logic and orchestration between different services?
   In the previous post I wrote the 4 roles of API Management — security, routing, logging, and monetization. I wrote that there are those who would place orchestration as the fifth role, and that I will address it in a later post. Here we go:
   There are several ways of business-logic and orchestration that we would expect API Management to assume responsibility for:
   I. Orchestration I: what if we have a case where one request requires hops to several different services — go to service A and get user data, service B will confirm that this user is eligible for the service, and the third service actually gets the data requested by the consumer. Should we ask the API Management to run through these hops, or expect the consumer to send three different requests?
   II. Orchestration II: What if we have a case where the provider expects the data as a header, but the request send the data as part of the payload, or (more acceptable) the data needed is an attribute in the JWT or signature, and we want to extract it and add it to the request header? What if there’s a mismatch between consumer and producer where one sends capital letters and the other expects lower-case, and changing either side is too expensive?
   III. Business Logic: Should the API Management system handle business-logic? There’s a very wide range of use-cases, but here’s one that is very likely to be API Management’s job: a request contains an ID number, or IBAN String. These parameters contain a digit (or more) that is there as a security number, just to make sure the number is correct. There’s a straight-forward algorithm to validate the sent parameter. A failure with an invalid parameter results in a 400 error response, clearly in the domain of API Management. Should API Management handle that?
   My answer on all three accounts is that it’s complicated. It really depends on the team and the dynamics within the organization. The reason to keep these tasks under the API Management’s responsibility is that this is stuff done before we “talk business”, and we want our providers to focus on providing their services and eliminate background noise that can be handled by others. However, there are two counter arguments for keeping API Management out of these tasks: One: We want the other 4 roles to be provided with a much higher priority than this one. An excellent system which allows for quick API development, serving routing, security, logging and monetization is far better than a good system where publishing new APIs takes longer and more error prone. Two: Tracing failures when they occur is very difficult when we have a lot of actors involved. Finding the cause of unexpected changes can take a long time and cost a lot of resources. Leaving the responsibility at both ends of the request/response with minimal intervention in between leads to a better maintained flow.
   To summarize this last point, I would say that yes — API Management can handle some orchestration and business-logic, but I would keep it to as little as possible, and use it only when it makes no sense to leave it to others. Still, this approach changes based on the people and the dynamics in the organization.

That’s it. This is the end of a 3-piece overview of API Management as I see it in a world of API-First and API Economy. There are many more issues to tackle in this field, but this should give us a good start.

Comments, questions, and additions are always welcome!