API Design Masterclass

Software engineers deal with APIs on a daily basis, whether it be in the context of designing, implementing, consuming or integration and strategy in the architecture context. It's important to understand what is out there and what goes into designing good APIs.

What is API?

API stands for Application Programming Interface. An API defines how to integrate with an application or service or library.

Here are three examples to drive this home:

First example is a web RESTful API for fetching stock prices. A consumer of the API can send a request to the API server to get current price of a stock. Typically, this would be a "HTTP" "GET" request to the API endpoint URI (e.g. GET /stocks). Input to the operation would be a stock symbol as string, which in this context would typically be passed as part of the URI (e.g. /stocks/AAPL). And API will respond with the current price of that stock among other details.

People typically refer to web APIs, most often REST when they talk about APIs. But the world of software is full of all types of APIs.

Second example of APIs is a library. Maybe you need to do some machine learning. You could pull in a library like svm or tensorflow. There is going to be a way to describe your model and feed training data to train and save your model. There is going to be a way to load your model and then use it for making predictions.

Third example is computing hardware. There is going to be a way of encoding instructions to the hardware so that it can run software. If we go one level up, programming languages provide a nice high level application programming interface to produce instructions that the hardware can understand. You specify your well defined program to a programming language compiler and it produces instructions that the hardware can understand.

Last one may feel weird for a second because it is a bit of a stretch. But think about GraphQL APIs for a second. API call is done with a Domain Specific Language (DSL) accompanied with input/output data in JSON. With that perspective, programming languages are that different.

In all three examples, the API defines how to get the API provider to do something of value for the API consumer.

A less vague metaphor might be that of remote procedure call. API consumer invokes a procedure on the API provider. Abstractions may be built on it (e.g. REST provides a resource management abstraction) but the underlying idea remains valid. Despite the abstractions, there is no way to service API calls without running some kind of procedure or code on the API provider side. If you don't have to deal with it, some abstraction you are using is doing it for you.

Layers: What level is your API at?

One of the great things to appreciate in the technology landscape is that things including APIs are always built in layers.

For example, RESTful APIs are layer 7 APIs. Here is a typical layer diagram of a REST API:

In this diagram, your RESTful API will define what operations the API supports. It will have many components such what data structures are used for each operation, how to authenticate requests, interpret responses and handle errors etc. They will all be defined in terms of HTTP concepts which is the layer/protocol/abstraction below it.

This next layer can be fulfilled by any HTTP compatible layer whether it be plain HTTP or HTTPS (HTTP on top of TLS) or HTTP/2/3/QUIC (5-6). This layer will typically sit on top of TCP/UDP (Layers 4-5) which in turn will sit on top of IP (Layer 3). These will in turn sit on lower level data link protocols such as MAC or PPP which typically connects you to the internet. These will be built on physical layer / connections which will concerns itself with physical transmission medium and channels and will define how bits and bytes are transmitted.

Isn't it amazing that you can design a RESTful API without having to worry how bits and bytes will be transmitted in the wires that connect you to the internet on the ocean bed? That is the beauty of layered architecture that when layers are designed properly, you can build complex systems with only knowledge of the layer below whatever you are designing.

You can also have higher level APIs defined on top of Generic REST APIs. For example, you could have a generic API for running tasks in a platform. You could then build a layer on top of it for running machine learning tasks.