Data APIs are going the way of the dodo: What you need to know

The API economy has been declared dead and resurrected a half dozen times over the last five years. From Netflix and Twitter turning off their fire hoses to API management tools like Apigee and Mashery being acquired by corporate giants, APIs have received more than their fair share of attention.

But not all APIs are created equal. At Nexla, we think of APIs are belonging to one of two categories: service or data. A service API is a building block for a developer, a way to hook into another application’s functionality. It’s how developers can build apps in Slack, or add google maps to a web app. Data APIs on the other hand are a bit more limited. They allow developers to pull data from a source and then use that data however they see fit. They don’t offer any additional functionality or services. And that’s why they’re facing extinction.

How Data APIs are Born

First, your developer needs to identify the data source, such as a database or a data warehouse,  from which the data will come. Next the developer will use their favorite programming language and libraries to access that data source. They may then modify and filter data and apply access control rules to restrict what data can be available to whom. Then the developer will test the API, write the output, document it, and finally ship it. Congratulations! A new data API is born. That was easy. Why would this fun process ever become extinct?

The Data API Grows Up- Sort Of

Partners love the new data API. Except, they would love it even more if you could just add this one little tweak. Oh, and they’d be willing to pay for it if you added these other fields. Geez, did you get that email from that one partner who needs an increase in his rate limit or he’s going to stop paying altogether? Ok, due to the popularity of the data API it’s time for some adjustments.

The developer will create and document a new version of the API- version 2.0. Because of the new bells and whistles, it now needs authorization and authentication. The auth code is more complex than the API, and your developer is starting to point out you need security. They’re probably also starting to raise concerns about scale for more customers and more frequent access. But nothing is as simple as it seems because you still need to support good ol’ version 1.0 since not all customers will move to a new version at the same time. Phew. This is starting to get complicated.

Mixing and Mingling

Where is all that sought-after data going, anyway? To your partners. When the API was new, the first integration was relatively easy for the partner. Your partners created great dashboards and models for this data in their favorite programming languages. Some of them have even provided those models and services to their partners. You could say it’s an ecosystem. That’s great, isn’t it? It is, until you release Data API version 3.0 and drop support for version 1.0. Partner X never migrated off the original API and their customers are now seeing errors in their models and dashboards. Partner X hits the panic button: It’s all hands on deck over the weekend to commit a rapid fix to the API. Developers are integrating, testing, and checking downstream performance until late Sunday night. Needless to say, there’s a post mortem scheduled for 3pm on Monday and the Partner X team is burnt out.

Survival of the Fittest

The above scenario is grossly oversimplified of course. Typically, a data API provider would use a service to help manage their APIs- to help with documentation, access, rate limiting, security, and other tasks. But the developer still needs to create new versions of the API and sunset old ones, creating a significant workload. What if, instead of writing and supporting code to move data, we connected to the source and the data destination directly? No code for the data owner to manage, no weekend fire drills. What if there was a service that could transform and filter the data, allowed for built-in, fine-grained user authentication, and was easily adjustable by non-engineers for each partner’s exact needs? Then the Data API would go the way of the dodo.