Delivering Data Products as an API

Nexla’s Data Feed API

Data and APIs can be found everywhere. In fact, Nexla’s CEO and cofounder, Saket Saurabh, describes how the API economy is really a data economy. Saket makes the case that APIs are ultimately a communication channel between applications and the language is data. For data product creators, the implication is that they will often need to consume source data from APIs as well as deliver their data products downstream via API.

In a previous post, we walked through how easy Nexla makes it to consume data from any API source. Carrying on with the same theme of data access via API, in this post, we’ll look at it from the other side of the coin: automatically making data products available via secure APIs.

Building APIs on top of databases or other data sources is not necessarily a novel idea. However, doing so with data from polyglot sources while providing a consistent interface can be a large undertaking, not to mention the operational aspects required to ensure security, uptime, and scalability.

Let’s see how Nexla makes it possible to access practically any data as an API.

First a quick primer on “Nexsets”

Using Nexla, you can rapidly connect to any data source. Once you do so, Nexla automatically generates a Nexset – logical data assets, which can also be thought of as ready-to-use data products. For more information, here’s a short intro video about Nexsets.

Now, let’s get to the APIs!

Example 1: Database table in Snowflake as an API

In our first example, I’ll show you how you can create an API on top of a table in a database. In this case, we will use a table from Snowflake.

Step 1: Select Snowflake as the Source Type. Snowflake is one of the hundreds of autogenerated out-of-the-box connectors.

Step 2: Provide the credentials:

Step 3: Select the Table and ingestion schedule and other settings:

Step 4: Drill into the autogenerated Nexset and find the “Data Feed URL”:

Click on “SHOW DATA FEED URL”:

And, there you go, in 4 simple steps, we have an API endpoint for a table in Snowflake:

Use this API to now fetch data from the table:

Example 2: Database Query results as an API

What if you wanted to provide an API on top of the results coming back from a specific query? What if that query had a “WHERE” clause? This would mean that the resulting API would need to accept parameters.

This is pretty easy to achieve in Nexla.

Step 1: Create another database source. This time, instead of selecting “Table Mode”, use “Query Mode”. Specify your query:

Note that the query has a special notation: SELECT * FROM “DEMO_DB”.”PUBLIC”.”SPACEX_FLIGHT_HISTORY_DEMO” WHERE EVENT_DATE_UTC > {FLIGHTS_SINCE=’2018-01-01′}. This notation: {FLIGHTS_SINCE=’2018-01-01′} allows you to specify a parameter called ‘FLIGHTS_SINCE’, the default value being ‘2018-01-01.

Now let’s go see what the API end point looks like. Follow the Step 4 from the first example and drill into the auto-generated Nexset to find the Data Feed URL:

This time you will notice that the REST method is “POST” instead of “GET”. Also, there is a sample payload with the default we specified in our query.

Example 3: Data from an API is transformed and served up as a new API

As you can see, once the Nexset is available from a source system, there’s a Data Feed API available automatically. You can also create new Nexsets (or derived data products) from the autogenerated Nexsets.

Let’s say, for example, you had data coming from an API which had some sensitive data fields which needed to be masked before making that data available for downstream consumption. Here’s how you can do that very easily in Nexla.

First, refer to this post on how to connect to an API source. Once you have that created, you can then “transform” the autogenerated Nexset to produce a new Nexset:

We will use the Nexla Transform Editor to copy some fields as-is, and also hash and/or mask some sensitive data. (Here are some videos if you are curious about the types of built-in transformations or how to build and share custom transformations using python/javascript check out these videos.)

This will create a new Nexset, which will have its own Data Feed API URL:

Now, you can serve the transformed data with this new API endpoint:

In this case you can see that the sensitive data has been masked and/or tokenized.

Wrap-Up

There are many use cases for accessing data as an API. The number and variety of such use cases continues to grow at a rapid pace. Nexla makes it easy to serve data downstream as an API. Not only is it easy to create these API endpoints, Nexla also takes care of making the APIs secure, scalable and highly available.

Want to learn more or try it out for yourself? Request a demo and free trial today!