How To Map Custom Field Names to Labels in JIRA

JIRA is a popular tool used by many software organizations to manage requirements, bugs, issues, and other agile development aspects. JIRA objects are customizable, and organizations often create custom fields for their issues or other objects.

Custom fields usually have useful labels, but the field names are not always easily identifiable. For example, you might have a field labeled “Project Stream”, but the name created by JIRA is something like “customfield_75801”. For some use cases, this can be confusing and inconvenient.

Companies often find value in pulling data out of JIRA for a variety of analytical use cases, including resource planning and operational metrics to drive efficiencies. Many of our customers use Nexla to easily extract data out of JIRA and load it to their data lake/warehouse/lakehouse.

The Challenge

One of the challenges that companies face is that the JIRA API used to fetch issues provides the internal id but not the associated label. Since the name is not easy to identify, it is difficult to relate the business analytics to the issues. JIRA does provide another API that produces the mapping between field names and labels, but the challenge then becomes renaming the target system column names to the contextual labels.

Before using Nexla, some customers dealt with this using complex SQL queries, which were hard to manage and scale.

The Solution

Nexla makes it easy to deal with JIRA custom field names when extracting data out of JIRA for analytics. Follow along with our example to see this for yourself.

At a high level, there are 4 main components of this procedure in Nexla:

1. Use the Nexla JIRA connector to ingest the field metadata, which contains the custom field name to label mapping, into a Nexla Dynamic Lookup.
2. Use the Nexla JIRA connector to create a source to ingest the issue records.
3. Create and use a Python transformation to transform the issue record column names from the custom name to the label.
4. Load the resulting data set into your target data lake, warehouse, lakehouse, or other system.

Step 1: Create a flow to pull JIRA metadata

First, we will walk through how to create a flow in Nexla to pull JIRA Field meta data and load it into a Nexla Dynamic Lookup.

The JIRA API for pulling field metadata is documented here: https://developer.atlassian.com/cloud/jira/platform/rest/v3/api-group-issue-fields/#api-group-issue-fields

In Nexla, create a new flow, select “JIRA” as the source, and click “Next”.

Provide your JIRA credentials, and click “Next”.

As you can see, Nexla has several built-in templates for reading data from JIRA. Since all of Nexla’s connectors are bi-directional, you can also push data into JIRA.

In addition to the templates, you can leverage the Advanced tab to query any available JIRA API. That’s what we will do here.

Click on Advanced, and set the URL to: https://<your-org>.atlassian.net/rest/api/latest/field  – Click “Test” to test it out.

Since we’re receiving an array of field metadata back, select the “$[*]” path suggestion to treat each element as an individual record.

Define the Schedule, and click “Create”.

Nexla will automatically create the first “Nexset”. From there, click on the transform icon.

We only need the “Name” and “Key” fields. Select these 2 fields on the left, click “Add Selected to Draft”, and then click “Save” on the upper right.

Activate the new Nexset that is created.

Click “Send to Destination”.

Search for “Dynamic Lookup”, select it, and then click “Next”.

Give your Dynamic Lookup a name, and set “key” as the primary key.

Click “Save”, and then Activate the destination:

At this point, the JIRA Issue Key & Name metadata will automatically be loaded into a Dynamic Lookup Table in Nexla. Each time it runs, it will upsert new values based on the Key field.

This concludes the first step.

Step 2: Connect JIRA to pull Issue Records

Now let’s use Nexla’s out-of-the-box JIRA connector to ingest JIRA Issue records.

Create a new Flow, select JIRA as the Source Type, and click “Next”.

Provide your credentials (or select one if you’ve already created a credential), and then click  “Save”.

There are a few built-in templates. In this example, we’ll use the one that fetches issues in a given project.

Select how often you want to ingest the JIRA Project data, and click “Create”.

Once the Nexset is detected, click on the “Transform” icon to apply the Python transformation.

Select the “Custom Code” radio button to go from the “no-code” editor to the code-based editor. In this example, we will use Python to build the custom transform.

Here’s the code that we will use:

def transform(input, metadata, args):
import json
output={}
output=input
fields = {}
fields = input.get(‘fields’)
fields_out = {}

for k, v in fields.items():
# Using the ‘nexla_fn.call()’ function, make a call to the “toMapValue2” Nexla function.
# This will allow us to look up values from a lookup table.
# 18973 is the ID of our dynamic lookup table created in step 1 of this example.
lookup_value = nexla_fn.call(‘toMapValue2’, 18973, k, ‘name’)
if lookup_value is not None:
fields_out[lookup_value] = fields[k]
else:
fields_out[k] = fields[k]

output.pop(‘fields’, None)
output[‘fields’] = fields_out
return output

Once you apply the changes, you will see that a lot of the “customfield_<id>” field names now have proper fields. However, some may still have deprecated fields, depending on how old the issues are.

Once you save this, you will have a new Nexset that represents your transformed data and can be transformed further or sent to the destination of your choice.

Conclusion

With Nexla, you can do the following to solve this type of complex use case very easily:

• Connect to pre-built sources like JIRA and ingest data using a variety of pre-built templates.
• Easily configure custom API sources for advanced use cases.
• Cache data in a ready to use “Dynamic Lookup” destination
• Not only do you have a wide variety of no-code transformations, you can also leverage custom code using Python or Javascript.

If you’re tired of default field names and the confusion that they cause during analytics, get a demo or book your free unified data solution strategy session today. For more on data and data solutions, check out the other articles on Nexla’s blog.