Writing a Custom NLP API Client

This document shows you how to write a simple custom NLP API client using JWT based authentication. This client can be used to submit documents to an NLP API server using the JSON input format.

JSON-based Document Submission

In this tutorial, we will submit documents to the NLP API using the JSON input format. This format uses the following fields:

1. id: a unique identifier for each document submitted. Must be unique within each submission, but can be re-used between submissions. This field is required.
2. category: one of the supported NLP API document categories must be used. If in doubt as to what kind of document category your document might be, a value of Clinical (case-insensitive) is suggested. This field is required.
3. subcategory: one of the supported NLP API document subcategories must be used. If in doubt as to what kind of document subcategory is present, a value of generic is suggested (case-insensitive) although only some of the categories accept generic as a subcategory. These are outlined in the NLP API specification. For subcategories with a space character in them please make sure they are provided inside quotes, e.g. "Discharge Summary" is a subcategory for the Clinical category. This field is required.
4. text - the text of the document, escaped properly for JSON formatting. This field is required.
5. type - a value of plain should be used. This field is optional and included for legacy reasons.

Multiple documents can be submitted in JSON-based document submission up to a maximum size of 2 MB of text per document and a maximum submission size of 500 MB.

An example of 2 NLP API input JSON-formatted documents is shown below. In the code examples later, save this JSON to a file and save the filename to an environment variable FILE_NAME.

[
  {
    "id": "18",
    "category": "Cardiology",
    "subcategory": "ECG",
    "type": "plain",
    "text": "Atrial fibrillation with RVR.  Left axis.  Non-specific ST-T wave abnormalities."
  },
  {
    "id": "7",
    "category": "Radiology",
    "subcategory": "CR",
    "type": "plain",
    "text": "INDICATION: Chest pain.\n\nCOMPARISON: None.\n\nFINDINGS:The lungs and pleural spaces are clear.  The cardiomediastinal silhouette is normal. No acute bony abnormality.\n\nIMPRESSION: Normal chest x-ray."
  }
]

Submission Process

The submission process involves the following steps:

1. Authentication with the NLP API IAM service to obtain a valid JWT which is used for authentication of all of the following steps
2. Submission of document(s) to the NLP API & obtaining the job task ID
3. Checking the NLP API repeatedly for job status using the task ID until status=success
4. Retrieval of processing results using the job task ID

A diagram showing these steps is shown below, omitting acknowledgements and status results:

Using cURL

To explain how to authenticate your credentials against the NLP API and submit documents for processing, this section provides a shell script that uses cURL for authentication and submitting documents that are in the JSON input format described above.

The script shows an example of submitting a document to the NLP API and getting the results. It uses only cURL and jq, which must be installed on the system running the script.

Some caveats about the script:

• It only supports JWT auth; HMAC auth not supported.
• It only supports JSON input format.

The command line option -F "file=@$FILE_NAME; type=application/json" in cURL sets the multipart form headers appropriately as per the NLP API specification.

#!/usr/bin/env bash
set -eou pipefail
IAM_HOST=iam.us.emtelligent.com
IAM_PORT=443
EMTELLIPRO_HOST=nlp.emtelligent.com
EMTELLIPRO_PORT=443
USER_NAME=<username>
PASSWORD='<password>'  #single-quotes needed to interpret password special characters as literals
FILE_NAME=<filename>
# Login; NB --data-urlencode needed to properly html encode special characters in password
AUTH_TOKEN=`curl -s --fail --show-error -d user_name=$USER_NAME --data-urlencode password=$PASSWORD -X POST "https://$IAM_HOST:$IAM_PORT/iam/v2/login" | jq --raw-output .auth_token`
# Submit job
TASK_ID=`curl -s --fail --show-error -H "Authorization: Bearer $AUTH_TOKEN" -F "file=@$FILE_NAME; type=application/json" -X POST "https://$EMTELLIPRO_HOST:$EMTELLIPRO_PORT/emtellipro/submit" | jq --raw-output .task_id`
# Wait for job to complete
while [ true ]; do
    STATUS=`curl -s --fail --show-error -H "Authorization: Bearer $AUTH_TOKEN"  -X GET "https://$EMTELLIPRO_HOST:$EMTELLIPRO_PORT/emtellipro/status/$TASK_ID?timeout=10" | jq --raw-output .state`
    case $STATUS in
        success)
            break;;
        failure)
            echo "Task failed."
            exit 1;;
        cancelled)
            echo "Task cancelled."
            exit 1;;
        *)
            continue;;
    esac
done
# Get results
curl -s --fail --show-error -H "Authorization: Bearer $AUTH_TOKEN" -X GET "https://$EMTELLIPRO_HOST:$EMTELLIPRO_PORT/emtellipro/result/$TASK_ID?format=emtellipro-json-2" | jq . 

This script has been validated to work with jq version 1.7 and cURL version 8.1.2, although it may work with other versions as well.

If debugging is required, users can dump out the cURL submission(s) by adding a --trace-ascii - command line option after curl.

Summary

The various authentication steps used in the cURL command above can be used to construct your own http requests using any http library in your favorite programming language. The shell script provides a step by step guide to writing your own custom client in any programming language of your choice. Please contact us if this guide does not meet your needs.