Quickstart

This guide shows you how to use the basic NLP API Python client to submit data for processing to the NLP API and examine the result returned by the NLP API.

This tutorial was developed and tested on Linux with Python 3.10 or higher but you can use it in other environments as long as you have access to command line tools.

Prerequisites

First, check that you have received the following from us:

1. A trial or license agreement from us to use the API
2. Public and secret keys for your API account
3. The API URL to which you’ll be submitting your reports. The URL will be something like this https://nlp.emtelligent.com/emtellipro

Set up the environment variables used in this guide for access to the NLP API. You will need to set the following environment variables for the NLP API and your access and secret keys. If you do not have access to the appropriate values for these variables please contact us.

$ export EMTELLIPRO_SERVER=''
$ export EMTELLIPRO_ACCESS_KEY=''
$ export EMTELLIPRO_SHARED_SECRET=''

If you don’t have the above information, please contact us at support@emtelligent.com to get your account set up.

Step 1. Download the Python SDK

Download the latest Python SDK package here.

Extract the zip file.

unzip /path/to/emtellipro-python-sdk-7.2.0.zip

The zip file extracts to a folder that contains:

• A python wheel which installs NLP API clients and libraries:
  • emtellipro-db-client: This client submits reports to the NLP API and inserts the structured output into a SQL database.
• Sample medical reports in /path/to/emtellipro-python-sdk/example-data (e.g. sample CT scan report)

Step 2. Install the Python SDK

Check the NLP API Python SDK system requirements.

Optionally create a virtual environment and then install the NLP API Python SDK.

$ python3 -m venv venv
$ source venv/bin/activate
(venv) $ pip install emtellipro-7.2.0-py3-none-any.whl

The emtellipro-client should now be installed. Check by printing out the client version number as follows:

(venv) $ emtellipro-client --version

Test that your API keys work with the NLP API:

(venv) $ emtellipro-db-client \
           get-user \
           --access-key $EMTELLIPRO_ACCESS_KEY \
           --secret-key $EMTELLIPRO_SHARED_SECRET \
           --server $EMTELLIPRO_SERVER

If you do not have these keys or they are invalid, please contact support@emtelligent.com.

Step 3. Process a Document

Let’s start off by processing a medical text file and turning it into structured data, returned by the engine as a JSON file (see Understanding the NLP API Output).

We will now process this report example-data/sample_ct_report.txt by calling the emtellipro-db-client as follows:

$ emtellipro-db-client \
    process \
    --access-key $EMTELLIPRO_ACCESS_KEY \
    --secret-key $EMTELLIPRO_SHARED_SECRET \
    --server $EMTELLIPRO_SERVER \
    --output raw+json://./sample_output/ \
    example-data/sample_ct_report.txt

Which produces the following output which tracks the progress of the submission and then provides the time taken for processing the document.

Generating state file path...
Using state file path: /tmp/emtellipro-state-2cpvfnbq.db (fresh, autodelete)
Testing emtelliPro connection...
Successful connection to https://emtellipro/
Connected as USERNAME
Looking for input documents...
Reading:    100%|██████████████████████████████████████| 1/1 files [00:00<00:00]
Loaded 1 files containing 1 documents.
Submitting 1 documents...
Processing: 100%|███████████████████████████████████████| 1/1 docs [00:01<00:00]
Storing:    100%|███████████████████████████████████████| 1/1 docs [00:01<00:00]
Processed 1 reports in 1.3s for an average per report processing time of 1.30172s per report
Saved 1 documents to database in .03s (.030668s / document)

The command we’ve run has created a directory called sample_output/ in which it stored the output of the processing job. Although we’re only processing one document, it creates a directory because we can use the command to process large numbers of documents that must be split into multiple processing tasks when submitting to the NLP API; each task will have its own JSON file in the directory.

Let’s look at the files created:

$ ls sample_output/
3b4ed20696dc4b1bba164cb0772a8244.input  3b4ed20696dc4b1bba164cb0772a8244.json  3b4ed20696dc4b1bba164cb0772a8244.map

The 3b4ed20696dc4b1bba164cb0772a8244 string is the task ID from the NLP API. Each task has a unique ID, so yours will be different.

The file we care about is 3b4ed20696dc4b1bba164cb0772a8244.json which contains the result from the NLP API for that task; the other files simply track the which input files were associated with the task.

For more detailed information about using the client see: Database client usage.

You can selectively enable extraction features and ontologies and specify the document type. For more information about processing options and supported document types, see Submitting Documents to NLP API.

Step 4. View the Results

Let’s take a look at the result file:

$ python -m json.tool ./sample_output/3b4ed20696dc4b1bba164cb0772a8244.json | less

You should get a prettyprint version that is a bit more readable. When you do that, the output you see should look something like this:

{
    "format": "emtellipro-json-2",
    "version": "2.28.0",
    "engine-version": "build-3036",
    "documents": [
        ...
    ]
}

To learn more about the JSON output see Understanding the NLP API Output.

Convert the output JSON into flattened TSV files. First change your directory

$ rm -rf tsv_output
$ emtellipro-flatten-json \
    -o tsv_output \
    -O tsv \
    sample_output/*.json

The structured data from the JSON file can be viewed as the following TSV files in the directory tsv_output that contain the individual types of information extracted by emtellipro:

concepts.tsv                           found-entity-concepts-rad_playbook.tsv     found-entity-concepts-umls_nci.tsv          found-entity-entity-types-snomed_icd10_cm.tsv  relations-experiencer.tsv  sentences.tsv
entities-assumed.tsv                   found-entity-concepts-rxnorm.tsv           found-entity-entity-types-emtelligent.tsv   found-entity-entity-types-snomed_icd10.tsv     relations-measurement.tsv
entities-found.tsv                     found-entity-concepts-snomed_icd10_cm.tsv  found-entity-entity-types-medcin.tsv        found-entity-entity-types-snomed.tsv           relations-medication.tsv
found-entity-concepts-emtelligent.tsv  found-entity-concepts-snomed_icd10.tsv     found-entity-entity-types-radlex.tsv        found-entity-entity-types-umls_loinc.tsv       relations-qualifier.tsv
found-entity-concepts-medcin.tsv       found-entity-concepts-snomed.tsv           found-entity-entity-types-rad_playbook.tsv  found-entity-entity-types-umls_nci.tsv         relations-temporality.tsv
found-entity-concepts-radlex.tsv       found-entity-concepts-umls_loinc.tsv       found-entity-entity-types-rxnorm.tsv        found-entity-entity-types-umls.tsv             sections.tsv

For example, one of the many structured data outputs is relations-measurement.tsv. The output (shown below) contains measurement relations extracted and structured from the text in the report. The other TSV files contain the many different types of structured output produced by the NLP API. To learn more about the NLP API output see Understanding the NLP API Output.

$ cat tsv_output/relations-measurement.tsv
sentences_text  subject value
Within the left parietal lobe, there is a 1.2 x 2.1 x 1.8 cm (ML x AP x CC) mass which demonstrates central low attenuation, and extensive surrounding white matter low attenuation change consistent with edema.       mass    1.2 x 2.1 x 1.8 cm (ML x AP x CC

Next Steps

For more complex processing, it’s useful to look at the database client, which provides all the functionality of the basic client, plus saving to a database.