Release notes for NLP API 2.23 | emtelligent Documentation

Summary

Added

New known_ambiguity entity attribute for found entities. Added in NLP API specification v2.13.0; JSON result output schema v2.15.0.
New qualifier relation type called location which identifies the location of various concepts, e.g. left, right, bilateral. Added in JSON result output schema v2.16.0
Support for new ontology snomed_icd10. Added in NLP API specification v2.14.0; JSON result output schema v2.17.0.
Updated medication relation model.
Several annotation updates to SNOMED and Medcin ontologies.

Fixed

Fix for temporal entities like 62 year old.
Bug fix for some entities that used to have empty SNOMED concept IDs.
Avoid false positives with adjectives and essential punctuation like commas and semi-colons.
Better detection of biomarkers and temporal expressions.

Compatibility

API version 2 Release 23
Python SDK and client v4.3.0 to v5.7.0. Latest version needed for new features.
Java SDK and client v3.0.2 to v5.2.0. Latest version needed for new features.
NLP API specification v2.14.0
JSON result output schema v2.17.0 for emtellipro-json-2

The latest version of the SDK is required for the new features in this release. The JSON result output schemas are backward compatible for all v2.x versions of emtellipro-json-2.

ICD10 codes

The processing feature for explicitly asking for ICD10 codes mapped from the SNOMED ontology is snomedicd10-ontology.

For the following example sentence:

Stage IV gastric adenocarcinoma.

The output from the engine using output json result format v2.17.0 adds a new ontology snomed_icd10 to the list of concepts. Each entity that has a corresponding ICD-10 code will have a reference to an ICD-10 code in the concept_links for that entity in the list of entities.

See the engine output for ICD10 codes below. The focus in the example below is on one entity and the linked concept that has the ICD10 code. The example output below is truncated to the relevant parts (the ... indicate the ellipsis of unnecessary details).

{
    "format": "emtellipro-json-2",
    "version": "2.17.0",
    "documents": [
        {
            ...
            "entities": {
                "found": [
                    ...
                    {
                        "label": "E1",
                        "spans": [ ... ],
                        "section_name": "Unknown Category",
                        "attributes": { ... }
                        "concept_links": [
                            ...
                            "Csnomed_icd100"
                        ],
                        "locations": [
                            "S0",
                            "SEC-0"
                        ],
                        "entity_type": { ... },
                        "text": [
                            "gastric adenocarcinoma"
                        ]
                    },
                    ...
                ],
                "assumed": [ ... ]
            },
            "concepts": {
                ...
                "snomed_icd10": [
                    {
                        "label": "Csnomed_icd100",
                        "concept_id": "C16.9",
                        "description": "TRUE || ALWAYS C16.9 | MAPPED FOLLOWING WHO GUIDANCE | POSSIBLE REQUIREMENT FOR MORPHOLOGY CODE (SNOMEDCT ICD10 code)"
                    },
                    ...
                ]
            },
            ...
        }
    ]
}

Understanding ICD-10 codes in the NLP API output

The concept_id value is the ICD-10 code which is created using the SNOMED to ICD-10 mapping published with the SNOMED CT ontology. The concept_id is the mapTarget ICD-10 classification code from this SNOMED CT to ICD-10 mapping table. The description in the NLP API output is the concatenation of the two following fields in the mapping table separated by a double bar ||. It contains useful information about the mapping into ICD-10 codes and can be used in postprocessing of the ICD-10 codes produced by the NLP API.

mapRule

A machine-processable truth statement created to evaluate to “true” or “false” at run-time and has one of three forms:

IFA Concept.id [= VALUE]: a mapRule which evaluates for the existence of one or more SNOMED CT concept instances including their descendant concepts, or an observable and value found in the patient record. The concept is designated by the SNOMED CT Identifier for a clinical finding or observable entity and the SNOMED CT fully specified name. In the case of rules for age at onset of clinical finding or current chronological age, it may include a range of allowable “VALUES”. If such an instance is found in the patient record at the time of rule evaluation, the rule is evaluated as “true” and the associated map target is selected for that mapGroup. Otherwise the rule evaluates as “false” and the run-time evaluation proceeds to the next Map member within the mapGroup. The mapAdvice for the record will include a readable statement relating the rule and map target.
TRUE: applied when a mapRule is not relevant.
OTHERWISE TRUE: employed when a series of rules must be evaluated to determine the valid map target. This is the case when none of the rules are satisfied or when there is no additional patient context information available.

mapAdvice

mapAdvice is human-readable textual advice that a software vendor may employ to inform the clinician user or the classification expert during a semi-automated mapping session. The mapAdvice used in the NLP API description field for ICD-10 codes have two components separated by vertical bar (|):

A summary statement of the mapRule logic in readable terms for the clinical user.
Supplementary metadata guidance intended to clarify the map for the coding professional. Metadata advice supported in the MAP includes the following values:

FIFTH CHARACTER REQUIRED TO FURTHER SPECIFY THE SITE
MAPPED FOLLOWING SNOMED GUIDANCE
MAPPED FOLLOWING WHO GUIDANCE
POSSIBLE REQUIREMENT FOR ADDITIONAL CODE TO FULLY DESCRIBE DISEASE OR CONDITION
POSSIBLE REQUIREMENT FOR AN EXTERNAL CAUSE CODE
POSSIBLE REQUIREMENT FOR CAUSATIVE AGENT CODE
POSSIBLE REQUIREMENT FOR MORPHOLOGY CODE
POSSIBLE REQUIREMENT FOR PLACE OF OCCURRENCE
THIS CODE IS NOT TO BE USED IN THE PRIMARY POSITION
THIS CODE MAY BE USED IN THE PRIMARY POSITION WHEN THE MANIFESTATION IS THE PRIMARY FOCUS OF CARE
THIS IS AN EXTERNAL CAUSE CODE FOR USE IN A SECONDARY POSITION
USE AS PRIMARY CODE ONLY IF SITE OF BURN UNSPECIFIED, OTHERWISE USE AS SUPPLEMENTARY CODE WITH CATEGORIES T20-T29 (BURNS)
THIS MAP REQUIRES A DAGGER CODE AS WELL AS AN ASTERISK CODE
MAP IS CONTEXT DEPENDENT FOR GENDER (obsolete)
DESCENDANTS NOT EXHAUSTIVELY MAPPED (obsolete)

New Qualifier Relation Type

Existing qualifier relation types supported by the NLP API are:

size/severity
status
duration/time
stage/grade
compliance

One new type of qualifier relations has been added:

location

For the following example sentences:

There is a stable nodule, demonstrated within the left axillary tail.
Unsucessful attempt at bilateral diagnostic mammogram.

The new qualifier relation type for location identifies that the axillary tail has location left and that mammogram location was bilateral.

For the sentence:

There is a stable nodule, demonstrated within the left axillary tail.

The output from the engine using output json result format v2.16.0 (or greater) is shown below. The example output below is truncated to the relevant parts (the ... indicate the ellipsis of unnecessary details).

{
    "format": "emtellipro-json-2",
    "version": "2.17.0",
    "documents": [
        {
            ...
            "entities": {
                "found": [
                    {
                        "label": "E1",
                        "spans": [ ... ],
                        ...
                        "text": [
                            "left"
                        ]
                    },
                    {
                        "label": "E2",
                        "spans": [ ... ],
                        ...
                        "text": [
                            "axillary tail"
                        ]
                    }
                ],
                "assumed": [ ... ]
            },
            "concepts": { ... },
                ...
                "qualifiers": [
                    {
                        "label": "RQ1",
                        "attributes": {
                            "qualifier_type": "location"
                        },
                        "args": {
                            "qualifier": {
                                "ref": "E1",
                                "text": [
                                    "left"
                                ]
                            },
                            "qualifies": {
                                "ref": "E2",
                                "text": [
                                    "axillary tail"
                                ]
                            }
                        }
                    }
                ],
                ...
            }
    ]
}

The processing feature to ask the engine to produce qualifer relations is qualifier-relations.

Known Ambiguity

We have introduced a new entity attribute called known_ambiguity (alongside polarity, uncertainty, etc.) which adds human expert annotations to describe if a mention that was found by the NLP API is an ambiguous concept or not. By ambiguous we mean that the mention span can be linked to more than one ontology concept.

Consider the following sentence:

62 year old patient has influ a and mrsa.

For this sentence, the NLP API identifies influ a with known_ambiguity of ambiguous and mrsa as unambiguous. This is produced in the output JSON as an attribute called known_ambiguity provided in the list of attributes associated with each found entity in the list of entities. The example output below is truncated to the relevant parts (the ... indicate the ellipsis of unnecessary details).

{
    "format": "emtellipro-json-2",
    "version": "2.17.0",
    "documents": [
        {
            ...
            "entities": {
                "found": [
                   {
                        "label": "E1",
                        "spans": [ ... ],
                        "section_name": "Unknown Category",
                        "attributes": {
                            "polarity": "asserted",
                            "uncertainty": "certain",
                            "known_ambiguity": "ambiguous"
                        },
                        "concept_links": [ ... ],
                        "locations": [ ... ],
                        "entity_type": { ... },
                        "text": [
                            "influ a"
                        ]
                    },
                    {
                        "label": "E2",
                        "spans": [ ... ],
                        "section_name": "Unknown Category",
                        "attributes": {
                            "polarity": "asserted",
                            "uncertainty": "certain",
                            "known_ambiguity": "unambiguous"
                        },
                        "concept_links": [ ... ],
                        "locations": [ ... ],
                        "entity_type": { ... },
                        "text": [
                            "mrsa"
                        ]
                    },
                    ...
                ],
                "assumed": [ ... ]
            },
            ...
    }
  ]
}

Python SDK updates

The new NLP API Python SDK is version 5.7.0. Refer to the changelog for the Python SDK for a full explanation of the updates made since the last public release which was version 5.5.0 released with the NLP API v2.22. You will need to use version 5.7.0 or later to take advantage of the new features described above.

ICD10 codes in Output Database

When using the emtellipro-db-client Python database client the table for foundentityconcept contains the ICD10 codes for each found entity.

 found_entity_id | concept_id | concept_ontology
-----------------+------------+------------------
              62 | I60.9      | snomed_icd10
              71 | I99        | snomed_icd10
              84 | I10        | snomed_icd10
             104 | G91.1      | snomed_icd10
             118 | I46.9      | snomed_icd10
             142 | J18.1      | snomed_icd10
            ...

See the tutorial page on Entity-level Tables to understand how to correctly identify and combine various entity-level information.

Qualifier relations of type “location” in Output Database

The new qualifier relation type appears in the qualifierrelation table in emtellipro_db (the Python database client):

 id | qualifier_type | qualifier_id | qualifies_id
----+----------------+--------------+--------------

The qualifier_type field can now contain location as the new type of qualifier relation.

Known ambiguity in Output Database

When using the emtellipro-db-client Python database client the table foundentity contains the attributes for each foundentity.id including the new known_ambiguity attribute. The foundentity information can be joined with other tables to access the information about the entity (span, concepts linked to the entity, etc.).

 id | polarity | section_name | uncertainty | known_ambiguity
----+----------+--------------+-------------+-----------------
  1 | asserted | INDICATION   | certain     | ambiguous
  2 | asserted | INDICATION   | certain     | unknown
  3 | asserted | INDICATION   | certain     | unknown
  4 | asserted | INDICATION   | certain     | unambiguous
  5 | asserted | INDICATION   | certain     | unambiguous
  6 | asserted | INDICATION   | certain     | unknown