> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.emtelligent.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.emtelligent.com/_mcp/server. # Release notes for NLP API 2.31 ## Contents * [Brief Summary](#brief-summary) * [New Features](#new-features) * [Changes](#changes) * [Python SDK updates](#python-sdk-updates) * [Java SDK updates](#java-sdk-updates) ## Brief Summary ### Release versions * NLP API version 2 Release 31. * Python SDK and client v5.26.0. * Java SDK and client v5.14.0. * Python SDK CCD reader plugin v2.2.0. * [NLP API specification v2.17.0](/nlp/api): added support for new processing features `radplaybook-ontology` and `entity-guidance`. * JSON result output schema v2.26.1 for [emtellipro-json-2](/nlp/reference/result-format): added new found entity attribute `guidance`, new ontology `radplaybook-ontology`, new temporality category `specific`, new `date_time` argument added to medication relations and added ontology version information. * [visualclient](https://visualclient.emtelligent.com) version `vc-v2023.03.17-dadc23b8` added support for new processing features `entity-guidance` and `radplaybook-ontology` and new `specific` category of temporality relation. * New version of the Snowflake CSV upload SQL script . * New description and relationship tables for SNOMED September 2022 release available on the [downloads page](https://legacy-docs.emtelligent.com/emtellipro/downloads.html). ### Highlights * Added the rad-playbook (LOINC/RSNA Radiology Playbook) ontology. * Guidance: new attribute for found entities. * New `specific` category for temporality relations. * The version of every ontology in the NLP API output is provided in the result JSON. * Added `date_time` argument to medication relations. ## New Features ### rad-playbook (LOINC/RSNA Radiology Playbook) Ontology From [the LOINC/RSNA Radiology Playbook website](https://www.rsna.org/practice-tools/data-tools-and-standards/radlex-radiology-lexicon/procedure-names-radlex-playbook) > The wide variation in the way radiology procedures are named in radiology departments makes ordering and protocoling less efficient and radiology records difficult to aggregate or compare across sites. The LOINC/RSNA Radiology Playbook was created to address this critical problem. Building on RadLex, RSNA developed names for nearly 1,000 commonly performed radiology procedures. Playbook provides a standard system for naming radiology procedures, based on the elements that define an imaging exam such as modality and body part. In the output json result format `v2.26.1` or later the NLP API adds a new ontology `rad_playbook` in the output controlled by the NLP API processing feature `radplaybook-ontology`. A link to the concept will appear for each found entity that has an rad-playbook code. The link will appear in the list of concept_links for each found entity (in the list found in the `entities` object). The example output below is truncated to the relevant parts (the `...` indicate the removal of unnecessary details). ```javascript { "documents": [ { ... "entities": { "found": [ { ... "concept_links": [ ... "Crad_playbook0" ], "entity_type": { "rad_playbook": "CT Chest" }, "text": [ "CT Chest" ] } ], } ... "concepts": { ... "rad_playbook": [ { "label": "Crad_playbook0", "concept_id": "24627-2", "description": "CT CHEST" } ] } ... } ], "format": "emtellipro-json-2", "version": "2.26.1", } ``` ### Guidance attribute (beta) This new attribute captures whether this entity is part of a guidance statement, where someone is suggesting that someone else do something. Say a caregiver writes 'If the patient experiences abdominal pain, she should return to the ER.' In this case, 'abdominal pain' is not something the patient currently has, but is part of a guidance statement, and thus the value of this attribute for 'abdominal pain' would be `is_guidance`. If an entity is not part of a guidance statement, it is assigned a value of `not_guidance`. This attribute is present in the output of the NLP API if the `entity-guidance` feature is enabled and only for Radiology reports and only in `FINDINGS`, `IMPRESSION` and `ADDENDUM` sections. For the following input processed as a `Radiology/generic` by the NLP API: ```text IMPRESSION: If there is ongoing concern for pneumothorax, an upright chest x-ray should be obtained. ``` The NLP API adds a new attribute `guidance` to the entity information for `found` entities in the output JSON result format `v2.26.1` or later. The output JSON for the above sentence is shown below. The focus in the example below is on just the new `guidance` attribute. The example output below is truncated to the relevant parts (the `...` indicate the removal of unnecessary details). ```javascript { "documents": [ { ... "entities": { "found": [ { ... "section_name": "IMPRESSION", "attributes": { ... "guidance": "is_guidance" }, ... "text": [ "chest x-ray" ] }, ... ], } ... "format": "emtellipro-json-2", "version": "2.26.1" } } ``` ### `specific` category for temporality relations The temporality relation in the NLP API indicates whether the `subject` entity is in scope of a `temporal_entity`. Currently the supported categories of temporality are `historical` and `specific`. The `historical` category was released previously in the NLP API v2.28. In this release we add a new category called `specific` to the temporality relation. The `specific` category indicates that the subject entity (like `diabetes`) is within the scope of a specific time, date, or date range (`since 1982`) in the sentence `He has had diabetes since 1982`. The `polarity` attribute for this relation indicates whether the temporality status itself is negated or asserted. This output will be present in NLP API output if the `temporality-relations` feature is enabled. For the following input processed as a `Clinical/generic` by the NLP API: ```text He has had diabetes since 1982. ``` The output JSON for the above sentence is shown below. The focus in the example below is on one relation, namely `temporalities`. The example output below is truncated to the relevant parts (the `...` indicate the removal of unnecessary details). ```javascript { "documents": [ { ... "relations": { ... "temporalities": [ { "label": "RT1", "attributes": { "polarity": "asserted", "category": "specific" }, "args": { "subject": { "ref": "E1", "text": [ "diabetes" ] }, "modifiers": [], "temporal_entity": { "ref": "E0", "text": [ "since 1982" ] } } } ] }, ... "format": "emtellipro-json-2", "version": "2.26.1" } } ``` ### Ontology versions The version of every ontology in the NLP API output is provided in the result JSON as a new object called `ontology_versions`. The output JSON for the above sentence is shown below. The focus in the example below is on one output feature, namely `ontology_versions`. The example output below is truncated to the relevant parts (the `...` indicate the removal of unnecessary details). ```javascript { "documents": [ { ... "ontology_versions": { "snomed": { "release_version": "2022-09-01", "release_type": "snapshot", "packaging_date": "2022-11-17", "language": "en_us" }, "radlex": { "release_version": "2020-11", "packaging_date": "2022-09-27" }, "rxnorm": { "release_version": "2022-11-07", "release_type": "full", "packaging_date": "2022-11-07" }, ... "rad_playbook": { "release_version": "2022-08-08", "packaging_date": "2022-11-10" } }, ... "format": "emtellipro-json-2", "version": "2.26.1" } } ``` Only the `release_version` is guaranteed to exist in each ontology description. All the other information is optional. ### New `date_time` argument in medication relations The new `date_time` arugment for medication relations appears as a new argument for medication relations and appears in the result JSON under `medications`. For the following input processed as a `Clinical/generic` by the NLP API: ```text • lisinopril 10 mg oral tablet, = 1 cap(s), PO, Once Daily, 10 cap(s), 09/22/2008, 10/02/2008, Ordered ``` The output JSON for the above sentence is shown below. The focus in the example below is on one relation, namely `medication-relations`. The example output below is truncated to the relevant parts (the `...`). ```javascript { "documents": [ { ... "relations": { "medications": [ { "label": "RMD0", "attributes": {}, "args": { "drug": { "ref": "E71", "text": [ "lisinopril" ] }, "dosage": [ { "ref": "E72", "text": [ "10 mg" ] } ], "route": [ { "ref": "E73", "text": [ "oral" ] }, { "ref": "E75", "text": [ "PO" ] } ], "quantity": [ { "ref": "E197", "text": [ "1" ] }, { "ref": "E198", "text": [ "10" ] } ], "mode": [ { "ref": "E74", "text": [ "tablet" ] }, { "ref": "E199", "text": [ "cap(s)" ] }, { "ref": "E200", "text": [ "cap(s)" ] } ], "frequency": [ { "ref": "E76", "text": [ "Once Daily" ] } ], "duration": [], "necessity": [], "modifier": [], "indication": [], "date_time": [ { "ref": "E78", "text": [ "09/22/2008" ] }, { "ref": "E79", "text": [ "10/02/2008" ] } ] } } ] } } ] "format": "emtellipro-json-2", "version": "2.26.1", } ``` ## Changes ### Ontology updates * SNOMED CT database updated to version `2022-09-01` US Edition. * Improved longer entity matching to SNOMED CT. * Added >50K new annotations of synonyms for SNOMED CT concepts. * RxNorm updated to November 2022 release. * UMLS updated to version `2022AB`. ### Changes to entities * Change SNOMED concept ID for temporal entities to `260863009 | Temporal relationship (attribute) |`. * Fix measurement value `emt_sem_type` (internal) / `semantic_type` (external) for SNOMED, corresponding to NLP API UMLS entity type. It was 'measurement' but is now 'Quantitative Concept'. ### Accuracy Improvements * Improved accuracy of imagelink relations. * Improved accuracy of measurement relations. * Improved accuracy of entity disambiguation for some common acronyms. * Improved accuracy of negation detection. * Improved accuracy of uncertainty detection. * Add new measurement units `CFU/ml` and `organisms/ml`. ### NLP API Improvements * New private interface for identity and access management. * Improvements to the NLP API admin UI. * Support for scalesets in cloud computing to use hundreds or thousands of workers as spot instances. ## Python SDK updates The latest NLP API Python SDK is version `5.26.0`. The last NLP API release `v2.30` was released with Python SDK version `5.25.0`. Refer to the [changelog for the Python SDK](/nlp/python/changelog) for a full explanation of the updates made since the last release. You will need to use at least version `5.7.0` of the Python SDK or later to send requests to the NLP API v2.29 but if you want to use the new features explained above you will need to use at least version `5.26.0`. We recommend using the latest version of the Python SDK `5.26.0` with the NLP API `v2.31`. > **Warning** > > To use existing NLP API databases with the new Python SDK you must do a data migration using the Python database client `migrate` subcommand: ```bash emtellipro-db-client --database postgresql://username:password@hostname:port/example_db migrate ``` ### rad-playbook (LOINC/RSNA Radiology Playbook) Ontology When using the `emtellipro-db-client` Python database client the `foundentityconcept` table contains `rad_playbook` and the values in the `found_entity_id` column can be used to find the mapping to the entity in the `foundentityconcept` table: ```bash # select * from foundentityconcept where concept_ontology='rad_playbook' limit 5; found_entity_id | concept_id | concept_ontology -----------------+------------+------------------ 2521 | 39026-0 | rad_playbook 5963 | 24627-2 | rad_playbook 5585 | 39026-0 | rad_playbook 6731 | 39026-0 | rad_playbook 226 | 39026-0 | rad_playbook (5 rows) # select * from foundentityspan where found_entity_id='2521'; id | start | end | text | found_entity_id ------+-------+------+------+----------------- 2521 | 1235 | 1237 | CT | 2521 (1 row) ``` ### Guidance attribute (beta) Support has been added for the `guidance` attribute of found entities. This is now found in the SDK as `FoundEntity.guidance` (where it is stored as a string), and it’s stored in the database in the new `foundentity.guidance` column. The new feature is called `entity-guidance`. Using this updated table will require a database migration. When using the `emtellipro-db-client` Python database client the `foundentity` table contains a column for the `guidance` attribute. The values can be `is_guidance` or `not_guidance`. `rad_playbook` and the values in the `found_entity_id` column can be used to find the mapping to the entity in the `foundentityconcept` table: ```bash # select * from foundentity where guidance='is_guidance' limit 2; id | polarity | section_name | uncertainty | known_ambiguity | question_status | guidance | document_id ----+----------+--------------+-------------+-----------------+-----------------+-------------+------------- 1 | asserted | FINDINGS | certain | unknown | not_question | is_guidance | 1 15 | asserted | FINDINGS | certain | unknown | not_question | is_guidance | 1 (2 rows) # select * from foundentity where guidance='not_guidance' limit 2; id | polarity | section_name | uncertainty | known_ambiguity | question_status | guidance | document_id ----+----------+--------------+-------------+-----------------+-----------------+--------------+------------- 2 | asserted | INDICATION | certain | unambiguous | not_question | not_guidance | 1 3 | asserted | TECHNIQUE | certain | unknown | not_question | not_guidance | 1 (2 rows) ``` You can use the `id` from the `foundentity` table to find out more information about the entity: ```bash # select * from foundentityspan where found_entity_id=1; id | start | end | text | found_entity_id ----+-------+-----+----------+----------------- 1 | 2 | 10 | CT CHEST | 1 (1 row) # select * from foundentityspan where found_entity_id=2; id | start | end | text | found_entity_id ----+-------+-----+--------+----------------- 2 | 53 | 59 | distal | 2 (1 row) ``` ### `specific` category for temporality relations The new `specific` category for temporality relations are stored in the same tables as the earlier `historical` category for this relation: * `temporalityrelation` with links to two argument entities: the subject argument `subject_id` and the temporal entity argument `temporal_entity_id`. * `temporalityrelationmodifier` (a list of modifier arguments for this relation) ```bash # select * from temporalityrelation where category='specific' limit 2; id | polarity | category | temporal_entity_id | subject_id ----+----------+----------+--------------------+------------ 7 | asserted | specific | 200 | 201 10 | asserted | specific | 565 | 440 (2 rows) ``` The `temporal_entity_id` can be viewed in the `entity` table: ```bash # select * from entity where id='200'; id | type_ | document_id | entity_id -----+-------+-------------+----------- 200 | found | 2 | 199 (1 row) ``` The text span information for this temporal entity argument can then be viewed in the `foundentityspan` table: ```bash # select * from foundentityspan where id='199'; id | start | end | text | found_entity_id -----+-------+-----+-----------+----------------- 199 | 151 | 160 | 2021-5-30 | 199 (1 row) ``` A sample SQL query for temporality relations is provided below: ```text SELECT d.filename, t.category AS category, sub_fes.text AS subject, temporal_fes.text AS temporal, s.text AS sentence FROM temporalityrelation t JOIN entity sub_e ON t.subject_id = sub_e.id AND sub_e.type_ = 'found' JOIN foundentity sub_fe ON sub_e.entity_id = sub_fe.id JOIN foundentityspan sub_fes ON sub_e.entity_id = sub_fes.id JOIN foundentitylocation sub_fel on sub_fe.id = sub_fel.found_entity_id JOIN location l on sub_fel.location_id = l.id AND l.type_ = 'sentence' JOIN sentencelocation s on l.location_id = s.id JOIN document d ON l.document_id = d.id JOIN entity temporal_e ON t.temporal_entity_id = temporal_e.id AND temporal_e.type_ = 'found' JOIN foundentity temporal_fe ON temporal_e.entity_id = temporal_fe.id JOIN foundentityspan temporal_fes ON temporal_fe.id = temporal_fes.found_entity_id; ``` ### Ontology versions Support is now available for the `ontology_versions` information returned by the NLP API for each processed document. These are available as `AnnotatedDocument.ontology_versions` and in the `ontologyversion` table. The database migration will insert rows in the `ontologyversion` table based on the `documentconcepts` table, but with `release_version = NULL` since previous ontology version data is not available. ```bash # select * from ontologyversion; document_id | ontology | release_version -------------+-----------------+----------------- 1 | snomed | 2022-09-01 1 | radlex | 2020-11 1 | rxnorm | 2022-11-07 1 | rad_playbook | 2022-08-08 ... ``` Since each document in a batch might be processed with a different set of ontologies, the release version information is repeated for each `document_id`. ### New `date_time` argument in medication relations The new `date_time` argument of medication relations is stored in the Python SDK in `emtellipro.data.MedicationRelation.date_times` (as a list), and in the `medicationrelationdatetime` database table when using `emtellipro-db-client`. ## Java SDK updates The latest NLP API Java SDK is version `5.14.0`. Refer to the [changelog for the Java SDK](https://legacy-docs.emtelligent.com/emtellipro-java-sdk/v5.15.0/) for a full explanation of the updates made since the last public release. You will need to use version `5.14.0` or later to take advantage of the new features described above.