> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.emtelligent.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.emtelligent.com/_mcp/server. # Release notes for NLP API 2.28 ## Contents * [Summary](#summary) * [Python SDK updates](#python-sdk-updates) * [Java SDK updates](#java-sdk-updates) ## Summary ### New Features #### Question Status attribute The new `question_status` attribute equals `is_question` in cases where the concept has been raised as a question to resolve. For example, `rule out appendicitis` where `appendicitis` will be `is_question`. In other cases, the attribute will have the value `not_question`. The `question_status` attribute is only applied to documents with the `Radiology` category and only in the `Indication` section. This new feature is available in the result output by asking for the `entity-question-status` processing feature when submitting to the NLP API. For the following input processed as a `Radiology/generic` by the NLP API: ```text INDICATION: r/o appendicitis. ``` in output json result format `v2.19.0` or later the NLP API adds a new attribute `question_status` to the list of `attributes`. See the output for the entity `appendicitis` below. The focus in the example below is on one entity and the new `question_status` attribute. The example output below is truncated to the relevant parts (the `...` indicate the ellipsis of unnecessary details). ```javascript { "documents": [ { ... "entities": { "found": [ ... { "label": "E3", "spans": [ { "start": 16, "end": 28 } ], "section_name": "INDICATION", "attributes": { "question_status": "is_question", ... }, ... "text": [ "appendicitis" ] }, ... ] } ... ] } "format": "emtellipro-json-2", "version": "2.19.0" } ``` #### Temporality relation (beta) Beta version of a new `temporality` relation indicating whether the `subject` entity is in scope of a `temporal_entity`. Currently the only supported category of `temporality` is `historical`. This relation indicates that the `subject` of this relation (e.g. an entity like `heart disease`) is within the scope of a historical `temporal_entity` (e.g. like `history of`) in a sentence `history of heart disease`. In some cases, the historical `temporal_entity` is not a `FoundEntity` but rather an `AssumedEntity` by virtue of occurring within a section that is categorized as historical e.g. `Family History:`. There is also a `modifier` argument which is currently always empty in this beta version of the relation. It will contain useful modifiers in future releases of the NLP API. The `polarity` attribute for this relation indicates whether the temporality status itself is negated or asserted. This new feature is available in the result output by asking for the `temporality-relations` processing feature when submitting to the NLP API. For the following input processed as a `Clinical/generic` by the NLP API: ```text history of heart disease. ``` in output json result format `v2.19.0` or later the NLP API adds a new relation `temporalities` to the list of `relations`. See the output for the above sentence below. The focus in the example below is on one relation, namely `temporalities`. The example output below is truncated to the relevant parts (the `...` indicate the ellipsis of unnecessary details). ```javascript { "documents": [ { ... "relations": { ... "temporalities": [ { "label": "RT0", "attributes": { "polarity": "asserted", "category": "historical" }, "args": { "subject": { "ref": "E1", "text": [ "heart disease" ] }, "modifiers": [], "temporal_entity": { "ref": "E0", "text": [ "history of" ] } } } ] }, ... "format": "emtellipro-json-2", "version": "2.19.0" } ``` ### Ontology updates * RadLex ontology updated to 4.1. * RxNorm update RxNorm Full Monthly Release January 3, 2022. Added `Indicator, Reagent, or Diagnostic Aid` as a valid type of drug for medication relations. * More flexible lemmatization options for each supported ontology. * Updated MEDCIN ontology database to new version with section-based disambiguation. * Added section based disambiguation for MEDCIN concepts. * Produce longer matches of mentions to MEDCIN concepts than previously. ### Changes * Disable `remote` as qualifier relation trigger since it will be a trigger for historical temporality relations. * Changed all measurement entities in SNOMED ontology to concept `246445000 Measure (attribute)`. ### Bug Fixes * Bug fix for some non-determinism issues leading to some missing output. * Bug fix to avoid crashes when there are no sections or no sentences (both of which occur when the input is empty or all whitespace). * Bug fix for sentence detection model which was creating spurious boundaries in very long documents. * Bug fix for rare cases where some plurals were being assigned the concept ID for the singular form. ### Accuracy #### Relations * Enable medication relations for `Calcium`. * Improved accuracy of medication relations. * Improvements to assumed family experiencers in experiencer relations. * Improved qualifier relations, including new triggers for `duration/time` qualifier, `normal` as value for `status` qualifiers. * Fix false positive qualifier relation with `old`. * Detect a larger set of SNOMED person concepts and add them as valid experiencer subjects. * Added new date, time and temporal expressions like `54 year old`, `1AM`, `5:33PM`, `21 days`, `4 to 6 hours`, etc. * Fix false positive for `2C/3C/4C` (number of chambers) in ejection fraction measurement relations when they are in parentheses. Also match `C` measurement units for `temperature` and `temp` measurement subjects. * Eliminate relation triggers that appear inside an entity span, e.g. `injection of technetium 99m tilmanocept` was previously creating a measurement relation between `injection` and `99m` even though there is an entity `technetium 99m tilmanocept` found in this input. * Improved measurement relations, including identification of pairs of measurement values for `FEV1` and `FVC`, added `FEV1/FVC` ratios, potassium `K`, `mg/kg/min` as measurement unit, `packs per day`, `g / dl` and `percent` as measurement units, `80's` and `80?s` as measurement values. * Added LVEF unitless measurements, e.g. `LVEF for 2C/4C was 75.` * Fix false positive trigger `2c/4c` for ejection fraction measurements. * Do not identify dates as measurement values. * Add ambiguous term `in` as measurement unit only for `height` in measurement relations. * Add `at stress` as a post-modifier of a measurement trigger for cases like `42% at rest and 48% at stress`. #### Section parsing * Change back section label from `immunizations` to `immunization`. * Improvements to section parsing with new headings added for all document types. * New additions and updates for CCD headings including: fixed level of CCD `Immunizations` heading, ignore CCD headings `Indication` and `Author`, `Problem List` mapped to `Problems` * Improvements to cardiology/echocardiogram headings. * Added new `Radiology` level 1 section headings. * New CRITICAL RESULTS COMMUNICATION heading. * New `wet read` and `pfi` radiology headings. * Add new `REFERENCE EXAMINATION` and `REFERENCE STUDY` variant headings. * New `Social` heading for clinical. #### Polarity * Polarity (negation) improvements, including `negatives include`, `no evidence by`, `no ... to indicate ...`, `free of`. * Ensure `never smoked` is an asserted entity. #### Misc * Annotation updates for SNOMED, MEDCIN, RxNorm including new biomarker concepts. * Fix `after fall` being picked up as temporal entity. * Allow British spellings when normalizing medical terms. * Improved uncertainty accuracy, including `with possible`. * Improve date finding to avoid false positives with dosages like `10-20-300`. * Improved speed and accuracy of sentence boundary detection, including the handling of ICD code, `f.u` as abbreviation, decimals and hyphens in CCDs, fix for `smoking-cessation` false positive, long sequences of hyphens and underscores from flat ascii tables, handle "fill in the blanks" cases with underscores. ### Infrastructure * We improved the stability and throughput of our API server to handle the case where a single user wants to run hundreds of simultaneous clients connecting to the NLP API service. * Improvements in the NLP API Admin UI including the ability to download and sort the logs. * Documentation web site is completely auto-built with versions for each sub-component like the NLP API description, the JSON result description, etc. * IAM command-line client has added as a new download in our Downloads page. * IAM support for custom JWT endpoints like the Visual Client. * For on-premises installations of the NLP API the audit logs have been available on the NLP API admin UI since v2.27. The audit logs can be now downloaded using the shell script provided below: ```bash #!/bin/sh # Make sure you have httpie and jq installed on your system: # sudo apt install httpie jq #USER_NAME="name" # set as environment variable #PASSWORD="pass" # set as environment variable IAM_URL="https://:1222" # Warning: this has changed in recent releases EMTELLIPRO_ADMIN_URL="https://:5051" echo “Getting auth token...” auth_token=$(http POST $IAM_URL/iam/v1/login user_name=$USER_NAME password=$PASSWORD | jq --raw-output .auth_token) echo “Downloading audit log...” http --timeout 120 GET $EMTELLIPRO_ADMIN_URL/admin/download-audit-log COOKIE:auth_token=$auth_token --download ``` * Extensive customized memory and time profiling of the NLP API was conducted. * Memory and time usage and time complexity improvements optimized to handle larger documents. Fix for potential memory leak in some rare cases. * Enabled support for longer documents without timing out. ## Python SDK updates The new NLP API Python SDK is version `5.22.0`. Refer to the [changelog for the Python SDK](/nlp/python/changelog) for a full explanation of the updates made since the last public release which was version `5.17.1` released with the NLP API v2.27. You will need to use at least version 5.7.0 of the Python SDK or later to send requests to the NLP API v2.28. > **Warning** > > To use existing NLP API databases with the new Python SDK you must do a data migration using the Python database client `migrate` subcommand: ```bash emtellipro-db-client --database postgresql://username:password@hostname:port/example_db migrate ``` ### Question Status in Output Database When using the `emtellipro-db-client` Python database client the table for `foundentity` contains the `question_status` attribute value. The updated `foundentity` table looks like this with an additional column for `question_status`: ```text id | polarity | section_name | uncertainty | known_ambiguity | question_status | document_id ----+----------+--------------+-------------+-----------------+-----------------+------------- 1 | asserted | INTRO | certain | unknown | not_question | 1 2 | asserted | INTRO | certain | unknown | not_question | 1 3 | asserted | INTRO | certain | unambiguous | not_question | 1 4 | asserted | INTRO | certain | unambiguous | not_question | 1 5 | asserted | INTRO | certain | unknown | not_question | 1 ... ``` ### Temporality relation in Output Database The new temporality relation appears in the `temporalityrelation` table and associated `temporalityrelationmodifier` table when using the `emtellipro-db-client` Python database client. The `temporalityrelation` table looks like this: ```text id | polarity | category | temporal_entity_id | subject_id ----+----------+------------+--------------------+------------ 1 | asserted | historical | 2 | 1 2 | asserted | historical | 18 | 17 3 | asserted | historical | 135 | 134 4 | asserted | historical | 138 | 139 5 | asserted | historical | 154 | 155 ... ``` The `temporalityrelationmodifier` table looks like this: ```text id | temporality_id | entity_id ----+----------------+----------- ... ``` ## Java SDK updates The latest NLP API Java SDK is version `5.8.0`. Refer to the [changelog for the Java SDK](https://legacy-docs.emtelligent.com/emtellipro-java-sdk/v5.15.0/) for a full explanation of the updates made since the last public release. You will need to use version `5.8.0` or later to take advantage of the new features described above.