Concept confidence

Concept confidences map a found entity’s concept links to a confidence score. These may be optionally be present if the processing feature entity-concept-link-confidence is enabled.

Example NLP API result file:

Specifically, the following snippet from the file contains the new heading_status fields.

"entities": {
  "found": [
    {
      "label": "E0",
      "spans": [
        {
          "start": 0,
          "end": 4
        }
      ],
      "section_name": "INTRO",
      "attributes": {
        "heading_status": "outside_heading",
        "polarity": "asserted",
        "uncertainty": "certain",
        "question_status": "not_question",
        "guidance": "not_guidance",
        "known_ambiguity": "unambiguous"
      },
      "concept_links": [
        "Csnomed0",
        "Cemtelligent0"
      ],
      "locations": [
        "S1",
        "SEC-0"
      ],
      "entity_type": {
        "snomed": "qualifier value",
        "umls": "Quantitative Concept",
        "emtelligent": "qualifier value"
      },
      "concept_confidences": {
        "Csnomed0": 0.8107627630233765
      },
      "text": [
        "Tiny"
      ]
    },

Changes

The new feature flag is entity-concept-link-confidence and may be passed using the command-line process --features flag, or the features parameter of submit.

Python API

The new concept confidences are presented in the annotated document using concept_confidences. This will be a mapping from Concept instances to a confidence score (a float or None if it doesn’t exist).

The concept confidences will contain all the same concepts from concepts, so it can be iterated over without having to check both attributes.

Example usage:

import json
import pathlib
from emtellipro.data import AnnotatedDocument
import emtellipro.load
json_data = pathlib.Path(
    'entity_confidence.json',
).read_text()
doc_dict = json.loads(json_data)['documents'][0]
ann_doc = AnnotatedDocument(doc_dict)
for fe in ann_doc.found_entities:
    for concept, confidence in fe.concept_confidences.items():
        if concept.ontology in ('snomed', 'umls_nci'):
            print(
                fe.text,
                concept.ontology,
                concept.id,
                confidence,
            )

title="Output of previous code. The "duplicates" are due to there being"

   two sentences in the input document with similar wording.
['Tiny'] snomed 255507004 0.8107627630233765
['Tiny'] snomed 255507004 0.7671563625335693
['pelvic'] umls_nci C12767 None
['veins'] umls_nci C12814 None
['heart attack'] umls_nci C27996 None
['heart attack'] snomed 22298006 0.9438971281051636
['pelvic'] umls_nci C12767 None
['veins'] umls_nci C12814 None
['phleboliths'] snomed 37876005 1.0
['pelvic veins'] snomed 13152008 1.0
['pelvic veins'] snomed 13152008 1.0

Schema changes

The new confidence is present in the foundentityconcept.confidence column.

The schema of that table now looks like (for PostgreSQL):

                     Table "public.foundentityconcept"
      Column      |          Type          | Collation | Nullable | Default
------------------+------------------------+-----------+----------+---------
 found_entity_id  | integer                |           | not null |
 concept_id       | character varying(255) |           | not null |
 concept_ontology | character varying(255) |           | not null |
 confidence       | double precision       |           |          |
Indexes:
    "pk_foundentityconcept" PRIMARY KEY, btree (found_entity_id, concept_id, concept_ontology)
Foreign-key constraints:
    "fk_foundentityconcept_concept_id_concept" FOREIGN KEY (concept_id, concept_ontology) REFERENCES concept(concept_id, ontology)
    "fk_foundentityconcept_found_entity_id_foundentity" FOREIGN KEY (found_entity_id) REFERENCES foundentity(id)

Example query showing the new confidence on the foundentityconcept table:

testdb=# select * from foundentityconcept where confidence is not null limit 5;
 found_entity_id | concept_id | concept_ontology |     confidence
-----------------+------------+------------------+--------------------
               1 | 255507004  | snomed           | 0.8107627630233765
               5 | 22298006   | snomed           | 0.9438971281051636
               9 | 13152008   | snomed           |                  1
               2 | 255507004  | snomed           | 0.7671563625335693
              10 | 13152008   | snomed           |                  1
(5 rows)