> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.emtelligent.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.emtelligent.com/_mcp/server.

# Concept confidence

Concept confidences map a found entity's concept links to a confidence score. These may be optionally be present if the processing feature `entity-concept-link-confidence` is enabled.

Example NLP API result file:

* entity_confidence.json
* entity_confidence.txt

Specifically, the following snippet from the file contains the new `heading_status` fields.

```text
"entities": {
  "found": [
    {
      "label": "E0",
      "spans": [
        {
          "start": 0,
          "end": 4
        }
      ],
      "section_name": "INTRO",
      "attributes": {
        "heading_status": "outside_heading",
        "polarity": "asserted",
        "uncertainty": "certain",
        "question_status": "not_question",
        "guidance": "not_guidance",
        "known_ambiguity": "unambiguous"
      },
      "concept_links": [
        "Csnomed0",
        "Cemtelligent0"
      ],
      "locations": [
        "S1",
        "SEC-0"
      ],
      "entity_type": {
        "snomed": "qualifier value",
        "umls": "Quantitative Concept",
        "emtelligent": "qualifier value"
      },
      "concept_confidences": {
        "Csnomed0": 0.8107627630233765
      },
      "text": [
        "Tiny"
      ]
    },
```

## Changes

The new feature flag is `entity-concept-link-confidence` and may be passed using the command-line `process --features` flag, or the `features` parameter of [submit](/nlp/python/reference/engine#emtelliprosubmit).

### Python API

The new concept confidences are presented in the annotated document using [concept_confidences](/nlp/python/reference/data#foundentityconcept_confidences). This will be a mapping from [Concept](/nlp/python/reference/data#emtelliprodataconcept) instances to a confidence score (a float or None if it doesn't exist).

The concept confidences will contain all the same concepts from [concepts](/nlp/python/reference/data#foundentityconcepts), so it can be iterated over without having to check both attributes.

Example usage:

```python
import json
import pathlib
from emtellipro.data import AnnotatedDocument
import emtellipro.load

json_data = pathlib.Path(
    'entity_confidence.json',
).read_text()
doc_dict = json.loads(json_data)['documents'][0]
ann_doc = AnnotatedDocument(doc_dict)

for fe in ann_doc.found_entities:
    for concept, confidence in fe.concept_confidences.items():
        if concept.ontology in ('snomed', 'umls_nci'):
            print(
                fe.text,
                concept.ontology,
                concept.id,
                confidence,
            )
```

**`Output of previous code. The `**

```text title="Output of previous code. The "duplicates" are due to there being"
   two sentences in the input document with similar wording.

['Tiny'] snomed 255507004 0.8107627630233765
['Tiny'] snomed 255507004 0.7671563625335693
['pelvic'] umls_nci C12767 None
['veins'] umls_nci C12814 None
['heart attack'] umls_nci C27996 None
['heart attack'] snomed 22298006 0.9438971281051636
['pelvic'] umls_nci C12767 None
['veins'] umls_nci C12814 None
['phleboliths'] snomed 37876005 1.0
['pelvic veins'] snomed 13152008 1.0
['pelvic veins'] snomed 13152008 1.0
```

### Schema changes

The new confidence is present in the `foundentityconcept.confidence` column.

The schema of that table now looks like (for PostgreSQL):

```text
                     Table "public.foundentityconcept"
      Column      |          Type          | Collation | Nullable | Default
------------------+------------------------+-----------+----------+---------
 found_entity_id  | integer                |           | not null |
 concept_id       | character varying(255) |           | not null |
 concept_ontology | character varying(255) |           | not null |
 confidence       | double precision       |           |          |
Indexes:
    "pk_foundentityconcept" PRIMARY KEY, btree (found_entity_id, concept_id, concept_ontology)
Foreign-key constraints:
    "fk_foundentityconcept_concept_id_concept" FOREIGN KEY (concept_id, concept_ontology) REFERENCES concept(concept_id, ontology)
    "fk_foundentityconcept_found_entity_id_foundentity" FOREIGN KEY (found_entity_id) REFERENCES foundentity(id)
```

Example query showing the new confidence on the `foundentityconcept` table:

```text
testdb=# select * from foundentityconcept where confidence is not null limit 5;
 found_entity_id | concept_id | concept_ontology |     confidence
-----------------+------------+------------------+--------------------
               1 | 255507004  | snomed           | 0.8107627630233765
               5 | 22298006   | snomed           | 0.9438971281051636
               9 | 13152008   | snomed           |                  1
               2 | 255507004  | snomed           | 0.7671563625335693
              10 | 13152008   | snomed           |                  1
(5 rows)
```