> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.emtelligent.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.emtelligent.com/_mcp/server. # Imagelink Relations In this section, we explain the imagelink relation data model and how it is implemented in the NLP API JSON output and in the Output Database schema. We also provide sample queries to illustrate the aspects of the data schema. After reading this page, we hope you will: 1. Understand what imagelink relations are. 2. Know how to retrieve image references and related findings for imagelink relations. 3. Know how to find imagelink relations in documents. The content in this section builds on the concepts and data models explained in earlier sections (see [Entity-level Tables](/nlp/database/schema/core/entity-tables), and [Document-level Tables](/nlp/database/schema/core/document-tables)). ## What are Imagelink Relations? Imagelink relations are radiology-specific relations. They are found in sentences where the dictating radiologist explicitly mentions the Series and Image where a finding was observed. Imagelink relations is an association between two types of arguments: image findings and references. Consider the following sentence: ```text 2.3 cm lesion in the liver (series 2, image 128) ``` From the above sentence, the NLP API extracts the arguments that comprise an imagelink relation as shown: ![Imagelink relation required and optional arguments](/_fern-files/emtelligent-prod.docs.buildwithfern.com/9496a7c4a822377c3201e751d98cb297f2d23b556c76af8c9966efecc58528cf/assets/nlp/imagelink_anno.png) In addition to the example above, the NLP API recognizes many variations in how images can be referenced in reports. The following are examples of image references recognized by the NLP API: ```text image 128 of series 2 (2:128) (2:120, 2:128) series 2, image 128 ``` Note, the list above is not exhaustive. An imagelink relation will typically have only one image reference. However, it can have many image finding entities. When using the SNOMED ontology, findings are entities with semantic type such as body structure, finding, disorder or morphologic abnormality. You may also use other semantic type categorizations provided by other ontologies enabled during processing such as UMLS, MEDCIN and RadLex. It should be noted that imagelink relations are only extracted from `FINDINGS` and `IMPRESSIONS` sections of radiology reports. ## JSON Data Model The JSON below shows the NLP API's JSON result output for a medication relation extracted from this statement: ```text 2.3 cm lesion in the liver (series 2, image 128) ``` ```json { "relations": { "imagelinks": [ { "label": "RIM0", "attributes": { "confidence": 1 }, "args": { "image_findings": [ { "ref": "E1", "text": [ "lesion" ] }, { "ref": "E3", "text": [ "liver" ] } ], "references": [ { "ref": "E2", "text": [ "(series 2, image 128)" ] } ] }, "concept_links": [] } ] } } ``` From the JSON output above, we see that: 1. The `imagelinks` array is an element of the `relations` object. Each element of the `imagelinks` array represents an imagelink relation instance. 2. An imagelink relation instance has two types of named arguments: `references` and `image_findings`. 3. The `references` argument is an entity that represents an image reference. 4. The `image_findings` arguments are entities that relate to an observed finding or structure. 5. There can be a one-to-many relation between `references` and `image_findings`, but at least one image reference must be present. ## ImageLink Relation Tables In the Output Database schema, imagelink relations are stored in the **imagelinkrelation** table. Its arguments are represented by the **imagelinkrelationimagefinding** and **imagelinkrelationreference** tables. The following ER diagram shows the relationship between these tables: ![Entity-Relation diagram showing imagelinkrelation and its associated tables.](/_fern-files/emtelligent-prod.docs.buildwithfern.com/a355eca2b85988456285b07b05fe583127288d1ffa5e51b2eec621340c5267cf/assets/nlp/imagelinkrelation_schema.png) From the ER diagram above, a few key points should be noted: 1. The primary key of the **imagelinkrelationimagefinding** table is a foreign key reference to the **imagelinkrelation** table's primary key. 2. Similarly, the primary key of the **imagelinkrelationreference** table is a foreign key reference to the **imagelinkrelation** table's primary key. 3. The **imagelinkrelationimagefinding** table **entity_id** column is a foreign key reference to the **entity** table. This foreign key constraint implements a one-to-many relation between imagelink relation and finding entities. 4. The **imagelinkrelationreference** table **entity_id** column is a foreign key reference to the **entity** table. This foreign key constraint implements a one-to-many relation between imagelink relation and image reference entities. 5. The **entity** table has a foreign key reference to the **foundentity** table which stores many entity attributes. By joining the **foundentity** with other entity-related tables, you can retrieve information associated with the concepts, entity types and locations for an entity. See [Entity-level Tables](/nlp/database/schema/core/entity-tables). Note: While the NLP API's database schema supports a one-to-many relation between imagelink relations and image references, in practice, we can consider the relation between imagelink relations and image references to be one-to-one. ## Sample Queries In this section, we provide sample SQL queries that illustrate how to retrieve data for imagelink relations. SQL queries will be discussed in order as follows: * [Query: Retrieve the annotated text for image references and findings](#query-retrieve-the-annotated-text-for-image-references-and-findings) * [Query: Count of image references by finding concept](#query-count-of-image-references-by-finding-concept) * [Query: The average number of image references per Radiology CT report](#query-the-average-number-of-image-references-per-radiology-ct-report) * [Query: Retrieve sentences containing imagelinks](#query-retrieve-sentences-containing-imagelinks) Note: Because imagelink relations are specific to Radiology reports, please use an Output Database that contains output from processing Radiology reports. ### Query: Retrieve the annotated text for image references and findings This query finds image references related to `umbilical hernia`: a partial or full collapse of the lung. It retrieves the terms that NLP API identified as the image reference, and the finding of umbilical hernia. To find information about imagelink relations we start at the **imagelinkrelation** table and perform a series of joins. The query involves the following steps: 1. First, construct an intermediate table containing the image term and a comma-delimited list of finding terms for all imagelink relations, using a CTE, as follows: 1. Get the finding terms by performing an inner join between the **imagelinkrelation** and **imagelinkrelationimagefinding** tables to get the **entity_id**. Then, perform a series of inner joins with entity-related tables to retrieve the **text** column from the **foundentity** table. 2. Get the image terms by performing an inner join between the **imagelinkrelation** and **imagelinkrelationreference** tables to get the **entity_id**. Again, perform the same series of inner joins to retrieve the annotated text from the **foundentity** table. 3. Use the `string_agg` Window function to aggregate the finding terms associated with a specific image reference into a comma-delimited list by partitioning on the **imagelinkrelation** table primary key. 2. In the outer query, use a WHERE clause to filter for the string `umbilical hernia` in the comma-delimited list of findings. ```sql WITH image_findings_table AS (SELECT imagelinkrelation.id AS imagerelation_id, image_fe.text AS image_reference_term, string_agg(finding_fe.text, ', ') OVER (PARTITION BY imagelinkrelation.id ) AS image_finding_terms FROM imagelinkrelation -- Joins to retrieve entity text for 'Image FINDING' foundentity JOIN imagelinkrelationimagefinding image_finding ON image_finding.imagelink_id = imagelinkrelation.id JOIN entity finding_e ON finding_e.id = image_finding.entity_id AND finding_e.type_='found' JOIN foundentity finding_fe ON finding_fe.id = finding_e.entity_id -- Joins to retrieve entity text for 'Image REFERENCE' JOIN imagelinkrelationreference image_ref ON image_ref.imagelink_id = imagelinkrelation.id JOIN entity image_e ON image_e.id = image_ref.entity_id AND image_e.type_='found' JOIN foundentity image_fe ON image_fe.id = image_e.entity_id ORDER BY imagelinkrelation.id) SELECT * FROM image_findings_table WHERE image_finding_terms LIKE '%%umbilical hernia%%' -- GROUP BY image_findings_table.imagerelation_id, image_reference_term, image_finding_terms; ``` | imagerelation_id | image_reference_term | image_finding_terms | | ---------------- | -------------------- | ------------------- | | 1 | series 3, image 53 | umbilical hernia | ### Query: Count of image references by finding concept This query finds the frequencies of image references grouped by findings. Since each imagelink relation is one image reference, we can count imagelink relations to get the number of image references. Then, instead of grouping by finding terms, we will group the imagelink relations by the finding's SNOMED CT concept. This query involves the following steps: 1. First, start with **imagelinkrelation** table and perform the series of table joins to retrieve the SNOMED CT concept from the **concept** table. 2. Then, perform a count distinct on the imagelink relation ID to count the number of imagelink relations for each concept, grouping by concept ID and description. Note, the DISTINCT keyword is required because it is unlikely, but possible, for an imagelink relation to have multiple findings with the same concept ID. ```sql SELECT finding_c.concept_id, finding_c.description, count(DISTINCT imagelinkrelation.id) as num_image_refs FROM imagelinkrelation JOIN imagelinkrelationimagefinding image_finding ON image_finding.imagelink_id = imagelinkrelation.id JOIN entity finding_e ON finding_e.id = image_finding.entity_id AND finding_e.type_='found' JOIN foundentity finding_fe ON finding_fe.id = finding_e.entity_id JOIN foundentityconcept finding_fec ON finding_fec.found_entity_id = finding_fe.id -- Join with concept table to retrieve concept id and description JOIN concept finding_c ON finding_c.concept_id = finding_fec.concept_id AND finding_c.ontology = finding_fec.concept_ontology --Ensure WHERE finding_c.ontology = 'snomed' GROUP BY finding_c.concept_id, finding_c.description ORDER BY 1 LIMIT 5; ``` | concept_id | description | num_image_refs | | ---------- | ------------------------------------------------------------------ | -------------- | | 107669003 | Degenerative abnormality (morphologic abnormality) | 1 | | 110495000 | Structure of lower inner quadrant of right breast (body structure) | 1 | | 113091000 | Magnetic resonance imaging (procedure) | 1 | | 118515002 | Structure of extensor compartment of forearm (body structure) | 2 | | 1201782000 | Structure of extensor tendon sheath (body structure) | 1 | ### Query: The average number of image references per Radiology CT report This query counts the imagelink occurrences by document in a CTE, and then gets the average number of imagelink occurrences across all documents in the database. ```sql WITH image_counts_by_doc AS (SELECT d.id, d.category, d.subcategory, count(imagelinkrelation.id) AS image_reference_count FROM imagelinkrelation JOIN imagelinkrelationreference image_ref ON image_ref.imagelink_id = imagelinkrelation.id JOIN entity finding_e ON finding_e.id = image_ref.entity_id AND finding_e.type_='found' JOIN document d ON d.id = finding_e.document_id GROUP BY d.id) SELECT round(AVG(image_reference_count)) AS average FROM image_counts_by_doc WHERE category = 'Radiology' AND subcategory = 'CT'; ``` | average | | ------- | | 5 | ### Query: Retrieve sentences containing imagelinks This query shows how to use the **location**, **document** and **sentencelocation** tables to retrieve sentences containing imagelink relations from documents. The query involves the following steps: 1. Starting with the **imagelinkrelation** table, perform a series of JOINs as shown in the SQL code below. 2. Note that when joining the **foundentitylocation** with the **location** table, in addition to joining on the foreign key fields, we need to check that the location type is `sentence` and not `section` as in the SQL code below. 3. Then, in the SELECT statement, use the **text** column from the **sentencelocation** table to retrieve the sentence containing each imagelink relation. For a more complete explanation of how to work with the **location**, **sentencelocation**, and **document** tables see [Document-level Tables](/nlp/database/schema/core/document-tables). ```sql SELECT image_doc.id AS doc_id, imagelinkrelation.id AS imagelink_id, image_fe.text, sl.text AS sentence FROM imagelinkrelation JOIN imagelinkrelationreference image_ref ON image_ref.imagelink_id = imagelinkrelation.id JOIN entity image_e ON image_e.id = image_ref.entity_id AND image_e.type_='found' JOIN foundentity image_fe ON image_fe.id = image_e.entity_id JOIN foundentitylocation image_fel ON image_fel.found_entity_id = image_fe.id JOIN location image_loc ON image_loc.id = image_fel.location_id AND image_loc.type_ = 'sentence' -- IMPORTANT JOIN sentencelocation sl ON image_loc.location_id = sl.id JOIN document image_doc ON image_doc.id = image_loc.document_id ``` | doc_id | imagelink_id | text | sentence | | ------ | ------------ | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 6 | 1 | series 3, image 53 | Moderate sized fat containing umbilical hernia is present on (series 3, image 53). | | 6 | 2 | series 3, image 104 | There is mild eccentric lower rectal wall thickening (series 3, image 104). | | 10 | 3 | series 68 image 29 | There are demonstrated bilateral deep lateral cervical nodes at the midlevel, measuring 0.6cm on the right side and 0.9cm on the left side (series 68 image 29). | | 10 | 4 | series 68 image 29 | There are demonstrated bilateral deep lateral cervical nodes at the midlevel, measuring 0.6cm on the right side and 0.9cm on the left side (series 68 image 29). | | 10 | 5 | series 68 image 29 | There are demonstrated bilateral deep lateral cervical nodes at the midlevel, measuring 0.6cm on the right side and 0.9cm on the left side (series 68 image 29). |