data
This module contains all data structures used to represent documents and annotations, including all forms of associated metadata.
The only class that should be created by user code should be InputDocument.
The other classes are used when parsing annotations returned by the server.
Module Contents
Classes
Functions
Data
_relation_handlers
API
emtellipro.data.__relation_handler_for
emtellipro.data.InputSource
Bases: object
Dataclass for storing the source of an input document.
This is used for cases where you have a PDF, you’ve extracted the text,
you’re using the text to process in InputDocument, but you’d like
to track the original data the text came from.
Attributes:
type: The MIME type of the input source.data: The raw bytes of the data in the input source.path: If present, this will be the path to the input source. This may be either a path on the filesystem, or some sort of URL/URI.
Initialization
InputSource.type
InputSource.data
InputSource.path
Value: None
InputSource.chunks
Iterate over the chunks in the input.
Returns:
(offset, chunk) tuples containing the offset of the chunk and the chunk data.
emtellipro.data.InputDocument
Bases: object
A document that needs to be annotated. It can be initialized either from plain text by passing the ‘text’ parameter, or from a file when passing the ‘filepath’ parameter.
If the path to a plaintext file is provided, then the ‘text’ attributed will be populated with the contents of the file for convenience.
Attributes:
category: The category the document was instantiated with.subcategory: The subcategory the document was instantiated with.type: The document type that was set when this class was instantiated.section_label: The section label this class was instantiated with.filepath: The filepath the documentated was loaded from. This may be None, iffilepathwasn’t passed when this class was instantiated.data: The data this class was instantiated with.text: If this class’sdatawas a string, this will reference the same data. Otherwise this will be None.metadata: The metadata information for this input document. This will always be present as a dictionary (or themetadataparameter that was passed when the class was instantiated). See documentation for parametermetadatafor details.filetype: The filetype for this document that will be used when submitting it to emtelliPro. Will be either ‘application/pdf’ or ‘text/plain’.source: The source this document came from.
Initialization
Parameters:
- id: A unique ID representing this particular document. This is useful for figuring out which document the returned annotations are for when providing multiple documents.
- category:
The category of document. This should be one of the categories in
emtellipro.engine.CATEGORIES. It’s not validated here, but will be when this document is submitted to emtelliPro. To have emtelliPro infer the category, this can be set to ‘auto’ or None. - subcategory:
The subcategory of the document. This should be one of the subcategories in
emtellipro.engine.CATEGORIES. It’s not validated here, but will be when this document is submitted to emtelliPro. To have emtelliPro infer the subcategory, setcategoryto ‘auto’ or None. This parameter will be ignored in that case. - data:
The contents of this document. May be a string for plaintext
files, or bytes for a PDF. If the data is binary, the
filetypeparameter must also be set. - type: The document type to be used for this document. If not specified, the API submit call will set it.
- filepath: The path of the file which is to be submitted. This is useful to track which path the document came from.
- filetype: The filetype to use for this document; maybe be either ‘pdf’ or ‘txt’. This is used when submitting the document to tell the server what type of file this is. This is optional for plaintext documents, but for binary documents this must be set.
- section_label:
The value sent as the
process_with_section_labelsheader for this document. - metadata: Any optional metadata that should be kept along with the document; this can be used for storing in a database along with the processed results. The metadata does not get sent to emtelliPro.
- There are two kinds of metadata: metadata with predefined keys and arbitrary key-value pairs.
- *Predefined metadata: * This is specified at the top level of
the metadata dictionary.
If stored in a database, the predefined key metadata is stored
in the
documentmetadatatable. Seepredefined_namesfor the list of valid names. This means that all the top-level keys should come from thatpredefined_nameslist and all the values here should be strings, except for the datetime cases which can be Pythondatetimeobjects. - *Arbitrary metadata: * For metadata that doesn’t fit in the
predefined list, you can put in a special
"extras"key. The value of this must be a list of pairs (tuples). If storing to a database, the arbitrary metadata will be stored in thedocumentstructuredmetadatatable (seeemtellipro.db.models.DocumentStructuredMetadata). See the examples below for how an example of a valid metadata object. - source: The source this document came from, especially useful if you’re processing plaintext, but wish to track the PDF data the text was extracted from.
- Examples: An example of a valid metadata dictionary may look like this.
InputDocument.nbytes
The number of bytes in this document. If it’s plaintext string, it will assume it’s utf-8 encoded.
InputDocument.file_data
The file contents as a file-like object.
Return type: io.BytesIO | io.StringIO
InputDocument.file_bytes
Return type: BytesIO
InputDocument.__eq__
InputDocument.serialize
Serialize for caching.
Parameters:
The maximum number of bytes to include in each data
The compression module to use. Default is ‘lzma’ for backwards compatibility, but ‘zlib’ is a better choice.
Returns:
A tuple (args, chunks) containing arguments (as a dictionary), and a generator that produces data chunks.
InputDocument.deserialize
Deserialize using the output produced by serialize.
Parameters:
The first element of the serialization tuple.
The second element of the serialization tuple.
Returns:
An InputDocument instance.
Return type: InputDocument
emtellipro.data.AnnotatedDocument
Bases: object
The annotated document returned by the Emtellipro API. This provides access to all returned entities and relations.
Attributes:
id: the ID for the associated document that was submitted for processingcategory: the category of the document as returned by the API, or None if not returned.subcategory: the subcategory of the document, or None if not returned by APIconcepts: List ofemtellipro.data.Conceptobjectsontology_versions: mapping from ontology name to a dict with version information. Onlyrelease_versionis guaranteed to be present; this key will also be present even if emtelliPro does not return any ontology version information.found_entities: List ofemtellipro.data.FoundEntityobjectsassumed_entities: List ofemtellipro.data.AssumedEntityobjectsrelations: the relations between entities found in the document. Keys are: -experiencer-follow-up, -measurement-imagelink-medication-qualifier-temporality-reportedevent-anatomic-sitelocations: Locations found in the document. Keys are: -sentence-section-heading-pagetext: The text content of the document, if returned by the API. This will be populated for PDF documents. Will be None if not returned by the API.processing_status: the processing status of the report as returned by the API
Initialization
Parameters:
- document_dict: A dictionary of the returned results for this document.
- text: The text of this document. This can be provided when the returned JSON from emtelliPro does not contain text data (usually because the ‘text’ feature was not enabled when processing). This text will be used for all contained annotations to ensure they have text contents.
AnnotatedDocument.supported_result_format
Value: emtellipro-json-2
AnnotatedDocument.__repr__
AnnotatedDocument.__str__
AnnotatedDocument.fail
Create a failed AnnotatedDocument.
AnnotatedDocument.as_dict
Returns the initial document_dict argument that was used to
instantiate this object.
emtellipro.data.Entity
Bases: _LabeledObject
Bases: emtellipro.data._LabeledObject
Base class for entities found in submitted document.
Entity.__slots__
Value: []
emtellipro.data.FoundEntity
Bases: Entity
Bases: emtellipro.data.Entity
Entities that were found concretely in the submitted document.
Attributes:
type_name: entity type names from the different ontologies returned by the API. Note that not all ontologies are always present, so it’s safer to use.get()than[].polarity: the polarity of the entity, or None if it was not returned by the APIuncertainty: the uncertainty of the entity, or None if it was not returned by the APImeasurement_unit: a list of measurement units in the found entity, or None if the API returned null.known_ambiguity: the known ambiguity status of the entity, or None if it was not returned by the APIquestion_status: the question status of the entity, or None if it was not returned by the APIguidance: the guidance attribute of the entity, or None if it was not returned by the APIheading_status: The heading status attribute of the entity, or None if it was not returned by the API.factuality: The factual status of the entity mention. Non-null only for entities with snomed entity type and where it was possible to find factuality.experiencer: The ‘experiencer’ of this entity mention, e.g. ‘patient’, ‘father’, ‘mother’. Non-null only for entities with snomed entity type and where it was possible to find experiencer.section_name: the name of the document section this entity was found inspans: list of spans associated with this entity
FoundEntity.__slots__
Value: ['type_name', 'polarity', 'uncertainty', 'measurement_unit', 'known_ambiguity', 'question_status', 'guidance', 'heading_status', 'factuality', 'experiencer', 'section_name', 'spans', 'text', '_locations', '_concept_links', '_concept_confidences']
FoundEntity.attributes
The names of the instance attributes which represent found entity attributes from emtelliPro.
FoundEntity.locations
Mapping from location type to list of locations of that type that contain this entity.
Keys are ‘section’, and ‘sentence’.
Return type: typing.Mapping[str, list]
FoundEntity.sentence
The sentence containg this entity.
FoundEntity.concepts
The concepts associated with this entity
FoundEntity.concept_confidences
Mapping from concept object to the confidence of that concept.
This property includes all the concepts returned by concepts,
however if emtelliPro didn’t include a confidence value for that
concept link, the value will be None.
FoundEntity.parts
A mapping of spans to text as returned by the API. If the text was not returned by the API, the values will be None.
FoundEntity.start
The start of the overarching span for this entity.
FoundEntity.stop
The end of the overarching span for this entity. This can be used as the end of a slice.
emtellipro.data.AssumedEntity
Bases: Entity
Bases: emtellipro.data.Entity
An entity that isn’t concretely present in the document text.
Attributes:
value: the text representing this entity
AssumedEntity.__slots__
Value: ['value']
emtellipro.data.Relation
Bases: _LabeledObject
Bases: emtellipro.data._LabeledObject
Base class for all relation types.
Relation.__slots__
Value: ['_args', '_attrs', '_concept_links']
Relation.__init_subclass__
Relation.arguments
The names of the instance attributes which represent relation arguments.
Relation.attributes
The names of the instance attributes which represent relation arguments.
Relation.concepts
The concepts associated with this relation
emtellipro.data.ExperiencerRelation
Bases: Relation
Bases: emtellipro.data.Relation
The experiencer/experienced relation between entities.
ExperiencerRelation.__slots__
Value: []
ExperiencerRelation.experiencer
Value: _RelationArg(...)
ExperiencerRelation.experienced
Value: _RelationArg(...)
ExperiencerRelation.polarity
Value: _RelationAttr(...)
ExperiencerRelation.confidence
Value: _RelationAttr(...)
emtellipro.data.FollowupRelation
Bases: Relation
Bases: emtellipro.data.Relation
The requested follow-up found in the submitted document
FollowupRelation.__slots__
Value: []
FollowupRelation.procedures
Value: _RelationArg(...)
FollowupRelation.time_expression
Value: _RelationArg(...)
FollowupRelation.reasons
Value: _RelationArg(...)
FollowupRelation.polarity
Value: _RelationAttr(...)
FollowupRelation.confidence
Value: _RelationAttr(...)
emtellipro.data.MeasurementRelation
Bases: Relation
Bases: emtellipro.data.Relation
A measurement found in the document.
MeasurementRelation.__slots__
Value: []
MeasurementRelation.subject
Value: _RelationArg(...)
MeasurementRelation.value
Value: _RelationArg(...)
MeasurementRelation.confidence
Value: _RelationAttr(...)
emtellipro.data.ImageLinkRelation
Bases: Relation
Bases: emtellipro.data.Relation
A reference to an image found in the text
ImageLinkRelation.__slots__
Value: []
ImageLinkRelation.image_findings
Value: _RelationArg(...)
ImageLinkRelation.references
Value: _RelationArg(...)
ImageLinkRelation.confidence
Value: _RelationAttr(...)
emtellipro.data.QualifierRelation
Bases: Relation
Bases: emtellipro.data.Relation
A relation between a qualifier and the entity it qualifiers.
QualifierRelation.__slots__
Value: []
QualifierRelation.qualifier
Value: _RelationArg(...)
QualifierRelation.qualifies
Value: _RelationArg(...)
QualifierRelation.qualifier_type
Value: _RelationAttr(...)
QualifierRelation.confidence
Value: _RelationAttr(...)
emtellipro.data.MedicationRelation
Bases: Relation
Bases: emtellipro.data.Relation
A relation between a medication and its arguments.
MedicationRelation.__slots__
Value: []
MedicationRelation.drug
Value: _RelationArg(...)
MedicationRelation.dosages
Value: _RelationArg(...)
MedicationRelation.frequencies
Value: _RelationArg(...)
MedicationRelation.modes
Value: _RelationArg(...)
MedicationRelation.quantities
Value: _RelationArg(...)
MedicationRelation.routes
Value: _RelationArg(...)
MedicationRelation.necessities
Value: _RelationArg(...)
MedicationRelation.modifiers
Value: _RelationArg(...)
MedicationRelation.durations
Value: _RelationArg(...)
MedicationRelation.indications
Value: _RelationArg(...)
MedicationRelation.date_times
Value: _RelationArg(...)
MedicationRelation.confidence
Value: _RelationAttr(...)
emtellipro.data.TemporalityRelation
Bases: Relation
Bases: emtellipro.data.Relation
A relation between a qualifier and the entity it qualifiers.
TemporalityRelation.__slots__
Value: []
TemporalityRelation.temporal_entity
Value: _RelationArg(...)
TemporalityRelation.subject
Value: _RelationArg(...)
TemporalityRelation.modifiers
Value: _RelationArg(...)
TemporalityRelation.polarity
Value: _RelationAttr(...)
TemporalityRelation.category
Value: _RelationAttr(...)
TemporalityRelation.confidence
Value: _RelationAttr(...)
emtellipro.data.ReportedEventRelation
Bases: Relation
Bases: emtellipro.data.Relation
A relation capturing the notion of a communication between two groups of entities.
ReportedEventRelation.__slot__
Value: []
ReportedEventRelation.subject
Value: _RelationArg(...)
ReportedEventRelation.to_entities
Value: _RelationArg(...)
ReportedEventRelation.from_entities
Value: _RelationArg(...)
ReportedEventRelation.methods
Value: _RelationArg(...)
ReportedEventRelation.time_expressions
Value: _RelationArg(...)
ReportedEventRelation.polarity
Value: _RelationAttr(...)
ReportedEventRelation.category
Value: _RelationAttr(...)
ReportedEventRelation.confidence
Value: _RelationAttr(...)
emtellipro.data.AnatomicSiteRelation
Bases: Relation
Bases: emtellipro.data.Relation
The anatomic site relation found in the processed document.
AnatomicSiteRelation.__slots__
Value: []
AnatomicSiteRelation.site
Value: _RelationArg(...)
AnatomicSiteRelation.situated_entity
Value: _RelationArg(...)
AnatomicSiteRelation.confidence
Value: _RelationAttr(...)
emtellipro.data.Concept
Bases: _LabeledObject
Bases: emtellipro.data._LabeledObject
Concept identified from input document.
Attributes:
id: The ID of concept found in the associated ontology.ontology: String containing the ontology’s name.description: The description of this concept from the ontology.map_rule: Present forsnomed_icd10_cmontology. Will be None if not returned by emtelliPro.map_advice: Present forsnomed_icd10_cmontology. Will be None if not returned by emtelliPro.map_group: Present forsnomed_icd10_cmontology. Will be None if not returned by emtelliPro.map_priority: Present forsnomed_icd10_cmontology. Will be None if not returned by emtelliPro.map_category: Present forsnomed_icd10_cmontology. Will be None if not returned by emtelliPro.
Concept.__slots__
Value: ['id', 'ontology', 'description', 'map_rule', 'map_advice', 'map_group', 'map_priority', 'map_category']
Concept.entity_type
Parsed entity type info from the description. Will be None if no entity type was found in the description.
Return type: str | None
Concept.__repr__
emtellipro.data.Span
Bases: object
A span representing a slice of the input document text.
s = Span(1, 45) start, end = s start, end (1, 45) slice(*s) slice(1, 45, None) s.slice slice(1, 45, None)
Initialization
Parameters:
- start: The start offset as integer.
- end: The end offset as integer. This is optional if the first argument is a span-like object.
Span.__slots__
Value: ['_start', '_end']
Span.start
The start index of the slice
Span.end
The end index of the slice (non-inclusive)
Span.stop
Alias for end.
Span.slice
A slice equivalent of this object. This can be used for indexing into texts more easily.
Span.asdict
Return a mapping of the attributes of this object:
Span.__iter__
Span.__eq__
Span.__hash__
Span.__repr__
emtellipro.data.Location
Bases: _LabeledObject
Bases: emtellipro.data._LabeledObject
Base class for different location types
Location.__slots__
Value: []
emtellipro.data.SentenceLocation
Bases: Location
Bases: emtellipro.data.Location
A sentence identified in the input document. May be discontinuous.
Attributes:
spans: list of Span objects representing parts of the sentencetext: list of strings containing text from the input document if returned by the API. If not, it will be None.
SentenceLocation.__slots__
Value: ['spans', 'text', '_sections_ref']
SentenceLocation.parts
A mapping of spans to text as returned by the API. If the text was not returned by the API, the values will be None.
SentenceLocation.sections
All sections containting this sentence.
emtellipro.data.SectionLocation
Bases: Location
Bases: emtellipro.data.Location
A section identified in the input document.
Attributes:
spans: list of Span objects representing parts of the sectiontext: list of strings containing text from the input document if returned by the API. If not, it will be None.name: the name of the sectionlevel: the level of the section in the document, starting with 0 for the outer-most section, and incremeting by 1 for each nested section.
SectionLocation.__slots__
Value: ['spans', 'text', 'level', 'name', '_parent_ref', '_page_refs', '_heading_ref']
SectionLocation.parent
The parent section that contains this sub-section, or None if this is an outer-most section
SectionLocation.parts
A mapping of spans to text as returned by the API. If the text was not returned by the API, the values will be None.
SectionLocation.pages
Pages linked to this section.
SectionLocation.heading
The heading for this section, if any.
emtellipro.data.HeadingLocation
Bases: Location
Bases: emtellipro.data.Location
A heading identified in the input document. May be discontinuous.
Attributes:
spans: List of Span objects representing parts of the heading.text: List of strings containing text from the input document if returned by the API. If not, it will be None.
HeadingLocation.__slots__
Value: ['spans', 'text', '_section_ref']
HeadingLocation.parts
A mapping of spans to text as returned by the API. If the text was not returned by the API, the values will be None.
HeadingLocation.section
The section containing this heading.
emtellipro.data.PageLocation
Bases: Location
Bases: emtellipro.data.Location
A page identified in the input document. May be discontinuous.
Attributes:
spans: List of Span objects representing parts of the page.page_index: The page index value returned by the API.text: List of strings containing text from the input document if returned by the API. If not, it will be None.
PageLocation.__slots__
Value: ['spans', 'page_index', 'text']
PageLocation.parts
A mapping of spans to text as returned by the API. If the text was not returned by the API, the values will be None.
emtellipro.data.UserPermissions
Bases: object
The permissions enabled for the user.
Initialization
UserPermissions.expires_time
The time this user’s access expires.
Value: None
UserPermissions.max_reports
The number of reports this user is allowed to process.
Value: None
UserPermissions.max_input_bytes
The number of bytes of input this user is allowed to process.
Value: None
UserPermissions.allowed_processing_features
The processing features that this user can use.
Value: field(...)
UserPermissions.__post_init__
emtellipro.data.User
Bases: object
Instances of this class represent the response of the /user API endpoint.
Initialization
User.username
The username as known by the server.
User.audit_reports
Whether reports are audited for this user.
Value: False
User.permissions
Type: UserPermissions | None
Permissions for this user.
Value: None
User.__repr__
User.__str__
emtellipro.data.ResultFile
Bases: object
A representation for the contents of a single emtelliPro JSON file.
Initialization
ResultFile.engine_version
The engine version string
ResultFile.docs
Type: typing.List[AnnotatedDocument]
The list of annotated documents contained in the file.
ResultFile.load
Load the results from a dictionary representation of the JSON results data.
Parameters:
Dictionary
Returns:
A result file instance.
ResultFile.loads
Load the results from a JSON string.
Parameters:
The contents of the JSON file as a string.
Returns:
A result file instance.
ResultFile.load_path
Load the results from a JSON file path.
Parameters:
The path to a JSON file containing an emtelliPro result.
Returns:
A result file instance.

