Input Reference | emtelligent Documentation

File Manifest Schema

File format: CSV (Comma-Separated Values)
Encoding: UTF-8
Delimiter: Comma-delimited
Header row: Required (first row contains column names)

Also see manifest file limits here.

Example of Minimal Manifest

This is an example of a simple manifest, containing required columns (document and subject_id).

document,subject_id,chartdate
dataset1/patient_001.pdf,P001,2025-01-15
dataset1/patient_002.pdf,P002,2025-02-20

Required and Recommended Columns

Column Name	Data Type	Description	Type
`document`	varchar(255)	The object key in the source bucket. Follow S3 naming guidelines: use web-safe characters, no whitespace, file suffix is required and must match the contents.	Required
`subject_id`	varchar(255)	Unique patient identifier in this dataset.	Required
`chartdate`	ISO Timestamp	Date in ISO timestamp format with optional time zone offset, e.g. `1999-01-08T04:05:06-8:00`	Recommended

Additional Optional Columns

Column Name	Data Type	Description
`subject_name`	String	Name of the subject
`subject_dob`	ISO Date	Date of birth, e.g. `1999-01-08`
`subject_gender`	varchar(255)	Gender
`source_document_id`	varchar(255)	Original document ID (e.g. document ID key in source CDR or EMR)
`original_category`	varchar(255)	Document category from source system
`original_subcategory`	varchar(255)	Document subcategory from source system
`clinical_domain`	varchar(255)	Clinical domain classification
`institution`	varchar(255)	Originating institution
`hadm_id`	varchar(255)	Hospital admission ID
`requestor`	varchar(255)	Requesting party
`author_name`	varchar(255)	Document author name

For information about file size limits, manifest row limits, and other constraints, see System Limits.

Input Validation

All manifest fields are validated before processing begins
If a manifest row fails validation, processing will not start
Validation checks include:
- Every file in the manifest must be present in the source bucket
- File types are correct for the pipeline (only .pdf files are supported)
- Files are non-zero bytes
- Additional file-type-specific requirements (e.g. text files must be UTF-8 encoded)
Any validation error causes the job to be cancelled
A new job must be submitted after correcting validation errors

Validation Schema

Each row in the manifest CSV is validated against the following JSON Schema:

{
  "type": "object",
  "properties": {
    "document": {
      "type": "string",
      "description": "Object key within the source bucket",
      "maxLength": 255
    },
    "subject_id": {
      "type": "string",
      "description": "Identifier that uniquely identifies a patient in this dataset",
      "maxLength": 255
    },
    "chartdate": {
      "type": "string",
      "format": "date-time",
      "description": "Chart date (ISO timestamp format)"
    },
    "subject_name": {
      "type": "string",
      "maxLength": 255,
      "description": "Name of the subject"
    },
    "subject_dob": {
      "type": "string",
      "format": "date-time",
      "description": "Subject DOB"
    },
    "subject_gender": {
      "type": "string",
      "maxLength": 255,
      "description": "Subject gender"
    },
    "source_document_id": {
      "type": "string",
      "maxLength": 255,
      "description": "Original document ID"
    },
    "original_category": {
      "type": "string",
      "maxLength": 255,
      "description": "Document category from source system"
    },
    "original_subcategory": {
      "type": "string",
      "maxLength": 255,
      "description": "Document subcategory from source system"
    },
    "clinical_domain": {
      "type": "string",
      "maxLength": 255,
      "description": "Clinical domain classification"
    },
    "institution": {
      "type": "string",
      "maxLength": 255,
      "description": "Originating institution"
    },
    "hadm_id": {
      "type": "string",
      "maxLength": 255,
      "description": "Hospital admission ID"
    },
    "requestor": {
      "type": "string",
      "maxLength": 255,
      "description": "Requesting party"
    },
    "author_name": {
      "type": "string",
      "maxLength": 255,
      "description": "Document author name"
    }
  },
  "required": [
    "document",
    "subject_id"
  ]
}

Key validation rules:

All string fields have a maximum length of 255 characters
document and subject_id are required fields
Date fields (chartdate, subject_dob) must use ISO 8601 date-time format
All other fields are optional

Troubleshooting Input Validation Errors

Manifest validation occurs on job submission; failures may result in one of the following errors:

Error Message	Description
CSV is missing ‘subject_id’ column	The manifest.csv must include the required `subject_id` column in the header row.
CSV is missing ‘document’ column	The manifest.csv must include the required `document` column in the header row.
Failed to retrieve manifest from S3	Verify that the provided AWS credentials are correct and that the manifest file exists at the specified S3 path.
Filenames exceeding length limit	File paths in the `document` column must not exceed 250 characters.
Document is not a PDF file based on file extension	The documents listed in the `document` column must have a `.pdf` extension.
Document is an invalid PDF file	The documents listed in the `document` column must be valid PDF files.
Input should be a valid datetime or date	Date fields must use ISO timestamp format: `YYYY-MM-DD`, `YYYY-MM-DD HH:MM:SS`, or `YYYY-MM-DDTHH:MM:SS`.
Manifest contains X rows, which exceeds the maximum allowed	The manifest.csv must contain no more than 1,000 rows.
Manifest file size (X MB) exceeds maximum limit	The manifest.csv file size must not exceed 20 MB.
Manifest CSV is not comma-delimited	The manifest file must use commas (`,`) as column delimiters, not tabs or other characters.
Object X was listed in the file manifest but does not exist	All S3 keys listed in the `document` column must exist in your input bucket. Verify the file paths match the actual S3 object keys.
String should have at most 255 characters	Text values in any column must not exceed 255 characters.