> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.emtelligent.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.emtelligent.com/_mcp/server. # Input Reference ## File Manifest Schema * **File format:** CSV (Comma-Separated Values) * **Encoding:** UTF-8 * **Delimiter:** Comma-delimited * **Header row:** Required (first row contains column names) Also see manifest file limits [here](/extract/reference/system-limits#manifest-limits). ### Example of Minimal Manifest This is an example of a simple manifest, containing required columns (`document` and `subject_id`). ```text document,subject_id,chartdate dataset1/patient_001.pdf,P001,2025-01-15 dataset1/patient_002.pdf,P002,2025-02-20 ``` ### Required and Recommended Columns | Column Name | Data Type | Description | Type | | ------------ | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | | `document` | varchar(255) | The object key in the source bucket. Follow S3 naming guidelines: use web-safe characters, no whitespace, file suffix is required and must match the contents. | Required | | `subject_id` | varchar(255) | Unique patient identifier in this dataset. | Required | | `chartdate` | ISO Timestamp | Date in ISO timestamp format with optional time zone offset, e.g. `1999-01-08T04:05:06-8:00` | Recommended | ### Additional Optional Columns | Column Name | Data Type | Description | | ---------------------- | ------------ | ---------------------------------------------------------------- | | `subject_name` | String | Name of the subject | | `subject_dob` | ISO Date | Date of birth, e.g. `1999-01-08` | | `subject_gender` | varchar(255) | Gender | | `source_document_id` | varchar(255) | Original document ID (e.g. document ID key in source CDR or EMR) | | `original_category` | varchar(255) | Document category from source system | | `original_subcategory` | varchar(255) | Document subcategory from source system | | `clinical_domain` | varchar(255) | Clinical domain classification | | `institution` | varchar(255) | Originating institution | | `hadm_id` | varchar(255) | Hospital admission ID | | `requestor` | varchar(255) | Requesting party | | `author_name` | varchar(255) | Document author name | > **Note** > > For information about file size limits, manifest row limits, and other constraints, see [System Limits](/extract/reference/system-limits). ## Input Validation * All manifest fields are validated before processing begins * If a manifest row fails validation, processing will not start * Validation checks include: * Every file in the manifest must be present in the source bucket * File types are correct for the pipeline (only `.pdf` files are supported) * Files are non-zero bytes * Additional file-type-specific requirements (e.g. text files must be UTF-8 encoded) * Any validation error causes the job to be cancelled * A new job must be submitted after correcting validation errors ### Validation Schema Each row in the manifest CSV is validated against the following JSON Schema: ```json { "type": "object", "properties": { "document": { "type": "string", "description": "Object key within the source bucket", "maxLength": 255 }, "subject_id": { "type": "string", "description": "Identifier that uniquely identifies a patient in this dataset", "maxLength": 255 }, "chartdate": { "type": "string", "format": "date-time", "description": "Chart date (ISO timestamp format)" }, "subject_name": { "type": "string", "maxLength": 255, "description": "Name of the subject" }, "subject_dob": { "type": "string", "format": "date-time", "description": "Subject DOB" }, "subject_gender": { "type": "string", "maxLength": 255, "description": "Subject gender" }, "source_document_id": { "type": "string", "maxLength": 255, "description": "Original document ID" }, "original_category": { "type": "string", "maxLength": 255, "description": "Document category from source system" }, "original_subcategory": { "type": "string", "maxLength": 255, "description": "Document subcategory from source system" }, "clinical_domain": { "type": "string", "maxLength": 255, "description": "Clinical domain classification" }, "institution": { "type": "string", "maxLength": 255, "description": "Originating institution" }, "hadm_id": { "type": "string", "maxLength": 255, "description": "Hospital admission ID" }, "requestor": { "type": "string", "maxLength": 255, "description": "Requesting party" }, "author_name": { "type": "string", "maxLength": 255, "description": "Document author name" } }, "required": [ "document", "subject_id" ] } ``` **Key validation rules:** * All string fields have a maximum length of 255 characters * `document` and `subject_id` are required fields * Date fields (`chartdate`, `subject_dob`) must use ISO 8601 date-time format * All other fields are optional ## Troubleshooting Input Validation Errors Manifest validation occurs on job submission; failures may result in one of the following errors: | Error Message | Description | | ------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | | *CSV is missing 'subject_id' column* | The manifest.csv must include the required `subject_id` column in the header row. | | *CSV is missing 'document' column* | The manifest.csv must include the required `document` column in the header row. | | *Failed to retrieve manifest from S3* | Verify that the provided AWS credentials are correct and that the manifest file exists at the specified S3 path. | | *Filenames exceeding length limit* | File paths in the `document` column must not exceed 250 characters. | | *Document is not a PDF file based on file extension* | The documents listed in the `document` column must have a `.pdf` extension. | | *Document is an invalid PDF file* | The documents listed in the `document` column must be valid PDF files. | | *Input should be a valid datetime or date* | Date fields must use ISO timestamp format: `YYYY-MM-DD`, `YYYY-MM-DD HH:MM:SS`, or `YYYY-MM-DDTHH:MM:SS`. | | *Manifest contains X rows, which exceeds the maximum allowed* | The manifest.csv must contain no more than 1,000 rows. | | *Manifest file size (X MB) exceeds maximum limit* | The manifest.csv file size must not exceed 20 MB. | | *Manifest CSV is not comma-delimited* | The manifest file must use commas (`,`) as column delimiters, not tabs or other characters. | | *Object X was listed in the file manifest but does not exist* | All S3 keys listed in the `document` column must exist in your input bucket. Verify the file paths match the actual S3 object keys. | | *String should have at most 255 characters* | Text values in any column must not exceed 255 characters. |