> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.emtelligent.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.emtelligent.com/_mcp/server.

# Preparing Files for Processing

A file manifest (.csv file) is the entry point for the document processing pipelines. It lists all the files to be processed, providing a path to where they are located within the input bucket. It is also a way for you to pass through metadata that you want to associate with the extracted output. This means you do not need to implement downstream processing steps to merge clinical extraction output with chartdate and other critical file or patient-based metadata.

## About this tutorial

This tutorial explains how to create a properly formatted manifest file, and how to upload it to your S3 bucket along with the documents it references. More details about the format and required columns can be found in [File Manifest Schema](/extract/reference/input#file-manifest-schema).

We will walk through completing these 4 steps:

1. Create the csv file
2. Add a header row
3. Add rows for each document you want to process
4. Upload to S3

### Step 1: Create the CSV File

Create a new file named `manifest.csv` using a text editor or spreadsheet application.

> **Note**
>
> Any file name is acceptable (e.g. `manifest_batch_1.csv`, `manifest_dataset_1.csv`); however, it must be valid CSV and you must provide the correct name in the job request.

### Step 2: Add the Header Row

The first row must contain column names. At minimum, these two columns are required:

* `document`, and
* `subject_id`

You may add any [optional columns](/extract/reference/input#additional-optional-columns). Let's add `subject_name`, `subject_dob` and `subject_gender`, so that your header row now contains:

```csv
document,subject_id,subject_name,subject_dob,subject_gender
```

### Step 3: Add Document Rows

For each PDF document, add a row with the corresponding metadata:

```csv
document,subject_id,subject_name,subject_dob,subject_gender
eptrial_pdf/patient_11110001_bundle.pdf,11110001,"Shui Chan","1966-08-01 04:10:48","male"
eptrial_pdf/patient_11110002_bundle.pdf,11110002,"Arabella Gabrielson","1960-05-23 15:04:07","female"
```

**Important**: The `document` column must specify S3 object keys (paths) to your PDF documents. These are the S3 object keys **relative to the root of your input bucket**. For example, if your files are organized as follows:

```text
s3://my-input-bucket/
├── eptrial_pdf/manifest.csv
├── eptrial_pdf/patient_11110001_bundle.pdf
└── eptrial_pdf/patient_11110002_bundle.pdf
```

Then, the object keys must be as follows:

```text
eptrial_pdf/patient_11110001_bundle.pdf
eptrial_pdf/patient_11110002_bundle.pdf
```

### Step 4: Upload to S3

Now, we will use the AWS CLI to upload the files your PDF documents and the corresponding manifest.csv to the designated S3 input bucket.

> **Tip**
>
> Before uploading, review the [Requirements Checklist](#requirements-checklist) below to ensure your manifest is properly formatted.

> **Note**
>
> During onboarding, you were provided with a bucket name to use for the input bucket. Make sure to use that one. The Extract API will only read from this bucket.

**Using AWS CLI**:

```bash
# Upload all PDF files and manifest from local directory to S3
aws s3 cp eptrial_pdf/ s3://my-input-bucket/eptrial_pdf/ --recursive
```

**Using AWS CLI with a specific profile**:

```bash
# If you have multiple AWS profiles configured
aws s3 cp eptrial_pdf/ s3://my-input-bucket/eptrial_pdf/ --recursive --profile my-profile
```

The examples above demonstrate the use of AWS CLI commands. You may use other methods to upload files to S3, including the AWS Console, AWS SDK, or other S3-compatible tools. For additional AWS CLI options, refer to the [AWS CLI S3 documentation](https://docs.aws.amazon.com/cli/latest/reference/s3/cp.html).

#### Validate S3 paths

Ensure the paths in the `document` column match the actual S3 keys.

**Verify your upload**:

```bash
# List files to confirm they were uploaded correctly
aws s3 ls s3://my-input-bucket/eptrial_pdf/ --recursive
```

**Example S3 bucket contents**:

```text
s3://my-input-bucket/
└── eptrial_pdf/
    ├── manifest.csv
    ├── patient_11110001_bundle.pdf
    └── patient_11110002_bundle.pdf
```

## Requirements Checklist

Before uploading your manifest, ensure these requirements are met:

1. All documents and manifest in the same S3 bucket with matching paths.

   Documents must be accessible from the same S3 bucket where manifest.csv is stored.

2. All required columns populated.

   Both `document` and `subject_id` columns must have values for every row.

3. Consistent date formats.

   Use ISO Timestamp format for all date fields.

4. Properly quoted text fields.

   Fields containing commas, quotes, or special characters must be enclosed in double quotes.

## Common Pitfalls

| Pitfall                                                            | How to Avoid                                                                                                                                                                                                        |
| ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Including full S3 URIs or absolute paths in the document column    | Use relative paths like `folder/file.pdf`, not `s3://bucket-name/folder/file.pdf` or `/folder/file.pdf`.                                                                                                            |
| Editing CSV in Excel and corrupting date formats                   | Excel may auto-format dates incorrectly. Use a text editor or ensure dates remain in ISO format (`YYYY-MM-DD`).                                                                                                     |
| Including spaces or special characters in S3 keys without encoding | Avoid spaces in filenames. Use underscores or hyphens: `patient_001.pdf` instead of `patient 001.pdf`.                                                                                                              |
| Mixing up the manifest location with document locations            | Documents must be in the same bucket as the manifest. If your manifest is at `s3://my-input-bucket/data/manifest.csv`, documents should be at `s3://my-input-bucket/data/file.pdf` (referenced as `data/file.pdf`). |
| Inconsistent date formats across rows                              | Pick one format and stick with it. All dates should use the same ISO timestamp pattern throughout the manifest.                                                                                                     |

## Best Practices

1. **Use consistent naming conventions** for your document S3 keys
2. **Include chartdate** when available for better temporal tracking
3. **Keep subject_ids unique**
4. **Avoid spaces in filenames**. Use underscores or hyphens: `patient_001.pdf` instead of `patient 001.pdf`.
5. **Avoid PHI in filenames**. Replace PII (personal identifiers) with generated IDs

## Troubleshooting Manifest Validation Errors

For guidance on troubleshooting validation errors, refer to the [Troubleshooting Input Validation Errors](/extract/reference/input#troubleshooting-input-validation-errors) section.