> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.emtelligent.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.emtelligent.com/_mcp/server.

# db.database

The database instance can be used for saving documents to a database.

## Module Contents

### Classes

| Name                                                              | Description                                                                                                                                                                                               |
| ----------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`MigrationTableState`](#emtelliprodbdatabasemigrationtablestate) | The state of the migration tables, if supported. Whether they're supported depends on both the schema and the database type; a file-based "database" will not support migrations even if the schema does. |
| [`DatabaseTestResult`](#emtelliprodbdatabasedatabasetestresult)   | The result of testing the database.                                                                                                                                                                       |
| [`DatabaseTransaction`](#emtelliprodbdatabasedatabasetransaction) | Represents a single database transaction. This is created using `db.begin()`.                                                                                                                             |
| [`Database`](#emtelliprodbdatabasedatabase)                       | The Database can be used for saving documents to a database. This may be a real DBMS or files on disk.                                                                                                    |

### API

#### `emtellipro.db.database.MigrationTableState`

```python
class emtellipro.db.database.MigrationTableState
```

**Bases:** object

The state of the migration tables, if supported. Whether they're supported
depends on both the schema and the database type; a file-based "database"
will not support migrations even if the schema does.

##### Initialization

```python
__init__(state, revision)
```

##### `MigrationTableState.state`

```python
state: typing.Literal['empty', 'absent', 'current', 'previous']
```

The alembic table can be:

* 'empty' when it's present, but contains no rows;
* 'absent' when it's not present (unrelated to the fact that maybe all
  other tables are also absent);
* 'current' when it contains the revision of the latest migration
  script; or
* 'previous' when it contains a revision ID that's older than the latest
  migration.

##### `MigrationTableState.revision`

```python
revision: str | None
```

The revision ID found in the alembic table, if it's present.

#### `emtellipro.db.database.DatabaseTestResult`

```python
class emtellipro.db.database.DatabaseTestResult
```

**Bases:** object

The result of testing the database.

This is truthy if the database is ready to use to save documents (this
means all the tables are in a consistent state). If the database is empty
and it's safe to create tables, `bool(result)` will be False.

##### Initialization

```python
__init__(*, migration=None, columns_found, columns_required)
```

##### `DatabaseTestResult.migration`

```python
migration: emtellipro.db.database.MigrationTableState | None
```

**Type:** [`MigrationTableState`](/nlp/python/reference/db-database#emtelliprodbdatabasemigrationtablestate) | None

This contains info about whether the migration-related tables are in a
consistent state. For schemas where database migrations are not
supported, this will be None. This will also be None for file-based
"databases", which don't support migrations regardless of schema.

**Value**: `None`

##### `DatabaseTestResult.columns_found`

```python
columns_found: set[str]
```

The columns found in the database, each one prefixed with the table name
like `"{table_name}.{column_name}"`.

##### `DatabaseTestResult.columns_required`

```python
columns_required: set[str]
```

These are the columns that are required by the latest schema to be
present.

##### `DatabaseTestResult.__bool__`

```python
__bool__()
```

##### `DatabaseTestResult.tables_found`

```python
property tables_found() -> set[str]
```

The table names found in the database.

**Return type:** `set[str]`

##### `DatabaseTestResult.tables_required`

```python
property tables_required() -> set[str]
```

The table names required by the schema.

**Return type:** `set[str]`

##### `DatabaseTestResult.can_create`

```python
property can_create
```

Whether it's safe to run the `Database.init()` method.

This checks if the tables that would be created do not already exist.

##### `DatabaseTestResult.can_migrate`

```python
property can_migrate
```

Whether it's safe to run the `Database.migrate()` method, according
to the contents of the `alembic` table (if migrations are supported).

Note that this does not check if the tables are consistent with an
earlier schema, so it may be possible that a migration will fail if
the schema was modified manually at some point.

#### `emtellipro.db.database.DatabaseTransaction`

```python
class emtellipro.db.database.DatabaseTransaction(engine_connection, schema)
```

**Bases:** object

Represents a single database transaction. This is created using
`db.begin()`.

> **Warning**
>
> This should not be instantiated by user code.

##### Initialization

**Parameters:**

* **engine_connection**: The connection to the engine that's within a
  transaction.
* **schema**:
  The schema to use when saving.

##### `DatabaseTransaction.bind`

```python
property bind
```

The underlying Engine to which this transaction is bound.

##### `DatabaseTransaction.connection`

```python
property connection
```

##### `DatabaseTransaction.dialect`

```python
property dialect
```

The underlying engine connection's dialect.

##### `DatabaseTransaction.get_bind`

```python
get_bind()
```

Return the underlying Engine to which this transaction is bound.

##### `DatabaseTransaction.execute`

```python
execute(
    *args, **kwargs
)
```

Calls out to the underlying Connection.execute() method.

##### `DatabaseTransaction.save`

```python
save(
    input_docs,
    annotated_docs,
    cache = None,
    options = None,
    **save_options
)
```

Save the documents to the database.

**Parameters:**

**`input_docs`**

A list of the input documents that were processed.

Note that if you wish to store document metadata in the
`documentmetadata` or `documentstructuredmetadata` tables,
that'll need to be included in the input documents.

See the `metadata` arugment of
[`InputDocument`](/nlp/python/reference/data#emtelliprodatainputdocument) for more details.

---

**`annotated_docs`**

A list of annotated documents from the results of processing
the `input_docs`. Note that the IDs of the input docs and
annotated docs must match: each annotated document's ID must be
found in the list of input documents.

When storing to a raw file engine (i.e. `raw://...`), this
must be the `Result` object instead.

---

**`cache`**

A cache used by the underlying schema save function to track
values that should be shared across calls. A potentially new
object of the same type will be returned by this function.

If all the calls to save are related in some way, the same
cache should be shared by all the calls.

This is useful for cases where (e.g.) there are multiple calls
within a processing job, and you want them all to be saved with
the same processing job ID, but the job ID is only determined
by inserting to a table. Then the ID is cached and the same
cache is shared by subsequent save calls so they all store
references to the same ID.

---

**`options`**

The SaveOptions instance for use with the schema's save
function. See `Database.schema.save_options` for the class.

---

**`**save_options`**

If `options` is None, then these keyword arguments (if
present) will be used to instantiate a SaveOptions instance.

---

**Returns:**

The cache instance.

#### `emtellipro.db.database.Database`

```python
class emtellipro.db.database.Database(uri, schema='legacy', engine=None, file_mode='a', **kwargs)
```

**Bases:** object

The Database can be used for saving documents to a database. This may be a
real DBMS or files on disk.

Example usage:

```python
db = Database(...)

db.init()       # create tables
# OR
db.migrate()    # if tables already exist and only need be migrated

with db.begin() as conn:
    conn.save(input_docs=..., annotated_docs=...)
```

##### Initialization

**Parameters:**

* **uri**:
  The URL for the database connection. If you're connecting to a
  database server, see [the SQLAlchemy documentation](https://docs.sqlalchemy.org/en/20/core/engines.html#database-urls)
  for full documentation on supported URL formats.
  This may also be a file-based URL, such as CSV/JSON. The supported formats are the same as for the `--output`
  command-line option, therefore see the documentation there for
  examples.
* **schema**:
  The name of the schema to use with this database, or the Schema
  object itself.
* **engine**:
  An Engine instance to use instead of connecting using the URI.
  This is for internal use.
* **file_mode**:
  When using a CSV or JSONL output, this can be set to `'w'` to
  enable write mode on the output files; this means each
  transaction will overwrite the previous one.
  It's important to note that if you leave `file_mode='a'`, you must call the [`init`](/nlp/python/reference/db-database#databaseinit) method to create the files (and
  write headers where necessary). Otherwise the files will only
  be created when there's data to write to them; depending on the
  processing features enabled this might not be the case.
  When seting file mode to 'w', all the files are unconditionally created when the [`begin`](/nlp/python/reference/db-database#databasebegin) context manager is used.
* **\*\*kwargs**:
  Any extra arguments to be used when instanting the underlying
  database engine.
  In the case of file-based databases (e.g. csv, json, jsonl),
  any extra arguments are ignored.
* **If you're using Snowflake, you can pass**: :
  snowflake_private_key_path="path/to/key"
* **If the private key is encrypted, you may also pass the passphrase**: :
  snowflake_private_key_passphrase="PASSWORD"
  If you don't pass in the passphrase, it'll be attempted to be
  read from the `SNOWFLAKE_PRIVATE_KEY_PASSPHRASE` or
  `SNOWSQL_PRIVATE_KEY_PASSPHRASE` environment variables.
  Any extra arguments are passed to SQLAlchemy's
  `create_engine()` function.

##### `Database.name`

```python
property name
```

The name of the underlying engine's dialect.

##### `Database.test`

```python
test() -> emtellipro.db.database.DatabaseTestResult
```

Attempt to connect to the database and validate it for use with the
schema.

**Returns:**

The validation results when checking the database.

**Raises:**

* [`emtellipro.exc.DatabaseConnectionError`](/nlp/python/reference/exc#emtelliproexcdatabaseconnectionerror): Connecting to the database failed for some reason. See the traceback for details.

**Return type:** [`DatabaseTestResult`](/nlp/python/reference/db-database#emtelliprodbdatabasedatabasetestresult)

##### `Database.init`

```python
init()
```

Create all the database tables that are needed.

If the database is already initialized with the current schema, this
function will do nothing; if the schema version marker is missing,
however, `DatabaseSchemaConflict` will be raised.

**Raises:**

* [`DatabaseSchemaConflict`](/nlp/python/reference/exc#emtelliproexcdatabaseschemaconflict): The database contains a schema that conflicts with the schema defined in this library.

##### `Database.create_indexes`

```python
create_indexes()
```

Create the appropriate full text search indexes for the database.

> **Note**
>
> Regular indexes useful for joining tables efficiently are always
> created by default when the tables are created.

**Raises:**

* `NotImplementedError`: The database type is not supported for index. This includes Snowflake and file-based "databases".
* [`MissingFullTextSupport`](/nlp/python/reference/exc#emtelliproexcmissingfulltextsupport): The database doesn't support full text indexes.

##### `Database.check_schema`

```python
check_schema()
```

Check whether the schema in the database is compatible with the current
SDK release.

"Compatible" means that all the tables and columns that are required
for saving annotated documents are present.

**Raises:**

* [`exc.DatabaseSchemaMissingColumnsError`](/nlp/python/reference/exc#emtelliproexcdatabaseschemamissingcolumnserror): This is raised when there are missing columns in the database.

  This will also include information about the missing columns and the schema version.

  If the schema version is a specific version, then it *may* be possible to migrate to the latest version.

  If the schema version number is 'missing', then it's not possible to automatically determine which schema version it was for a migration to the latest schema (or if the database is compatible with any older schema version).

  If the schema version is "latest" and there are missing columns, this may indicate that the columns were improperly dropped by some other tool or individual.
* [`exc.DatabaseMissingVersionError`](/nlp/python/reference/exc#emtelliproexcdatabasemissingversionerror): Raised when the schemna version number stored in `alembic_version.version_num` is missing, but all the expected columns are present.

  This indicates that it may be possible to manually add the latest schema version number to that table.
* [`exc.DatabaseSchemaValidationError`](/nlp/python/reference/exc#emtelliproexcdatabaseschemavalidationerror): May be raised as a catch-all error.
* `NotImplementedError`: For file-based output formats, where schema checking is not possible.

##### `Database.migrate`

```python
migrate()
```

Migrate the database to the latest schema version.

[`create_indexes`](/nlp/python/reference/db-database#databasecreate_indexes) should be called after migration to
rebuild the indexes.

##### `Database.drop`

```python
drop()
```

Will drop all tables managed by the SDK.

If the database is file-based (e.g. CSV/JSON), the entire directory
will be removed.

##### `Database.clear`

```python
clear()
```

Will DELETE all data rows from the database. This includes only the
tables this library manages; any other tables will be left untouched.

This only works for real databases connected through SQLAlchemy. For
file-based databases, you should delete the files instead.

##### `Database.begin`

```python
begin()
```

Begin a transaction, returning a context manager. The context manager
must be used for storing documents and results in the database.

Example usage:

```python
db = Database(...)

with db.begin() as conn:
    conn.save(input_docs=..., annotated_docs=...)
```