flatten

This module provides a method for flattening JSON files from emtelliPro, in a similar method to emtellipro-flatten-json.

Examples:

Simple usage involves reading a JSON file, and saving the documents contained within to a TSV file.

from emtellipro.data import ResultFile
from emtellipro.flatten import Flattener
result_file = ResultFile.load_path('path/to/result.json')
flattener = Flattener(include=('document_id', 'spans'))
flattener.add(*result_file.docs)
tables = flattener.flatten()
# write TSV files to the given directory
tables.write('path/to/output/directory/', 'tsv')

However, if you prefer to work with Pandas DataFrames, you can iterate over the tables and convert them to dataframes.

for name, table in tables:
    print(name)
    print(table.to_df())

Module Contents

Classes

API

emtellipro.flatten.Flattener

class emtellipro.flatten.Flattener(include=())

Bases: object

Flattens annotated documents from emtelliPro into a set of tables.

Initialization

Parameters:

• include: A list of extra columns to include when generating the flattened tables. document_id The ID of the document containg each item. spans The character spans for each item. spans_in_sentences The character spans for each item, adjusted to be relative to the containing sentence.

Flattener.add

add(*docs: emtellipro.data.AnnotatedDocument)

Add documents to the flattener.

Parameters:

Flattener.flatten

flatten() -> emtellipro.flatten.Tables

Flatten the documents into tables.

Returns:

The flattened tables.

Return type: Tables

emtellipro.flatten.Tables

class emtellipro.flatten.Tables(tables)

Bases: object

The flattened tables, produced by Flattener.flatten.

Examples:

This table can be iterated over, which is helpful for storing to custom file paths, or for converting the table to a pandas DataFrame.

tables: Tables = flattener.flatten()
for name, table in tables:
    print(table.to_df())

Initialization

Parameters:

• tables: The tables (internal data structure).

Tables.names

property names

A list of all the table names.

Tables.__iter__

__iter__()

Iterate over tuples containing the table name, and Table instance.

Tables.write

write(
    outdir,
    format,
    values_separator = '::',
    dewhitespace = False
)

Write the tables to the given output directory.

Parameters:

emtellipro.flatten.Table

class emtellipro.flatten.Table(items)

Bases: object

A single table in a Tables collection.

Initialization

Parameters:

• items: The items in the table (private data structure).

Table.write

write(
    path,
    format,
    values_separator = '::',
    dewhitespace = False
)

Write the table to the given output file.

Parameters:

Table.to_df

to_df(
    values_separator = '::', dewhitespace = False
)

Return a pandas DataFrame containing the data.

Parameters: