# ferc_xbrl_extractor.cli

A command line interface (CLI) to the xbrl extractor.

## Functions

| [`parse`](#ferc_xbrl_extractor.cli.parse)(→ argparse.Namespace)                                                                             | Process base commands from the CLI.                                   |
|---------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------|
| [`write_to_sqlite`](#ferc_xbrl_extractor.cli.write_to_sqlite)(→ None)                                                                       | Write one table to a SQLite database.                                 |
| [`write_to_duckdb`](#ferc_xbrl_extractor.cli.write_to_duckdb)(→ None)                                                                       | Write one table to a duckdb database.                                 |
| [`load_extracted`](#ferc_xbrl_extractor.cli.load_extracted)(→ None)                                                                         | Write extracted data to SQLite/Duckdb databases.                      |
| [`run_main`](#ferc_xbrl_extractor.cli.run_main)(→ None)                                                                                     | Log setup, taxonomy finding, and SQL IO.                              |
| [`convert_duckdb_into_parquet`](#ferc_xbrl_extractor.cli.convert_duckdb_into_parquet)(→ None)                                               | Convert the duckdb into a directory of parquet files.                 |
| [`convert_and_validate_datapackage_sqlite_to_parquet`](#ferc_xbrl_extractor.cli.convert_and_validate_datapackage_sqlite_to_parquet)(→ dict) | Convert the SQLite datapackage into one that points at Parquet files. |
| [`write_datapackage`](#ferc_xbrl_extractor.cli.write_datapackage)(→ None)                                                                   | Write a datapackage to <output_dir>/datapackage.json.                 |
| [`main`](#ferc_xbrl_extractor.cli.main)(→ None)                                                                                             | Parse arguments and pass to run_main.                                 |

## Module Contents

### ferc_xbrl_extractor.cli.parse() → argparse.Namespace

Process base commands from the CLI.

### ferc_xbrl_extractor.cli.write_to_sqlite(sqlite_engine: sqlalchemy.engine.Engine, table_name: str, table_data: pandas.DataFrame) → None

Write one table to a SQLite database.

### ferc_xbrl_extractor.cli.write_to_duckdb(duckdb_path: str | pathlib.Path, table_name: str, table_data: pandas.DataFrame) → None

Write one table to a duckdb database.

### ferc_xbrl_extractor.cli.load_extracted(extracted: [ferc_xbrl_extractor.xbrl.ExtractOutput](../xbrl/index.md#ferc_xbrl_extractor.xbrl.ExtractOutput), sqlite_uri: str, duckdb_path: str | pathlib.Path | None) → None

Write extracted data to SQLite/Duckdb databases.

### ferc_xbrl_extractor.cli.run_main(filings: list[pathlib.Path] | list[io.BytesIO], taxonomy: str | pathlib.Path | io.BytesIO, output_dir: pathlib.Path, sqlite_path: pathlib.Path, duckdb_path: pathlib.Path | None, form_number: int, workers: int | None, batch_size: int | None, loglevel: str, logfile: pathlib.Path | None, requested_tables: list[str] | None = None, instance_pattern: str = '') → None

Log setup, taxonomy finding, and SQL IO.

### ferc_xbrl_extractor.cli.convert_duckdb_into_parquet(duckdb_path: pathlib.Path, parquet_dir: pathlib.Path) → None

Convert the duckdb into a directory of parquet files.

We do this using COPY. We tried using EXPORT DATABASE, but it unfortunately
sanitizes the table names, which removes the schedule numbers in the table
names so we can’t use it.

### ferc_xbrl_extractor.cli.convert_and_validate_datapackage_sqlite_to_parquet(datapackage_path: pathlib.Path) → dict

Convert the SQLite datapackage into one that points at Parquet files.

* instead of `path` pointing at monolithic SQLite db, point at individual Parquet files instead
* update format/metadata fields
* remove irrelevant dialect field

### ferc_xbrl_extractor.cli.write_datapackage(datapackage: dict, output_dir: pathlib.Path) → None

Write a datapackage to <output_dir>/datapackage.json.

output_dir must exist.

### ferc_xbrl_extractor.cli.main() → None

Parse arguments and pass to run_main.