pudl.dagster.assets.deploy.ferceqr#

Define deployment helper assets for publishing FERC EQR outputs.

These assets run during batch builds to publish transformed FERC EQR outputs, notify Zulip of success or failure, and create status files that tell the batch job when deployment handling is complete.

Attributes#

`logger`
`FERCEQR_TRANSFORM_ASSETS`

Functions#

`_get_logfile_list`(→ str)	Return pointer to logs to send in Zulip message.
`_get_etl_status_csv_path`(→ pathlib.Path)
`_write_status_file`(status, pudl_paths)	Notify build script that job is complete by creating a status file.
`deployment_status_asset`(→ dagster.AssetsDefinition)	Create a wrapper for deployment handler assets.
`deploy_ferceqr`(pudl_paths, zulip)	Publish EQR outputs to cloud storage.
`handle_ferceqr_deployment_failure`(pudl_paths, zulip)	Send notification if EQR deployment failed.

Module Contents#

pudl.dagster.assets.deploy.ferceqr.logger[source]#

pudl.dagster.assets.deploy.ferceqr.FERCEQR_TRANSFORM_ASSETS = ['core_ferceqr__contracts', 'core_ferceqr__transactions', 'core_ferceqr__quarterly_identity',...[source]#

pudl.dagster.assets.deploy.ferceqr._get_logfile_list(build_id: str) → str[source]#: Return pointer to logs to send in Zulip message.

pudl.dagster.assets.deploy.ferceqr._get_etl_status_csv_path(pudl_paths: pudl.workspace.setup.PudlPaths) → pathlib.Path[source]#

pudl.dagster.assets.deploy.ferceqr._write_status_file(status: Literal['SUCCESS', 'FAILURE'], pudl_paths: pudl.workspace.setup.PudlPaths)[source]#: Notify build script that job is complete by creating a status file.

pudl.dagster.assets.deploy.ferceqr.deployment_status_asset(handler: collections.abc.Callable) → dagster.AssetsDefinition[source]#

Create a wrapper for deployment handler assets.

This is useful to gracefully handle errors if the deployment status assets fail for any reason. When these assets fail, sometimes the logs don’t show up in the batch job appropriately, and the status file never gets created, so the job keeps running until it eventually times out.

pudl.dagster.assets.deploy.ferceqr.deploy_ferceqr(pudl_paths: pudl.workspace.setup.PudlPaths, zulip: pudl.dagster.resources.ZulipNotificationResource)[source]#: Publish EQR outputs to cloud storage.

pudl.dagster.assets.deploy.ferceqr.handle_ferceqr_deployment_failure(pudl_paths: pudl.workspace.setup.PudlPaths, zulip: pudl.dagster.resources.ZulipNotificationResource)[source]#: Send notification if EQR deployment failed.