pudl.dagster.sensors

Dagster sensors for PUDL.

This module defines sensor-based automation that watches the state of the PUDL code location and requests follow-on work when specific conditions are met. Add sensor definitions here when they poll Dagster state, external state, or partition progress to trigger a run, rather than when the logic belongs inside an asset or job itself. Keep the module focused on automation entrypoints and their shared defaults.

For the underlying Dagster concept, see https://docs.dagster.io/guides/automate/sensors

Attributes

ferceqr_deployment_sensor_status
default_sensors

Functions

ferceqr_deployment_sensor(context)

Monitor a batch FERC EQR backfill and trigger deployment handling.

Module Contents

pudl.dagster.sensors.ferceqr_deployment_sensor_status

pudl.dagster.sensors.ferceqr_deployment_sensor(context: dagster.RunStatusSensorContext)

Monitor a batch FERC EQR backfill and trigger deployment handling.

This sensor is only enabled when FERCEQR_BUILD is set, which is the case for the Google Batch jobs that run the published FERC EQR build. Every 60 seconds it polls Dagster for the partition status of the quarterly extract asset and the core transformed FERC EQR assets.

While any required partition is still in progress, or has not yet been attempted, the sensor does nothing. Once the backfill is complete, it requests exactly one of the deployment handler assets: deploy_ferceqr after a successful backfill, or handle_ferceqr_deployment_failure if any partition failed.

pudl.dagster.sensors.default_sensors