Data Access

We publish the PUDL pipeline outputs in several ways to serve different users and use cases. We’re always trying to increase the accessibility of the PUDL data, so if you have a suggestion, please open a GitHub issue. If you have a question, you can create a GitHub discussion.

We recommend working with tables with the out_ prefix, as these tables contain the most complete and easiest to work with data. For more information about the different types of tables, read through PUDL’s naming conventions.

Quick Reference

Platform	Format	Version	User Types	Use Cases
PUDL Data Viewer	Parquet, CSV	nightly	Data Explorer, Spreadsheet Analyst, Jupyter Notebook User	Explore PUDL data interactively in a web browser, including hourly timeseries data. Select data to download as CSVs for local analysis in spreadsheets. Download full tables as Parquet files to play with programmatically.
Kaggle	Parquet, DuckDB, SQLite	nightly	Data Scientist, Data Analyst, Jupyter Notebook User	Work with PUDL data products in Jupyter Notebooks via the web with minimal setup. Explore curated and contributed analyses and visualizations using PUDL data. notebooks. Create and share your own interactive notebooks using PUDL data.
Cloud Storage	Parquet, DuckDB, SQLite	nightly, stable	Data Scientist, Analytics Engineer, Data Engineer, Cloud Developer	Performant remote queries of clearly versioned PUDL Parquet outputs from cloud computing platforms or GitHub Actions. Fast bulk download of outputs for local use. Parquet based data warehouse for large-scale data analysis in the cloud. Integrates well with Pandas, Polars, DuckDB, and other dataframe libraries.
Zenodo Archives	Parquet, DuckDB, SQLite	stable	Researcher, Publisher, Archivist	Access a specific, immutable version of the PUDL data by DOI for citation in academic publications or other applications where long-term reproducibility is needed. Web-based bulk download of data for local analysis.
Coding Agents (PUDL Agent Skill)	Parquet	nightly, stable	Coding Agent User	Let an AI coding agent (Claude Code, OpenCode, Pi, etc.) discover PUDL tables, look up column meanings and data-quality caveats, and load the Parquet outputs into a notebook or script.

How Should You Access PUDL Data?

In order to serve a wider variety of users, we provide several ways to access PUDL data. When choosing an access method you’ll want to consider:

• What tool or platform do you want to use to access the data?

• What data format are you most comfortable with?

• Which historical version of the data do you want?

Data Platform

PUDL data is distributed on a number of different platforms to accommodate a variety of different use cases. These include PUDL Data Viewer, Kaggle, Cloud Storage, Zenodo Archives, and the PUDL agent skill for use with AI coding agents.

Data Format

PUDL data is distributed in several file formats:

• Apache Parquet: a compressed, columnar storage format in which each file stores a single table. Parquet supports rich data types and metadata, and is highly performant. Parquet is PUDL’s primary output format, and the format we recommend for all new work.

• DuckDB: a fast, self-contained analytical database that holds many tables in a single file. We publish the fully processed PUDL data as a single pudl.duckdb database, assembled from the Parquet outputs. It preserves the full set of column types and primary key constraints, but omits foreign key constraints to keep the file size down.

• SQLite: a self-contained relational database that holds many tables in a single file, supported by many programming languages and tools.

Warning

The ``pudl.sqlite`` database is being deprecated. It is currently provided purely for backwards compatibility, and we will stop producing SQLite versions of the fully processed PUDL data in 2027. Please migrate to the Parquet outputs or the new pudl.duckdb database. This deprecation does not affect the minimally processed raw FERC data, which will continue to be distributed as SQLite for the time being.

Not every table is available in every format:

• Parquet only: The hourly data tables are distributed only as Parquet files. These tables have hourly in their names, and are excluded from the pudl.duckdb and pudl.sqlite databases.

• SQLite only: The minimally processed FERC data which we have converted from XBRL and DBF into SQLite are only available in SQLite (and, experimentally, DuckDB — see Raw FERC XBRL data converted to DuckDB).

All Parquet data is available through PUDL Data Viewer for previewing. It can be downloaded as a CSV through that platform if you need to work with it in spreadsheets. For programmatic use we strongly recommend that you access the Parquet files in S3 directly. See Cloud Storage.

The pudl.duckdb and pudl.sqlite databases and the raw FERC SQLite databases can be downloaded from S3 (see Cloud Storage) or our regular versioned releases (see Zenodo Archives). All the converted FERC databases can also be accessed through PUDL Data Viewer.

Data Version

We assign a version number to our quarterly data releases so they can be easily identified. These versions are based on the date of publication. For example, v2024.11.0 would be the first release of the data that happened in November 2024. These are referred to as stable releases, and are archived for long-term access and citation.

We also provide access to a nightly development build of the data, which is updated most weekday mornings. These builds are useful for beta testing new outputs, but are ephemeral and may not be as well validated as the stable releases.

PUDL Data Viewer

The PUDL Data Viewer provides flexible search of table metadata, live data preview with filtering and sorting, and CSV export of up to 5 million rows. It provides access to all of the PUDL Parquet outputs, the minimally processed FERC Form 1, 2, 6, 60, and 714 data, and the FERC EQR.

It also provides links to download Parquet files for each table, which you can view locally with tools like Tad.

Kaggle

Are you comfortable with Jupyter Notebooks? Want to explore a fresh version of all available PUDL data without needing to do any environment setup? We provide several example notebooks on Kaggle. (they are also pushed to our PUDL Examples Repo on GitHub). These notebooks pull data directly from the Parquet outputs in S3 (see below) and so can also be used locally if you’re familiar with setting up a Python environment and running Jupyter.

Our nightly build outputs also automatically update the PUDL Kaggle dataset once a week. This dataset contains all the PUDL outputs, so it’s quite large and can take a few minutes to copy into your Kaggle notebook’s private workspace, but once it’s copied, access will be fast.

Cloud Storage

All PUDL data products are freely available in the AWS Open Data Registry including both stable and nightly outputs and multiple years of past stable releases. These include data in Parquet, DuckDB, and SQLite formats. The AWS S3 bucket is:

s3://pudl.catalyst.coop

The same outputs are available in a similarly named “requester pays” Google Cloud Storage bucket. However, you will need to authenticate your GCP account. The GCS bucket is:

gs://pudl.catalyst.coop

The SQLite databases must be downloaded for local use, but Parquet and DuckDB files can be queried remotely using a number of different tools. Some examples below:

Pandas

Using Pandas read_parquet()

Note

You will need to install pandas with the extra cloud dependencies.

import pandas as pd

# Outputs from the most recent nightly build:
nightly_df = pd.read_parquet("s3://pudl.catalyst.coop/nightly/core_eia__codes_energy_sources.parquet")
# Outputs from the most recent stable data release:
stable_df = pd.read_parquet("s3://pudl.catalyst.coop/stable/core_eia__codes_energy_sources.parquet")
# A specific stable version of the data:
versioned_df = pd.read_parquet("s3://pudl.catalyst.coop/v2024.11.0/core_eia__codes_energy_sources.parquet")

DuckDB

DuckDB with the httpfs extension can query the PUDL outputs in place on S3 — you don’t need to download anything first.

Note

Our bucket name contains dots (pudl.catalyst.coop), which breaks DuckDB’s default virtual-host S3 addressing: the request goes to pudl.catalyst.coop.s3.us-west-2.amazonaws.com, and AWS’s wildcard TLS certificate (*.s3.us-west-2.amazonaws.com) only covers a single label, so the handshake fails. Create an anonymous S3 secret that uses path-style addressing to avoid this. It applies to both read_parquet() and ATTACH:

-- Install the httpfs extension once and it will be available in subsequent sessions
INSTALL httpfs;
LOAD httpfs;
CREATE SECRET (TYPE s3, PROVIDER config, REGION 'us-west-2', URL_STYLE 'path');

To query an individual Parquet file:

SELECT * FROM read_parquet('s3://pudl.catalyst.coop/nightly/core_eia__codes_energy_sources.parquet');

To query the full pudl.duckdb database, attach it read-only and refer to its tables by name:

ATTACH 's3://pudl.catalyst.coop/nightly/pudl.duckdb' AS pudl (READ_ONLY);
SELECT
    report_date,
    plant_id_eia,
    generator_id,
    capacity_mw
FROM pudl.out_eia__yearly_generators
WHERE technology_description = 'Nuclear';

The same works from the DuckDB Python API:

import duckdb

con = duckdb.connect()
con.execute("INSTALL httpfs; LOAD httpfs;")
con.execute("CREATE SECRET (TYPE s3, PROVIDER config, REGION 'us-west-2', URL_STYLE 'path')")
con.execute("ATTACH 's3://pudl.catalyst.coop/nightly/pudl.duckdb' AS pudl (READ_ONLY)")
df = con.execute(
    "SELECT * FROM pudl.out_eia__yearly_generators WHERE technology_description = 'Nuclear'"
).df()

Downloading pudl.duckdb for local use works too, and will be faster if you plan to run many queries — see Direct Links for Bulk Download.

Polars

Polars can scan the Parquet outputs directly from S3, using lazy evaluation so that filters and column selection are pushed down and only the data you actually need is downloaded:

import polars as pl

lf = pl.scan_parquet(
    "s3://pudl.catalyst.coop/nightly/out_eia__yearly_generators.parquet",
    storage_options={"aws_region": "us-west-2", "aws_skip_signature": "true"},
)
df = (
    lf.filter(pl.col("technology_description") == "Nuclear")
    .select("report_date", "plant_id_eia", "generator_id", "capacity_mw")
    .collect()
)

R (dplyr)

The arrow package lets you open a Parquet file directly from S3 with dplyr verbs. Column selection and row filters are pushed down to Arrow, so only the data you ask for is read; call collect() to pull the result into a regular data frame:

library(arrow)
library(dplyr)

generators <- open_dataset(
  "s3://pudl.catalyst.coop/nightly/out_eia__yearly_generators.parquet",
  format = "parquet"
)

df <- generators |>
  filter(technology_description == "Nuclear") |>
  select(report_date, plant_id_eia, generator_id, capacity_mw) |>
  collect()

The AWS CLI

You can also use the AWS CLI to see what data is available and download it locally. For example, to list the contents of the AWS S3 bucket to see what historic versions are available:

aws s3 ls --no-sign-request s3://pudl.catalyst.coop/

To list the contents of a particular version:

aws s3 ls --no-sign-request s3://pudl.catalyst.coop/v2026.8.0/

To download the full PUDL DuckDB database (9 GB) from the nightly build outputs:

aws s3 cp --no-sign-request s3://pudl.catalyst.coop/nightly/pudl.duckdb .

Direct Links for Bulk Download

The links below allow bulk download the most recent nightly builds of the PUDL Parquet, DuckDB, and SQLite outputs, as well as their associated metadata in JSON.

Fully Processed Parquet Data

• PUDL Parquet Datapackage (JSON): a Frictionless Data Package v2 descriptor listing all PUDL Parquet tables with full column types, constraints, and foreign keys. Browse the schema of every table without downloading any data.

• PUDL Parquet Archive (ZIP): all PUDL Parquet files bundled together with the datapackage.json descriptor inside. Suitable for bulk download to a local machine.

Fully Processed DuckDB Database

• Main PUDL Database

Fully Processed SQLite Databases

Warning

The fully processed pudl.sqlite database is deprecated and will no longer be produced starting in 2027. See Data Format. Use the Parquet outputs or the pudl.duckdb database instead.

• Main PUDL Database (~3GB)

• US Census DP1 Database (2010)

Raw FERC DBF & XBRL data converted to SQLite

• FERC Form 1:

  • FERC-1 SQLite derived from DBF (1994-2020)

  • FERC-1 SQLite derived from XBRL (2021-2024)

  • FERC-1 Datapackage (JSON) describing SQLite derived from XBRL

  • FERC-1 XBRL Taxonomy Metadata as JSON (2021-2024)

• FERC Form 2:

  • FERC-2 SQLite derived from DBF (1996-2020)

  • FERC-2 SQLite derived from XBRL (2021-2024)

  • FERC-2 Datapackage (JSON) describing SQLite derived from XBRL

  • FERC-2 XBRL Taxonomy Metadata as JSON (2021-2024)

• FERC Form 6:

  • FERC-6 SQLite derived from DBF (2000-2020)

  • FERC-6 SQLite derived from XBRL (2021-2024)

  • FERC-6 Datapackage (JSON) describing SQLite derived from XBRL

  • FERC-6 XBRL Taxonomy Metadata as JSON (2021-2024)

• FERC Form 60:

  • FERC-60 SQLite derived from DBF (2006-2020)

  • FERC-60 SQLite derived from XBRL (2021-2024)

  • FERC-60 Datapackage (JSON) describing SQLite derived from XBRL

  • FERC-60 XBRL Taxonomy Metadata as JSON (2021-2024)

• FERC Form 714:

  • FERC-714 SQLite derived from XBRL (2021-2024)

  • FERC-714 Datapackage (JSON) describing SQLite derived from XBRL

  • FERC-714 XBRL Taxonomy Metadata as JSON (2021-2024)

Raw FERC XBRL data converted to DuckDB

To enable remote querying of the converted FERC databases, we have started using DuckDB as an output format. Currently it only includes the more recent XBRL data. Within DuckDB you can now do queries like this:

INSTALL httpfs;
LOAD httpfs;
-- Path-style S3 addressing is required because our bucket name contains dots.
-- See the DuckDB notes under Cloud Storage above.
CREATE SECRET (TYPE s3, PROVIDER config, REGION 'us-west-2', URL_STYLE 'path');
ATTACH 's3://pudl.catalyst.coop/nightly/ferc1_xbrl.duckdb' AS ferc1_xbrl (READ_ONLY);
SELECT * FROM ferc1_xbrl.transmission_lines_added_during_year_424_duration;

You can also use the DuckDB Python API to create a connection and execute the above statements, and then use that connection with pandas.read_sql() and other libraries that understand DBAPI connections.

• FERC Form 1: s3://pudl.catalyst.coop/nightly/ferc1_xbrl.duckdb (direct download)

• FERC Form 2: s3://pudl.catalyst.coop/nightly/ferc2_xbrl.duckdb (direct download)

• FERC Form 6: s3://pudl.catalyst.coop/nightly/ferc6_xbrl.duckdb (direct download)

• FERC Form 60: s3://pudl.catalyst.coop/nightly/ferc60_xbrl.duckdb (direct download)

• FERC Form 714: s3://pudl.catalyst.coop/nightly/ferc714_xbrl.duckdb (direct download)

FERC EQR (Form 920)

In early 2026 we started processing and distributing the FERC Electric Quarterly Reports (EQR) dataset as a partitioned Apache Parquet dataset. Even as Parquet the EQR is close to 100 GB, so unlike the other PUDL data products, we can’t archive it on Zenodo or provide access to multiple historical versions through S3. Instead, we update a single set of outputs in S3 on a quarterly basis. The current outputs can be found at:

aws s3 ls --no-sign-request s3://pudl.catalyst.coop/ferceqr/

Each of the subdirectories corresponds to a table. Like the main PUDL database, these tables are documented in the PUDL Data Dictionary:

• core_ferceqr__contracts

• core_ferceqr__quarterly_identity

• core_ferceqr__quarterly_index_pub

• core_ferceqr__transactions

The subdirectories contain a number of Parquet files (2025q3.parquet, 2025q2.parquet, etc.), each containing one quarter of data for that table. Most tools for reading Parquet files out of cloud storage are able to query and read from many files sharing the same schema at the same time, typically with file globbing wildcards. Note that the EQR tables (particularly core_ferceqr__transactions) can be much larger than memory, so you will need to select only a subset of the data and/or use tools that are designed to work with large data efficiently like DuckDB or Polars. Some brief examples:

SQL (DuckDB)

INSTALL httpfs;
LOAD httpfs;
-- Path-style S3 addressing is required because our bucket name contains dots.
CREATE SECRET (TYPE s3, PROVIDER config, REGION 'us-west-2', URL_STYLE 'path');
SELECT * FROM 's3://pudl.catalyst.coop/ferceqr/core_ferceqr__contracts/*.parquet'
WHERE seller_company_name LIKE '%Bonneville%'
LIMIT 10;

Python (DuckDB/Pandas)

import duckdb
# Path-style S3 addressing is required because our bucket name contains dots.
con = duckdb.connect()
con.execute("INSTALL httpfs; LOAD httpfs;")
con.execute("CREATE SECRET (TYPE s3, PROVIDER config, REGION 'us-west-2', URL_STYLE 'path')")
# Query S3 with DuckDB and convert the result to pandas
df = con.execute("""
   SELECT *
   FROM 's3://pudl.catalyst.coop/ferceqr/core_ferceqr__contracts/*.parquet'
   WHERE seller_company_name LIKE '%Bonneville%'
   LIMIT 10
""").to_df()

Python (Polars)

import polars as pl
# Use scan_parquet (lazy evaluation) and filter
df = (
   pl.scan_parquet(
         "s3://pudl.catalyst.coop/ferceqr/core_ferceqr__contracts/*.parquet",
         storage_options={"aws_region": "us-west-2", "aws_skip_signature": "True"},
   )
   .filter(pl.col("seller_company_name").str.contains("Bonneville"))
   .head(10)
   .collect()
)

Coding Agents (PUDL Agent Skill)

If you already work with an AI coding agent (Claude Code, OpenCode, Pi, and similar tools), you can install the PUDL agent skill from the catalyst-cooperative/agent-skills repository on GitHub. The skill teaches your agent how to:

• discover which PUDL tables exist and what each table and column means,

• surface data-quality caveats and usage warnings recorded in the table metadata, and

• load the Parquet outputs directly from S3 or a local directory into a notebook or script.

It reads the same published Parquet outputs and Frictionless metadata described elsewhere on this page, and does not require the pudl Python package to be installed. See the skill’s README for installation and usage instructions. It builds on a companion datapackage skill defined in the same repository which provides similar help for any dataset described by a datapackage.json descriptor.

Zenodo Archives

If you want a specific, immutable version of our data for any reason, you can find them all here on Zenodo. Zenodo assigns long-lived DOIs to each archive, suitable for citation in academic journals and other publications. The most recent versioned PUDL data release can always be found using this Concept DOI: https://doi.org/10.5281/zenodo.3653158

From Zenodo you can download the pudl.duckdb database, individual SQLite databases, a zipfile containing all the Parquet files bundled together (pudl_parquet.zip, which includes a datapackage.json descriptor), and the standalone pudl_parquet_datapackage.json descriptor for browsing the schema without downloading any data.

The documentation for the latest such stable build is here. You can access the documentation for a specific version by hovering over the version selector at the bottom left of the page.

Raw Data

Sometimes you want to see the raw data that is published by the government, but it’s hard to find or difficult to download, or you want to see what an older version of the published data looked like prior to being revised or deleted.

We use Zenodo to archive and version our raw data inputs. You can find all of our archives in the Catalyst Cooperative Community.

These have been minimally processed - in some cases, we’ve compressed them or grouped them into ZIP archives to fit the Zenodo repository requirements. In all cases we’ve added some metadata to help identify the resources you’re looking for. But, apart from that, these datasets are unmodified.

Development Environment

If you want to run the PUDL data processing pipeline yourself from scratch, run the software tests, or make changes to the source code, you’ll need to set up our development environment. This is a bit involved, so it has its own separate documentation.

Most users shouldn’t need to do this, and will probably find working with the pre-processed data via one of the other access modes easier. But if you want to contribute to the project, please give it a shot!