pudl.workspace.resource_cache
=============================

.. py:module:: pudl.workspace.resource_cache

.. autoapi-nested-parse::

   Implementations of datastore resource caches.



Attributes
----------

.. autoapisummary::

   pudl.workspace.resource_cache.logger


Classes
-------

.. autoapisummary::

   pudl.workspace.resource_cache.PudlResourceKey
   pudl.workspace.resource_cache.AbstractCache
   pudl.workspace.resource_cache.UPathCache
   pudl.workspace.resource_cache.LayeredCache


Module Contents
---------------

.. py:data:: logger

.. py:class:: PudlResourceKey

   Bases: :py:obj:`NamedTuple`


   Uniquely identifies a specific resource.


   .. py:attribute:: dataset
      :type:  str


   .. py:attribute:: doi
      :type:  str


   .. py:attribute:: name
      :type:  str


   .. py:method:: __repr__() -> str

      Returns string representation of PudlResourceKey.



   .. py:method:: get_local_path() -> pathlib.Path

      Returns (relative) path that should be used when caching this resource.



.. py:class:: AbstractCache(read_only: bool = False)

   Bases: :py:obj:`abc.ABC`


   Defines interaface for the generic resource caching layer.


   .. py:attribute:: _read_only
      :value: False



   .. py:method:: is_read_only() -> bool

      Returns true if the cache is read-only and should not be modified.



   .. py:method:: get(resource: PudlResourceKey) -> bytes
      :abstractmethod:


      Retrieves content of given resource or throws KeyError.



   .. py:method:: add(resource: PudlResourceKey, content: bytes) -> None
      :abstractmethod:


      Adds resource to the cache and sets the content.



   .. py:method:: delete(resource: PudlResourceKey) -> None
      :abstractmethod:


      Removes the resource from cache.



   .. py:method:: contains(resource: PudlResourceKey) -> bool
      :abstractmethod:


      Returns True if the resource is present in the cache.



.. py:class:: UPathCache(storage_upath: upath.UPath, **kwargs: Any)

   Bases: :py:obj:`AbstractCache`


   Implements file cache using UPath for unified access to multiple storage backends.

   This cache uses universal_pathlib's UPath to provide a unified interface
   for accessing data stored in S3, GCS, or local filesystems. It handles backend-specific
   authentication and credential management internally.

   Requires UPath objects with explicit protocols:
       - s3://bucket-name/path/prefix
       - gs://bucket-name/path/prefix
       - file:///local/path


   .. py:attribute:: supported_protocols
      :type:  set[str]


   .. py:attribute:: _protocol


   .. py:attribute:: _storage_options


   .. py:attribute:: _base_path


   .. py:method:: __repr__() -> str

      Returns string representation of UPathCache.



   .. py:method:: _setup_credentials() -> dict[str, Any]

      Set up backend-specific credentials and storage options.

      This should be the only place where backend-specific logic is required.

      :returns: Dictionary of storage options to pass to UPath



   .. py:method:: _resource_path(resource: PudlResourceKey) -> upath.UPath

      Get the UPath for a given resource.

      :param resource: The resource to get the path for

      :returns: UPath object pointing to the resource location



   .. py:method:: is_anonymous() -> bool

      Returns True if the cache is using anonymous access (no credentials).



   .. py:method:: get(resource: PudlResourceKey) -> bytes

      Retrieves value associated with given resource.

      :param resource: The resource to retrieve

      :returns: The content of the resource as bytes

      :raises KeyError: if the resource doesn't exist
      :raises Exception: for other storage backend errors



   .. py:method:: add(resource: PudlResourceKey, content: bytes)

      Adds (or updates) resource to the cache with given content.

      :param resource: The resource to add
      :param content: The content to store

      :raises RuntimeError: if cache is read-only or credentials are insufficient



   .. py:method:: delete(resource: PudlResourceKey)

      Deletes resource from the cache.

      :param resource: The resource to delete

      :raises RuntimeError: if cache is read-only or credentials are insufficient



   .. py:method:: contains(resource: PudlResourceKey) -> bool

      Returns True if resource is present in the cache.

      :param resource: The resource to check

      :returns: True if the resource exists, False otherwise



.. py:class:: LayeredCache(*caches: AbstractCache, **kwargs: Any)

   Bases: :py:obj:`AbstractCache`


   Implements multi-layered system of caches.

   This allows building multi-layered system of caches. The idea is that you can have
   faster local caches with fall-back to the more remote or expensive caches that can
   be accessed in case of missing content.

   Only the closest layer is being written to (set, delete), while all remaining layers
   are read-only (get).


   .. py:attribute:: _caches
      :type:  list[AbstractCache]
      :value: []



   .. py:method:: add_cache_layer(cache: AbstractCache)

      Adds caching layer.

      The priority is below all other.



   .. py:method:: num_layers()

      Returns number of caching layers that are in this LayeredCache.



   .. py:method:: get(resource: PudlResourceKey) -> bytes

      Returns content of a given resource.

      When a resource is found in a distant cache layer, it is automatically populated
      into all closer (higher-priority) cache layers that are writable.  This ensures
      optimal cache performance for subsequent accesses.



   .. py:method:: add(resource: PudlResourceKey, content)

      Adds (or replaces) resource into the cache with given content.



   .. py:method:: delete(resource: PudlResourceKey)

      Removes resource from the cache if the cache is not in the read_only mode.



   .. py:method:: contains(resource: PudlResourceKey) -> bool

      Returns True if resource is present in the cache.



   .. py:method:: is_optimally_cached(resource: PudlResourceKey) -> bool

      Return True if resource is contained in the closest write-enabled layer.