esmvalcore.io.local#

Find files on the local filesystem.

Example configuration to find CMIP6 data on a personal computer:

Listing 10 Contents of data-local.yml#

projects:
  CMIP6:
    data:
      local:
        type: "esmvalcore.io.local.LocalDataSource"
        rootpath: ~/climate_data
        dirname_template: "{project}/{activity}/{institute}/{dataset}/{exp}/{ensemble}/{mip}/{short_name}/{grid}/{version}"
        filename_template: "{short_name}_{mip}_{dataset}_{exp}_{ensemble}_{grid}*.nc"

The module will find files matching the glob.glob() pattern formed by rootpath/dirname_template/filename_template, where the facets defined inside the curly braces of the templates are replaced by their values from the Dataset or the recipe plus any facet-value pairs that can be automatically added using augment_facets(). Note that the name of the data source, local-data in the example above, must be unique within each project but can otherwise be chosen freely.

To start using this module on a personal computer, copy the example configuration file into your configuration directory by running the command:

esmvaltool config copy data-local.yml

and tailor it for your own system if needed.

Example configuration files for popular HPC systems and some supported climate models are also available. View the list of available files by running the command:

esmvaltool config list

Further information is available in Data sources.

Data:

GRIB_FORMATS

GRIB file extensions.

Classes:

`LocalDataSource`(name, project, priority, ...)	Data source for finding files on a local filesystem.
`LocalFile`(args, *kwargs)	File on the local filesystem.

esmvalcore.io.local.GRIB_FORMATS = ('.grib2', '.grib', '.grb2', '.grb', '.gb2', '.gb')#: GRIB file extensions.

class esmvalcore.io.local.LocalDataSource(name: str, project: str, priority: int, rootpath: ~pathlib.Path, dirname_template: str, filename_template: str, ignore_warnings: list[dict[str, ~typing.Any]] | None = <factory>)[source]#

Bases: DataSource

Data source for finding files on a local filesystem.

Attributes:

`debug_info`	A string containing debug information when no data is found.
`dirname_template`	The template for the directory names.
`filename_template`	The template for the file names.
`ignore_warnings`	Warnings to ignore when loading the data.
`name`	A name identifying the data source.
`priority`	The priority of the data source.
`project`	The project that the data source provides data for.
`rootpath`	The path where the directories are located.

Methods:

find_data(**facets)

Find data locally.

Parameters:

name (str)
project (str)
priority (int)
rootpath (Path)
dirname_template (str)
filename_template (str)
ignore_warnings (list[dict[str, Any]] | None)

debug_info: str = ''#: A string containing debug information when no data is found.

dirname_template: str#: The template for the directory names.

filename_template: str#: The template for the file names.

find_data(**facets: FacetValue) → list[LocalFile][source]#

Find data locally.

Parameters:: **facets (FacetValue) – Find data matching these facets.
Returns:: A list of files.
Return type:: list[LocalFile]

ignore_warnings: list[dict[str, Any]] | None#

Warnings to ignore when loading the data.

The list should contain dict`s with keyword arguments that will be passed to the :func:`warnings.filterwarnings function when calling LocalFile.to_iris().

name: str#: A name identifying the data source.

priority: int#: The priority of the data source. Lower values have priority.

project: str#: The project that the data source provides data for.

rootpath: Path#: The path where the directories are located.

class esmvalcore.io.local.LocalFile(*args, **kwargs)[source]#

Bases: PosixPath, DataElement

File on the local filesystem.

Attributes:

`attributes`	Attributes are key-value pairs describing the data.
`facets`	Facets are key-value pairs that were used to find this data.
`ignore_warnings`	Warnings to ignore when loading the data.

Methods:

`prepare`()	Prepare the data for access.
`to_iris`()	Load the data as Iris cubes.

property attributes: dict[str, Any]#: Attributes are key-value pairs describing the data.

property facets: Facets#: Facets are key-value pairs that were used to find this data.

property ignore_warnings: list[dict[str, Any]] | None#

Warnings to ignore when loading the data.

The list should contain dict`s with keyword arguments that will be passed to the :func:`warnings.filterwarnings function when calling the to_iris method.

prepare() → None[source]#

Prepare the data for access.

Return type:: None

to_iris() → CubeList[source]#

Load the data as Iris cubes.

Returns:: The loaded data.
Return type:: iris.cube.CubeList

esmvalcore.io.local#

This Page