model_catalogs

Provides access through Intake catalogs to a set of ocean models, especially the NOAA OFS models.

Specific functionality includes:

• Sets up an Intake catalog for known models to provide direct access to model output.
• Provides access to model output as an xarray Dataset.
• Models are known by their catalog files; see set here. They include
  • NOAA OFS Models:
    • CBOFS
    • CIOFS
    • CREOFS
    • DBOFS
    • GOMOFS
    • LEOFS
    • LMHOFS
    • LOOFS
    • NGOFS2
    • NYOFS
    • SFBOFS
    • TBOFS
    • WCOFS
    • Full 3D fields, or regularly gridded or 2D versions when available
  • GFS models
  • Global GOFS HYCOM
• Multiple time ranges and sources of model output are provided when known. For example for the NOAA OFS models there are both forecast and historical sources for all models, and some have others as well.
• model_catalogs knows how to aggregate NOAA OFS model output between nowcast and forecast files.
• Known models have cleaned up and filled-in metadata so they are easy to work with in xarray and with cf-xarray.
  • cf-xarray will understand dimension and coordinate names, as well as a set of standard_names mapped to the variables.
• Metadata about models is included in the Intake catalogs, such as:
  • polygon boundary of numerical domain
  • grid parameters
  • arguments for optimal read-in with xarray
• Can request the availability of each model source.

Installation

PyPI

To install from PyPI:

pip install model_catalogs

Subsequently install a few packages by saving the pip-requirements.txt file locally and:

pip install -r pip-requirements.txt

These need to be installed separately because some of the packages on GitHub have updates that have not been included in any releases that are available otherwise.

Install Optional Dependencies

Install additional dependencies for full functionality and running the demonstration notebooks. Activate your Python environment, then:

$ mamba install -c conda-forge --file model_catalogs/conda-requirements-opt.txt

or use conda in place of mamba if you don't have mamba installed.

Develop Package

To use provided environment

Clone the repo:

$ git clone http://github.com/NOAA-ORR-ERD/model_catalogs.git

In the model_catalogs directory, install conda environment:

$ conda env create -f environment.yml

Alternatively, if you have an existing environment you want to add to:

$ conda install --file conda-requirements.txt
$ pip install -r pip-requirements.txt

Install model_catalogs into new environment (still in model_catalogs directory):

$ conda activate model_catalogs
$ pip install -e .

Install development packages

To develop the code, follow instructions above for "To use provided environment". Then you can install additional dependencies for development and testing with

$ conda install --file requirements-dev.txt

Run tests

Run tests that haven't been marked as "slow" with

$ pytest

Run all tests, including slow tests, with:

$ pytest --runslow

Check precommits locally before pushing

To then check code before committing and pushing it to github, locally run

$ pre-commit run --all-files

These checks can change your files so it is best to check the changes before pushing to github.