xscen 0.6.0

xscen is now offered as a conda package available through Anaconda.org. Refer to the installation documentation for more information. (GH/149, PR/171).

Deprecation: Release 0.6.0 of xscen will be the last version to support xscen.extract.clisops_subset. Use xscen.spatial.subset instead. (PR/182, PR/184).

Deprecation: The argument region, used in multiple functions, has been slightly reformatted. Release 0.6.0 of xscen will be the last version to support the old format. (GH/99, GH/101, PR/184).

New ‘cos-lat’ averaging in spatial_mean. (GH/94, PR/125).

Support for computing anomalies in compute_deltas. (PR/165).

Add function diagnostics.measures_improvement_2d. (PR/167).

Add function regrid.create_bounds_rotated_pole and automatic use in regrid_dataset and spatial_mean. This is temporary, while we wait for a functionning method in cf_xarray. (PR/174, GH/96).

Add spatial submodule with functions creep_weights and creep_fill for filling NaNs using neighbours. (PR/174).

Allow passing GeoDataFrame instances in spatial_mean’s region argument, not only geospatial file paths. (PR/174).

Allow searching for periods in catalog.search. (GH/123, PR/170).

Allow searching and extracting multiple frequencies for a given variable. (GH/168, PR/170).

New masking feature in extract_dataset. (GH/180, PR/182).

New function xs.spatial.subset to replace xs.extract.clisops_subset and add method “sel”. (GH/180, PR/182).

Add long_name attribute to diagnostics. ( PR/189).

Added a new YAML-centric notebook (GH/8, PR/191).

New utils.standardize_periods to standardize that argument across multiple functions. (GH/87, PR/192).

New coverage_kwargs argument added to search_data_catalogs to allow modifying the default values of subset_file_coverage. (GH/87, PR/192).

‘mean’ averaging has been deprecated in spatial_mean. (PR/125).

‘interp_coord’ has been renamed to ‘interp_centroid’ in spatial_mean. (PR/125).

The ‘datasets’ dimension of the output of diagnostics.measures_heatmap is renamed ‘realization’. (PR/167).

_subset_file_coverage was renamed subset_file_coverage and moved to catalog.py to prevent circular imports. (PR/170).

extract_dataset doesn’t fail when a variable is in the dataset, but not variables_and_freqs. (PR/185).

The argument period, used in multiple function, is now always a single list, while periods is more flexible. (GH/87, PR/192).

The parameters reference_period and simulation_period of xscen.train and xscen.adjust were renamed period/periods to respect the point above. (GH/87, PR/192).

Forbid pandas v1.5.3 in the environment files, as the linux conda build breaks the data catalog parser. (GH/161, PR/162).

Only return requested variables when using DataCatalog.to_dataset. (PR/163).

compute_indicators no longer crashes if less than 3 timesteps are produced. (PR/125).

xarray is temporarily pinned below v2023.3.0 due to an API-breaking change. (GH/175, PR/173).

xscen.utils.unstack_fill_nan` can now handle datasets that have non dimension coordinates. (GH/156, PR/175).

extract_dataset now skips a simulation way earlier if the frequency doesn’t match. (PR/170).

extract_dataset now correctly tries to extract in reverse timedelta order. (PR/170).

compute_deltas no longer creates all NaN values if the input dataset is in a non-standard calendar. (PR/188).

xscen now manages packaging for PyPi and TestPyPI via GitHub workflows. (PR/159).

Pre-load coordinates in extract.clisops_subset (PR/163).

Minimal documentation for templates. (PR/163).

xscen is now indexed in Zenodo, under the ouranos community of projects. (PR/164).

Added a few relevant Shields to the README.rst. (PR/164).

Better warning messages in _subset_file_coverage when coverage is insufficient. (PR/125).

The top-level Makefile now includes a linkcheck recipe, and the ReadTheDocs configuration no longer reinstalls the llvmlite compiler library. (PR/173).

The checkups on coverage and duplicates can now be skipped in subset_file_coverage. (PR/170).

Changed the ProjectCatalog docstrings to make it more obvious that it needs to be created empty. (GH/99, PR/184).

Added parse_config to creep_fill, creep_weights, and reduce_ensemble (PR/191).

Possibility of excluding variables read from file from the catalog produced by parse_directory. (PR/107).

New functions extract.subset_warming_level and aggregate.produce_horizon. (PR/93).

add round_var to xs.clean_up. (PR/93).

New “timeout_cleanup” option for save_to_zarr, which removes variables that were in the process of being written when receiving a TimeoutException. (PR/106).

New scripting.skippable context, allowing the use of CTRL-C to skip code sections. (PR/106).

Possibility of fields with underscores in the patterns of parse_directory. (PR/111).

New utils.show_versions function for printing or writing to file the dependency versions of xscen. (GH/109, PR/112).

Added previously private notebooks to the documentation. (PR/108).

Notebooks are now tested using pytest with nbval. (PR/108).

New restrict_warming_level argument for extract.search_data_catalogs to filter dataset that are not in the warming level csv. (GH/105, PR/138).

Set configuration value programmatically through CONFIG.set. (PR/144).

New to_dataset method on DataCatalog. The same as to_dask, but exposing more aggregation options. (PR/147).

New templates folder with one general template. (GH/151, PR/158).

Functions that are called internally can no longer parse the configuration. (PR/133).

clean_up now converts the calendar of variables that use “interpolate” in “missing_by_var” at the same time.

• Hence, when it is a conversion from a 360_day calendar, the random dates are the same for all of the these variables. (GH/102, PR/104).

properties_and_measures no longer casts month coordinates to string. (PR/106).

search_data_catalogs no longer crashes if it finds nothing. (GH/42, PR/92).

Prevented fixed fields from being duplicated during _dispatch_historical_to_future (GH/81, PR/92).

Added missing parse_config to functions in reduce.py (PR/92).

Added deepcopy before skipna is popped in spatial_mean (PR/92).

subset_warming_level now validates that the data exists in the dataset provided (GH/117, PR/119).

Adapt stack_drop_nan for the newest version of xarray (2022.12.0). (GH/122, PR/126).

Fix stack_drop_nan not working if intermediate directories don’t exist (GH/128).

Fixed a crash when compute_indicators produced fixed fields (PR/139).

compute_deltas skips the unstacking step if there is no time dimension and cast object dimensions to string. (PR/9)

Added the “2sem” frequency to the translations CVs. (PR/111).

Skip files we can’t read in parse_directory. (PR/111).

Fixed non-numpy-standard Docstrings. (PR/108).

Added more metadata to package description on PyPI. (PR/108).

Faster search_data_catalogs and extract_dataset through a faster DataCatalog.unique, date parsing and a rewrite of the ensure_correct_time logic. (PR/127).

The search_data_catalogs function now accepts str or pathlib.Path variables (in addition to lists of either data type) for performing catalog lookups. (PR/121).

produce_horizons now supports fixed fields (PR/139).

Rewrite of unstack_dates for better performance with dask arrays. (PR/144).

New functions diagnostics.properties_and_measures, diagnostics.measures_heatmap and diagnostics.measures_improvement. (GH/5, PR/54).

Add argument resample_methods to xs.extract.resample. (GH/57, PR/57)

Added a ReadTheDocs configuration to expose public documentation. (GH/65, PR/66).

xs.utils.stack_drop_nans/ xs.utils.unstack_fill_nan will now format the to_file/coords string to add the domain and the shape. (GH/59, PR/67).

New unstack_dates function to “extract” seasons or months from a timeseries. (PR/68).

Better spatial_mean for cases using xESMF and a shapefile with multiple polygons. (PR/68).

Yet more changes to parse_directory: (PR/68).

• Better parallelization by merging the finding and name-parsing step in the same dask tree.

• Allow cvs for the variable columns.

• Fix parsing the variable names from datasets.

• Sort the variables in the tuples (for a more consistent output)

In extract_dataset, add option ensure_correct_time to ensure the time coordinate matches the expected freq. Ex: monthly values given on the 15th day are moved to the 1st, as expected when asking for “MS”. (:issue: 53).

In regrid_dataset: (PR/68).

• Allow passing skipna to the regridder kwargs.

• Do not fail for any grid mapping problem, includin if a grid_mapping attribute mentions a variable that doesn’t exist.

Default email sent to the local user. (PR/68).

Special accelerated pathway for parsing catalogs with all dates within the datetime64[ns] range. (PR/75).

New functions reduce_ensemble and build_reduction_data to support kkz and kmeans clustering. (GH/4, PR/63).

ensemble_stats can now loop through multiple statistics, support functions located in xclim.ensembles._robustness, and supports weighted realizations. (PR/63).

New function ensemble_stats.generate_weights that estimates weights based on simulation metadata. (PR/63).

New function catalog.unstack_id to reverse-engineer IDs. (PR/63).

generate_id now accepts Datasets. (PR/63).

Add rechunk option to properties_and_measures (PR/76).

Add create argument to ProjectCatalog (GH/11, PR/77).

Add percentage deltas to compute_deltas (GH/82, PR/90).

statistics / stats_kwargs have been changed/eliminated in ensemble_stats, respectively. (PR/63).

Add a missing dependencies to the env (pyarrow, for faster string handling in catalogs). (PR/68).

Allow passing compute=False to save_to_zarr. (PR/68).

Small bugfixes in aggregate.py. (PR/55, PR/56).

Default method of xs.extract.resample now depends on frequency. (GH/57, PR/58).

Bugfix for _restrict_by_resolution with CMIP6 datasets (PR/71).

More complete check of coverage in _subset_file_coverage. (GH/70, PR/72)

The code that performs common_attrs_only in ensemble_stats has been moved to clean_up. (PR/63).

Removed the default to_level in clean_up. (PR/63).

xscen now has an official logo. (PR/69).

Use numpy max and min in properties_and_measures (PR/76).

Cast catalog date_start and date_end to “%4Y-%m-%d %H:00” when writing to disk. (GH/83, PR/79)

Skip test of coverage on the sum if the list of select files is empty. (PR/79)

Added missing CMIP variable names in conversions.yml and added the ability to provide a custom file instead (GH/86, PR/88)

Changed ‘allow_conversion’ and ‘allow_resample’ default to False in search_data_catalogs (GH/86, PR/88)

New function clean_up added. (GH/22, PR/25).

parse_directory: Fixes to xr_open_kwargs and support for wildcards (*) in the directories. (PR/19).

New function xscen.ensemble.ensemble_stats added. (GH/3, PR/28).

New functions spatial_mean, climatological_mean and deltas added. (GH/4, PR/35).

Add argument intermediate_reg_grids to xscen.regridding.regrid. (GH/34, PR/39).

Add argument moving_yearly_window to xscen.biasadjust.adjust. (PR/39).

Many adjustments to parse_directory: better wildcards (GH/24), allow custom columns, fastpaths for parse_from_ds, and more (PR/30).

Documentation now makes better use of autodoc to generate package index. (PR/41).

periods argument added to compute_indicators to support datasets with jumps in time (PR/35).

Patterns in parse_directory start at the end of the paths in directories. (PR/30).

Argument extension of parse_directory has been renamed globpattern. (PR/30).

The xscen API and filestructure have been significantly refactored. (GH/40, PR/41). The following functions are available from the top-level:

• adjust, train, ensemble_stats, clisops_subset, dispatch_historical_to_future, extract_dataset, resample, restrict_by_resolution, restrict_multimembers, search_data_catalogs, save_to_netcdf, save_to_zarr, rechunk, compute_indicators, regrid_dataset, and create_mask.

xscen now requires geopandas and shapely (PR/35).

Following a change in intake-esm xscen now uses “cat:” to prefix the dataset attributes extracted from the catalog. All catalog-generated attributes should now be valid when saving to netCDF. (GH/13, PR/51).

parse_directory: Fixes to xr_open_kwargs. (PR/19).

Fix for indicators removing the ‘time’ dimension. (PR/23).

Security scanning using CodeQL and GitHub Actions is now configured for the repository. (PR/21).

Bumpversion action now configured to automatically augment the version number on each merged pull request. (PR/21).

Add align_on = 'year' argument in bias adjustment converting of calendars. (PR/39).

GitHub Actions using Ubuntu-22.04 images are now configured for running testing ensemble using tox-conda. (PR/44).

import xscen smoke test is now run on all pull requests. (PR/44).

Fix for create_mask removing attributes (PR/35).

This is the first official release for xscen!

Supports workflows with YAML configuration files for better transparency, reproducibility, and long-term backups.

Intake_esm-based catalog to find and manage climate data.

Climate dataset extraction, subsetting, and temporal aggregation.

Calculate missing variables through Intake-esm’s DerivedVariableRegistry.

Regridding with xESMF.

Bias adjustment with xclim.

N/A

N/A