dask-ms

xarray Dataset from CASA Tables

These details have not been verified by PyPI

Project description

https://img.shields.io/pypi/v/dask-ms.svg

https://github.com/ratt-ru/dask-ms/actions/workflows/ci.yml/badge.svg

Constructs xarray Datasets from CASA Tables via python-casacore. The Variables contained in the Dataset are dask arrays backed by deferred calls to pyrap.tables.table.getcol.

Supports writing Variables back to the respective column in the Table.

The intention behind this package is to support the Measurement Set as a data source and sink for the purposes of writing parallel, distributed Radio Astronomy algorithms.

Installation

To install with xarray support:

$ pip install dask-ms[xarray]

Without xarray similar, but reduced Dataset functionality is replicated in dask-ms itself. Expert users may wish to use this option to reduce python package dependencies.

$ pip install dask-ms

Documentation

https://dask-ms.readthedocs.io

Gitter Page

https://gitter.im/dask-ms/community

Example Usage

  import dask.array as da
  from daskms import xds_from_table, xds_to_table

  # Create xarray datasets from Measurement Set "WSRT.MS"
  ds = xds_from_table("WSRT.MS")
  # Set the flag Variable on first Dataset to it's inverse
  ds[0]['flag'] = (ds[0].flag.dims, da.logical_not(ds[0].flag))
  # Write the flag column back to the Measurement Set
  xds_to_table(ds, "WSRT.MS", "FLAG").compute()

  print ds

[<xarray.Dataset>
 Dimensions:         (chan: 64, corr: 4, row: 6552, uvw: 3)
 Coordinates:
     ROWID           (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
 Dimensions without coordinates: chan, corr, row, uvw
 Data variables:
     IMAGING_WEIGHT  (row, chan) float32 dask.array<shape=(6552, 64), chunksize=(6552, 64)>
     ANTENNA1        (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
     STATE_ID        (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
     EXPOSURE        (row) float64 dask.array<shape=(6552,), chunksize=(6552,)>
     MODEL_DATA      (row, chan, corr) complex64 dask.array<shape=(6552, 64, 4), chunksize=(6552, 64, 4)>
     FLAG_ROW        (row) bool dask.array<shape=(6552,), chunksize=(6552,)>
     CORRECTED_DATA  (row, chan, corr) complex64 dask.array<shape=(6552, 64, 4), chunksize=(6552, 64, 4)>
     PROCESSOR_ID    (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
     WEIGHT          (row, corr) float32 dask.array<shape=(6552, 4), chunksize=(6552, 4)>
     FLAG            (row, chan, corr) bool dask.array<shape=(6552, 64, 4), chunksize=(6552, 64, 4)>
     TIME            (row) float64 dask.array<shape=(6552,), chunksize=(6552,)>
     SIGMA           (row, corr) float32 dask.array<shape=(6552, 4), chunksize=(6552, 4)>
     SCAN_NUMBER     (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
     INTERVAL        (row) float64 dask.array<shape=(6552,), chunksize=(6552,)>
     OBSERVATION_ID  (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
     TIME_CENTROID   (row) float64 dask.array<shape=(6552,), chunksize=(6552,)>
     ARRAY_ID        (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
     ANTENNA2        (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
     DATA            (row, chan, corr) complex64 dask.array<shape=(6552, 64, 4), chunksize=(6552, 64, 4)>
     FEED1           (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
     FEED2           (row) int32 dask.array<shape=(6552,), chunksize=(6552,)>
     UVW             (row, uvw) float64 dask.array<shape=(6552, 3), chunksize=(6552, 3)>
 Attributes:
     FIELD_ID:      0
     DATA_DESC_ID:  0]

Limitations

Many Measurement Sets columns are defined as variably shaped, but the actual data is fixed. dask-ms will infer the shape of the data from the first row and must be consistent with that of other rows. For example, this may be issue where multiple Spectral Windows are present in the Measurement Set with differing channels per SPW.

dask-ms works around this by partitioning the Measurement Set into multiple datasets. The first row’s shape is used to infer the shape of the partition. Thus, in the case of multiple Spectral Window’s, we can partition the Measurement Set by DATA_DESC_ID to create a dataset for each Spectral Window.

Project details

These details have not been verified by PyPI

Release history Release notifications | RSS feed

0.2.23

Nov 11, 2024

0.2.22

Nov 7, 2024

0.2.21

Jun 18, 2024

0.2.20

Jan 30, 2024

0.2.19

Nov 13, 2023

0.2.18

Sep 20, 2023

0.2.17

Aug 2, 2023

This version

0.2.16

May 26, 2023

0.2.15

Oct 19, 2022

0.2.14

Oct 4, 2022

0.2.13 yanked

Oct 4, 2022

Reason this release was yanked:

version tag was incorrect on github (v0.2.13)

0.2.12

Sep 9, 2022

0.2.11

Jul 27, 2022

0.2.10

Jul 18, 2022

0.2.9

Jun 28, 2022

0.2.8

Apr 6, 2022

0.2.7

Jan 13, 2022

0.2.6

Oct 20, 2020

0.2.5

May 11, 2020

0.2.4

Apr 24, 2020

0.2.3

Dec 9, 2019

0.2.2

Oct 25, 2019

0.2.1

Oct 23, 2019

0.2.0

Sep 30, 2019

0.2.0rc3 pre-release

Sep 10, 2019

0.2.0rc2 pre-release

Aug 27, 2019

0.2.0rc1 pre-release

Aug 26, 2019

0.2.0a2 pre-release

Aug 21, 2019

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dask-ms-0.2.16.tar.gz (102.3 kB view details)

Uploaded May 26, 2023 Source

Built Distribution

dask_ms-0.2.16-py3-none-any.whl (130.1 kB view details)

Uploaded May 26, 2023 Python 3

File details

Details for the file dask-ms-0.2.16.tar.gz.

File metadata

Download URL: dask-ms-0.2.16.tar.gz
Upload date: May 26, 2023
Size: 102.3 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/4.0.2 CPython/3.9.16

File hashes

Hashes for dask-ms-0.2.16.tar.gz
Algorithm	Hash digest
SHA256	`474f5fbe370e6422938a7ef8770215ef7dfa266a106da5b985bdc3e6f6323525`
MD5	`bec3e1c0b44135a254aa864986b9720d`
BLAKE2b-256	`794df444b36e4ce677ecf0638b1344364185d9617a57859c971d4d867fa4a73f`

See more details on using hashes here.

File details

Details for the file dask_ms-0.2.16-py3-none-any.whl.

File metadata

Download URL: dask_ms-0.2.16-py3-none-any.whl
Upload date: May 26, 2023
Size: 130.1 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: twine/4.0.2 CPython/3.9.16

File hashes

Hashes for dask_ms-0.2.16-py3-none-any.whl
Algorithm	Hash digest
SHA256	`4f0867e00ee984705bd13efe50132692c652747b030cd2908251344a10c1d854`
MD5	`f979765aeee5c77989cfbf8e1b57515d`
BLAKE2b-256	`f4128e73536feb3dccad11d5afaa927cc1555c55f87357629f359368a1cddf51`