High Level Expressions for Dask

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for fjetter from gravatar.com fjetter Avatar for jrbourbeau from gravatar.com jrbourbeau Avatar for phofl from gravatar.com phofl Avatar for rjzamora from gravatar.com rjzamora

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: BSD License (BSD)
  • Maintainer: Matthew Rocklin
  • Tags dask, pandas
  • Requires: Python >=3.9
Classifiers

Project description

Dask Expressions

Dask DataFrames with query optimization.

This is a proof-of-concept rewrite of Dask DataFrame that includes query optimization and generally improved organization.

More in our blog posts:

Example

import dask_expr as dx

df = dx.datasets.timeseries()
df.head()

df.groupby("name").x.mean().compute()

Query Representation

Dask-expr encodes user code in an expression tree:

>>> df.x.mean().pprint()

Mean:
  Projection: columns='x'
    Timeseries: seed=1896674884

This expression tree will be optimized and modified before execution:

>>> df.x.mean().optimize().pprint()

Div:
  Sum:
    Fused(375f9):
    | Projection: columns='x'
    |   Timeseries: dtypes={'x': <class 'float'>} seed=1896674884
  Count:
    Fused(375f9):
    | Projection: columns='x'
    |   Timeseries: dtypes={'x': <class 'float'>} seed=1896674884

Stability

This project is a work in progress and will be changed without notice or deprecation warning. Please provide feedback, but it's best to avoid use in production settings.

API Coverage

Dask-Expr covers almost everything of the Dask DataFrame API. The only missing features are:

  • melt
  • named GroupBy Aggregations

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for fjetter from gravatar.com fjetter Avatar for jrbourbeau from gravatar.com jrbourbeau Avatar for phofl from gravatar.com phofl Avatar for rjzamora from gravatar.com rjzamora

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: BSD License (BSD)
  • Maintainer: Matthew Rocklin
  • Tags dask, pandas
  • Requires: Python >=3.9
Classifiers

Release history Release notifications | RSS feed

1.1.18

1.1.17 yanked

Reason this release was yanked:

Critical performance regression

1.1.16

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.8

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.14

1.0.13

1.0.12

1.0.11

1.0.10

1.0.9

1.0.7

1.0.6

1.0.5

1.0.4

1.0.3

1.0.2

1.0.1

1.0

0.5.3

0.5.2

0.5.1

0.5.0 yanked

Reason this release was yanked:

Wrong Dask Version Pin

0.4.2

This version

0.4.1

0.4.0

0.3.5

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.9

0.2.8

0.2.7

0.2.6

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.12

0.1.11

0.1.10

0.1.9

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dask-expr-0.4.1.tar.gz (150.7 kB view hashes)

Uploaded Source

Built Distribution

dask_expr-0.4.1-py3-none-any.whl (164.9 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page