A Distributed DataFrame library for large scale complex data processing.

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for jaychia from gravatar.com jaychia Avatar for samster25 from gravatar.com samster25

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache-2.0)
  • Author: Eventual Inc
  • Maintainer: Sammy Sidhu
  • Requires: Python >=3.7.1, <4.0.0
Classifiers

Project description

daft

Welcome to Daft

Daft is a fast, ergonomic and scalable open-source dataframe library: built for Python and Complex Data/Machine Learning workloads.

Daft is currently in its Alpha release phase - please expect bugs and rapid improvements to the project. We welcome user feedback/feature requests in our Discussions forums.

Frame 113

Installation

Install Daft with pip install getdaft.

Documentation

Learn more about Daft in our documentation.

Community

For questions about Daft, please post in our community hosted on GitHub Discussions. We look forward to meeting you there!

Why Daft?

Processing Complex Data such as images/audio/pointclouds often requires accelerated compute for geometric or machine learning algorithms, much of which leverages existing tooling from the Python/C++ ecosystem. However, many workloads such as analytics, model training data curation and data processing often also require relational query operations for loading/filtering/joining/aggregations.

Daft marries the two worlds with a Dataframe API, enabling you to run both large analytical queries and powerful Complex Data algorithms from the same interface.

  1. Python-first: Python and Jupyter notebooks are first-class citizens. Daft handles any Python libraries and datastructures natively - use any Python library such as Numpy, OpenCV and PyTorch for Complex Data processing.

  2. Laptop to Cloud: Daft is built to run as easily on your laptop for interactive development and on your own Ray cluster or Eventual deployment for terabyte-scale production workloads.

  3. Open Data Formats: Daft loads from and writes to open data formats such as Apache Parquet and Apache Iceberg. It also supports all major cloud vendors' object storage options, allowing you to easily integrate with your existing storage solutions.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for jaychia from gravatar.com jaychia Avatar for samster25 from gravatar.com samster25

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache-2.0)
  • Author: Eventual Inc
  • Maintainer: Sammy Sidhu
  • Requires: Python >=3.7.1, <4.0.0
Classifiers

Release history Release notifications | RSS feed

0.3.14

0.3.13

0.3.12 yanked

Reason this release was yanked:

daft.context.set_runner_ray is broken, which leads to a bad experience with using Daft on Ray

0.3.11 yanked

Reason this release was yanked:

daft.context.set_runner_ray is broken, which leads to a bad experience with using Daft on Ray

0.3.10

0.3.9

0.3.8

0.3.7

0.3.6

0.3.5

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.33

0.2.32 yanked

Reason this release was yanked:

This release is broken for Linux ARM machines because of a linking issue in the SIMD crate

0.2.31

0.2.30

0.2.29

0.2.28

0.2.27

0.2.26

0.2.25

0.2.24

0.2.23

0.2.22

0.2.21

0.2.20

0.2.19

0.2.18

0.2.17

0.2.16

0.2.15

0.2.14

0.2.13

0.2.12

0.2.11

0.2.10

0.2.9

0.2.8

0.2.7

0.2.6

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

0.1.13

0.1.12

0.1.11

0.1.10

0.1.9

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.24

0.0.23

0.0.22

0.0.21

0.0.20

0.0.19

0.0.18

0.0.17

This version

0.0.16

0.0.15 yanked

Reason this release was yanked:

Broken build for python 3.8 and 3.9

0.0.14

0.0.13

0.0.12 yanked

Reason this release was yanked:

Binary Wheel is built with wrong version of pyarrow (9.0 vs 6.0)

0.0.11

0.0.10

0.0.9

0.0.8

0.0.7

0.0.6

0.0.5 yanked

Reason this release was yanked:

nonvalid release

0.0.4 yanked

Reason this release was yanked:

nonvalid release

0.0.3 yanked

Reason this release was yanked:

nonvalid release

0.0.2 yanked

Reason this release was yanked:

nonvalid release

0.0.1 yanked

Reason this release was yanked:

nonvalid release

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

getdaft-0.0.16.tar.gz (143.2 kB view details)

Uploaded Source

Built Distributions

getdaft-0.0.16-cp310-cp310-manylinux_2_17_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

getdaft-0.0.16-cp310-cp310-macosx_11_0_x86_64.whl (286.2 kB view details)

Uploaded CPython 3.10 macOS 11.0+ x86-64

getdaft-0.0.16-cp310-cp310-macosx_11_0_arm64.whl (272.9 kB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

getdaft-0.0.16-cp39-cp39-manylinux_2_17_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

getdaft-0.0.16-cp39-cp39-macosx_11_0_x86_64.whl (286.6 kB view details)

Uploaded CPython 3.9 macOS 11.0+ x86-64

getdaft-0.0.16-cp39-cp39-macosx_11_0_arm64.whl (273.2 kB view details)

Uploaded CPython 3.9 macOS 11.0+ ARM64

getdaft-0.0.16-cp38-cp38-manylinux_2_17_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

getdaft-0.0.16-cp38-cp38-macosx_11_0_arm64.whl (273.1 kB view details)

Uploaded CPython 3.8 macOS 11.0+ ARM64

getdaft-0.0.16-cp38-cp38-macosx_10_16_x86_64.whl (286.7 kB view details)

Uploaded CPython 3.8 macOS 10.16+ x86-64

getdaft-0.0.16-cp37-cp37m-manylinux_2_17_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

getdaft-0.0.16-cp37-cp37m-macosx_10_16_x86_64.whl (286.1 kB view details)

Uploaded CPython 3.7m macOS 10.16+ x86-64

File details

Details for the file getdaft-0.0.16.tar.gz.

File metadata

  • Download URL: getdaft-0.0.16.tar.gz
  • Upload date:
  • Size: 143.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.10.7

File hashes

Hashes for getdaft-0.0.16.tar.gz
Algorithm Hash digest
SHA256 079a215c68c29f8a8b35da00a24889cb1caa300b85bad35511d1c676f1420c45
MD5 d24f5922403ac3ae95c4c8d757215b1e
BLAKE2b-256 ade43c762f96cc4c14c105bab41479bc1c564875a678874c895294411169c4c1

See more details on using hashes here.

File details

Details for the file getdaft-0.0.16-cp310-cp310-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.16-cp310-cp310-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 bf8b0df3a9fec32f3a3fc88bc00fd4deaf7dd93ea4a80a9753251f767f59d79c
MD5 7228af31972a516b8778150eec2842b3
BLAKE2b-256 25085df6c2c43d853f278f4792060865930b73300428ad0c77f85a4395060e07

See more details on using hashes here.

File details

Details for the file getdaft-0.0.16-cp310-cp310-macosx_11_0_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.16-cp310-cp310-macosx_11_0_x86_64.whl
Algorithm Hash digest
SHA256 664a48c01301e7d88797dab56abf6ed11aecabf158ab8184b8e71bdbad64f9fe
MD5 14464d798a0fdf82ae69b4c13ed095ac
BLAKE2b-256 e6cadd70f05e609807f45a203640db02ab07ca6a45cf82f7bb6dfd7fae254dd7

See more details on using hashes here.

File details

Details for the file getdaft-0.0.16-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.16-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 391c85ad2be33dab37896fc2fa9ab4cde336eef111016722136a42fd2f1f6215
MD5 4b6817a40529eda883fa79c64d3e37c2
BLAKE2b-256 168ce27a7a98518f2c34fd4ecf603a45749292a53bfb1627c1a1b51ce30b4e9e

See more details on using hashes here.

File details

Details for the file getdaft-0.0.16-cp39-cp39-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.16-cp39-cp39-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 7d6441634b51187456587f8dc852663773304d4992c9fc3d728d825ff0416ba6
MD5 f318793a3ccb502ad8da2ff8946c7055
BLAKE2b-256 ab86bf16ec2a7154c3c22cb624145052ffef03a252a50c4bdfd93e19bc02258d

See more details on using hashes here.

File details

Details for the file getdaft-0.0.16-cp39-cp39-macosx_11_0_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.16-cp39-cp39-macosx_11_0_x86_64.whl
Algorithm Hash digest
SHA256 0dc9e6ab17496b18596578fd49b21e8eebe4cb219a7e64cf6c112ff9aef83f24
MD5 45c99edba86f5d630ffd7d160f775fd6
BLAKE2b-256 af94b701ad319c0ac95e5048a97384ecdfc37a483d5de14a79d819cca2833782

See more details on using hashes here.

File details

Details for the file getdaft-0.0.16-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.16-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 9c203208679a65d71688fb9defd910a6a8c8c3d9e7d73e0881e4b9aedca5b49c
MD5 86e18aad3540813233fa655360159f21
BLAKE2b-256 cf9e9db2cd33d79acf4cf10dd17a8149484e969f41407614bb4115a85ef2feaa

See more details on using hashes here.

File details

Details for the file getdaft-0.0.16-cp38-cp38-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.16-cp38-cp38-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 40949d5517d3e8efa843cfa9e12abd245a4e2edaef171c21b4e5d9219fea6f44
MD5 b121a6485213bda584f1e082d45973af
BLAKE2b-256 dd8dfbe34fbc4c2ea0327c00c1956525ff7687c9f874e00ee8c7431daf0e8060

See more details on using hashes here.

File details

Details for the file getdaft-0.0.16-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.16-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 6238d96e6199423b29261405dfccc0ccafd9bb1de868dec499ccb3af4ea89ac7
MD5 84399cb09789b56a4af9fbeb993ce530
BLAKE2b-256 619ec203c84167b96201d85b88bc4f4adc3f7c4e20cd4e1bbe8258212d6319ef

See more details on using hashes here.

File details

Details for the file getdaft-0.0.16-cp38-cp38-macosx_10_16_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.16-cp38-cp38-macosx_10_16_x86_64.whl
Algorithm Hash digest
SHA256 dcd1f1551d2384b28a35978e828e5f5b740d4fe25776947a74231445f100e4f7
MD5 c3fbbb0f4f937a94f570103f3ce2606b
BLAKE2b-256 877c0c020ffcee744ed9aede993c3505c468458cd852cd8773b122086e6929bc

See more details on using hashes here.

File details

Details for the file getdaft-0.0.16-cp37-cp37m-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.16-cp37-cp37m-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 0e9a87b87b77020e7253a9284cfe22e0ef467317729542b137770494d40e4655
MD5 f134596c5c3dec95d373f6863c1648e1
BLAKE2b-256 d60783df83c48edbe62b3cf2615af907ae7bdb79c4927ecd56a5d24a332d5662

See more details on using hashes here.

File details

Details for the file getdaft-0.0.16-cp37-cp37m-macosx_10_16_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.16-cp37-cp37m-macosx_10_16_x86_64.whl
Algorithm Hash digest
SHA256 c9534114ae36dc9d3578e5aac75dd978595da94ca4cf76cc99b598287680d7d3
MD5 96158940078130dd17104c5ae94a41a9
BLAKE2b-256 57e6dc06da5465ab42be3eaada2f8762a1b0e7bfa2b5984838c80c3ef3295606

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page