A Distributed DataFrame library for large scale complex data processing.

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for jaychia from gravatar.com jaychia Avatar for samster25 from gravatar.com samster25

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache-2.0)
  • Author: Eventual Inc
  • Maintainer: Sammy Sidhu
  • Requires: Python >=3.7.1, <4.0.0
Classifiers

Project description

daft

Welcome to Daft

Daft is a fast, ergonomic and scalable open-source dataframe library: built for Python and Complex Data/Machine Learning workloads.

Frame 113

Installation

Install Daft with pip install getdaft.

Documentation

Learn more about Daft in our documentation.

Community

For questions about Daft, please post in our community hosted on GitHub Discussions. We look forward to meeting you there!

Why Daft?

Processing Complex Data such as images/audio/pointclouds often requires accelerated compute for geometric or machine learning algorithms, much of which leverages existing tooling from the Python/C++ ecosystem. However, many workloads such as analytics, model training data curation and data processing often also require relational query operations for loading/filtering/joining/aggregations.

Daft marries the two worlds with a Dataframe API, enabling you to run both large analytical queries and powerful Complex Data algorithms from the same interface.

  1. Python-first: Python and Jupyter notebooks are first-class citizens. Daft handles any Python libraries and datastructures natively - use any Python library such as Numpy, OpenCV and PyTorch for Complex Data processing.

  2. Laptop to Cloud: Daft is built to run as easily on your laptop for interactive development and on your own Ray cluster or Eventual deployment for terabyte-scale production workloads.

  3. Open Data Formats: Daft loads from and writes to open data formats such as Apache Parquet and Apache Iceberg. It also supports all major cloud vendors' object storage options, allowing you to easily integrate with your existing storage solutions.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for jaychia from gravatar.com jaychia Avatar for samster25 from gravatar.com samster25

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache-2.0)
  • Author: Eventual Inc
  • Maintainer: Sammy Sidhu
  • Requires: Python >=3.7.1, <4.0.0
Classifiers

Release history Release notifications | RSS feed

0.3.14

0.3.13

0.3.12 yanked

Reason this release was yanked:

daft.context.set_runner_ray is broken, which leads to a bad experience with using Daft on Ray

0.3.11 yanked

Reason this release was yanked:

daft.context.set_runner_ray is broken, which leads to a bad experience with using Daft on Ray

0.3.10

0.3.9

0.3.8

0.3.7

0.3.6

0.3.5

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.33

0.2.32 yanked

Reason this release was yanked:

This release is broken for Linux ARM machines because of a linking issue in the SIMD crate

0.2.31

0.2.30

0.2.29

0.2.28

0.2.27

0.2.26

0.2.25

0.2.24

0.2.23

0.2.22

0.2.21

0.2.20

0.2.19

0.2.18

0.2.17

0.2.16

0.2.15

0.2.14

0.2.13

0.2.12

0.2.11

0.2.10

0.2.9

0.2.8

0.2.7

0.2.6

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

0.1.13

0.1.12

0.1.11

0.1.10

0.1.9

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.24

0.0.23

0.0.22

0.0.21

0.0.20

0.0.19

0.0.18

0.0.17

0.0.16

0.0.15 yanked

Reason this release was yanked:

Broken build for python 3.8 and 3.9

0.0.14

This version

0.0.13

0.0.12 yanked

Reason this release was yanked:

Binary Wheel is built with wrong version of pyarrow (9.0 vs 6.0)

0.0.11

0.0.10

0.0.9

0.0.8

0.0.7

0.0.6

0.0.5 yanked

Reason this release was yanked:

nonvalid release

0.0.4 yanked

Reason this release was yanked:

nonvalid release

0.0.3 yanked

Reason this release was yanked:

nonvalid release

0.0.2 yanked

Reason this release was yanked:

nonvalid release

0.0.1 yanked

Reason this release was yanked:

nonvalid release

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

getdaft-0.0.13.tar.gz (141.4 kB view details)

Uploaded Source

Built Distributions

getdaft-0.0.13-cp310-cp310-manylinux_2_17_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

getdaft-0.0.13-cp310-cp310-macosx_11_0_x86_64.whl (285.2 kB view details)

Uploaded CPython 3.10 macOS 11.0+ x86-64

getdaft-0.0.13-cp310-cp310-macosx_11_0_arm64.whl (272.0 kB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

getdaft-0.0.13-cp39-cp39-manylinux_2_17_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

getdaft-0.0.13-cp39-cp39-macosx_11_0_x86_64.whl (285.7 kB view details)

Uploaded CPython 3.9 macOS 11.0+ x86-64

getdaft-0.0.13-cp39-cp39-macosx_11_0_arm64.whl (272.4 kB view details)

Uploaded CPython 3.9 macOS 11.0+ ARM64

getdaft-0.0.13-cp38-cp38-manylinux_2_17_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

getdaft-0.0.13-cp38-cp38-macosx_11_0_arm64.whl (272.2 kB view details)

Uploaded CPython 3.8 macOS 11.0+ ARM64

getdaft-0.0.13-cp38-cp38-macosx_10_16_x86_64.whl (285.7 kB view details)

Uploaded CPython 3.8 macOS 10.16+ x86-64

getdaft-0.0.13-cp37-cp37m-manylinux_2_17_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

getdaft-0.0.13-cp37-cp37m-macosx_10_16_x86_64.whl (285.1 kB view details)

Uploaded CPython 3.7m macOS 10.16+ x86-64

File details

Details for the file getdaft-0.0.13.tar.gz.

File metadata

  • Download URL: getdaft-0.0.13.tar.gz
  • Upload date:
  • Size: 141.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.10.7

File hashes

Hashes for getdaft-0.0.13.tar.gz
Algorithm Hash digest
SHA256 91cfbed9365835c24ee8ae107e9ae7f69d07bfdff51435128fed09384cccb25d
MD5 a288cba27655ba8129366e4b4e981fbd
BLAKE2b-256 caeb61835ba9110541c28b20ad9fda1dfeb3ceb91c18f65b40c2f5625cda79ed

See more details on using hashes here.

File details

Details for the file getdaft-0.0.13-cp310-cp310-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.13-cp310-cp310-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 71f52b0131404c8119711ef7b10af4651649f0c14844514b53754f15c163cbe9
MD5 dba2c97e0ac24069c7f902163338c6a0
BLAKE2b-256 cd65e93262cf3099ce010961f8da6e4cef7d5ebbadd64b20d3a3952d02c2e594

See more details on using hashes here.

File details

Details for the file getdaft-0.0.13-cp310-cp310-macosx_11_0_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.13-cp310-cp310-macosx_11_0_x86_64.whl
Algorithm Hash digest
SHA256 9c72b6eb7b8f969ad45d5895c29550c2603bd2166dd4a5f9b416ddb61beb85bf
MD5 3e3e4e2df99b180d2aeea703408abdd8
BLAKE2b-256 5744f4218b13cd1634f44f6456ce0a859222ed37b851edad76fa17886c48524e

See more details on using hashes here.

File details

Details for the file getdaft-0.0.13-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.13-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 311b225bc4a05add1ac209642cc6ac82b579eba171a454af70e07f4e263dab21
MD5 286aa45bd2fda0e5ebfd5c7593ebf3ec
BLAKE2b-256 35604a08e7a4b03105f19305a053b3adbb9cff5d5e3e0f8b29cfb6e42ca8fb5e

See more details on using hashes here.

File details

Details for the file getdaft-0.0.13-cp39-cp39-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.13-cp39-cp39-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 43e6574d49902f63df58ed70519b38860048a4bc51c37c6441a60e1421f17951
MD5 3247d3181f85c7f6e6b5edb4bb67ef99
BLAKE2b-256 cd03e80bf36199bb6fcf758d2dcf96a5e2e8662ab38bd5f0842d25b4392d08e9

See more details on using hashes here.

File details

Details for the file getdaft-0.0.13-cp39-cp39-macosx_11_0_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.13-cp39-cp39-macosx_11_0_x86_64.whl
Algorithm Hash digest
SHA256 c0d29300474262e75c37262145885af6eb4ff5a80e670f0dfc37ec025a489edb
MD5 fc9872ec0b0f53a806bb88b4716aca9d
BLAKE2b-256 10649ece9302cab24e1b3618ffb4ec7e763b6bb92a96bd7234ba2a3b205c5884

See more details on using hashes here.

File details

Details for the file getdaft-0.0.13-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.13-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 6710343aae9a41e08b797a0da3ee9a10b3eb15a5d4b6dcf834014a57be59c93d
MD5 d4fd18a7d8cd3f05cac9cae1497c8a77
BLAKE2b-256 6247d5baedc3ce5fba0b27c020b7eaee4411bd8c34a5b70a45ec95dee4629465

See more details on using hashes here.

File details

Details for the file getdaft-0.0.13-cp38-cp38-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.13-cp38-cp38-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 b8970e461b51ab8a3c4e8d8cee89c945caf006d0960d9a64b75c6a393b3047bb
MD5 de734b01b56edea382209f353835523b
BLAKE2b-256 17b75da05b33c63917cbde2a5136c60fc8731b9805abd36af480b55ae1fd2341

See more details on using hashes here.

File details

Details for the file getdaft-0.0.13-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.13-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 52260d90c9dfe635246592655f445ab65f08664ec26c44e266722477d70d3c44
MD5 6c1a419633b5aaaf7971410cae7cffe2
BLAKE2b-256 abb75bda23951b279729185eeaf5c8b70a5add2ff9c262490fe9e735f78540e1

See more details on using hashes here.

File details

Details for the file getdaft-0.0.13-cp38-cp38-macosx_10_16_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.13-cp38-cp38-macosx_10_16_x86_64.whl
Algorithm Hash digest
SHA256 9ffbc264ba0abed426f76878157cb8f7f33fccfbdf0ce169c2b6bdf4d4356b00
MD5 ae474e4a616069f73b3bebaf863b5b51
BLAKE2b-256 2aef800a5bc6e11d1acaee5bb9b73b1c902888dfaa73262d69305f0fd976b73d

See more details on using hashes here.

File details

Details for the file getdaft-0.0.13-cp37-cp37m-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.13-cp37-cp37m-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 8afb1b57ad8a8a36c17b605d4a5aa3f9f5135c39e879443312300a88c48c6448
MD5 0548817dfa7f1d7f7b826ccc7d0d706f
BLAKE2b-256 b6c2e68926297042bd021232ec38a1babb5dd45af13abf3ae4c16cecb6479589

See more details on using hashes here.

File details

Details for the file getdaft-0.0.13-cp37-cp37m-macosx_10_16_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.13-cp37-cp37m-macosx_10_16_x86_64.whl
Algorithm Hash digest
SHA256 49ba11bb2016074672889789bd6881f54b51cf68f3cd3ff74f529a572068d399
MD5 627b43e22d6a3ef5051fc71f959622c2
BLAKE2b-256 a5faa5bd84ce71cc715c6f990a4823e3e0275bc525706b54a877ebf006a0fd26

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page