A Distributed DataFrame library for large scale complex data processing.

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for jaychia from gravatar.com jaychia Avatar for samster25 from gravatar.com samster25

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache-2.0)
  • Author: Eventual Inc
  • Maintainer: Sammy Sidhu
  • Requires: Python >=3.7.1, <4.0.0
Classifiers

Project description

daft

Daft

Frame 113

Daft is a fast, ergonomic and scalable open-source dataframe library: built for Python and Complex Data/Machine Learning workloads.

Installation

Install Daft with pip install getdaft.

Documentation

Learn more about Daft in our documentation.

Community

For questions about Daft, please post in our community hosted on GitHub Discussions! We look forward to meeting you there.

Why Daft?

Processing Complex Data such as images/audio/pointclouds often requires accelerated compute for geometric or machine learning algorithms, much of which leverages existing tooling from the Python/C++ ecosystem. However, many workloads such as analytics, model training data curation and data processing often also require relational query operations for loading/filtering/joining/aggregations.

Daft marries the two worlds with a Dataframe API, enabling you to run both large analytical queries and powerful Complex Data algorithms from the same interface.

  1. Python-first: Python and Jupyter notebooks are first-class citizens. Daft handles any Python libraries and datastructures natively - use any Python library such as Numpy, OpenCV and PyTorch for Complex Data processing.

  2. Laptop to Cloud: Daft is built to run as easily on your laptop for interactive development and on your own Ray cluster or Eventual deployment for terabyte-scale production workloads.

  3. Open Data Formats: Daft loads from and writes to open data formats such as Apache Parquet and Apache Iceberg. It also supports all major cloud vendors' object storage options, allowing you to easily integrate with your existing storage solutions.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for jaychia from gravatar.com jaychia Avatar for samster25 from gravatar.com samster25

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache-2.0)
  • Author: Eventual Inc
  • Maintainer: Sammy Sidhu
  • Requires: Python >=3.7.1, <4.0.0
Classifiers

Release history Release notifications | RSS feed

0.3.14

0.3.13

0.3.12 yanked

Reason this release was yanked:

daft.context.set_runner_ray is broken, which leads to a bad experience with using Daft on Ray

0.3.11 yanked

Reason this release was yanked:

daft.context.set_runner_ray is broken, which leads to a bad experience with using Daft on Ray

0.3.10

0.3.9

0.3.8

0.3.7

0.3.6

0.3.5

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.33

0.2.32 yanked

Reason this release was yanked:

This release is broken for Linux ARM machines because of a linking issue in the SIMD crate

0.2.31

0.2.30

0.2.29

0.2.28

0.2.27

0.2.26

0.2.25

0.2.24

0.2.23

0.2.22

0.2.21

0.2.20

0.2.19

0.2.18

0.2.17

0.2.16

0.2.15

0.2.14

0.2.13

0.2.12

0.2.11

0.2.10

0.2.9

0.2.8

0.2.7

0.2.6

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

0.1.13

0.1.12

0.1.11

0.1.10

0.1.9

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.24

0.0.23

0.0.22

0.0.21

0.0.20

0.0.19

0.0.18

0.0.17

0.0.16

0.0.15 yanked

Reason this release was yanked:

Broken build for python 3.8 and 3.9

0.0.14

0.0.13

0.0.12 yanked

Reason this release was yanked:

Binary Wheel is built with wrong version of pyarrow (9.0 vs 6.0)
This version

0.0.11

0.0.10

0.0.9

0.0.8

0.0.7

0.0.6

0.0.5 yanked

Reason this release was yanked:

nonvalid release

0.0.4 yanked

Reason this release was yanked:

nonvalid release

0.0.3 yanked

Reason this release was yanked:

nonvalid release

0.0.2 yanked

Reason this release was yanked:

nonvalid release

0.0.1 yanked

Reason this release was yanked:

nonvalid release

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

getdaft-0.0.11.tar.gz (128.2 kB view details)

Uploaded Source

Built Distributions

getdaft-0.0.11-cp310-cp310-manylinux_2_17_x86_64.whl (799.8 kB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

getdaft-0.0.11-cp310-cp310-macosx_11_0_x86_64.whl (195.3 kB view details)

Uploaded CPython 3.10 macOS 11.0+ x86-64

getdaft-0.0.11-cp310-cp310-macosx_11_0_arm64.whl (191.9 kB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

getdaft-0.0.11-cp39-cp39-manylinux_2_17_x86_64.whl (799.5 kB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

getdaft-0.0.11-cp39-cp39-macosx_11_0_x86_64.whl (195.1 kB view details)

Uploaded CPython 3.9 macOS 11.0+ x86-64

getdaft-0.0.11-cp39-cp39-macosx_11_0_arm64.whl (191.9 kB view details)

Uploaded CPython 3.9 macOS 11.0+ ARM64

getdaft-0.0.11-cp38-cp38-manylinux_2_17_x86_64.whl (797.7 kB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

getdaft-0.0.11-cp38-cp38-macosx_11_0_arm64.whl (191.8 kB view details)

Uploaded CPython 3.8 macOS 11.0+ ARM64

getdaft-0.0.11-cp38-cp38-macosx_10_16_x86_64.whl (195.1 kB view details)

Uploaded CPython 3.8 macOS 10.16+ x86-64

getdaft-0.0.11-cp37-cp37m-manylinux_2_17_x86_64.whl (796.1 kB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

getdaft-0.0.11-cp37-cp37m-macosx_10_16_x86_64.whl (194.8 kB view details)

Uploaded CPython 3.7m macOS 10.16+ x86-64

File details

Details for the file getdaft-0.0.11.tar.gz.

File metadata

  • Download URL: getdaft-0.0.11.tar.gz
  • Upload date:
  • Size: 128.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.10.6

File hashes

Hashes for getdaft-0.0.11.tar.gz
Algorithm Hash digest
SHA256 4a888338d5e868bc2903e7fd54357a617763480fb0f4c34114c16755f8001406
MD5 c7a784c47a71910dce7ffba1086adfd7
BLAKE2b-256 c10b1bf0a48510a0730e5397cde65de6abfc04bac2ebc0d77cf81ff0faea514b

See more details on using hashes here.

File details

Details for the file getdaft-0.0.11-cp310-cp310-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.11-cp310-cp310-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 65a2ebd808b783f21ef22cde6f35a8a1b728bc24aa95cac22db94a1b4a134567
MD5 d931885cd82ac876c38352be5532d15b
BLAKE2b-256 b519b9fca2edac630d04e88a82a24359c150fd1ce76bf43bc594d9b167f14da7

See more details on using hashes here.

File details

Details for the file getdaft-0.0.11-cp310-cp310-macosx_11_0_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.11-cp310-cp310-macosx_11_0_x86_64.whl
Algorithm Hash digest
SHA256 9d9f263f1082aec0fdc79da424708af73d87fe1bbe20857c30c9f4a6814cb431
MD5 5297c288248c53a9e5fa1fe89d44f4b2
BLAKE2b-256 15e0323643b423e5a822e3fda68e0342e5e4a4b5a73116d54a8485cb32be0852

See more details on using hashes here.

File details

Details for the file getdaft-0.0.11-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.11-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 45047ffa3c68473067f282ac496fac454a852a39cff76f8601d52249e5571956
MD5 0ff0a241ca062c77f2a76023c1366002
BLAKE2b-256 669681e9165c777e57e7a7b6ad66ddab08df70d9c626f6bc3009e13b769803dc

See more details on using hashes here.

File details

Details for the file getdaft-0.0.11-cp39-cp39-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.11-cp39-cp39-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 11f826f0924c4b719d352fe8595b1ee7d46a880620bb3f697700d3e78b673baf
MD5 3b04ac992b45a9651375e9a3dd515afe
BLAKE2b-256 a0fada29bd7bf61c622477244859eec8d4322d6e0bb2d0c7310bc8c0502fccc3

See more details on using hashes here.

File details

Details for the file getdaft-0.0.11-cp39-cp39-macosx_11_0_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.11-cp39-cp39-macosx_11_0_x86_64.whl
Algorithm Hash digest
SHA256 070c813578c7813873c6ba6050253a938119f4fdc650ecd249f1dd5e10bffe6c
MD5 0bf4f807c00d5aa26193a068daf71a45
BLAKE2b-256 6eb0905b6e2d750b914f7de5ad95a4692e10635d0c1ea7be7d58bc555aebd99a

See more details on using hashes here.

File details

Details for the file getdaft-0.0.11-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.11-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 7dd6d6ca05108e21135a99f1d64d0c00b6be8d1d4a44c35044b0f04a8d16871c
MD5 837dd050563a07c199e24483b43d1c27
BLAKE2b-256 41a0c1bddf0b6f77407c312df7064df51a9613370e3aa16efee8c84647706098

See more details on using hashes here.

File details

Details for the file getdaft-0.0.11-cp38-cp38-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.11-cp38-cp38-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 c3fd236133ccdaa60182fd7c00f3cf129690969bf0491874541ca60cfd4ab9bd
MD5 336058b9c364a2d7f97991d983677222
BLAKE2b-256 1b21471cb73ca2d991aab3696464ad3c09028063f3c098a6ca3a3211329db8d1

See more details on using hashes here.

File details

Details for the file getdaft-0.0.11-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.11-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 56d1a49da663666a5cbbc468d44351736b9fbadb9952e8d0767132416e599be9
MD5 5b17f6d6b2c6fcc0f50e909d201b4c97
BLAKE2b-256 92a70b931c3658444b8fd3267da3fa2b809451fd8ee2449587ae806918ad8145

See more details on using hashes here.

File details

Details for the file getdaft-0.0.11-cp38-cp38-macosx_10_16_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.11-cp38-cp38-macosx_10_16_x86_64.whl
Algorithm Hash digest
SHA256 fd9ccf17f8eeb1b218c8e53c93815676e2693c3cb0c38e6a74211482173719f1
MD5 70ba7144de79e460aeb5c66f4bd474c3
BLAKE2b-256 a0ae698e6097662ffbcff791a89c8c80e53ff2da58b5dadd685447df3c9da601

See more details on using hashes here.

File details

Details for the file getdaft-0.0.11-cp37-cp37m-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.11-cp37-cp37m-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 d7c68b364b5d279ec25426659ed0d752f2793fa4cb9913eee1304169c967b4e3
MD5 96f1e20d458618af508f88fbe3d63647
BLAKE2b-256 7f2f535f68475f2cecba0bf3cf6e81fb326beadbbbbea31596ac8e07f850be7f

See more details on using hashes here.

File details

Details for the file getdaft-0.0.11-cp37-cp37m-macosx_10_16_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.11-cp37-cp37m-macosx_10_16_x86_64.whl
Algorithm Hash digest
SHA256 2b528b1de3a6fbdfd188fa961344150a577fe38f359e553ff2010ebcd0e967b7
MD5 3e3b67977a23ab9bc39421b75982ad95
BLAKE2b-256 5df801ee4d368e2ece0feac79c128de38d63c1588741382f173ff7f83052aeb4

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page