A Distributed DataFrame library for large scale complex data processing.

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for jaychia from gravatar.com jaychia Avatar for samster25 from gravatar.com samster25

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache-2.0)
  • Author: Eventual Inc
  • Maintainer: Sammy Sidhu
  • Requires: Python >=3.7.1, <4.0.0
Classifiers

Project description

daft

Welcome to Daft

Daft is a fast, ergonomic and scalable open-source dataframe library: built for Python and Complex Data/Machine Learning workloads.

Daft is currently in its Alpha release phase - please expect bugs and rapid improvements to the project. We welcome user feedback/feature requests in our Discussions forums.

Frame 113

Installation

Install Daft with pip install getdaft.

Documentation

Learn more about Daft in our documentation.

Community

For questions about Daft, please post in our community hosted on GitHub Discussions. We look forward to meeting you there!

Why Daft?

Processing Complex Data such as images/audio/pointclouds often requires accelerated compute for geometric or machine learning algorithms, much of which leverages existing tooling from the Python/C++ ecosystem. However, many workloads such as analytics, model training data curation and data processing often also require relational query operations for loading/filtering/joining/aggregations.

Daft marries the two worlds with a Dataframe API, enabling you to run both large analytical queries and powerful Complex Data algorithms from the same interface.

  1. Python-first: Python and Jupyter notebooks are first-class citizens. Daft handles any Python libraries and datastructures natively - use any Python library such as Numpy, OpenCV and PyTorch for Complex Data processing.

  2. Laptop to Cloud: Daft is built to run as easily on your laptop for interactive development and on your own Ray cluster or Eventual deployment for terabyte-scale production workloads.

  3. Open Data Formats: Daft loads from and writes to open data formats such as Apache Parquet and Apache Iceberg. It also supports all major cloud vendors' object storage options, allowing you to easily integrate with your existing storage solutions.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for jaychia from gravatar.com jaychia Avatar for samster25 from gravatar.com samster25

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License (Apache-2.0)
  • Author: Eventual Inc
  • Maintainer: Sammy Sidhu
  • Requires: Python >=3.7.1, <4.0.0
Classifiers

Release history Release notifications | RSS feed

0.3.14

0.3.13

0.3.12 yanked

Reason this release was yanked:

daft.context.set_runner_ray is broken, which leads to a bad experience with using Daft on Ray

0.3.11 yanked

Reason this release was yanked:

daft.context.set_runner_ray is broken, which leads to a bad experience with using Daft on Ray

0.3.10

0.3.9

0.3.8

0.3.7

0.3.6

0.3.5

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.33

0.2.32 yanked

Reason this release was yanked:

This release is broken for Linux ARM machines because of a linking issue in the SIMD crate

0.2.31

0.2.30

0.2.29

0.2.28

0.2.27

0.2.26

0.2.25

0.2.24

0.2.23

0.2.22

0.2.21

0.2.20

0.2.19

0.2.18

0.2.17

0.2.16

0.2.15

0.2.14

0.2.13

0.2.12

0.2.11

0.2.10

0.2.9

0.2.8

0.2.7

0.2.6

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

0.1.13

0.1.12

0.1.11

0.1.10

0.1.9

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.24

0.0.23

0.0.22

0.0.21

0.0.20

0.0.19

0.0.18

0.0.17

0.0.16

0.0.15 yanked

Reason this release was yanked:

Broken build for python 3.8 and 3.9
This version

0.0.14

0.0.13

0.0.12 yanked

Reason this release was yanked:

Binary Wheel is built with wrong version of pyarrow (9.0 vs 6.0)

0.0.11

0.0.10

0.0.9

0.0.8

0.0.7

0.0.6

0.0.5 yanked

Reason this release was yanked:

nonvalid release

0.0.4 yanked

Reason this release was yanked:

nonvalid release

0.0.3 yanked

Reason this release was yanked:

nonvalid release

0.0.2 yanked

Reason this release was yanked:

nonvalid release

0.0.1 yanked

Reason this release was yanked:

nonvalid release

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

getdaft-0.0.14.tar.gz (142.8 kB view details)

Uploaded Source

Built Distributions

getdaft-0.0.14-cp310-cp310-manylinux_2_17_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

getdaft-0.0.14-cp310-cp310-macosx_11_0_x86_64.whl (286.5 kB view details)

Uploaded CPython 3.10 macOS 11.0+ x86-64

getdaft-0.0.14-cp310-cp310-macosx_11_0_arm64.whl (273.3 kB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

getdaft-0.0.14-cp39-cp39-manylinux_2_17_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

getdaft-0.0.14-cp39-cp39-macosx_11_0_x86_64.whl (287.0 kB view details)

Uploaded CPython 3.9 macOS 11.0+ x86-64

getdaft-0.0.14-cp39-cp39-macosx_11_0_arm64.whl (273.7 kB view details)

Uploaded CPython 3.9 macOS 11.0+ ARM64

getdaft-0.0.14-cp38-cp38-manylinux_2_17_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

getdaft-0.0.14-cp38-cp38-macosx_11_0_arm64.whl (273.5 kB view details)

Uploaded CPython 3.8 macOS 11.0+ ARM64

getdaft-0.0.14-cp38-cp38-macosx_10_16_x86_64.whl (286.9 kB view details)

Uploaded CPython 3.8 macOS 10.16+ x86-64

getdaft-0.0.14-cp37-cp37m-manylinux_2_17_x86_64.whl (1.7 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

getdaft-0.0.14-cp37-cp37m-macosx_10_16_x86_64.whl (286.3 kB view details)

Uploaded CPython 3.7m macOS 10.16+ x86-64

File details

Details for the file getdaft-0.0.14.tar.gz.

File metadata

  • Download URL: getdaft-0.0.14.tar.gz
  • Upload date:
  • Size: 142.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.10.7

File hashes

Hashes for getdaft-0.0.14.tar.gz
Algorithm Hash digest
SHA256 8522b6258eaed5253fe0efa4c2569c395b0a6eceb78cf53355befc8cf6892fad
MD5 e80966685d023f0402d7a87dbfa0a243
BLAKE2b-256 dc63ffbb7ce0bc034ac4a399aec6bb66fe18cf1fbf11cc72a97addfca04407b8

See more details on using hashes here.

File details

Details for the file getdaft-0.0.14-cp310-cp310-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.14-cp310-cp310-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 dad26617007d2b57988d36d0c78132ef94129126125b513e3193a38031d177aa
MD5 dea2d0815d3df7b05b5a63734097374a
BLAKE2b-256 26c57dd778c2d51f2d2a1bc389d005238fbecae948f4c0794a8ce49918986d1d

See more details on using hashes here.

File details

Details for the file getdaft-0.0.14-cp310-cp310-macosx_11_0_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.14-cp310-cp310-macosx_11_0_x86_64.whl
Algorithm Hash digest
SHA256 9492048ab637dcd51dceed5e70a2d9bbf6289c746bc93ae2329fb00f45ed0a3c
MD5 b7bf34172e58613792d9ebe91770eb33
BLAKE2b-256 b31597675fa62e3b2b950b29e2a9e6e88e7cdd81329fd500c915ebb63aeecd38

See more details on using hashes here.

File details

Details for the file getdaft-0.0.14-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.14-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 e95b876b9b53c4d0e167c8af2eca6e25301a199cc763d0087222d4eb7adf2e21
MD5 288c94873869bb0ba39beafd79901500
BLAKE2b-256 a28edb718b8b276657449c47e8fbefe1ee676308d8bfa6cdff97869ba0960ab9

See more details on using hashes here.

File details

Details for the file getdaft-0.0.14-cp39-cp39-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.14-cp39-cp39-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 c0ae62bc3a3ed8bf004d6892b55468f1f23d37e92e27c3bffc1d3bbd64c95698
MD5 e3400dcd62c480113f220c17694d427b
BLAKE2b-256 a685c632933b932e3d0307b24fc163bebf0061a8f9c1a3f782846a6fe72032c9

See more details on using hashes here.

File details

Details for the file getdaft-0.0.14-cp39-cp39-macosx_11_0_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.14-cp39-cp39-macosx_11_0_x86_64.whl
Algorithm Hash digest
SHA256 77ae8f4c5c5aa31ae336b1bee8dc78cb1404f19538f362f0b1512aa72d363c9d
MD5 b3df777adc81a001172b715a087473cd
BLAKE2b-256 353e5c9db79caf743703d2ce365fc99099ed43d91c134ca8639e0ee198128061

See more details on using hashes here.

File details

Details for the file getdaft-0.0.14-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.14-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 25a2020797f518ab7d1fa4d3ff740a6c29087377146ff680e11d84b947933967
MD5 2d47f15e5cefca0a3236017a9216313b
BLAKE2b-256 bc2fd0f8c14cc1db3eb5af5006c714679a26a27d9a2e8e169b6c9a3e8c57df04

See more details on using hashes here.

File details

Details for the file getdaft-0.0.14-cp38-cp38-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.14-cp38-cp38-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 51f12f11aa6f31a9196361f3bab99979f1100bde6fd0ddcef9c3a0b9068c4cc1
MD5 7d4e337a88e09e95eb1620a6a26c331c
BLAKE2b-256 fddb3d76d3a662de20be78fe607e52c6a0656a64c7da1fb8d77be132d6402907

See more details on using hashes here.

File details

Details for the file getdaft-0.0.14-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.14-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 db5cf5cd8429563a4a2462b97e4c0f7f6b8c5197954d13d37d4b8168ba846f9e
MD5 348b818576f750b204a7f56e5f40cdc6
BLAKE2b-256 97540597969fc6a5da12dafc11d838f59a9e0619bdec704d51ebe955047e2d3d

See more details on using hashes here.

File details

Details for the file getdaft-0.0.14-cp38-cp38-macosx_10_16_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.14-cp38-cp38-macosx_10_16_x86_64.whl
Algorithm Hash digest
SHA256 0a41b30a3f723e2c00e2cc84246b94579df13049afed58e78072f041b30aed97
MD5 3563b07648afb15749de7fa7a801afaa
BLAKE2b-256 42606f1981a730cf8cdbc203aa592cfc7f048b89882797b0e2e89304e6df9e96

See more details on using hashes here.

File details

Details for the file getdaft-0.0.14-cp37-cp37m-manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.14-cp37-cp37m-manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 2b7317fd63af0eccdee49a92364833e577fcab2a517729413df1a6226fdafa38
MD5 fe4bb3b7a5e4e221f8b9235162d5a1de
BLAKE2b-256 dac13cfee855fb53ca8d530bc04693fad12514b8ff9403b98fe5955ac3814edf

See more details on using hashes here.

File details

Details for the file getdaft-0.0.14-cp37-cp37m-macosx_10_16_x86_64.whl.

File metadata

File hashes

Hashes for getdaft-0.0.14-cp37-cp37m-macosx_10_16_x86_64.whl
Algorithm Hash digest
SHA256 365246cd0a1f04cbe232aeeafdfe6216df020fc6a516ce55ae6daa0030fa0a89
MD5 c40c7e7877d14bd27b2b832acffc9df3
BLAKE2b-256 eb274758e3f7f138ebbae210b21301da233aaa6203005118c146ef16876a2162

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page