polars-deltalake

Native polars deltalake reader

These details have not been verified by PyPI

Project links

Project description

A native deltalake reader built with Pola-rs and Delta-RS crates.

Example

import polars as pl
import polars_deltalake as pldl

df = pl.DataFrame(
    [
        pl.Series("foo", [1, 2, 3], dtype=pl.Int64),
        pl.Series("bar", ['1', '2', '3'], dtype=pl.String),
        pl.Series("datetime", [datetime(2010, 1, 1, 0, 0)]*3, dtype=pl.Datetime(time_unit='us', time_zone=None)),
        pl.Series("datetime_tz", [datetime(2010, 1, 1, 0, 0, tzinfo=timezone.utc)]*3, dtype=pl.Datetime(time_unit='us', time_zone='UTC')),
        pl.Series("date_month", [201001, 201002, 201003], dtype=pl.Int32),
        pl.Series("static_part", ['A', 'A', 'A'], dtype=pl.String),
        pl.Series("list", [['5', 'B'], ['5', 'B'], ['5', 'B']], dtype=pl.List(pl.String)),
    ]
)

df.write_delta("test_table", delta_write_options={"partition_by":["foo"]})

table = pldl.scan_delta("test_table")

table.filter(pl.col("foo") == 1).collect()

shape: (1, 7)
┌─────┬─────────────────────┬─────────────────────┬────────────┬─────────────┬────────────┬─────┐
│ bar ┆ datetime            ┆ datetime_tz         ┆ date_month ┆ static_part ┆ list       ┆ foo │
│ --- ┆ ---                 ┆ ---                 ┆ ---        ┆ ---         ┆ ---        ┆ --- │
│ str ┆ datetime[μs]        ┆ datetime[μs, UTC]   ┆ i32        ┆ str         ┆ list[str]  ┆ i64 │
╞═════╪═════════════════════╪═════════════════════╪════════════╪═════════════╪════════════╪═════╡
│ 1   ┆ 2010-01-01 00:00:00 ┆ 2010-01-01 00:00:00 ┆ 201001     ┆ A           ┆ ["5", "B"] ┆ 1   │
│     ┆                     ┆ UTC                 ┆            ┆             ┆            ┆     │
└─────┴─────────────────────┴─────────────────────┴────────────┴─────────────┴────────────┴─────┘

Project details

These details have not been verified by PyPI

Project links

Release history Release notifications | RSS feed

This version

0.2.0

Mar 17, 2024

0.1.0

Mar 16, 2024

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

polars_deltalake-0.2.0.tar.gz (31.4 kB view details)

Uploaded Mar 17, 2024 Source

Built Distributions

polars_deltalake-0.2.0-cp38-abi3-win_amd64.whl (11.7 MB view details)

Uploaded Mar 17, 2024 CPython 3.8+ Windows x86-64

polars_deltalake-0.2.0-cp38-abi3-win32.whl (10.5 MB view details)

Uploaded Mar 17, 2024 CPython 3.8+ Windows x86

polars_deltalake-0.2.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (15.5 MB view details)

Uploaded Mar 17, 2024 CPython 3.8+ manylinux: glibc 2.17+ x86-64

polars_deltalake-0.2.0-cp38-abi3-manylinux_2_17_i686.manylinux2014_i686.whl (16.0 MB view details)

Uploaded Mar 17, 2024 CPython 3.8+ manylinux: glibc 2.17+ i686

polars_deltalake-0.2.0-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (14.9 MB view details)

Uploaded Mar 17, 2024 CPython 3.8+ manylinux: glibc 2.17+ ARMv7l

polars_deltalake-0.2.0-cp38-abi3-macosx_11_0_arm64.whl (11.7 MB view details)

Uploaded Mar 17, 2024 CPython 3.8+ macOS 11.0+ ARM64

polars_deltalake-0.2.0-cp38-abi3-macosx_10_12_x86_64.whl (12.7 MB view details)

Uploaded Mar 17, 2024 CPython 3.8+ macOS 10.12+ x86-64

File details

Details for the file polars_deltalake-0.2.0.tar.gz.

File metadata

Download URL: polars_deltalake-0.2.0.tar.gz
Upload date: Mar 17, 2024
Size: 31.4 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: maturin/1.5.0

File hashes

Hashes for polars_deltalake-0.2.0.tar.gz
Algorithm	Hash digest
SHA256	`c3b43a808b51074b48e0404b90275428c51ca313bf4e9c176eec7b1fc98366dd`
MD5	`c559162afc1f5539a5b6b81f968c5e8e`
BLAKE2b-256	`ea3e2889666d19a71fb4292c7957980a7f16a33abd097d3356a64e6a53e52f85`

See more details on using hashes here.

File details

Details for the file polars_deltalake-0.2.0-cp38-abi3-win_amd64.whl.

File metadata

Download URL: polars_deltalake-0.2.0-cp38-abi3-win_amd64.whl
Upload date: Mar 17, 2024
Size: 11.7 MB
Tags: CPython 3.8+, Windows x86-64
Uploaded using Trusted Publishing? No
Uploaded via: maturin/1.5.0

File hashes

Hashes for polars_deltalake-0.2.0-cp38-abi3-win_amd64.whl
Algorithm	Hash digest
SHA256	`caca382d2f95fab17eccfe8b13146b59b24476d2292ac291f52e02a1b47a3bf1`
MD5	`5f7bce801a85f4b2efe5de7128e9a7fa`
BLAKE2b-256	`09e55a8f7d976933b890ba9138e93c8d0ced96b267f001ea418e083b84606604`

See more details on using hashes here.

File details

Details for the file polars_deltalake-0.2.0-cp38-abi3-win32.whl.

File metadata

Download URL: polars_deltalake-0.2.0-cp38-abi3-win32.whl
Upload date: Mar 17, 2024
Size: 10.5 MB
Tags: CPython 3.8+, Windows x86
Uploaded using Trusted Publishing? No
Uploaded via: maturin/1.5.0

File hashes

Hashes for polars_deltalake-0.2.0-cp38-abi3-win32.whl
Algorithm	Hash digest
SHA256	`dcd95ede7772b31871358650c8199770d1d5cea69ce4c8809887d82893e1d5dd`
MD5	`aa37667f32db268b63bf196fd3e5b706`
BLAKE2b-256	`0e319dfac621eef96b64e3333c4dda6f1c3775d44d63a6833900055ff2887a01`

See more details on using hashes here.

File details

Details for the file polars_deltalake-0.2.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

Download URL: polars_deltalake-0.2.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Upload date: Mar 17, 2024
Size: 15.5 MB
Tags: CPython 3.8+, manylinux: glibc 2.17+ x86-64
Uploaded using Trusted Publishing? No
Uploaded via: maturin/1.5.0

File hashes

Hashes for polars_deltalake-0.2.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm	Hash digest
SHA256	`1791f22479bf445fb2d9a43b9c5bd83f2891a4b34a2a1119bf71d74385e61deb`
MD5	`e3c10ba9dd0b4607f3553927d1b044a5`
BLAKE2b-256	`e2e83bac3bdea36e00959b79721ac8281aa634ae81694eb8f265ef9c7f3d49f6`

See more details on using hashes here.

File details

Details for the file polars_deltalake-0.2.0-cp38-abi3-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

Download URL: polars_deltalake-0.2.0-cp38-abi3-manylinux_2_17_i686.manylinux2014_i686.whl
Upload date: Mar 17, 2024
Size: 16.0 MB
Tags: CPython 3.8+, manylinux: glibc 2.17+ i686
Uploaded using Trusted Publishing? No
Uploaded via: maturin/1.5.0

File hashes

Hashes for polars_deltalake-0.2.0-cp38-abi3-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm	Hash digest
SHA256	`c7869174c05cc9734042e00bdbf5a541160a12d34484bd120b3244cdc61c7955`
MD5	`3fc64807837f367339a54d53edc51ff2`
BLAKE2b-256	`82dce4be2a73a3052fbfbe08a7d6345b721b8cf7af4d46539c0814fe9c6cbcd5`

See more details on using hashes here.

File details

Details for the file polars_deltalake-0.2.0-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

Download URL: polars_deltalake-0.2.0-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Upload date: Mar 17, 2024
Size: 14.9 MB
Tags: CPython 3.8+, manylinux: glibc 2.17+ ARMv7l
Uploaded using Trusted Publishing? No
Uploaded via: maturin/1.5.0

File hashes

Hashes for polars_deltalake-0.2.0-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm	Hash digest
SHA256	`868db9b0913d12e7c2da97433125a9b2bdc388cf31c8f81c17c1b9d56966444f`
MD5	`e9cf2648a913667c3141db28a78c408f`
BLAKE2b-256	`ecc1cb97eada8c18a28c0a72ed3caf182113aec8b127efb194433b1efc759708`

See more details on using hashes here.

File details

Details for the file polars_deltalake-0.2.0-cp38-abi3-macosx_11_0_arm64.whl.

File metadata

Download URL: polars_deltalake-0.2.0-cp38-abi3-macosx_11_0_arm64.whl
Upload date: Mar 17, 2024
Size: 11.7 MB
Tags: CPython 3.8+, macOS 11.0+ ARM64
Uploaded using Trusted Publishing? No
Uploaded via: maturin/1.5.0

File hashes

Hashes for polars_deltalake-0.2.0-cp38-abi3-macosx_11_0_arm64.whl
Algorithm	Hash digest
SHA256	`71fb2cfc273cadcd620b93838aec42903365cb8f8923f6b8f853c4112deb985a`
MD5	`95cc4ed41bdc4cedfcc340453e034ee5`
BLAKE2b-256	`8e7f077a68215c72cbd067cfadc3455b417618ad7cb86b6f9ccce83f715192f4`

See more details on using hashes here.

File details

Details for the file polars_deltalake-0.2.0-cp38-abi3-macosx_10_12_x86_64.whl.

File metadata

Download URL: polars_deltalake-0.2.0-cp38-abi3-macosx_10_12_x86_64.whl
Upload date: Mar 17, 2024
Size: 12.7 MB
Tags: CPython 3.8+, macOS 10.12+ x86-64
Uploaded using Trusted Publishing? No
Uploaded via: maturin/1.5.0

File hashes

Hashes for polars_deltalake-0.2.0-cp38-abi3-macosx_10_12_x86_64.whl
Algorithm	Hash digest
SHA256	`e93b792156662c0eda83ffeba1dfb59fb351f511d6296db291e0b4cb3ec83eaf`
MD5	`288f38f7022b9ba98a300655b0126914`
BLAKE2b-256	`6b9f5a31f06946977387fbadc1f008c2b7422271cc9c5e611696a4c6e727ca7f`