ArcticDB DataFrame Database

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Business Source License 1.1 (See LICENSE.txt)
  • Author: Man Alpha Technology
  • Provides-Extra: testing
Classifiers

Project description

ArcticDB Website | 📘 ArcticDB Docs | 📰: ArcticDB Blog | Press Release | Press Release | Community

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability

PyPI (Python 3.6 - 3.11) conda-forge (Python 3.8 - 3.11)
Linux (Intel/AMD) ✔️ ✔️
Windows (Intel/AMD) ✔️ ➖
MacOS ➖ Beta️

For conda-forge see the release-info.

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ➖

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs

Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from Man Group Operations Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact arcticdb@man.com.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026
4.4 Business Source License 1.1 Apr 5, 2026

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at arcticdb@man.com.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at arcticdb@man.com or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Business Source License 1.1 (See LICENSE.txt)
  • Author: Man Alpha Technology
  • Provides-Extra: testing
Classifiers

Release history Release notifications | RSS feed

5.1.0rc0 pre-release

5.0.0

5.0.0rc1 pre-release

5.0.0rc0 pre-release

4.5.1

4.5.1rc4 pre-release

4.5.1rc3 pre-release

4.5.1rc0 pre-release

This version

4.5.0

4.5.0rc4 pre-release

4.5.0rc3 pre-release

4.5.0rc2 pre-release

4.5.0rc1 pre-release

4.5.0rc0 pre-release

4.4.6

4.4.5

4.4.4

4.4.4rc6 pre-release

4.4.4rc5 pre-release

4.4.4rc4 pre-release

4.4.4rc3 pre-release

4.4.4rc2 pre-release

4.4.4rc1 pre-release

4.4.4rc0 pre-release

4.4.3

4.4.3rc1 pre-release

4.4.3rc0 pre-release

4.4.2

4.4.2rc0 pre-release

4.4.1

4.4.0

4.4.0rc2 pre-release

4.4.0rc1 pre-release

4.3.2rc2 pre-release

4.3.1

4.3.0 yanked

Reason this release was yanked:

bad build

4.3.0rc2 pre-release

4.3.0rc1 pre-release

4.3.0rc0 pre-release

4.2.1

4.2.0

4.2.0rc0 pre-release

4.1.0

4.1.0rc2 pre-release

4.1.0rc1 pre-release

4.1.0rc0 pre-release

4.1.0.dev0 pre-release

4.0.5

4.0.4

4.0.3

4.0.2

4.0.2rc0 pre-release

4.0.1

4.0.0

4.0.0rc2 pre-release

4.0.0rc1 pre-release

4.0.0rc0 pre-release

3.0.0

3.0.0rc2 pre-release

3.0.0rc0 pre-release

2.0.0

2.0.0rc0 pre-release

1.6.2

1.6.2rc0 pre-release

1.6.1

1.6.1rc2 pre-release

1.6.1rc1 pre-release

1.6.1rc0 pre-release

1.6.0

1.5.0

1.4.1

1.4.0

1.3.0

1.2.1

1.2.0

1.1.0

1.0.1

1.0.0

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

arcticdb-4.5.0-cp311-cp311-win_amd64.whl (6.7 MB view details)

Uploaded CPython 3.11 Windows x86-64

arcticdb-4.5.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.0 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

arcticdb-4.5.0-cp310-cp310-win_amd64.whl (6.8 MB view details)

Uploaded CPython 3.10 Windows x86-64

arcticdb-4.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

arcticdb-4.5.0-cp39-cp39-win_amd64.whl (6.7 MB view details)

Uploaded CPython 3.9 Windows x86-64

arcticdb-4.5.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

arcticdb-4.5.0-cp38-cp38-win_amd64.whl (6.8 MB view details)

Uploaded CPython 3.8 Windows x86-64

arcticdb-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

arcticdb-4.5.0-cp37-cp37m-win_amd64.whl (6.7 MB view details)

Uploaded CPython 3.7m Windows x86-64

arcticdb-4.5.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.2 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

arcticdb-4.5.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64

File details

Details for the file arcticdb-4.5.0-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.5.0-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 6.7 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-4.5.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 090a7f4a01a4c435ab0f36582dc5a855f96411fe6bf9e7ff62b9497d4082931a
MD5 ae0de908df2d41112c809084cee9ad13
BLAKE2b-256 637bce36ad1792142edefedf3d4392c999b78bb829673f1324300b83fcc058dc

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.5.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 691e46f62f6e2632cf54cffd13e4cba5ebbc356a06d86b567a6d105fdf4a4507
MD5 e876b9931ba4c3168fe508961f30969b
BLAKE2b-256 841cdc85757092ec5b74f210b8d76b0ff56d982d53bae485c06a66f556fe22c3

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.5.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 6.8 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-4.5.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 273f8b57874dc9168df2f58a3c97ef375938b8684637428befd9271dc22b5014
MD5 bb07f01d08ddc4a9e89f5995f508a28a
BLAKE2b-256 39009d7c2185b1fdaf5fd032d92e321d55c11bb45bfbc2393e8834bf656bb6ce

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 7205b9f1ddbe79687be12f58353fc766d5e3395e1e57760e7873ec334c6c6b08
MD5 af410d2313ac5c594e94787cb810474f
BLAKE2b-256 b043d41a42f58883f8b2c01e0d7149cc7ccd8882e905502390a546204eba89f1

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.0-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.5.0-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 6.7 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-4.5.0-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 97669a9f8b35d1bae858b8d3d9e0fac4e7017fe85324dbf27c378e1f5a4bebae
MD5 f79333258e76f105a922b2e91e770538
BLAKE2b-256 eb10895efe0b70643e5e053200b889f7e98100fbc2612568ff306d8a8c8ecfe4

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.5.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 2d37551689edb35e7dfd4cdbf7eec45463a62de37873350ec3aeec44422a31da
MD5 c95b0ad887beec05397b7a5abea67da4
BLAKE2b-256 63921c5025f6f33c44ee22ddaecd354ecea51b1b913eae16460620706836fc37

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.0-cp38-cp38-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.5.0-cp38-cp38-win_amd64.whl
  • Upload date:
  • Size: 6.8 MB
  • Tags: CPython 3.8, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-4.5.0-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 a6c6194f9e1fd6a69ee2e8c490e020f393915698b9240bc85cf87f696ae14844
MD5 36f71f63d8df14bf903487eae59ab987
BLAKE2b-256 25b008e4f4bc85ab0c9f7f72d1bfa5beea67b5cff84c77e8748a645843649df7

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.5.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 0402853f13451069e79dbd20ce755211bdf03f17df34cb68eb1c1bacb28ae434
MD5 eca8b7486acc209c8137ada59eb5dae2
BLAKE2b-256 adcf1bf37347ad1070ffd37cc04a89a9eff26a46198a40cb269a8df1b7abaaa4

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.0-cp37-cp37m-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.5.0-cp37-cp37m-win_amd64.whl
  • Upload date:
  • Size: 6.7 MB
  • Tags: CPython 3.7m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-4.5.0-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 bc9ef38044c452a81abac8e2d7800aff6cde5d29b7383a93f6f8188d441d3cd2
MD5 720860e5868c0c2b67269f944aeae4b1
BLAKE2b-256 84af4f8e46842b53090dea8eecf10b6ae5d6a241ca28cf59a8963286e0597e81

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.5.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 8e7aa1ba6c0630ea83bbda00ba83b251a94ed582893119441a054f2f5d84fc6e
MD5 0ed4835b19a907bd77c5e00f39226121
BLAKE2b-256 cee67e933bbe17d0b2eb3ee3b4394f80557cc95fdd935cf30c2107b82f67963e

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.5.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 bbc153a40e5194244a9c3d7c7401f104b99e0cef4b0d66dfd5586b9f89dcb777
MD5 b0052ce646553fa9679a709f0a57d69d
BLAKE2b-256 88e3879e1655af532d18d322ce8e822c608f84cc91368eaf028722490b9ce2df

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page