ArcticDB DataFrame Database

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Reason this release was yanked:

bad build

Project description

ArcticDB Website | ArcticDB Blog | Press Release | Press Release | Community

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability

PyPI (Python 3.6 - 3.11) conda-forge (Python 3.8 - 3.11)
Linux ✔️ ✔️
Windows ✔️ ➖
MacOS ➖ Beta️

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ➖

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs

Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from Man Group Operations Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact arcticdb@man.com.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at arcticdb@man.com.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at arcticdb@man.com or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

5.1.0rc0 pre-release

5.0.0

5.0.0rc1 pre-release

5.0.0rc0 pre-release

4.5.1

4.5.1rc4 pre-release

4.5.1rc3 pre-release

4.5.1rc0 pre-release

4.5.0

4.5.0rc4 pre-release

4.5.0rc3 pre-release

4.5.0rc2 pre-release

4.5.0rc1 pre-release

4.5.0rc0 pre-release

4.4.6

4.4.5

4.4.4

4.4.4rc6 pre-release

4.4.4rc5 pre-release

4.4.4rc4 pre-release

4.4.4rc3 pre-release

4.4.4rc2 pre-release

4.4.4rc1 pre-release

4.4.4rc0 pre-release

4.4.3

4.4.3rc1 pre-release

4.4.3rc0 pre-release

4.4.2

4.4.2rc0 pre-release

4.4.1

4.4.0

4.4.0rc2 pre-release

4.4.0rc1 pre-release

4.3.2rc2 pre-release

4.3.1

This version

4.3.0 yanked

Reason this release was yanked:

bad build

4.3.0rc2 pre-release

4.3.0rc1 pre-release

4.3.0rc0 pre-release

4.2.1

4.2.0

4.2.0rc0 pre-release

4.1.0

4.1.0rc2 pre-release

4.1.0rc1 pre-release

4.1.0rc0 pre-release

4.1.0.dev0 pre-release

4.0.5

4.0.4

4.0.3

4.0.2

4.0.2rc0 pre-release

4.0.1

4.0.0

4.0.0rc2 pre-release

4.0.0rc1 pre-release

4.0.0rc0 pre-release

3.0.0

3.0.0rc2 pre-release

3.0.0rc0 pre-release

2.0.0

2.0.0rc0 pre-release

1.6.2

1.6.2rc0 pre-release

1.6.1

1.6.1rc2 pre-release

1.6.1rc1 pre-release

1.6.1rc0 pre-release

1.6.0

1.5.0

1.4.1

1.4.0

1.3.0

1.2.1

1.2.0

1.1.0

1.0.1

1.0.0

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

arcticdb-4.3.0-cp311-cp311-win_amd64.whl (7.0 MB view details)

Uploaded CPython 3.11 Windows x86-64

arcticdb-4.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.7 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

arcticdb-4.3.0-cp310-cp310-win_amd64.whl (7.0 MB view details)

Uploaded CPython 3.10 Windows x86-64

arcticdb-4.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.7 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

arcticdb-4.3.0-cp39-cp39-win_amd64.whl (7.0 MB view details)

Uploaded CPython 3.9 Windows x86-64

arcticdb-4.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.7 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

arcticdb-4.3.0-cp38-cp38-win_amd64.whl (7.0 MB view details)

Uploaded CPython 3.8 Windows x86-64

arcticdb-4.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.7 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

arcticdb-4.3.0-cp37-cp37m-win_amd64.whl (7.0 MB view details)

Uploaded CPython 3.7m Windows x86-64

arcticdb-4.3.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.8 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

arcticdb-4.3.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.8 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64

File details

Details for the file arcticdb-4.3.0-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for arcticdb-4.3.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 ce6c52fa495aa5566bd0cec75c40d5449bdbfd6d2701d56795a8d02726c55bc2
MD5 50e139bb1fba308cce43833a1201ffae
BLAKE2b-256 bd531e9aeaeaa40a5b924b01fcceb80aff4ad343dd4a2c71e32f5931fb1b6130

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 0283b8d2283dd920f418b6e186c61b9f5cdaaf4d7247af8443217d2708e1e05a
MD5 3f7e7a407227fea56f91892ac24f7069
BLAKE2b-256 b2084624076573e4a7a10b8eab7cdd274f2ef65f415e73dca990abb134009b69

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.3.0-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for arcticdb-4.3.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 e68012601cbc6effadd86b9111975f541c638f678035e58b0c2d5413cfc934ed
MD5 951a00507d33d8f4399d2fbefdc949de
BLAKE2b-256 935a2479a418ba4b644b81e96bad482206457a231a38b07379830793656e51cb

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 42f8a7fdcfd52d40ada5983d09524b962792945e6e66f3bd53d4afa993cd865f
MD5 8d221d4b08d25bc46106f3eb5a0dc489
BLAKE2b-256 479425934ed1285a102272c66782f3f5d7ac9ed1b2acde3c0ea737b1d79b2596

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.3.0-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.3.0-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 7.0 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.12

File hashes

Hashes for arcticdb-4.3.0-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 1748814e99b5b63b2e9d9f2529388d1945e4e907f6e95ef5b6b0f52f5e3ee242
MD5 8588611738e58ad595fff60c9544df8c
BLAKE2b-256 9230f95ddf8aee230860dbbbe15c108a931436bf3fedc28109e162a219837ba8

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 1e9278ca14ae0a9ff943716529155d531034193a51f6bb0a6a40cfc16182f528
MD5 f544076b980f9a6550b1d2899c9982ea
BLAKE2b-256 d7d1cf645274eee31a7c4884ce023eacbfa9bcc14149c3a157ad3baa10f18e7d

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.3.0-cp38-cp38-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.3.0-cp38-cp38-win_amd64.whl
  • Upload date:
  • Size: 7.0 MB
  • Tags: CPython 3.8, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.12

File hashes

Hashes for arcticdb-4.3.0-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 1c7ab3692b1dd1e2b04d4da76bc60dfe05f0b05ca0fdfe73a499d2b961c50ed3
MD5 56c1ea9c7a8c393f08240685ee35b950
BLAKE2b-256 8e26de415c714ba3646c451adebadea8254f0b2859db943654d31599d9c4e37d

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.3.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 12100c671f3fcbcb4fa42723b69e6df1cdeba16f71820e1fe91d4ac09a961e0e
MD5 be0f4c3f652cfb2b95b1bc0e37c6bbae
BLAKE2b-256 ff329e996130d3500b36e6dd20208eb6c35f9a2a4d5676bf2f457d74698c493e

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.3.0-cp37-cp37m-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.3.0-cp37-cp37m-win_amd64.whl
  • Upload date:
  • Size: 7.0 MB
  • Tags: CPython 3.7m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.12

File hashes

Hashes for arcticdb-4.3.0-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 701301d45071b23daff07314b6d2a13d8275704e0d37f58cc6c3e0397d1dc1c1
MD5 f3947a22202c1934ccca5e7f06c1892a
BLAKE2b-256 bca892889be3272157f32589f8d515316a983c5317f3576edd044ac05b2e81e6

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.3.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.3.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 c334815b64ada6b13296aa7e7a8f4a0c487a2f1079a8e2c62e8e7ed2a4d0dfe3
MD5 6e9f32d6ca195bf9822ef1e2166e9519
BLAKE2b-256 8a2a31528375cddad9cca116407742de92d515542dc8da9d470bb9805e3b8ba2

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.3.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.3.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 4f4cf5c3b20c434c08c334b7af017a8eab6c5ee0f9c4a30f295da8dfce9eaa6f
MD5 8c34b316bff56774f279be0a94e10502
BLAKE2b-256 07ab66bc74be9990ff20142654b87d00356e53e8a9c2214cb4ea328b6ef35f7f

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page