ArcticDB DataFrame Database

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Project description

ArcticDB Website | ArcticDB Blog | Press Release | Press Release | Community

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability

PyPI (Python 3.6 - 3.11) conda-forge (Python 3.8 - 3.11)
Linux (Intel/AMD) ✔️ ✔️
Windows (Intel/AMD) ✔️ ➖
MacOS ➖ Beta️

For conda-forge see the release-info.

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ➖

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs

Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from Man Group Operations Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact arcticdb@man.com.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at arcticdb@man.com.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at arcticdb@man.com or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

5.1.0rc0 pre-release

5.0.0

5.0.0rc1 pre-release

5.0.0rc0 pre-release

4.5.1

4.5.1rc4 pre-release

4.5.1rc3 pre-release

4.5.1rc0 pre-release

4.5.0

4.5.0rc4 pre-release

4.5.0rc3 pre-release

4.5.0rc2 pre-release

4.5.0rc1 pre-release

4.5.0rc0 pre-release

4.4.6

4.4.5

4.4.4

4.4.4rc6 pre-release

4.4.4rc5 pre-release

4.4.4rc4 pre-release

4.4.4rc3 pre-release

4.4.4rc2 pre-release

4.4.4rc1 pre-release

4.4.4rc0 pre-release

4.4.3

4.4.3rc1 pre-release

4.4.3rc0 pre-release

4.4.2

4.4.2rc0 pre-release

This version

4.4.1

4.4.0

4.4.0rc2 pre-release

4.4.0rc1 pre-release

4.3.2rc2 pre-release

4.3.1

4.3.0 yanked

Reason this release was yanked:

bad build

4.3.0rc2 pre-release

4.3.0rc1 pre-release

4.3.0rc0 pre-release

4.2.1

4.2.0

4.2.0rc0 pre-release

4.1.0

4.1.0rc2 pre-release

4.1.0rc1 pre-release

4.1.0rc0 pre-release

4.1.0.dev0 pre-release

4.0.5

4.0.4

4.0.3

4.0.2

4.0.2rc0 pre-release

4.0.1

4.0.0

4.0.0rc2 pre-release

4.0.0rc1 pre-release

4.0.0rc0 pre-release

3.0.0

3.0.0rc2 pre-release

3.0.0rc0 pre-release

2.0.0

2.0.0rc0 pre-release

1.6.2

1.6.2rc0 pre-release

1.6.1

1.6.1rc2 pre-release

1.6.1rc1 pre-release

1.6.1rc0 pre-release

1.6.0

1.5.0

1.4.1

1.4.0

1.3.0

1.2.1

1.2.0

1.1.0

1.0.1

1.0.0

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

arcticdb-4.4.1-cp311-cp311-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.11 Windows x86-64

arcticdb-4.4.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.1-cp310-cp310-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.10 Windows x86-64

arcticdb-4.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.5 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.1-cp39-cp39-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.9 Windows x86-64

arcticdb-4.4.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.1-cp38-cp38-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.8 Windows x86-64

arcticdb-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.1-cp37-cp37m-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.7m Windows x86-64

arcticdb-4.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.5 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

arcticdb-4.4.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.5 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64

File details

Details for the file arcticdb-4.4.1-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.1-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.1-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 8686fddcbe75b3de08a70824fbe8eec7013b6336675ca70d6ca42be643aac1a3
MD5 3c199f699659e09fa56097087f245043
BLAKE2b-256 6e704e58f1eb5b0950655f015fec52643a89d0a393a1be2f875447da2cc18bc5

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 7cae6efea1db3f96ce316d09ad07f633201eeea7bff8dc3070014df557f89053
MD5 0f9c8dd73d7c4556ca9fef3cdbd9159f
BLAKE2b-256 0e2d6a903b937e917ba51fb4fc5e2685681571e1d6f03794ead064ca73358c25

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.1-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.1-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.1-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 1c20b89ec4988f5bc938e5b2cbbf90194abd36e7d3969bb2a6d400d6e6eea2c4
MD5 417b5d31211053a2d423281f1bf7bebe
BLAKE2b-256 43f5dba4e8e0c9a27ec2ca6f8568d4ab6735d2b6090b8a6f0efba33f798536d1

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 8226b61aebcfcad05ea6021709ca1326c7d3a96f3c97155a58538f909eeadfa3
MD5 d6d23faefd19cbaacc4d1af6db974bd0
BLAKE2b-256 d32f0867348b12a436e3289c7e231ce7639aa06ecca02639303291e86e7005f8

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.1-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.1-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.1-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 d66fb33c4fcc0d2bf89978dc420b493ab7a0524136ece9ca25dac61b5045184f
MD5 2917542f236c2a80150a66da9aa583d8
BLAKE2b-256 dff035bfdab59bcc301422b15bbc68608113f8a62c4826cc3ba30d07f08959de

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 97c20c6eff415e401d85fc1e75a21141e2cc8ddd607ae09e8775fe58c7b77250
MD5 b43cb090d3da0f679aa61832e76f5cd7
BLAKE2b-256 5520e2c86a67304e2d9e29b4d70fdfdb1f1cb8a0f792b5414b1165c73351ae44

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.1-cp38-cp38-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.1-cp38-cp38-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.8, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.1-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 04e1d93d5f246ce26708d8d71cbd4af3a59343546919c47d95f88d026e2b0121
MD5 66990b2eb1ecbf921374e52565e537be
BLAKE2b-256 8ac2d75ac05a0b914d64c6af20b19632f9f7f713b552c21258574b51835d1659

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 56363f011178b3f65852ab1d6a9ed2abac6f4acb01f91937721c13b40348c5bf
MD5 06e8941f9bd3509fe2439d3242fa489c
BLAKE2b-256 1a1776fc3fc787aaaa81dd8d38fedb908503ccf37c2ca1f5ca417d2b28b11b1d

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.1-cp37-cp37m-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.1-cp37-cp37m-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.7m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.1-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 1100a0fb7f6297dc38ba31a72969de39a897f737e19610b5358768ead8ddcb81
MD5 60ef9be77c556213e881e85861f6c785
BLAKE2b-256 69e0ee9b8be55221dadd033b89128327753156549bc98296d882f4df255e1b60

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 b4ed2d7a311491fefb5825683a64280a91452533c3da579e28765eb5d316e6c4
MD5 dab0f9753665851fd97c1f6e1174a3c3
BLAKE2b-256 c64d0270b3aa868d40c5325a7a907d0c19317826ba1961c8f5037cca4a0dd671

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d1a819beb80eea305584e8c15dc35d8284134a4bf51cc53bbe4080c6a44e4314
MD5 ef86c5ecd81ba49d6df3ce8f407f1926
BLAKE2b-256 c5fe67beed23a7f15a05173393f1d2eb19a6d59516449d39cf3e786f59fe9290

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page