ArcticDB DataFrame Database

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Project description

ArcticDB Website | ArcticDB Blog | Press Release | Press Release | Community

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability

PyPI (Python 3.6 - 3.11) conda-forge (Python 3.8 - 3.11)
Linux ✔️ ✔️
Windows Beta ➖
MacOS (Apple Silicon) ➖ ✔️

Storage compatibility

PyPI conda-forge
S3 ✔️ ✔️
LMDB ✔️ ✔️
Azure Blob Storage ✔️ ✔️

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> from arcticdb import Arctic

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs

Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from Man Group Operations Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact arcticdb@man.com.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at arcticdb@man.com.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at arcticdb@man.com or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

5.1.0rc0 pre-release

5.0.0

5.0.0rc1 pre-release

5.0.0rc0 pre-release

4.5.1

4.5.1rc4 pre-release

4.5.1rc3 pre-release

4.5.1rc0 pre-release

4.5.0

4.5.0rc4 pre-release

4.5.0rc3 pre-release

4.5.0rc2 pre-release

4.5.0rc1 pre-release

4.5.0rc0 pre-release

4.4.6

4.4.5

4.4.4

4.4.4rc6 pre-release

4.4.4rc5 pre-release

4.4.4rc4 pre-release

4.4.4rc3 pre-release

4.4.4rc2 pre-release

4.4.4rc1 pre-release

4.4.4rc0 pre-release

4.4.3

4.4.3rc1 pre-release

4.4.3rc0 pre-release

4.4.2

4.4.2rc0 pre-release

4.4.1

4.4.0

4.4.0rc2 pre-release

4.4.0rc1 pre-release

4.3.2rc2 pre-release

4.3.1

4.3.0 yanked

Reason this release was yanked:

bad build

4.3.0rc2 pre-release

4.3.0rc1 pre-release

4.3.0rc0 pre-release

4.2.1

4.2.0

4.2.0rc0 pre-release

4.1.0

4.1.0rc2 pre-release

4.1.0rc1 pre-release

4.1.0rc0 pre-release

4.1.0.dev0 pre-release

4.0.5

This version

4.0.4

4.0.3

4.0.2

4.0.2rc0 pre-release

4.0.1

4.0.0

4.0.0rc2 pre-release

4.0.0rc1 pre-release

4.0.0rc0 pre-release

3.0.0

3.0.0rc2 pre-release

3.0.0rc0 pre-release

2.0.0

2.0.0rc0 pre-release

1.6.2

1.6.2rc0 pre-release

1.6.1

1.6.1rc2 pre-release

1.6.1rc1 pre-release

1.6.1rc0 pre-release

1.6.0

1.5.0

1.4.1

1.4.0

1.3.0

1.2.1

1.2.0

1.1.0

1.0.1

1.0.0

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

arcticdb-4.0.4-cp311-cp311-win_amd64.whl (6.6 MB view details)

Uploaded CPython 3.11 Windows x86-64

arcticdb-4.0.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.3 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

arcticdb-4.0.4-cp310-cp310-win_amd64.whl (6.7 MB view details)

Uploaded CPython 3.10 Windows x86-64

arcticdb-4.0.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.3 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

arcticdb-4.0.4-cp39-cp39-win_amd64.whl (6.6 MB view details)

Uploaded CPython 3.9 Windows x86-64

arcticdb-4.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.3 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

arcticdb-4.0.4-cp38-cp38-win_amd64.whl (6.7 MB view details)

Uploaded CPython 3.8 Windows x86-64

arcticdb-4.0.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.3 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

arcticdb-4.0.4-cp37-cp37m-win_amd64.whl (6.7 MB view details)

Uploaded CPython 3.7m Windows x86-64

arcticdb-4.0.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

arcticdb-4.0.4-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64

File details

Details for the file arcticdb-4.0.4-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.0.4-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 6.6 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.0.4-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 4f3baa3f6ce1765b73c501470563f63cbf5cfbd2001be8bb81e33563489c29dc
MD5 f7fc8bd0eb9e16d6ac0fa225e9c0214f
BLAKE2b-256 1e6609697d57de985f07ba2f9c301d5f7341251602c511a1c5d0a77d1a4f4567

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.0.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.0.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f00d89257e66a25c110b539d5388e706e9da85fc7a0094c6c6e0c1f12fe6ac30
MD5 ccfcc3d150f9396e59ec380578ec8cd0
BLAKE2b-256 3cb296fed4deef6ce52d09dea1bc0df183860c74e1b203e58581045f2bab57fa

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.0.4-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.0.4-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 6.7 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.0.4-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 09eede7f259ec10d186a87c7fc6f5d945ee474e2ebc8ee0a00d53fbc7d922cac
MD5 77d0a3f6c7222287af36411e86c91b19
BLAKE2b-256 589225c9c75e4250ccab6d84c0473e7cede7803e5d5baff740ffd21df0cd1fdd

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.0.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.0.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 6f5e597a2c85ab1796dc36b6bc7dc079baf22c2e3ab51fe9dc5726d6f921a867
MD5 e4800f9b996e17172697e806937ddd4d
BLAKE2b-256 c31e23235b9af94dc524c5d32e015ca426ca235840de695cbdcaa2eefabe95d3

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.0.4-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.0.4-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 6.6 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.0.4-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 2d588aacc4b035070d07ec093110520085c356d2479dba6b9887c6f3b0e21818
MD5 d0264d1138e6b1001750115390fff4b6
BLAKE2b-256 50395bd391b0ddd30f10a7199cf35ee0e5acae5306116440306819e58a7e9a7e

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d93e5167492fb43ed2411e22220ea9398c64a3e46bbc5cc0901231fa9c4baa12
MD5 13961752675965e380768fbff9b60fb9
BLAKE2b-256 04fe71afb178279cde00e5cf7af5704b4d59ce65be6629f364cae51ba231da9c

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.0.4-cp38-cp38-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.0.4-cp38-cp38-win_amd64.whl
  • Upload date:
  • Size: 6.7 MB
  • Tags: CPython 3.8, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.0.4-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 1e17371859d1fa6029cd28c1a059dcfbd3fd2c0fd4a3528882e8e0d0be55e722
MD5 9407bb606a2b0acfc2972a9ccdcfc107
BLAKE2b-256 1250ac3498d2be3d4ac026db76cdfba0d75236532b0f79f30cfceaa7c1ee0098

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.0.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.0.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 384effc692a391911f0295a316a63a4243af82fa20bb98758749f5ad5268760d
MD5 178c505d48129111a82adc53d4a879e0
BLAKE2b-256 d5064037f2a5e4b2f775fa8d3ae863c705c8c76520cf495c72cbfb52d9a7e6d8

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.0.4-cp37-cp37m-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.0.4-cp37-cp37m-win_amd64.whl
  • Upload date:
  • Size: 6.7 MB
  • Tags: CPython 3.7m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.0.4-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 569bb92374e0c086f18afc52e43dd69ccb6af7dbd930b9a5fb1ef7d2f7d74886
MD5 8ce0be58b42ec1c704456aaa8d7f0ff4
BLAKE2b-256 849e2d5133cb19519e6c8fe90492ac367339de45e5fd16297cfd18d464215e2c

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.0.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.0.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 2cea69003944eada3ee78981b4d30e953e02cb144db694f0f5c5d452ef067958
MD5 d3e227f15802ef27ef46d24cfe68bd21
BLAKE2b-256 879d91f886de71aeb73166e5ea0f2b9a9345f9bb14d11885e3d8eab9197f89c6

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.0.4-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.0.4-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 4a72bfad2c6e02685241e8ff09f2c44a9c77c3ec0a605050015c60d79038841e
MD5 3a1a0ad732f608021d872618ad60f79e
BLAKE2b-256 c2d9daceff540df415cce5be82a33a353a0972eefd5298228573417455a1673e

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page