ArcticDB DataFrame Database

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Business Source License 1.1 (See LICENSE.txt)
  • Author: Man Alpha Technology
  • Provides-Extra: testing
Classifiers

Project description

ArcticDB Website | ArcticDB Blog | Press Release | Press Release | Community

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability

PyPI (Python 3.6 - 3.11) conda-forge (Python 3.8 - 3.11)
Linux (Intel/AMD) ✔️ ✔️
Windows (Intel/AMD) ✔️ ➖
MacOS ➖ Beta️

For conda-forge see the release-info.

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ➖

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs

Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from Man Group Operations Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact arcticdb@man.com.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at arcticdb@man.com.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at arcticdb@man.com or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Business Source License 1.1 (See LICENSE.txt)
  • Author: Man Alpha Technology
  • Provides-Extra: testing
Classifiers

Release history Release notifications | RSS feed

5.1.0rc0 pre-release

5.0.0

5.0.0rc1 pre-release

5.0.0rc0 pre-release

4.5.1

4.5.1rc4 pre-release

4.5.1rc3 pre-release

4.5.1rc0 pre-release

4.5.0

4.5.0rc4 pre-release

4.5.0rc3 pre-release

4.5.0rc2 pre-release

4.5.0rc1 pre-release

4.5.0rc0 pre-release

4.4.6

4.4.5

4.4.4

4.4.4rc6 pre-release

4.4.4rc5 pre-release

4.4.4rc4 pre-release

4.4.4rc3 pre-release

4.4.4rc2 pre-release

4.4.4rc1 pre-release

4.4.4rc0 pre-release

This version

4.4.3

4.4.3rc1 pre-release

4.4.3rc0 pre-release

4.4.2

4.4.2rc0 pre-release

4.4.1

4.4.0

4.4.0rc2 pre-release

4.4.0rc1 pre-release

4.3.2rc2 pre-release

4.3.1

4.3.0 yanked

Reason this release was yanked:

bad build

4.3.0rc2 pre-release

4.3.0rc1 pre-release

4.3.0rc0 pre-release

4.2.1

4.2.0

4.2.0rc0 pre-release

4.1.0

4.1.0rc2 pre-release

4.1.0rc1 pre-release

4.1.0rc0 pre-release

4.1.0.dev0 pre-release

4.0.5

4.0.4

4.0.3

4.0.2

4.0.2rc0 pre-release

4.0.1

4.0.0

4.0.0rc2 pre-release

4.0.0rc1 pre-release

4.0.0rc0 pre-release

3.0.0

3.0.0rc2 pre-release

3.0.0rc0 pre-release

2.0.0

2.0.0rc0 pre-release

1.6.2

1.6.2rc0 pre-release

1.6.1

1.6.1rc2 pre-release

1.6.1rc1 pre-release

1.6.1rc0 pre-release

1.6.0

1.5.0

1.4.1

1.4.0

1.3.0

1.2.1

1.2.0

1.1.0

1.0.1

1.0.0

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

arcticdb-4.4.3-cp311-cp311-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.11 Windows x86-64

arcticdb-4.4.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.3-cp310-cp310-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.10 Windows x86-64

arcticdb-4.4.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.3-cp39-cp39-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.9 Windows x86-64

arcticdb-4.4.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.3-cp38-cp38-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.8 Windows x86-64

arcticdb-4.4.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.3-cp37-cp37m-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.7m Windows x86-64

arcticdb-4.4.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.5 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

arcticdb-4.4.3-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.5 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64

File details

Details for the file arcticdb-4.4.3-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.3-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.3-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 d4e4e253dc053bcacc9e0bb4a3536ee42826cb2f2c8b7c3cd1e045c95a9dea58
MD5 f1454d8610f36c8c8016722f56f2dafc
BLAKE2b-256 2ecf567bf745513e9810db080f56e209bb6bb844066a1c91824b6130859d0822

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 2a080f5ed1c7a3b9d799698ffce650a0f20a53957400cdf0dd5b376731ffb2b3
MD5 258b9ba85bbfea85def8e5e68b64c5ec
BLAKE2b-256 d99ba178e2067956a2821d31350ce5ee0e796927702528c742d6531069452af6

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.3-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.3-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.3-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 c8a54b1642c1bad762d80863aba57a57a5e13dfab39c5bcec3f8c063f77cfe40
MD5 6e473690769a234e346a3041c2de812a
BLAKE2b-256 8fd4f34acfa4e34bb1edbe53f26e3b5ace3e625188637b4b930b205791709a68

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f8d21d18627cc7360f037ae05debc26be92bc7e707860f4910144ca7748cf0d2
MD5 7ff4af334a305e40fcb54fea22619632
BLAKE2b-256 73d53ba47d0fd5d87d8782f0a14594c3e635372994871e3cd1cd1d1f362f215a

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.3-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.3-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.3-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 008ff0939be7d804b2a57905e81536cabdb80a8f610c14676fcf00fefbd8596c
MD5 55c3da2fb287e17772f6747b2da6a2d4
BLAKE2b-256 b4ac7173ef479560afa813b60c2367544c58b19dd9fa3cc895112cd2747a0a1a

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d04ab7c8eef20649ea452aad27377c39fcf3ce049572cad5d00537f19cd39cf7
MD5 26bb377808618a0569b7a056e5ac407c
BLAKE2b-256 5dbd1282d2dc95b10aeab9f03922aa832d00260098337d1dc4f8105441b71e38

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.3-cp38-cp38-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.3-cp38-cp38-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.8, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.3-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 0ee4499cd7e4ceb928364714efc95fd9e1ea5786d98d1d98bb76876a6418776e
MD5 de61f034e3b9aee7928e8c3b1e300a03
BLAKE2b-256 d699e471e1f0407dae6ab44967e326a6d8b34b5dbdb63bb181dbabe58db78012

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 17db8b6107397fecd41d5521a2beb6909ea5c424e4bceaf76331dde806cf9504
MD5 7a19d63804fae2eab5331f0a932c2b96
BLAKE2b-256 a8e6e842e71bc797f5025244638df9db818301cea8b6e481ea3d296c58390f58

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.3-cp37-cp37m-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.3-cp37-cp37m-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.7m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.3-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 cbc088d326d2223efd9f5e828c15ec7308e84c8a8b3788e247d563064ebbcbd5
MD5 c3f389563df702259ccfa7027c5c78c9
BLAKE2b-256 e216643dc8d98cbfe3a68aabbb688bd3252a5f1d5415537f9be7e9496d850353

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 8646670e759c842e82f08d9a16144675c242c88c2971b6fc273cef3b4ade0135
MD5 2c15c1b08508f02b264754d377c632a6
BLAKE2b-256 b25b8ec664e72625c70dbc3e18b558bb3ee96c7caff7157bcb7a19762ff21798

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.3-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.3-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 cbe00fb1b7fa5764eee54a7501d7e0f22c24d1102dcffa9654c83ae2ae45920b
MD5 ce656666b66c1538dc1c99a5df423ce0
BLAKE2b-256 e5fb24235d62e7246a2cfd168609cf95071336c7b5500e22d7f59b88112ec046

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page