ArcticDB DataFrame Database

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Business Source License 1.1 (See LICENSE.txt)
  • Author: Man Alpha Technology
  • Provides-Extra: testing
Classifiers

Project description

Three minute ArcticDB demo from PyQuantNews

Options data in ArcticDB

ArcticDB Website | 📘 ArcticDB Docs | 📰: ArcticDB Blog | Press Release | Press Release | Community

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability

PyPI (Python 3.6 - 3.11) conda-forge (Python 3.8 - 3.11)
Linux (Intel/AMD) ✔️ ✔️
Windows (Intel/AMD) ✔️ ➖
MacOS ➖ Beta️

For conda-forge see the release-info.

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ➖

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs

Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from Man Group Operations Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact arcticdb@man.com.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026
4.4 Business Source License 1.1 Apr 5, 2026
4.5 Business Source License 1.1 Aug 14, 2026

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at arcticdb@man.com.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at arcticdb@man.com or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Business Source License 1.1 (See LICENSE.txt)
  • Author: Man Alpha Technology
  • Provides-Extra: testing
Classifiers

Release history Release notifications | RSS feed

5.1.0rc0 pre-release

This version

5.0.0

5.0.0rc1 pre-release

5.0.0rc0 pre-release

4.5.1

4.5.1rc4 pre-release

4.5.1rc3 pre-release

4.5.1rc0 pre-release

4.5.0

4.5.0rc4 pre-release

4.5.0rc3 pre-release

4.5.0rc2 pre-release

4.5.0rc1 pre-release

4.5.0rc0 pre-release

4.4.6

4.4.5

4.4.4

4.4.4rc6 pre-release

4.4.4rc5 pre-release

4.4.4rc4 pre-release

4.4.4rc3 pre-release

4.4.4rc2 pre-release

4.4.4rc1 pre-release

4.4.4rc0 pre-release

4.4.3

4.4.3rc1 pre-release

4.4.3rc0 pre-release

4.4.2

4.4.2rc0 pre-release

4.4.1

4.4.0

4.4.0rc2 pre-release

4.4.0rc1 pre-release

4.3.2rc2 pre-release

4.3.1

4.3.0 yanked

Reason this release was yanked:

bad build

4.3.0rc2 pre-release

4.3.0rc1 pre-release

4.3.0rc0 pre-release

4.2.1

4.2.0

4.2.0rc0 pre-release

4.1.0

4.1.0rc2 pre-release

4.1.0rc1 pre-release

4.1.0rc0 pre-release

4.1.0.dev0 pre-release

4.0.5

4.0.4

4.0.3

4.0.2

4.0.2rc0 pre-release

4.0.1

4.0.0

4.0.0rc2 pre-release

4.0.0rc1 pre-release

4.0.0rc0 pre-release

3.0.0

3.0.0rc2 pre-release

3.0.0rc0 pre-release

2.0.0

2.0.0rc0 pre-release

1.6.2

1.6.2rc0 pre-release

1.6.1

1.6.1rc2 pre-release

1.6.1rc1 pre-release

1.6.1rc0 pre-release

1.6.0

1.5.0

1.4.1

1.4.0

1.3.0

1.2.1

1.2.0

1.1.0

1.0.1

1.0.0

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

arcticdb-5.0.0-cp311-cp311-win_amd64.whl (7.0 MB view details)

Uploaded CPython 3.11 Windows x86-64

arcticdb-5.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.6 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

arcticdb-5.0.0-cp310-cp310-win_amd64.whl (7.0 MB view details)

Uploaded CPython 3.10 Windows x86-64

arcticdb-5.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.7 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

arcticdb-5.0.0-cp39-cp39-win_amd64.whl (7.0 MB view details)

Uploaded CPython 3.9 Windows x86-64

arcticdb-5.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.7 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

arcticdb-5.0.0-cp38-cp38-win_amd64.whl (7.0 MB view details)

Uploaded CPython 3.8 Windows x86-64

arcticdb-5.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.7 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

arcticdb-5.0.0-cp37-cp37m-win_amd64.whl (7.0 MB view details)

Uploaded CPython 3.7m Windows x86-64

arcticdb-5.0.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

arcticdb-5.0.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.7 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64

File details

Details for the file arcticdb-5.0.0-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-5.0.0-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 7.0 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-5.0.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 58ece8d978d4a1fb7f612780f9755643a914181fb83058a93a59942a983a5669
MD5 6fd31a329729db1e5a83ce60e9421136
BLAKE2b-256 e8dc0d14631cf5147f2308aca98fee8537f787b0ce1caad8240d5e72dd389f09

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-5.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-5.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 2b153b96135db1b4d2254c2e54720286e9a06f5fc24a5172c4a25491e85215b5
MD5 0e738be942b86746a9905487e05ed0d9
BLAKE2b-256 e4cba799aeae1be68f2c911819d889584758b7ce1f0cd135a1d4c65dd08fa8e0

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-5.0.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-5.0.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 7.0 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-5.0.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 fb262161db73cf283ddfee94bd35d9a7fd949a6c90181c196ae97ff09adbb630
MD5 669f134b1a927d9ebed6809f83feee8f
BLAKE2b-256 2e0fe9b2c8cc1a197b1a689a5393aba9011f8560b9157ad34ea1e2f3028a7274

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-5.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-5.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 4e3cd9b9438e5c37b246b175eb7a3828a40cf0c6d529d4f78e1a33bb2c930407
MD5 6148a70c6ec361cc95f67d6955086a3d
BLAKE2b-256 38fa6dcf72c322f72eb422cfc0b20e433057636de5b8a418374cfc2edb1b96ca

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-5.0.0-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-5.0.0-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 7.0 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-5.0.0-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 392baa0d312bc2b370fed9eee7122dabe5095f593eb4b0dabe45ab130015569b
MD5 8456a2fdc0638cea60eb0e70e65e286f
BLAKE2b-256 865ad886370a09220ed4c7f459db151fd76725de10153bf3f3adf6d33aa0ab9f

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-5.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-5.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 ce0f807e78ea1bc287f575d85bba50fcde9f5545f501e42d6a48855e057af0da
MD5 a56d4c0da8413566d4203f5c497753ac
BLAKE2b-256 c8cc8bdc9fdec5f1d418b1a78bbc3f7d26258e22db3538887b9559038f38ae2e

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-5.0.0-cp38-cp38-win_amd64.whl.

File metadata

  • Download URL: arcticdb-5.0.0-cp38-cp38-win_amd64.whl
  • Upload date:
  • Size: 7.0 MB
  • Tags: CPython 3.8, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-5.0.0-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 6635a2189f29da6a0d3af8cc9f63f139ee4690994cfeb159208eeee2bb9e51f1
MD5 f5cf93ba997a6cc9b4413a7a2e018d23
BLAKE2b-256 827f9850e6927f52f3bd18582dff06ccee9bbb93b618259489629b86c1a4d370

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-5.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-5.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 68d2f7927a3fb364c5e50e289a7b41cece6decb1106a735a1aa60caf624e590d
MD5 07956bf9481d7f2b25ccefe04dd7f60e
BLAKE2b-256 d0e5ea3da71677353e708b1a59109726b6d0f8a6f7a25931b96f3148d3905e79

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-5.0.0-cp37-cp37m-win_amd64.whl.

File metadata

  • Download URL: arcticdb-5.0.0-cp37-cp37m-win_amd64.whl
  • Upload date:
  • Size: 7.0 MB
  • Tags: CPython 3.7m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-5.0.0-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 a23c68aa1b403a4269dd1b28de4b9618d8f2cbf84185d6fc91b4217617528771
MD5 fcc21ac9d8f31847ca9388bb3dbf7982
BLAKE2b-256 0d214c35023f064eb2a4bfe58014a8621b6319994226c530e0fc1ba18390c0c1

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-5.0.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-5.0.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 40c605b1adadf1dc643aea611baea5f7970b0c10041d686db101d7477dea03ad
MD5 88a56a8aca2ac3b19c3bce22abc0a3b4
BLAKE2b-256 0f7f84b518836e1273cff7ec5e7b06e7bc2d81cebbc5bb29e6ce1acde0249c84

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-5.0.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-5.0.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 1f0f2bf1fb6c79ae6ed65777d9a84bb930b89620e365b43c65920e0d64478afa
MD5 fd2115780eb54e6da2605eb2118af60d
BLAKE2b-256 b89e3ff5982d9ce69903121df5c91d33c7b93e650b4b7b8431b9cd421d807bbb

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page