ArcticDB DataFrame Database

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Business Source License 1.1 (See LICENSE.txt)
  • Author: Man Alpha Technology
  • Provides-Extra: testing
Classifiers

Project description

ArcticDB Website | 📘 ArcticDB Docs | 📰: ArcticDB Blog | Press Release | Press Release | Community

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability

PyPI (Python 3.6 - 3.11) conda-forge (Python 3.8 - 3.11)
Linux (Intel/AMD) ✔️ ✔️
Windows (Intel/AMD) ✔️ ➖
MacOS ➖ Beta️

For conda-forge see the release-info.

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ➖

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs

Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from Man Group Operations Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact arcticdb@man.com.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026
4.4 Business Source License 1.1 Apr 5, 2026

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at arcticdb@man.com.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at arcticdb@man.com or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Business Source License 1.1 (See LICENSE.txt)
  • Author: Man Alpha Technology
  • Provides-Extra: testing
Classifiers

Release history Release notifications | RSS feed

5.1.0rc0 pre-release

5.0.0

5.0.0rc1 pre-release

5.0.0rc0 pre-release

This version

4.5.1

4.5.1rc4 pre-release

4.5.1rc3 pre-release

4.5.1rc0 pre-release

4.5.0

4.5.0rc4 pre-release

4.5.0rc3 pre-release

4.5.0rc2 pre-release

4.5.0rc1 pre-release

4.5.0rc0 pre-release

4.4.6

4.4.5

4.4.4

4.4.4rc6 pre-release

4.4.4rc5 pre-release

4.4.4rc4 pre-release

4.4.4rc3 pre-release

4.4.4rc2 pre-release

4.4.4rc1 pre-release

4.4.4rc0 pre-release

4.4.3

4.4.3rc1 pre-release

4.4.3rc0 pre-release

4.4.2

4.4.2rc0 pre-release

4.4.1

4.4.0

4.4.0rc2 pre-release

4.4.0rc1 pre-release

4.3.2rc2 pre-release

4.3.1

4.3.0 yanked

Reason this release was yanked:

bad build

4.3.0rc2 pre-release

4.3.0rc1 pre-release

4.3.0rc0 pre-release

4.2.1

4.2.0

4.2.0rc0 pre-release

4.1.0

4.1.0rc2 pre-release

4.1.0rc1 pre-release

4.1.0rc0 pre-release

4.1.0.dev0 pre-release

4.0.5

4.0.4

4.0.3

4.0.2

4.0.2rc0 pre-release

4.0.1

4.0.0

4.0.0rc2 pre-release

4.0.0rc1 pre-release

4.0.0rc0 pre-release

3.0.0

3.0.0rc2 pre-release

3.0.0rc0 pre-release

2.0.0

2.0.0rc0 pre-release

1.6.2

1.6.2rc0 pre-release

1.6.1

1.6.1rc2 pre-release

1.6.1rc1 pre-release

1.6.1rc0 pre-release

1.6.0

1.5.0

1.4.1

1.4.0

1.3.0

1.2.1

1.2.0

1.1.0

1.0.1

1.0.0

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

arcticdb-4.5.1-cp311-cp311-win_amd64.whl (6.8 MB view details)

Uploaded CPython 3.11 Windows x86-64

arcticdb-4.5.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.2 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

arcticdb-4.5.1-cp310-cp310-win_amd64.whl (6.8 MB view details)

Uploaded CPython 3.10 Windows x86-64

arcticdb-4.5.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.2 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

arcticdb-4.5.1-cp39-cp39-win_amd64.whl (6.8 MB view details)

Uploaded CPython 3.9 Windows x86-64

arcticdb-4.5.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.2 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

arcticdb-4.5.1-cp38-cp38-win_amd64.whl (6.8 MB view details)

Uploaded CPython 3.8 Windows x86-64

arcticdb-4.5.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.2 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

arcticdb-4.5.1-cp37-cp37m-win_amd64.whl (6.8 MB view details)

Uploaded CPython 3.7m Windows x86-64

arcticdb-4.5.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.3 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

arcticdb-4.5.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.3 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64

File details

Details for the file arcticdb-4.5.1-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.5.1-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 6.8 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-4.5.1-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 c24bf815406f206e1cf7236fc77bb12cd6071e5c7259b10e1aad878e4cfc4067
MD5 51c13d485541353f7ef5cb8a76a5165a
BLAKE2b-256 40e373f042caa13e7bdd3c35abc73c35ed25eb6c2665be39f06a4aa7ad94ba10

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.5.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 3d74d7ed43c46202005027ced3fc742cf4eb91b3192683a26e4fd6baccea915f
MD5 4b1b479c7c558ebe48d49e05170a5cb3
BLAKE2b-256 9e0de80b3bf904d44a6958c27a4b7e9de281488c8e65a72b57d6d85240fdcadd

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.1-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.5.1-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 6.8 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-4.5.1-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 ec67c97c43ef17db3eb4aeebfc810a48d9dd0937a1f6cfb9dd3153d737969204
MD5 df799600349289e37e7eb239b4eadbfa
BLAKE2b-256 1c7b94a2f2601f582035e8968b3f1549515366542d2950b399a94f2598ee7216

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.5.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d55afed48ff9600eb567cf86c5cf4564c4e181ed2f7573e3df4d542f051cd91d
MD5 31a664aadd58efc2d57e26d5e9bf19d5
BLAKE2b-256 ed58bd43919a9ce1fb8fc770ccf5f6b7d4aa59890e9e52ef6689d9f01778e048

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.1-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.5.1-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 6.8 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-4.5.1-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 2f204bc158f04972c6b2158d48663622bfef9031542b800e84d463105f54fa0e
MD5 08ae6cf69671721e9e8d6005fda24698
BLAKE2b-256 63a76b1bfbfb4bec4bb3a23d556ff284c5fc926994cffc3383d22a4513f0617c

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.5.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f9c746c49ba452615872ca35fc8f038a8166a29e056decc23b4c298717f80bc8
MD5 6d1e89ea71c81e670465b30ef6c5a9e0
BLAKE2b-256 90555d44c01035d018b1ebfa526ab439a7ae1ff71455f90d253e995407f57586

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.1-cp38-cp38-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.5.1-cp38-cp38-win_amd64.whl
  • Upload date:
  • Size: 6.8 MB
  • Tags: CPython 3.8, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-4.5.1-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 2627c7f9f3b4b43f9eb9c13fb1f474511a9e3eee3c2bf762793d0e95e7ed98f5
MD5 0ef6de4c50bd1c0e738fff2aefcfb7f3
BLAKE2b-256 e8593d6a54e8f74dd418338f0cfe202c104c0709b937f957e693bfa2b6ad8d9a

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.5.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 a184fb32fe1be42fa7d64dc90adbc5394b5f00213f397da6dcbc52fb69611786
MD5 0d8844d3b2e4d38abbc579b6a5772706
BLAKE2b-256 a096d79a5ad8d8876ebf27d2004f89dbd305de3d0854c861c0f0028861b78073

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.1-cp37-cp37m-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.5.1-cp37-cp37m-win_amd64.whl
  • Upload date:
  • Size: 6.8 MB
  • Tags: CPython 3.7m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-4.5.1-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 a2c0fba45f0ad4fc3cc3707aef860c371e8bf09b48108c3ed8cd83b0a1c5d391
MD5 3822521b6d54394e4f909e7745821f62
BLAKE2b-256 116399e69e8f3066b98acbb7aee8897c1df45b78ebe4f09e525379a0bb322069

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.5.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 53d5737ee8b002605910ef2102db75c078accc36e354e89d2c84eec364f1a808
MD5 dbf5305d963a529f30dbc9306bcaff44
BLAKE2b-256 220492fb41652a4fa2b019f31b46b6310696bee50564cec6b87b95d66a713678

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.5.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.5.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 1d343f9d58ffbc902b17659cb6ae818a281412c4c4c4114aaeb83f7309910247
MD5 17fe661f829fd03fe5600bb60fcecf64
BLAKE2b-256 ba64b7c9012339f61228e99ec91445bf78bf4a85a9aa464ee981adebd12fc225

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page