ArcticDB DataFrame Database

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Project description

ArcticDB Website | ArcticDB Blog | Press Release | Press Release | Community

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability

PyPI (Python 3.6 - 3.11) conda-forge (Python 3.8 - 3.11)
Linux ✔️ ✔️
Windows Beta ➖
MacOS (Apple Silicon) ➖ ✔️

Storage compatibility

PyPI conda-forge
S3 ✔️ ✔️
LMDB ✔️ ✔️
Azure Blob Storage ✔️ ➖

Azure Blob Storage will be available on conda-forge as of 3.0.0.

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> from arcticdb import Arctic

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs

Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from Man Group Operations Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact arcticdb@man.com.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Aug 31, 2025

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at arcticdb@man.com.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at arcticdb@man.com or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

5.1.0rc0 pre-release

5.0.0

5.0.0rc1 pre-release

5.0.0rc0 pre-release

4.5.1

4.5.1rc4 pre-release

4.5.1rc3 pre-release

4.5.1rc0 pre-release

4.5.0

4.5.0rc4 pre-release

4.5.0rc3 pre-release

4.5.0rc2 pre-release

4.5.0rc1 pre-release

4.5.0rc0 pre-release

4.4.6

4.4.5

4.4.4

4.4.4rc6 pre-release

4.4.4rc5 pre-release

4.4.4rc4 pre-release

4.4.4rc3 pre-release

4.4.4rc2 pre-release

4.4.4rc1 pre-release

4.4.4rc0 pre-release

4.4.3

4.4.3rc1 pre-release

4.4.3rc0 pre-release

4.4.2

4.4.2rc0 pre-release

4.4.1

4.4.0

4.4.0rc2 pre-release

4.4.0rc1 pre-release

4.3.2rc2 pre-release

4.3.1

4.3.0 yanked

Reason this release was yanked:

bad build

4.3.0rc2 pre-release

4.3.0rc1 pre-release

4.3.0rc0 pre-release

4.2.1

4.2.0

4.2.0rc0 pre-release

4.1.0

4.1.0rc2 pre-release

4.1.0rc1 pre-release

4.1.0rc0 pre-release

4.1.0.dev0 pre-release

4.0.5

4.0.4

4.0.3

4.0.2

4.0.2rc0 pre-release

4.0.1

4.0.0

4.0.0rc2 pre-release

4.0.0rc1 pre-release

4.0.0rc0 pre-release

This version

3.0.0

3.0.0rc2 pre-release

3.0.0rc0 pre-release

2.0.0

2.0.0rc0 pre-release

1.6.2

1.6.2rc0 pre-release

1.6.1

1.6.1rc2 pre-release

1.6.1rc1 pre-release

1.6.1rc0 pre-release

1.6.0

1.5.0

1.4.1

1.4.0

1.3.0

1.2.1

1.2.0

1.1.0

1.0.1

1.0.0

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

arcticdb-3.0.0-cp311-cp311-win_amd64.whl (6.6 MB view details)

Uploaded CPython 3.11 Windows x86-64

arcticdb-3.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.2 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

arcticdb-3.0.0-cp310-cp310-win_amd64.whl (6.6 MB view details)

Uploaded CPython 3.10 Windows x86-64

arcticdb-3.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.2 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

arcticdb-3.0.0-cp39-cp39-win_amd64.whl (6.6 MB view details)

Uploaded CPython 3.9 Windows x86-64

arcticdb-3.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.2 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

arcticdb-3.0.0-cp38-cp38-win_amd64.whl (6.6 MB view details)

Uploaded CPython 3.8 Windows x86-64

arcticdb-3.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.2 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

arcticdb-3.0.0-cp37-cp37m-win_amd64.whl (6.6 MB view details)

Uploaded CPython 3.7m Windows x86-64

arcticdb-3.0.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.3 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

arcticdb-3.0.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.3 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64

File details

Details for the file arcticdb-3.0.0-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: arcticdb-3.0.0-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 6.6 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.12

File hashes

Hashes for arcticdb-3.0.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 c8ca304752b80fcf7534b4697bb99b94230989a12f48727acb799cbfa67e08a6
MD5 4dc2087fbd41da59ac47d5424f7dc32a
BLAKE2b-256 26f71cd0d763d68773b7445b93e8668d32786f3bbc281fe0cba8642d0bcfb7c0

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-3.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-3.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 e2703e869ecc56894d32799a5ebd8520caa12218b65d90833ffcf9bdf7b6103b
MD5 3e0efa5d93012c3038db3ecb814c73bf
BLAKE2b-256 19949af6226f131086ac91906a42c0987e85520faf429df37babcf91888739eb

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-3.0.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: arcticdb-3.0.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 6.6 MB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.12

File hashes

Hashes for arcticdb-3.0.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 08e87ba8efe945fc0b59196c8b4c7ce952f8be83bb4878441c6ce2660b5a5b3a
MD5 0c598c223439445cc7731e1519b34838
BLAKE2b-256 c230cc5500b4a5b8dbd892932b1f717c75b91f4230020f7fe106e04840ac60ef

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-3.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-3.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 2da1f5e5c4c3b7920200e21d80a97de4e1ce62ca02aa9bfe909ad85e2d1be89d
MD5 71942178d0d2d8b084ddbaec2fa4e37f
BLAKE2b-256 ca47430bf7a0d7d28f2de4e86ad57a4959359a4a0510b38f3771242f52fb571c

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-3.0.0-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-3.0.0-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 6.6 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.12

File hashes

Hashes for arcticdb-3.0.0-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 3fff76c80200488694cb7561ad380a24315dd438eb97885df699994078d89cbf
MD5 34e3b9d292b4f22ddecd5a675d8a7e29
BLAKE2b-256 760cc494132c7ce98701d3f510ab7543e51db54c580ebb8a31b056cd8545df29

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-3.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-3.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 46a38aa963c1b19bbb50791b104f1dc4e3f65922052598c1ff5bfd36ca547c05
MD5 e666a66b88f175ce2660cf2052ee9489
BLAKE2b-256 180d6c338fc4b014e205ff99bc4ba37d6662ebdbe142320a8226f21979e00695

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-3.0.0-cp38-cp38-win_amd64.whl.

File metadata

  • Download URL: arcticdb-3.0.0-cp38-cp38-win_amd64.whl
  • Upload date:
  • Size: 6.6 MB
  • Tags: CPython 3.8, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.12

File hashes

Hashes for arcticdb-3.0.0-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 3b5429c5f60a5cd1c8f7a4acf31e7d949efe20da18d5a8fb3d70aff9eb7aa5f0
MD5 527d9f413cf15a5542e32f76e40092ed
BLAKE2b-256 dc60fd658f3d3223dc9ed15fe2b2f4498bd5e6df0c97089f4f8a3b117f345e1c

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-3.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-3.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d8c02f90c9df5a567797ecae950e2edcf13623cde9d2bbc3fa081d49e32a6eaa
MD5 6a70f9d20ffe63d016ea90812cd9c9cf
BLAKE2b-256 b05805e70e839e774d7b9a2e61e0a19b0310fef9ea3d69adfcefe13b8fd4816e

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-3.0.0-cp37-cp37m-win_amd64.whl.

File metadata

  • Download URL: arcticdb-3.0.0-cp37-cp37m-win_amd64.whl
  • Upload date:
  • Size: 6.6 MB
  • Tags: CPython 3.7m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.12

File hashes

Hashes for arcticdb-3.0.0-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 c0f1fb2222eb0bd0ba9edb919f45ee403585d83d536ef906066dea1258ab5c82
MD5 c462a144cf5be57d879ff5090ea696a9
BLAKE2b-256 5029882fa3c77cf3d8835d1e842d0908024a02a1147800c9cd3bcd0593b29317

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-3.0.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-3.0.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 0b6daaaf9236a5b9de798d7b1c1849640e7b6170cfa278ac35fd1e8b9d3a27ee
MD5 c659b0d1a794e53dbb307179be6cc5ec
BLAKE2b-256 f7886e18d70c0c70a36745f5ea9f2f0b7e1491037ff4e9ba4771544bf14d17ed

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-3.0.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-3.0.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 529ad31199af399c098e0eee1307d64f3d5a1e67a64168d6dc1e19e33614878a
MD5 21cda0886dd347dc94b30f0464d97050
BLAKE2b-256 0a4c6a2acd7aa44b041bf7f25d139d16af098f9ba4d0524740ed45bf588c59b1

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page