ArcticDB DataFrame Database

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Project description

ArcticDB Website | ArcticDB Blog | Press Release | Press Release | Community

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability

PyPI (Python 3.6 - 3.11) conda-forge (Python 3.8 - 3.11)
Linux (Intel/AMD) ✔️ ✔️
Windows (Intel/AMD) ✔️ ➖
MacOS ➖ Beta️

For conda-forge see the release-info.

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ➖

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs

Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from Man Group Operations Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact arcticdb@man.com.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at arcticdb@man.com.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at arcticdb@man.com or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

5.1.0rc0 pre-release

5.0.0

5.0.0rc1 pre-release

5.0.0rc0 pre-release

4.5.1

4.5.1rc4 pre-release

4.5.1rc3 pre-release

4.5.1rc0 pre-release

4.5.0

4.5.0rc4 pre-release

4.5.0rc3 pre-release

4.5.0rc2 pre-release

4.5.0rc1 pre-release

4.5.0rc0 pre-release

4.4.6

4.4.5

4.4.4

4.4.4rc6 pre-release

4.4.4rc5 pre-release

4.4.4rc4 pre-release

4.4.4rc3 pre-release

4.4.4rc2 pre-release

4.4.4rc1 pre-release

4.4.4rc0 pre-release

4.4.3

4.4.3rc1 pre-release

4.4.3rc0 pre-release

4.4.2

4.4.2rc0 pre-release

4.4.1

This version

4.4.0

4.4.0rc2 pre-release

4.4.0rc1 pre-release

4.3.2rc2 pre-release

4.3.1

4.3.0 yanked

Reason this release was yanked:

bad build

4.3.0rc2 pre-release

4.3.0rc1 pre-release

4.3.0rc0 pre-release

4.2.1

4.2.0

4.2.0rc0 pre-release

4.1.0

4.1.0rc2 pre-release

4.1.0rc1 pre-release

4.1.0rc0 pre-release

4.1.0.dev0 pre-release

4.0.5

4.0.4

4.0.3

4.0.2

4.0.2rc0 pre-release

4.0.1

4.0.0

4.0.0rc2 pre-release

4.0.0rc1 pre-release

4.0.0rc0 pre-release

3.0.0

3.0.0rc2 pre-release

3.0.0rc0 pre-release

2.0.0

2.0.0rc0 pre-release

1.6.2

1.6.2rc0 pre-release

1.6.1

1.6.1rc2 pre-release

1.6.1rc1 pre-release

1.6.1rc0 pre-release

1.6.0

1.5.0

1.4.1

1.4.0

1.3.0

1.2.1

1.2.0

1.1.0

1.0.1

1.0.0

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

arcticdb-4.4.0-cp311-cp311-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.11 Windows x86-64

arcticdb-4.4.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.0-cp310-cp310-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.10 Windows x86-64

arcticdb-4.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.5 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.0-cp39-cp39-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.9 Windows x86-64

arcticdb-4.4.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.0-cp38-cp38-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.8 Windows x86-64

arcticdb-4.4.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.0-cp37-cp37m-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.7m Windows x86-64

arcticdb-4.4.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.5 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

arcticdb-4.4.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.5 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64

File details

Details for the file arcticdb-4.4.0-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 f486b6d275e0ff05b40a8f9306f32a917d7c65eb8803827b70801fe015212974
MD5 8bd110f88abe002ac3bff82f3c04c8c8
BLAKE2b-256 7f69c5eaa9f9c8b580a6c955fe85d2397fceb2873e25f942eb488b0a63db92f9

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 fd36acd407086a719b341ff6c6c426af60835241da3e71e70e353e971460de93
MD5 c139b70f02dbb0736d15bc38337e95f8
BLAKE2b-256 b0c218da472da48cb484f6d0465edf0c7dba1b4e3b38974042e15bc17e2b3293

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.0-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 a8e1bcfe416fdf4e0dcdfdd16509b0397b3348da6cacf98a49d6543f0f6fa47f
MD5 771cf3ffa295e0c9e65f0aa04d09d2d8
BLAKE2b-256 316280399cfa3a0f5889d6477eb641e13dfdad77c66c20dba2ca90e21a89aef0

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 028ab64d2c71e76175746fd737758080715c09c937ea66ffda05a900b950bcbe
MD5 6904e6935a1f516d13a754bfb6e9ef9e
BLAKE2b-256 87aaaeffed3a67fd02ad7e4a4f6c94271a1a07580e87ea48747d35131f0c8f17

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.0-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.0-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.0-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 9352031e6cbe104adcf56c9ed7872a798890c0b45e9b0c3de2c4bdb74da74e4a
MD5 74ba07897d7fc3497ee658c43896d9f2
BLAKE2b-256 62d01356468ab3231ef47e7316f81848fa9f1044a535200d567ecaea0dfbd6eb

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 68b0ae3ff43c7acddada5c316555880d3653676970b39c96477c630a9174acee
MD5 511aa63da20adc59ca0e9af8d7590f44
BLAKE2b-256 f4b90848a18a4a1b40a3b951ce4149fbaed562cbb50e0ad05d4c6f6594c2813e

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.0-cp38-cp38-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.0-cp38-cp38-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.8, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.0-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 76a7d0d6b0a5eefecb02ccb18bcfd79b3a4363edafe2db4387ac4f6f2f770223
MD5 09bdbf1ae3e17465782818f31ec94fb9
BLAKE2b-256 99d6e6ba15af250d40b9b0043a83caac7e968f161abb13a8d315ab66202ae8fb

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 e5b6aad2f47d1d6a672b58d46bd435b35cb4efe8da70d43a765a9936f1c62081
MD5 90524e4db83d3db420fddbfed2f6a899
BLAKE2b-256 8a997330a14adcacc3d466a005b8521c255452e6e89c8b48ba75e220e5eff69b

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.0-cp37-cp37m-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.0-cp37-cp37m-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.7m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.0-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 f14b88209a1256a7d79156d98143d8b969795fddc4e2c4bd53f749a90d6794ba
MD5 a9fcaaae5d97ddd9b2104837f27b72e2
BLAKE2b-256 8ea5505cb56e371732ec9d0d18fb40897ba1d898170b98970e7b971a6917dd05

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 5e0591fa41bf55b647741a8d57137f34366b3a23c0c3042cf9e19e42db851d76
MD5 660db5969519896b7a8f370398ba8b9c
BLAKE2b-256 3163810cabd19f73a7fa33f4c8161ecd2a9b035b7369aaa33410e81224fc5a0f

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 acc84da20117e86839a0b6eb1976e71b530b1fad971d297404b23f0bed1e9517
MD5 2114490e8484a4552c7394329223fb4a
BLAKE2b-256 933afa22e33e51cf162928755546a926812b1d0189a2685800b299dd06c92564

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page