ArcticDB DataFrame Database

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Business Source License 1.1 (See LICENSE.txt)
  • Author: Man Alpha Technology
  • Provides-Extra: testing
Classifiers

Project description

ArcticDB Website | ArcticDB Blog | Press Release | Press Release | Community

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem. Launched in March 2023, it is the successor to Arctic.

ArcticDB offers an intuitive Python-centric API enabling you to read and write Pandas DataFrames to S3 or LMDB utilising a fast C++ data-processing and compression engine.

ArcticDB allows you to:

  • Pandas in, Pandas out: Read and write Pandas DataFrames, NumPy arrays and native types to S3 and LMDB without leaving Python.
  • Built for time-series data: Efficiently index and query time-series data across billions of rows
  • Time travel: Travel back in time to see previous versions of your data and create customizable snapshots of the database
  • Schemaless Database: Append, update and modify data without being constrained by the existing schema
  • Optimised for streaming data: Built in support for efficient sparse data storage
  • Powerful processing: Filter, aggregate and create new columns on-the-fly with a Pandas-like syntax
  • C++ efficiency: Accelerate analytics though concurrency in the C++ data-processing engine

ArcticDB handles data that is big in both row count and column count, so a 20-year history of more than 400,000 unique securities can be stored in a single symbol. Each symbol is maintained as a separate entity with no shared data which means ArcticDB can scale horizontally across symbols, maximising the performance potential of your compute, storage and network.

ArcticDB is designed from the outset to be resilient; there is no single point of failure, and persistent data structures in the storage mean that once a version of a symbol has been written, it can never be corrupted by subsequent updates. Pulling compressed data directly from storage to the client means that there is no server to overload, so your data is always available when you need it.

Quickstart

Prebuilt binary availability

PyPI (Python 3.6 - 3.11) conda-forge (Python 3.8 - 3.11)
Linux (Intel/AMD) ✔️ ✔️
Windows (Intel/AMD) ✔️ ➖
MacOS ➖ Beta️

For conda-forge see the release-info.

Storage compatibility

Linux Windows Mac
S3 ✔️ ✔️ ✔️
LMDB ✔️ ✔️ ✔️
Azure Blob Storage ✔️ ✔️ ➖

We have tested against the following S3 backends:

  • AWS S3
  • Ceph
  • MinIO on Linux
  • Pure Storage S3
  • Scality S3
  • VAST Data S3

Installation

Install ArcticDB:

$ pip install arcticdb

or using conda-forge

$ conda install -c conda-forge arcticdb

Import ArcticDB:

>>> import arcticdb as adb

Create an instance on your S3 storage (with or without explicit credentials):

# Leave AWS to derive credential information
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?aws_auth=true')

# Manually specify creds
>>> ac = adb.Arctic('s3://MY_ENDPOINT:MY_BUCKET?region=YOUR_REGION&access=ABCD&secret=DCBA')

Or create an instance on your local disk:

>>> ac = adb.Arctic("lmdb:///<path>")

Create your first library and list the libraries in the instance:

>>> ac.create_library('travel_data')
>>> ac.list_libraries()

Create a test dataframe:

>>> import numpy as np
>>> import pandas as pd
>>> NUM_COLUMNS=10
>>> NUM_ROWS=100_000
>>> df = pd.DataFrame(np.random.randint(0,100,size=(NUM_ROWS, NUM_COLUMNS)), columns=[f"COL_{i}" for i in range(NUM_COLUMNS)], index=pd.date_range('2000', periods=NUM_ROWS, freq='h'))

Get the library, write some data to it, and read it back:

>>> lib = ac['travel_data']
>>> lib.write("my_data", df)
>>> data = lib.read("my_data")

To find out more about working with data, visit our docs

Documentation

The source code for the ArcticDB docs are located in the docs folder, and are hosted at docs.arcticdb.io.

License

ArcticDB is released under a Business Source License 1.1 (BSL)

BSL features are free to use and the source code is available, but users may not use ArcticDB for production use or for a Database Service, without agreement with Man Group Operations Limited.

Use of ArcticDB in production or for a Database Service requires a paid for license from Man Group Operations Limited and is licensed under the ArcticDB Software License Agreement. For more information please contact arcticdb@man.com.

The BSL is not certified as an open-source license, but most of the Open Source Initiative (OSI) criteria are met. Please see version conversion dates in the below table:

ArcticDB Version License Converts to Apache 2.0
1.0 Business Source License 1.1 Mar 16, 2025
1.2 Business Source License 1.1 May 22, 2025
1.3 Business Source License 1.1 Jun 9, 2025
1.4 Business Source License 1.1 Jun 23, 2025
1.5 Business Source License 1.1 Jul 11, 2025
1.6 Business Source License 1.1 Jul 25, 2025
2.0 Business Source License 1.1 Aug 29, 2025
3.0 Business Source License 1.1 Sep 13, 2025
4.0 Business Source License 1.1 Sep 27, 2025
4.1 Business Source License 1.1 Nov 1, 2025
4.2 Business Source License 1.1 Nov 12, 2025
4.3 Business Source License 1.1 Feb 7, 2026

Code of Conduct

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behaviour that you have experienced in the project, please contact us at arcticdb@man.com.

Contributing/Building From Source

We welcome your contributions to help us improve and extend this project!

Please refer to the Contributing page and feel free to open issues on GitHub.

We are also always looking for feedback from our dedicated community! If you have used ArcticDB please let us know, we would love to hear about your experience!

Our release process is documented here.

Community

We would love to hear how your ArcticDB journey evolves, email us at arcticdb@man.com or come chat to us on Twitter!

Interested in learning more about ArcticDB? Head over to our blog!

Do you have any questions or issues? Chat to us and other users through our dedicated Slack Workspace - sign up for Slack access on our website.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ahldev from gravatar.com ahldev Avatar for alexbseaton from gravatar.com alexbseaton Avatar for alexowens90 from gravatar.com alexowens90 Avatar for arcticdb from gravatar.com arcticdb Avatar for jamesmunro from gravatar.com jamesmunro Avatar for williamdealtry from gravatar.com williamdealtry

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Business Source License 1.1 (See LICENSE.txt)
  • Author: Man Alpha Technology
  • Provides-Extra: testing
Classifiers

Release history Release notifications | RSS feed

5.1.0rc0 pre-release

5.0.0

5.0.0rc1 pre-release

5.0.0rc0 pre-release

4.5.1

4.5.1rc4 pre-release

4.5.1rc3 pre-release

4.5.1rc0 pre-release

4.5.0

4.5.0rc4 pre-release

4.5.0rc3 pre-release

4.5.0rc2 pre-release

4.5.0rc1 pre-release

4.5.0rc0 pre-release

4.4.6

4.4.5

This version

4.4.4

4.4.4rc6 pre-release

4.4.4rc5 pre-release

4.4.4rc4 pre-release

4.4.4rc3 pre-release

4.4.4rc2 pre-release

4.4.4rc1 pre-release

4.4.4rc0 pre-release

4.4.3

4.4.3rc1 pre-release

4.4.3rc0 pre-release

4.4.2

4.4.2rc0 pre-release

4.4.1

4.4.0

4.4.0rc2 pre-release

4.4.0rc1 pre-release

4.3.2rc2 pre-release

4.3.1

4.3.0 yanked

Reason this release was yanked:

bad build

4.3.0rc2 pre-release

4.3.0rc1 pre-release

4.3.0rc0 pre-release

4.2.1

4.2.0

4.2.0rc0 pre-release

4.1.0

4.1.0rc2 pre-release

4.1.0rc1 pre-release

4.1.0rc0 pre-release

4.1.0.dev0 pre-release

4.0.5

4.0.4

4.0.3

4.0.2

4.0.2rc0 pre-release

4.0.1

4.0.0

4.0.0rc2 pre-release

4.0.0rc1 pre-release

4.0.0rc0 pre-release

3.0.0

3.0.0rc2 pre-release

3.0.0rc0 pre-release

2.0.0

2.0.0rc0 pre-release

1.6.2

1.6.2rc0 pre-release

1.6.1

1.6.1rc2 pre-release

1.6.1rc1 pre-release

1.6.1rc0 pre-release

1.6.0

1.5.0

1.4.1

1.4.0

1.3.0

1.2.1

1.2.0

1.1.0

1.0.1

1.0.0

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

arcticdb-4.4.4-cp311-cp311-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.11 Windows x86-64

arcticdb-4.4.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.4-cp310-cp310-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.10 Windows x86-64

arcticdb-4.4.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.4-cp39-cp39-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.9 Windows x86-64

arcticdb-4.4.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.4-cp38-cp38-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.8 Windows x86-64

arcticdb-4.4.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

arcticdb-4.4.4-cp37-cp37m-win_amd64.whl (6.3 MB view details)

Uploaded CPython 3.7m Windows x86-64

arcticdb-4.4.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.5 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

arcticdb-4.4.4-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.5 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64

File details

Details for the file arcticdb-4.4.4-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.4-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 eba19f2c81fdee39cf92980a6aec6feaafc9e0daf7566c529beea9027ba03eaa
MD5 b313bc7318c50580b55bbe152e323551
BLAKE2b-256 9f85764bd6bd77e8fb93d60e4663cc2f238f363771fe7df808fd974c611c13fa

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 6c0d11abdbebf21bd7b9aa968952776f02ad94f032ec8f02843b59d513f13dbc
MD5 b0299165b551d5d5a726d9e3417e062a
BLAKE2b-256 9d72b2f931a9eec1b5f789102f97dc7d20e5ca96e52941410505a966a8e6ffe3

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.4-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.4-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 9db1e042b3f116e5d0c1f5f50a93bba4807936f205bbd183c832e820eff00373
MD5 bf11fe7177e813a975cfd456d3e9e80f
BLAKE2b-256 055ccdc229db389469e11e816ba7575f60c9b56ce76e775e7f00a3a48879f575

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 a280409febbcac30428bb54b7a6bdb657f769aa0228096f657a320cea868fa07
MD5 3e1c199372df4d002196b739f19ef2a1
BLAKE2b-256 2798519d33d410e991fa43a3ee38eb8d769a1f1fb395cf5316c02366e41360e0

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.4-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.4-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.4-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 9f6e7783e51c3871a795c74bcbc5bf1e6df798b277ee455fd36055ea666703b5
MD5 850c0c2f97546bd54204c0e0bedddaa5
BLAKE2b-256 5897ed3a24e70eb4b05ca8c590432ce21a701c63e7a1c50f001156e81cfee33c

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 1b71bc64c0db9a39137099055eaf16c7386455fa16947e6212077d85d45a5ec6
MD5 6c45a201122bfa778ecc01a404f9656a
BLAKE2b-256 c7763b698028be5ac635bc586b406604d92329c792315c0f15b09fa71d3f8fa7

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.4-cp38-cp38-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.4-cp38-cp38-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.8, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.4-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 b83d198dde21c5a7d6d4969c696d1874d4eb833b136a331c628d0a5381c9a24a
MD5 feb061774beab8e982dcd710b7a2fc44
BLAKE2b-256 214580bab246fd544d3621d4295c6eedf9954acf7183a1c462251c882ae9f7e0

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 90257cf71e2ac94a1e28de357dda51dd9641255d1878e4811e7826192933b3c8
MD5 99d1e4cf08221c67a2923577a88b5042
BLAKE2b-256 7801ded8d657d98cc8436894d5925fcd94ded23cb1ce5e6ef9216c4237ef4fff

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.4-cp37-cp37m-win_amd64.whl.

File metadata

  • Download URL: arcticdb-4.4.4-cp37-cp37m-win_amd64.whl
  • Upload date:
  • Size: 6.3 MB
  • Tags: CPython 3.7m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.12

File hashes

Hashes for arcticdb-4.4.4-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 39abb467929e8dad34fecd6991fb87637ba5bcdf7cc10cf73838ce37e4de09e3
MD5 c7ff7baa7bdf153ca592410a25fc10fb
BLAKE2b-256 3348f1d330311e6cdef10c076646267d81d68dc8a413cb089718de78e9bc963f

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 6a5d16235250f41c09624973cad3b6e409ada90375a2cbe58d97ed8fc896b782
MD5 1a3403f06a3009e53031e45f3cc9b1f1
BLAKE2b-256 629e59a1f611a497e2ad2bc31e8cb455753937fccb7e72c8b3c8c1361606e2ac

See more details on using hashes here.

Provenance

File details

Details for the file arcticdb-4.4.4-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arcticdb-4.4.4-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 102a08403220780fce1cf2cbfe4419f0b4f1d272315623f26bea9a8e6fbbead9
MD5 0e608ad7bab66e4feff82b148d0eb3a3
BLAKE2b-256 98b52d1d33e97945f17a65372eb141b46456d29f40eac2890e83a17e0135076a

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page