Orderbook

A fast L2/L3 orderbook data structure, in C, for Python

Basic Usage

from decimal import Decimal

import requests
from order_book import OrderBook

ob = OrderBook()

# get some orderbook data
data = requests.get("https://api.pro.coinbase.com/products/BTC-USD/book?level=2").json()

ob.bids = {Decimal(price): size for price, size, _ in data['bids']}
ob.asks = {Decimal(price): size for price, size, _ in data['asks']}

# OR

for side in data:
    # there is additional data we need to ignore
    if side in {'bids', 'asks'}:
        ob[side] = {Decimal(price): size for price, size, _ in data[side]}


# Data is accessible by .index(), which returns a tuple of (price, size) at that level in the book
price, size = ob.bids.index(0)
print(f"Best bid price: {price} size: {size}")

price, size = ob.asks.index(0)
print(f"Best ask price: {price} size: {size}")

print(f"The spread is {ob.asks.index(0)[0] - ob.bids.index(0)[0]}\n\n")

# Data is accessible via iteration
# Note: bids/asks are iterators

print("Bids")
for price in ob.bids:
    print(f"Price: {price} Size: {ob.bids[price]}")


print("\n\nAsks")
for price in ob.asks:
    print(f"Price: {price} Size: {ob.asks[price]}")


# Data can be exported to a sorted dictionary
# In Python3.7+ dictionaries remain in insertion ordering. The
# dict returned by .to_dict() has had its keys inserted in sorted order
print("\n\nRaw asks dictionary")
print(ob.asks.to_dict())

Main Features

• Sides maintained in correct order
• Can perform orderbook checksums
• Supports max depth and depth truncation

Installation

The preferable way to install is via pip - pip install order-book. Installing from source will require a compiler and can be done with setuptools: python setup.py install.

Running code coverage

The script coverage.sh will compile the source using the -coverage CFLAG, run the unit tests, and build a coverage report in HTML. The script uses tools that may need to be installed (coverage, lcov, genhtml).

Running the performance tests

You can run the performance tests like so: python perf/performance_test.py. The program will profile the time to run for random data samples of various sizes as well as the construction of a sorted orderbook using live L2 orderbook data from Coinbase.

The performance of constructing a sorted orderbook (using live data from Coinbase) using this C library, versus a pure Python sorted dictionary library:

The performance of constructing sorted dictionaries using the same libraries, as well as the cost of building unsorted, python dictionaies for dictionaries of random floating point data:

This represents a roughly 2x speedup compared to a pure python implementation, and in many cases is close to the performance of an unsorted python dictionary.

For other performance metrics, run performance_test.py as well as the other performance tests in perf/

Changelog

0.5.0 (2022-08-23)

• Bugfix: fix segmentation fault when calculating checksum on empty orderbook
• Bugfix: fix missing reference decrement
• Performance: Improvement to marking dirty keys

0.4.3 (2022-05-29)

• Bugfix: handle scientific notation of small values in Kraken checksum
• Update: calculate Kraken checksum on order books less than 10 levels deep
• Bugfix: fix occasional incorrect checksums for OKX, FTX and Bitget

0.4.2 (2022-04-17)

• Update: OKEx renamed OKX (for checksum validation)
• Feature: Add support for orderbook checksums with Bitget

0.4.1 (2021-10-12)

• Bugfix: unnecessary reference counting prevented sorted dictionaries from being deallocated
• Bugfix: setting ordering on a sorted dict before checking that it was created successfully

0.4.0 (2021-09-16)

• Feature: changes to code and setup.py to enable compiling on windows
• Feature: add from_type/to_type kwargs to the to_dict methods, allowing for type conversion when creating the dictionary

0.3.2 (2021-09-04)

• Bugfix: depth was incorrectly ignored when converting sorteddict to python dict

0.3.1 (2021-09-01)

• Bugfix: truncate and max_depth not being passed from orderbook to sorteddict object correctly
• Feature: let checksum_format kwarg be set to None

0.3.0 (2021-07-16)

• Update classifiers to indicate this projects only supports MacOS/Linux
• Bugfix: Using less than the minimum number of levels for a checksum with Kraken not raising error correctly
• Update: add del examples to test code

0.2.1 (2021-03-29)

• Bugfix: Invalid deallocation of python object

0.2.0 (2021-03-12)

• Feature: Add branch prediction hints around error handling code
• Bugfix: Fix regression from adding branch predictors
• Bugfix: Fix error corner case when iterating twice on an empty dataset
• Feature: Add contains function for membership test
• Bugfix: Fix issues around storing L3 data
• Feature: Enhance testing, add in L3 book test cases

0.1.1 (2021-02-12)

• Feature: Checksum support for orderbooks
• Feature: FTX checksum support
• Feature: Kraken checksum support
• Feature: OkEX/OKCoin checksum support
• Perf: Use CRC32 table to improve performance of checksum code

0.1.0 (2021-01-18)

• Minor: Use enums to make code more readable
• Bugfix: Add manifest file to ensure headers and changes file are included in sdist builds
• Feature: Add support for max depth and depth truncation

0.0.2 (2020-12-27)

• Bugfix: Fix sorted dictionary arg parsing
• Feature: Coverage report generation for C library
• Bugfix: Fix reference counting in index method in SortedDict
• Feature: New unit tests to improve SortedDict coverage
• Feature: Modularize files
• Feature: Add ability to set bids/asks to dictionaries via attributes or [ ]
• Docs: Update README with simple usage example

0.0.1 (2020-12-26)

• Initial Release