Skip to main content

Kaitai Struct runtime for Awkward Arrays

Project description

Kaitai Struct: runtime library for Awkward

This library building Awkward Arrays using Kaitai Struct API for Awkward using C++/STL.

Steps

1. Write a .ksy file for your custom file format. Refer to the Kaitai User Guide for more details.

Here, we take an example of animal.ksy

meta:
  id: animal
  endian: le
  license: CC0-1.0
  ks-version: 0.8

seq:
  - id: entry
    type: animal_entry
    repeat: eos

types:
  animal_entry:
    seq:
      - id: str_len
        type: u1

      - id: species
        type: str
        size: str_len
        encoding: UTF-8

      - id: age
        type: u1

      - id: weight
        type: u2

2. Clone kaitai_struct_awkward_runtime repository:

git clone --recursive https://github.com/ManasviGoyal/kaitai_struct_awkward_runtime.git
cd kaitai_struct_awkward_runtime
git checkout ManasviGoyal/test

3. Update submodule and compile scala code (only the first time)

git submodule update --init
cd kaitai_struct_compiler
sbt package
cd ../

4. Generate the source and header files for Awkward target

./kaitai-struct-compiler -t awkward --outdir src-animal example_data/schemas/animal.ksy

5. Install the library

pip install .

6. Build awkward_kaitai by passing the path of the main .cpp from the generated code.

awkward-kaitai-build src-animal/animal.cpp -b build

Note:

awkward-kaitai-build [-h] [-d DEST] [-b BUILD] file

options:

  • -h, --help: shows help message
  • -d DEST, --dest DEST: explicitly specify a destination for the build shared library.
  • -b BUILD, --build BUILD: explicitly specify a build location.

7. Open python and print the returned ak.Array:

python
import awkward_kaitai

animal = awkward_kaitai.Reader("./src-animal/libanimal.so") # pass the path of the shared file
awkward_array = animal.load("example_data/data/animal.raw")

print(awkward_array.to_list())

Output

[{'animalA__Zentry': [{'animal_entryA__Zstr_len': 3, 'animal_entryA__Zspecies': 'cat', 'animal_entryA__Zage': 5, 'animal_entryA__Zweight': 12}, {'animal_entryA__Zstr_len': 3, 'animal_entryA__Zspecies': 'dog', 'animal_entryA__Zage': 3, 'animal_entryA__Zweight': 43}, {'animal_entryA__Zstr_len': 6, 'animal_entryA__Zspecies': 'turtle', 'animal_entryA__Zage': 10, 'animal_entryA__Zweight': 5}]}]

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

awkward_kaitai-0.1.0.tar.gz (1.1 MB view details)

Uploaded Source

Built Distribution

awkward_kaitai-0.1.0-py3-none-any.whl (9.8 kB view details)

Uploaded Python 3

File details

Details for the file awkward_kaitai-0.1.0.tar.gz.

File metadata

  • Download URL: awkward_kaitai-0.1.0.tar.gz
  • Upload date:
  • Size: 1.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.10.12

File hashes

Hashes for awkward_kaitai-0.1.0.tar.gz
Algorithm Hash digest
SHA256 617b3538b7eb5f94ac1ce5e636d60976b30548efc6d09fd6c39733c2ca2bb481
MD5 f43160ff8a81ee03028b4af2ec8f146f
BLAKE2b-256 d62e24d8b73d4ad0d8ea8c04831bf1688e8f778928d0083375e6ae373abcda59

See more details on using hashes here.

File details

Details for the file awkward_kaitai-0.1.0-py3-none-any.whl.

File metadata

File hashes

Hashes for awkward_kaitai-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 ff2a41e917f49b1009c949df67295b92abcfa7ffcff081ceef1c9e6d873a253a
MD5 e925e76293b3722bce04280d31f9705b
BLAKE2b-256 d0460a97256c71027f33063f026c02bdceac2293166706e68d0557e0b7f03754

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page