Skip to main content

TileDB Cloud platform Python client

Project description

TileDB-Cloud-Py

This repository contains the Python client for the TileDB Cloud Service.

Quick Links

Installation

See Installation Instructions

Quickstart

See Quickstart section of the docs.

API Documentation

See https://tiledb-inc.github.io/TileDB-Cloud-Py/

Contribute to documentation

Documentation uses Quarto style documentation.

Testing

  • Selection:
    • All tests: pytest
    • All tests with verbosity: pytest -vs
    • Tests with taskgraphs in their names: pytest -vsk taskgraphs
  • Credentials:
    • These tests run on TileDB Cloud using your current environment variable TILEDB_REST_TOKEN -- you will need a valid API token for the tests to pass
    • For continuous integration, the token is configured for the unittest user and all tests should pass
    • For interactive use, if your TILEDB_REST_TOKEN points to your own account, most tests will run, except for those that explicitly check against contents of the unittest account which are skipped

Ingestion testing

This package contains modules and functions that will be run in the TileDB Cloud as UDFs. Local ingestor changes can be tested in the cloud by using a feature of cloudpickle. Below is a runnable example. TILEDB_NAMESPACE is your TileDB namespace ("TileDB-Inc", for example). TILEDB_ACCESS_CREDENTIAL_NAME is the name of the stored credentials for accessing AWS_BUCKET. IMAGE_FILE_KEY is the key for an object in that bucket and OUTPUT_GROUP_KEY is the key to be used for the group that the ingest() UDF will create in AWS_BUCKET.

import cloudpickle

import tiledb.cloud.bioimg

cloudpickle.register_pickle_by_value(tiledb.cloud.bioimg)

tiledb.cloud.bioimg.ingest(
    source="s3://AWS_BUCKET/IMAGE_FILE_KEY",
    output="s3://AWS_BUCKET/OUTPUT_GROUP_KEY",
    config=None,
    namespace="TILEDB_NAMESPACE",
    acn="TILEDB_ACCESS_CREDENTIAL_NAME",
    ingest_resources={"cpu": "8", "memory": "32Gi"},
)

In this case tiledb.cloud.bioimg.ingest() uses cloudpickle to send a local function to TileDB Cloud, and cloudpickle.register_pickle_by_value(tiledb.cloud.bioimg) directs cloudpickle to bring the currently imported tiledb.cloud.bioimg module along with the function. Your local version of the module will be used instead of the version currently deployed in TileDB Cloud.

Note: your local changes to the Cloud-Py package will need to be installed in order for cloudpickle to serialize them, as cloudpickle needs to find them at runtime.

Releasing

Releasing is entirely automated. Releases made on GitHub using tags that start with "v", like "v0.12.28", trigger sdist and wheel builds and upload of those distributions to the Python Package Index.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tiledb_cloud-0.12.29.tar.gz (400.1 kB view details)

Uploaded Source

Built Distribution

tiledb_cloud-0.12.29-py3-none-any.whl (789.5 kB view details)

Uploaded Python 3

File details

Details for the file tiledb_cloud-0.12.29.tar.gz.

File metadata

  • Download URL: tiledb_cloud-0.12.29.tar.gz
  • Upload date:
  • Size: 400.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for tiledb_cloud-0.12.29.tar.gz
Algorithm Hash digest
SHA256 a260d6c2d5848bbdd35546e740a4f8b816ded26bd38d423b9077ebc0521138d6
MD5 46f8a568c62444e06d9930577d37e1e6
BLAKE2b-256 c0f906473ba0342546bfd7f00d41df7c16fbafd9a6044b3a7c59f27e0c6a5224

See more details on using hashes here.

Provenance

File details

Details for the file tiledb_cloud-0.12.29-py3-none-any.whl.

File metadata

File hashes

Hashes for tiledb_cloud-0.12.29-py3-none-any.whl
Algorithm Hash digest
SHA256 7d7e3166a7c898e9157051a626f7fa8a8ba9c542e1098acb76d65d5e7c14061d
MD5 90b3265cf284d09f95b85dc6524402ff
BLAKE2b-256 59d7d5240e316e6722eb978443eee7a9333bf9438e7f11ed59018aeaa306a26b

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page