Skip to main content

linkml-store

Project description

linkml-store

An AI-ready data management and integration platform. LinkML-Store provides an abstraction layer over multiple different backends (including DuckDB, MongoDB, and local filesystems), allowing for common query, index, and storage operations.

For full documentation, see https://linkml.io/linkml-store/

Warning LinkML-Store is still undergoing changes and refactoring, APIs and command line options are subject to change!

Quick Start

Install, add data, query it:

pip install linkml-store[all]
linkml-store -d duckdb:///db/my.db -c persons insert data/*.json
linkml-store -d duckdb:///db/my.db -c persons query -w "occupation: Bricklayer"

Index it, search it:

linkml-store -d duckdb:///db/my.db -c persons index -t llm
linkml-store -d duckdb:///db/my.db -c persons search "all persons employed in construction"

Validate it:

linkml-store -d duckdb:///db/my.db -c persons validate

Basic usage

Features

Multiple Adapters

LinkML-Store is designed to work with multiple backends, giving a common abstraction layer

Coming soon: any RDBMS, any triplestore, Neo4J, HDF5-based stores, ChromaDB/Vector dbs ...

The intent is to give a union of all features of each backend. For example, analytic faceted queries are provided for all backends, not just Solr.

Composable indexes

Many backends come with their own indexing and search schemes. Classically this was Lucene-based indexes, now it is semantic search using LLM embeddings.

LinkML store treats indexing as an orthogonal concern - you can compose different indexing schemes with different backends. You don't need to have a vector database to run embedding search!

See How to Use-Semantic-Search

Use with LLMs

TODO - docs

Validation

LinkML-Store is backed by LinkML, which allows for powerful expressive structural and semantic constraints.

See Indexing JSON

and Referential Integrity

Web API

There is a preliminary API following HATEOAS principles implemented using FastAPI.

To start you should first create a config file, e.g. db/conf.yaml:

Then run:

export LINKML_STORE_CONFIG=./db/conf.yaml
make api

The API returns links as well as data objects, it's recommended to use a Chrome plugin for JSON viewing for exploring the API. TODO: add docs here.

The main endpoints are:

  • http://localhost:8000/ - the root of the API
  • http://localhost:8000/pages/ - browse the API via HTML
  • http://localhost:8000/docs - the Swagger UI

Streamlit app

make app

Background

See these slides for more details

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

linkml_store-0.1.13.tar.gz (65.0 kB view details)

Uploaded Source

Built Distribution

linkml_store-0.1.13-py3-none-any.whl (86.9 kB view details)

Uploaded Python 3

File details

Details for the file linkml_store-0.1.13.tar.gz.

File metadata

  • Download URL: linkml_store-0.1.13.tar.gz
  • Upload date:
  • Size: 65.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.0 CPython/3.12.4

File hashes

Hashes for linkml_store-0.1.13.tar.gz
Algorithm Hash digest
SHA256 6fe5f66bea13fdb2846bc6b4c08c2f174bfd23766fa9854fc5b7cf7c852c7263
MD5 21a8aebad4d58a0c773e3946f99f104b
BLAKE2b-256 914d8577f73727ca17a7429a8c7cc348614c112b13561f245c6a82bdad7788d8

See more details on using hashes here.

Provenance

File details

Details for the file linkml_store-0.1.13-py3-none-any.whl.

File metadata

File hashes

Hashes for linkml_store-0.1.13-py3-none-any.whl
Algorithm Hash digest
SHA256 e18fe10b1bd99d34cdb2083e27804ad98e9714a09623e7c93457b475c7eb2ca8
MD5 673fe3a170a34ace73268b7efbcc45f3
BLAKE2b-256 681cd9e0619702a1833c6c3387c92392ffd6d42406c8ce7422393fb82cf9a3d3

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page