linkml-store

An AI-ready data management and integration platform. LinkML-Store provides an abstraction layer over multiple different backends (including DuckDB, MongoDB, and local filesystems), allowing for common query, index, and storage operations.

For full documentation, see https://linkml.io/linkml-store/

Warning LinkML-Store is still undergoing changes and refactoring, APIs and command line options are subject to change!

Quick Start

Install, add data, query it:

pip install linkml-store[all]
linkml-store -d duckdb:///db/my.db -c persons insert data/*.json
linkml-store -d duckdb:///db/my.db -c persons query -w "occupation: Bricklayer"

Index it, search it:

linkml-store -d duckdb:///db/my.db -c persons index -t llm
linkml-store -d duckdb:///db/my.db -c persons search "all persons employed in construction"

Validate it:

linkml-store -d duckdb:///db/my.db -c persons validate

Basic usage

• Command Line
• Python
• API
• Streamlit applications

Features

Multiple Adapters

LinkML-Store is designed to work with multiple backends, giving a common abstraction layer

• MongoDB
• DuckDB
• Solr
• Filesystem

Coming soon: any RDBMS, any triplestore, Neo4J, HDF5-based stores, ChromaDB/Vector dbs ...

The intent is to give a union of all features of each backend. For example, analytic faceted queries are provided for all backends, not just Solr.

Composable indexes

Many backends come with their own indexing and search schemes. Classically this was Lucene-based indexes, now it is semantic search using LLM embeddings.

LinkML store treats indexing as an orthogonal concern - you can compose different indexing schemes with different backends. You don't need to have a vector database to run embedding search!

See How to Use-Semantic-Search

Validation

LinkML-Store is backed by LinkML, which allows for powerful expressive structural and semantic constraints.

See Indexing JSON

and Referential Integrity

Background

See these slides for more details