Tools for making OPTIMADE APIs from raw structural data.
Project description
optimade-maker
Tools for making OPTIMADE APIs from various formats of structural data (e.g. an archive of CIF files).
This repository contains the src/optimade-maker
Python package and the corresponding CLI tool optimake
that work towards this aim. Features include
- definition of a config file format (
optimade.yaml
) for annotating data archives to be used in the OPTIMADE ecosystem; - conversion of the raw data into corresponding OPTIMADE types using pre-existing parsers (e.g., ASE for structures);
- conversion of the annotated data archive into an intermediate JSONLines file format that can be ingested into a database and used to serve a full OPTIMADE API.
- serving either an annotated data archive or a JSONLines file as an OPTIMADE API (using the
optimade-python-tools
reference server implementation).
Usage
See ./examples
for a more complete set of supported formats and corresponding optimade.yaml
config files.
Annotating with optimade.yaml
To annotate your structural data for optimade-maker
, the data archive needs to be accompanied by an optimade.yaml
config file. The following is a simple example for a zip archive (structures.zip
) of cif files together with an optional property file (data.csv
):
config_version: 0.1.0
database_description: Simple database
entries:
- entry_type: structures
entry_paths:
- file: structures.zip
matches:
- cifs/*/*.cif
# (optional) property file and definitions:
property_paths:
- file: data.csv
property_definitions:
- name: energy
title: Total energy per atom
description: The total energy per atom as computed by DFT
unit: eV/atom
type: float
Structure id
s and property files
optimade-maker
will assign an id
for each structure based on its full path in the archive, following a simple deterministic rule: from the set of all archive paths, the maximum common path prefix and postfix (including file extensions) are removed. E.g.
structures.zip/cifs/set1/101.cif
structures.zip/cifs/set2/102.cif
produces ["set1/101", "set2/102"]
.
The property files need to either refer to these id
s or the full path in the archive to be associated with a structure. E.g. a possible property csv
file could be
id,energy
set1/101,2.5
structures.zip/cifs/set2/102.cif,3.2
Installing and running optimake
Install with
pip install optimade-maker
this will also make the optimake
CLI utility available.
For a folder containing the data archive and the optimade.yaml
file (such as in /examples
), run
optimake convert .
to just convert the entry into the JSONL format (see below).optimake serve .
to start the OPTIMADE API (this also first converts the entry, if needed);
For more detailed information see also optimake --help
.
optimade-maker
JSONLines Format
As described above, optimade-maker
works via an intermediate JSONLines file representation of an OPTIMADE API (see also the corresponding issue in the specification).
This file should provide enough metadata to spin up an OPTIMADE API with many different entry types.
The format is as follows:
- First line must be a dictionary with the key
x-optimade
, containing a sub-dictionary of metadata (such as the OPTIMADE API version). - Second line contains the
info/structures
endpoint. - Third line contains the
info/references
endpoint, if present. - Then each line contains an entry from the corresponding individual structure/reference endpoints.
{"x-optimade": {"meta": {"api_version": "1.1.0"}}}
{"type": "info", "id": "structures", "properties": {...}}
{"type": "info", "id": "references", "properties": {...}}
{"type": "structures", "id": "1234", "attributes": {...}}
{"type": "structures", "id": "1235", "attributes": {...}}
{"type": "references", "id": "sfdas", "attributes": {...}}
NOTE: the info/
endpoints in OPTIMADE v1.2.0 will include type
and id
as well.
Relevant links
Contributors
Initial prototype was created at the Paul Scherrer Institute, Switzerland in the week of 12th-16th June 2023.
Authors (alphabetical):
- Kristjan Eimre
- Matthew Evans
- Giovanni Pizzi
- Gian-Marco Rignanese
- Jusong Yu
- Xing Wang
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file optimade_maker-0.3.0.tar.gz
.
File metadata
- Download URL: optimade_maker-0.3.0.tar.gz
- Upload date:
- Size: 23.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.10.14
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8bc30a074e92dae3103634c747f4153db3bcfdd30e6fed8daa7c9539c671f25d |
|
MD5 | c30cfcfd4fbbe0fa30b533a6e4a7d7cb |
|
BLAKE2b-256 | 2620ef437f60f09b811fbb2a306ee1331b314c3c002e52ba8336bc8bf6f7f5a4 |
File details
Details for the file optimade_maker-0.3.0-py3-none-any.whl
.
File metadata
- Download URL: optimade_maker-0.3.0-py3-none-any.whl
- Upload date:
- Size: 22.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.10.14
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | ca7739a72596d8ada7d5d3a080f1fd0d902807197e6fdb0597b34d59204c65ba |
|
MD5 | 782633c7487e28d84b99b7056a37affc |
|
BLAKE2b-256 | cf6c39035c4e4028f7027c5fd4f57cfc20188d187c7ef8e86aefa015fcfefe21 |