Storage and database adapters available in project Thoth
Project description
This library provides a library called thoth-storages used in project Thoth. The library exposes core queries and methods for Dgraph database as well as adapters for manipulating with Ceph via its S3 compatible API.
Installation and Usage
The library can be installed via pip or Pipenv from PyPI:
pipenv install thoth-storagesThe library does not provide any CLI, it is rather a low level library supporting other parts of Thoth.
You can run prepared testsuite via the following command:
pipenv install --dev
pipenv run python3 setup.py test
# To generate docs:
pipenv run python3 setup.py build_sphinxAutomatically generate schema for Graph database
To automatically generate schema for the graph database from models defined in this module, run:
PYTHONPATH=. pipenv run python3 ./create_schema.py --output thoth/storages/graph/schema.rdfAfter running this command, the RDF file describing schema will be updated based on changes in model.
from thoth.storages import GraphDatabase
# Also provide configuration if needed.
graph = GraphDatabase()
graph.connect()
graph.initialize_schema()Running Dgraph locally
You can use docker-compose present in this repository to run a local Dgraph instance. It does not use TLS certificates (so you must not to provide GRAPH_TLS_PATH environment variable).
$ docker-compose upAfter running the command above (make sure your big fat daemon is up using systemctl start docker), you should be able to access a local Dgraph instance at localhost:9080. This is also the default configuration for Dgraph’s adapter - you don’t need to provide GRAPH_SERVICE_HOST explicitly.
The provided docker-compose has also Ratel enabled for to have an UI for graph database content. To access it visit http://localhost:8000/.
The provided docker-compose uses volume mounted from /tmp. After you computer restart, the content will not be available anymore.
If you would like to experiment with Dgraph programatically, you can use the following code snippet as a starting point:
from thoth.storages import GraphDatabase
graph = GraphDatabase()
graph.connect()
# To clear database:
# graph.drop_all()
# To initialize schema in the graph database:
# graph.initialize_schema()Schema adjustment in deployment
It’s possible to perform adjustments of schema in a deployemnt. It’s important that there are no open transactions (simply retry schema creation until it succeeds). You can use relevant endpoint on Management API for this purpose.
If there are changes in types, Dgraph tries to automatically perform conversion from an old type to the new one as described in the new schema (e.g. a float to string). Invalid schema changes (e.g. parsing string into a float, but the string cannot be parsed as a float) result in schema change errors. These errors need to be handled programatically by deployment administrator (ideally avoid such conversions).
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file thoth-storages-0.14.8.tar.gz.
File metadata
- Download URL: thoth-storages-0.14.8.tar.gz
- Upload date:
- Size: 36.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/36.5.0 requests-toolbelt/0.9.1 tqdm/4.32.2 CPython/3.6.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 7ca092d81aaf95d90baa8f8472836e2e0545efafbf89efe216bc622e6fadd9f4 |
|
| MD5 | 33ef1875e2ca54ff941116e4f4e5bfa2 |
|
| BLAKE2b-256 | b9d4ccd47f90ad01198b20d5966797a2692b3f901357119ec8b4104f135ecf78 |
File details
Details for the file thoth_storages-0.14.8-py3-none-any.whl.
File metadata
- Download URL: thoth_storages-0.14.8-py3-none-any.whl
- Upload date:
- Size: 57.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/36.5.0 requests-toolbelt/0.9.1 tqdm/4.32.2 CPython/3.6.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | a2f49447f82d1fd297b91c18956f3443ee7e1bb69a251adf38ba98da72e8f9fd |
|
| MD5 | 667f5cc83b284add80deb67790fb5698 |
|
| BLAKE2b-256 | c671dfa821e731e9b21e49010c8160adf88a4030ff1daaed688df4532dfba7f6 |
