Index DBnomics data into Apache Solr for full-text and faceted search.
Project description
DBnomics Solr
Index DBnomics data into Apache Solr for full-text and faceted search.
Requirements:
- a running instance of Apache Solr; at the time this documentation is written, we use the version 7.3.
See dbnomics-docker to run a local DBnomics instance with Docker that includes a service for Apache Solr.
Configuration
Environment variables:
DEBUG_PYSOLR: display pysolr DEBUG logging messages (cf https://github.com/django-haystack/pysolr)
Index a provider
Replace wto by the real provider slug in the following command:
dbnomics-solr index-provider /path/to/wto-json-data
Full mode vs incremental mode
When data is stored in a regular directory, the script always indexes all datasets and series of a provider. This is called full mode.
When data is stored in a Git repository, the script runs by default in incremental mode: it indexes only the datasets modified since the last indexation.
It is possible to force the full mode with the --full option.
Bare repositories
The script has an option --bare-repo-fallback which tries to add .git at the end of the storage dir name, if not found.
Remove all data from a provider
To remove all the documents related to a provider (type:provider, type:dataset and type:series):
./delete_provider.sh <provider_code>
Example:
./delete_provider.sh WTO
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file dbnomics-solr-1.0.3.tar.gz.
File metadata
- Download URL: dbnomics-solr-1.0.3.tar.gz
- Upload date:
- Size: 50.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/49.2.0 requests-toolbelt/0.9.1 tqdm/4.48.0 CPython/3.8.4
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | d880840814b88225ee90a2473d8337c63b344c775b376366cbdd5556fe6a46b4 |
|
| MD5 | 60b025154aeccd6bca9a296358204510 |
|
| BLAKE2b-256 | 9d429355e595eecdc0faeb65528e0559d24e1e92604bdd71c9b743ff12d9432e |
File details
Details for the file dbnomics_solr-1.0.3-py2.py3-none-any.whl.
File metadata
- Download URL: dbnomics_solr-1.0.3-py2.py3-none-any.whl
- Upload date:
- Size: 20.8 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/49.2.0 requests-toolbelt/0.9.1 tqdm/4.48.0 CPython/3.8.4
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 49afcb87b3831d631a9b9abc0ea85a19c8c2e33157472cc54739859640e9c26f |
|
| MD5 | 72b352631f1c4456eddec5cef82eb30b |
|
| BLAKE2b-256 | 0e17190eb542fa7112bdcdc1852e9d65c18997eade7e0d55a7be13239c43d9ec |
