A webapp to query datanommer
Project description
datagrepper
Datagrepper is a web application and JSON API to retrieve historical messages sent via Fedora Messaging. Datanommer is a seperate project and service that consumes messages from the Fedora Messaging queue and puts them in a database. These messages is what datagrepper queries.
Datagrepper is curently running in production at https://apps.fedoraproject.org/datagrepper/
Development Environment
Vagrant allows contributors to get quickly up and running with a datagrepper development environment by automatically configuring a virtual machine.
The datagrepper Vagrant environment configures configures and enables a datanommer service and database. The datanommer instance is configured to be empty when first provisioned, but to consume messages from the stage Fedora Messaging queue.
Install vagrant
To get started, run the following commands to install the Vagrant and Virtualization packages needed, and start the libvirt service:
$ sudo dnf install ansible libvirt vagrant-libvirt vagrant-sshfs vagrant-hostmanager
$ sudo systemctl enable libvirtd
$ sudo systemctl start libvirtd
Checkout and Provision
Next, check out the datagrepper code and run vagrant up:
$ git clone https://github.com/fedora-infra/datagrepper
$ cd datanommer
$ vagrant up
Interacting with your development datagrepper
After successful provisioning of the Datagrepper vagrant setup, the datagrepper web application will be accessible from your host machine's web browser at
Using the development environment
SSH into your newly provisioned development environment:
$ vagrant ssh
The vagrant setup also defines 4 handy commands to interact with the service that runs the datagrepper flask application:
$ datagrepper-start
$ datagrepper-stop
$ datagrepper-restart
$ dataprepper-logs
Additionally, the following commands are also available for interacting with the datanommer service:
$ datanommer-consumer-start
$ datanommer-consumer-stop
$ datanommer-consumer-restart
$ datanommer-consumer-logs
Running the tests
Datanommer is comprised of 3 seperate modules in this single repository. There is a handy script in the top directory of this repo to run the tests on all 3 modules:
$ ./runtests.sh
However, tests can also be run on a single module by invotking tox in that modules' directory. For example:
$ cd datanommer.models/
$ tox
Note, that the tests use virtual environments that are not created from scratch with every subsequent run of the tests. Therefore, when changes happen to dependencies, the tests may fail to run correctly. To recreate the virtual envrionments, run the tests commands with the -r flag, for example:
$ ./runtests.sh -r
or
$ cd datanommer.models/
$ tox -r
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file datagrepper-1.1.0.tar.gz.
File metadata
- Download URL: datagrepper-1.1.0.tar.gz
- Upload date:
- Size: 3.1 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.5 Linux/6.4.15-200.fc38.x86_64
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 3b461cbb5f54ef1b133bb462a8f2ed872c8832ae4bddce110c086d822b98ec35 |
|
| MD5 | d90529efbe63919a721267ca047928a3 |
|
| BLAKE2b-256 | d92b6d36f46610750d75b755cee90e156eece31e13e474d4427943d8563de035 |
Provenance
File details
Details for the file datagrepper-1.1.0-py3-none-any.whl.
File metadata
- Download URL: datagrepper-1.1.0-py3-none-any.whl
- Upload date:
- Size: 3.1 MB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.5 Linux/6.4.15-200.fc38.x86_64
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 8c774363ca9094294ad02d6b49373e6feb5e8491b8536ddd607b48bc0ba7b832 |
|
| MD5 | 9289d60b8fc20670df1913eb061def18 |
|
| BLAKE2b-256 | f6e600e2e1c58501b32ee93e3542866742ecc7299c2c754590e451709943c561 |
