Toolbox for Serenata de Amor project
Project description
Serenata de Amor Toolbox
pip installable package to support Serenata de Amor and Rosie development.
Serenata_toolbox is compatible with Python 3.6+
Installation
$ pip install -U serenata-toolboxIf you are a regular user you are ready to get started after pip install.
If you are a core developer willing to upload datasets to the cloud you need to configure AMAZON_ACCESS_KEY and AMAZON_SECRET_KEY environment variables before running the toolbox.
Usage
We have plenty of them ready for you to download from our servers. And this toolbox helps you get them. Here some examples:
Example 1: Using the command line wrapper
# without any arguments will download our pre-processed datasets and store into data/ folder
$ serenata-toolbox
# will download these specific datasets and store into /tmp/serenata-data folder
$ serenata-toolbox /tmp/serenata-data --module federal_senate chamber_of_deputies
# you can specify a dataset and a year
$ serenata-toolbox --module chamber_of_deputies --year 2009
# or specify all options simultaneously
$ serenata-toolbox /tmp/serenata-data --module federal_senate --year 2017
# getting help
$ serenata-toolbox --helpExample 2: How do I download the datasets?
Another option is creating your own Python script:
from serenata_toolbox.datasets import Datasets
datasets = Datasets('data/')
# now lets see what are the latest datasets available
for dataset in datasets.downloader.LATEST:
print(dataset) # and you'll see a long list of datasets!
# and let's download one of them
datasets.downloader.download('2018-01-05-reimbursements.xz') # yay, you've just downloaded this dataset to data/
# you can also get the most recent version of all datasets:
latest = list(datasets.downloader.LATEST)
datasets.downloader.download(latest)Example 3: Using shortcuts
If the last example doesn’t look that simple, there are some fancy shortcuts available:
from serenata_toolbox.datasets import fetch, fetch_latest_backup
fetch('2018-01-05-reimbursements.xz', 'data/')
fetch_latest_backup( 'data/') # yep, we've just did exactly the same thingExample 4: Generating datasets
If you ever wonder how did we generated these datasets, this toolbox can help you too (at least with the more used ones — the other ones are generated in our main repo):
from serenata_toolbox.federal_senate.dataset import Dataset as SenateDataset
from serenata_toolbox.chamber_of_deputies.reimbursements import Reimbursements as ChamberDataset
chamber = ChamberDataset('2018', 'data/')
chamber()
senate = SenateDataset('data/')
senate.fetch()
senate.translate()
senate.clean()Documentation (WIP)
The full documentation is still a work in progress. If you wanna give us a hand you will need Sphinx:
$ cd docs
$ make clean;make rst;rm source/modules.rst;make htmlContributing
Firstly, you should create a development environment with Python’s venv module to isolate your development. Then clone the repository and build the package by running:
$ git clone https://github.com/okfn-brasil/serenata-toolbox.git
$ cd serenata-toolbox
$ python setup.py developAlways add tests to your contribution — if you want to test it locally before opening the PR:
$ pip install tox
$ toxWhen the tests are passing, also check for coverage of the modules you edited or added — if you want to check it before opening the PR:
$ tox
$ open htmlcov/index.htmlFollow PEP8 and best practices implemented by Landscape in the veryhigh strictness level — if you want to check them locally before opening the PR:
$ pip install prospector
$ prospector -s veryhigh serenata_toolboxIf this report includes issues related to import section of your files, isort can help you:
$ pip install isort
$ isort **/*.py --diffAlways suggest a version bump. We use Semantic Versioning – or in Elm community words:
MICRO: the API is the same, no risk of breaking code
MINOR: values have been added, existing values are unchanged
MAJOR: existing values have been changed or removed
This is really important because every new code merged to master triggers the CI and then the CI triggers a new release to PyPI. The attemp to roll out a new version of the toolbox will fail without a version bump. So we do encorouge to add a version bump even if all you have changed is the README.rst — this is the way to keep the README.rst updated in PyPI.
If you are not changing the API or README.rst in any sense and if you really do not want a version bump, you need to add [skip ci] to you commit message.
And finally take The Zen of Python into account:
$ python -m thisRelease history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file serenata-toolbox-15.1.6.tar.gz.
File metadata
- Download URL: serenata-toolbox-15.1.6.tar.gz
- Upload date:
- Size: 19.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/44.0.0 requests-toolbelt/0.9.1 tqdm/4.41.1 CPython/3.8.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 746b1723bc8c871e6badf9d4a19b042e474bfaf88f23f8bdd8a80e1442034f1c |
|
| MD5 | 9de64df2e50a5a2c760b7cb2a499e8e2 |
|
| BLAKE2b-256 | 3242960db8e8b74e8cd04af25f8da03ff92e99e70bac338d411ff5735b8e5d1d |
File details
Details for the file serenata_toolbox-15.1.6-py3-none-any.whl.
File metadata
- Download URL: serenata_toolbox-15.1.6-py3-none-any.whl
- Upload date:
- Size: 23.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/44.0.0 requests-toolbelt/0.9.1 tqdm/4.41.1 CPython/3.8.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 6a2714350e973d02652ea95bbc7728522368e73e9beb8128b82613eb5df6ea2c |
|
| MD5 | 6a6a0ff418080e91c2e21abd3c845548 |
|
| BLAKE2b-256 | 53ac14880ec0aab83b56130e6fecf0a3da7a452c796674ee13b17ba199e0d9c0 |
