A bunch of custom harvesters for ckanext-harvest
Project description
The harvest_basket
extension comes with a few custom harvesters for different data portals.
The list of available harvesters, that must be enabled with ckan.plugins
in your config:
dkan_harvester
junar_harvester
socrata_harvester
arcgis_harvester
This extension also adds some features to extend the basic harvester:
- Source checkup preview. During the source creation stage the harvester will try to access the remote portal and harvest one dataset to check if it's accessible or not. (TODO: the source checkup interface will be documented soon).
- Restriction for anonymous users to visit harveser pages
Installation
To install ckanext-harvest-basket:
-
Activate your CKAN virtual environment, for example:
. /usr/lib/ckan/default/bin/activate
-
Clone the source and install it on the virtualenv
git clone https://github.com/mutantsan/ckanext-harvest-basket.git cd ckanext-harvest-basket pip install -e . pip install -r requirements.txt
-
Add
harvest_basket
to theckan.plugins
setting in your CKAN config file (by default the config file is located at/etc/ckan/default/ckan.ini
). -
Restart CKAN. For example if you've deployed CKAN with Apache on Ubuntu:
sudo service apache2 reload
Config settings
Available config options:
# You can disallow accessing harvester pages for anonymous users.
# (optional, default: 1).
ckanext.harvest_basket.allow_anonymous = 0
Developer installation
To install ckanext-harvest-basket for development, activate your CKAN virtualenv and do:
git clone https://github.com/mutantsan/ckanext-harvest-basket.git
cd ckanext-harvest-basket
python setup.py develop
pip install -r requirements.txt
Tests
To run the tests, do:
pytest --ckan-ini=test.ini
License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ckanext-harvest-basket-1.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 65153ec4ed82dc59941cbc315389f015025557d8ee80ba7ab511a216f3a8d19b |
|
MD5 | ab49e68ec06b62adc82705062ea6e394 |
|
BLAKE2b-256 | 70987cac5157171bcec0fec3e731ddfcb344c794137618084be02023ba44c735 |
Hashes for ckanext_harvest_basket-1.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 401393a273b943df8630e6e664adf20f843b2973de9754865d2d83bb515d2363 |
|
MD5 | 8b77ec7eb799dc3cd384aac563af1547 |
|
BLAKE2b-256 | da254036b9d1bbd706d9985cc7ee955ee9696c2e6bf929e2816a211644c82b32 |