A bunch of custom harvesters for ckanext-harvest
Project description
The harvest_basket
extension comes with a few custom harvesters for different data portals.
The list of available harvesters, that must be enabled with ckan.plugins
in your config:
dkan_harvester
junar_harvester
socrata_harvester
arcgis_harvester
This extension also adds some features to extend the basic harvester:
- Source checkup preview. During the source creation stage the harvester will try to access the remote portal and harvest one dataset to check if it's accessible or not. (TODO: the source checkup interface will be documented soon).
- Restriction for anonymous users to visit harveser pages
Installation
To install ckanext-harvest-basket:
-
Activate your CKAN virtual environment, for example:
. /usr/lib/ckan/default/bin/activate
-
Clone the source and install it on the virtualenv
git clone https://github.com/mutantsan/ckanext-harvest-basket.git cd ckanext-harvest-basket pip install -e . pip install -r requirements.txt
-
Add
harvest_basket
to theckan.plugins
setting in your CKAN config file (by default the config file is located at/etc/ckan/default/ckan.ini
). -
Restart CKAN. For example if you've deployed CKAN with Apache on Ubuntu:
sudo service apache2 reload
Config settings
Available config options:
# You can disallow accessing harvester pages for anonymous users.
# (optional, default: 1).
ckanext.harvest_basket.allow_anonymous = 0
Developer installation
To install ckanext-harvest-basket for development, activate your CKAN virtualenv and do:
git clone https://github.com/mutantsan/ckanext-harvest-basket.git
cd ckanext-harvest-basket
python setup.py develop
pip install -r requirements.txt
Tests
To run the tests, do:
pytest --ckan-ini=test.ini
License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ckanext-harvest-basket-1.3.4.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 193a8cf878d304091c0b0a9c1bb7fe68e04c020a9d82e7d3776f011b7f61abb7 |
|
MD5 | 8f755bdff97ef868e513b59d560ecbfb |
|
BLAKE2b-256 | 10d81861ae1250e72a89cd0182b6dfa3ca487b9309645fcc1c90a0f4297c28ff |
Hashes for ckanext_harvest_basket-1.3.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 96b6341d158839a15cdfdab772f42fbdf11065095246efdc8325813795cb3100 |
|
MD5 | da9b4c0ec9f0f994ce1b4106a85b43fa |
|
BLAKE2b-256 | 5308b792abad3ce5b8d6243e3c543f5bf40292a930c5c46858a57d8965f0967b |