A bunch of custom harvesters for ckanext-harvest
Project description
The harvest_basket
extension comes with a few custom harvesters for different data portals.
The list of available harvesters, that must be enabled with ckan.plugins
in your config:
dkan_harvester
junar_harvester
socrata_harvester
arcgis_harvester
This extension also adds some features to extend the basic harvester:
- Source checkup preview. During the source creation stage the harvester will try to access the remote portal and harvest one dataset to check if it's accessible or not. (TODO: the source checkup interface will be documented soon).
- Restriction for anonymous users to visit harveser pages
Installation
To install ckanext-harvest-basket:
-
Activate your CKAN virtual environment, for example:
. /usr/lib/ckan/default/bin/activate
-
Clone the source and install it on the virtualenv
git clone https://github.com/mutantsan/ckanext-harvest-basket.git cd ckanext-harvest-basket pip install -e . pip install -r requirements.txt
-
Add
harvest_basket
to theckan.plugins
setting in your CKAN config file (by default the config file is located at/etc/ckan/default/ckan.ini
). -
Restart CKAN. For example if you've deployed CKAN with Apache on Ubuntu:
sudo service apache2 reload
Config settings
Available config options:
# You can disallow accessing harvester pages for anonymous users.
# (optional, default: 1).
ckanext.harvest_basket.allow_anonymous = 0
Developer installation
To install ckanext-harvest-basket for development, activate your CKAN virtualenv and do:
git clone https://github.com/mutantsan/ckanext-harvest-basket.git
cd ckanext-harvest-basket
python setup.py develop
pip install -r requirements.txt
Tests
To run the tests, do:
pytest --ckan-ini=test.ini
License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ckanext-harvest-basket-1.2.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | f08d3d7019984b087ae6caa1110699a8cae9664e12dc37460fde2bd6bb4924c9 |
|
MD5 | 0d2870be23cb6a024d19d34fc05164a2 |
|
BLAKE2b-256 | c784f155aebfa65abb0dafcd3289684fd7c9209737583525d1d4e294d5a60319 |
Hashes for ckanext_harvest_basket-1.2.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2aa96f763d199d84029c05d547b863142a1361a24b25bc2e564c5548abb9a38b |
|
MD5 | ca8d0cc34f89e0606121dc7701d28936 |
|
BLAKE2b-256 | 3dd25d23d498847b7c3b0f9bc5cd782bbfd8718979ce0ef0a825aec260cf4f73 |