A bunch of custom harvesters for ckanext-harvest
Project description
The harvest_basket
extension comes with a few custom harvesters for different data portals.
The list of available harvesters, that must be enabled with ckan.plugins
in your config:
dkan_harvester
junar_harvester
socrata_harvester
arcgis_harvester
This extension also adds some features to extend the basic harvester:
- Source checkup preview. During the source creation stage the harvester will try to access the remote portal and harvest one dataset to check if it's accessible or not. (TODO: the source checkup interface will be documented soon).
- Restriction for anonymous users to visit harveser pages
Installation
To install ckanext-harvest-basket:
-
Activate your CKAN virtual environment, for example:
. /usr/lib/ckan/default/bin/activate
-
Clone the source and install it on the virtualenv
git clone https://github.com/mutantsan/ckanext-harvest-basket.git cd ckanext-harvest-basket pip install -e . pip install -r requirements.txt
-
Add
harvest_basket
to theckan.plugins
setting in your CKAN config file (by default the config file is located at/etc/ckan/default/ckan.ini
). -
Restart CKAN. For example if you've deployed CKAN with Apache on Ubuntu:
sudo service apache2 reload
Config settings
Available config options:
# You can disallow accessing harvester pages for anonymous users.
# (optional, default: 1).
ckanext.harvest_basket.allow_anonymous = 0
Developer installation
To install ckanext-harvest-basket for development, activate your CKAN virtualenv and do:
git clone https://github.com/mutantsan/ckanext-harvest-basket.git
cd ckanext-harvest-basket
python setup.py develop
pip install -r requirements.txt
Tests
To run the tests, do:
pytest --ckan-ini=test.ini
License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ckanext-harvest-basket-1.3.5.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 833eb7fd0d4fa68cbbbb640dbbdc294a926707a0cf6a103f82291f0ff8056f04 |
|
MD5 | b70ad6d3fc5c39b3a526e39b5ab1ed50 |
|
BLAKE2b-256 | d8e2b56440eaf9632a77bd887bc8977fe1b12e2dd8932f196dfd3e13da07d209 |
Hashes for ckanext_harvest_basket-1.3.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 148655fa51f0cdb5740e8054e389d8755e1f3977bb5e8508f980949d4d9ad1eb |
|
MD5 | f7e070db85943639c043a5760ef4f403 |
|
BLAKE2b-256 | 9e37ba4dc919c6be9f2be375cd2d28e0709025320e929f497bd971c51da5b3de |