A bunch of custom harvesters for ckanext-harvest
Project description
The harvest_basket
extension comes with a few custom harvesters for different data portals.
The list of available harvesters, that must be enabled with ckan.plugins
in your config:
dkan_harvester
junar_harvester
socrata_harvester
arcgis_harvester
This extension also adds some features to extend the basic harvester:
- Source checkup preview. During the source creation stage the harvester will try to access the remote portal and harvest one dataset to check if it's accessible or not. (TODO: the source checkup interface will be documented soon).
- Restriction for anonymous users to visit harveser pages
Installation
To install ckanext-harvest-basket:
-
Activate your CKAN virtual environment, for example:
. /usr/lib/ckan/default/bin/activate
-
Clone the source and install it on the virtualenv
git clone https://github.com/mutantsan/ckanext-harvest-basket.git cd ckanext-harvest-basket pip install -e . pip install -r requirements.txt
-
Add
harvest_basket
to theckan.plugins
setting in your CKAN config file (by default the config file is located at/etc/ckan/default/ckan.ini
). -
Restart CKAN. For example if you've deployed CKAN with Apache on Ubuntu:
sudo service apache2 reload
Config settings
Available config options:
# You can disallow accessing harvester pages for anonymous users.
# (optional, default: 1).
ckanext.harvest_basket.allow_anonymous = 0
Developer installation
To install ckanext-harvest-basket for development, activate your CKAN virtualenv and do:
git clone https://github.com/mutantsan/ckanext-harvest-basket.git
cd ckanext-harvest-basket
python setup.py develop
pip install -r requirements.txt
Tests
To run the tests, do:
pytest --ckan-ini=test.ini
License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ckanext-harvest-basket-1.2.4.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d2a783f5e14635a2fb83b8cee66ecf3f4e12290d4021a9e34c33684e70f7d813 |
|
MD5 | e4418c27fae9cf02ff3d15c97963e1b8 |
|
BLAKE2b-256 | 1170fe878a095f1403ba8b09322b511dcf4e7a60abded46829d2fec61e1f68f5 |
Hashes for ckanext_harvest_basket-1.2.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 43323f845852bc4713cc38c5872a9051fda60ae97167dc1d474c950110c51515 |
|
MD5 | 5ffbe6321cc1ecf4d656e3dcf11d1fbf |
|
BLAKE2b-256 | 6253d86f4056d6781096f1137c874ca6f76cbdc88495a2a4b1fe56a514477f9e |