Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
parsing job data from environment
processing job args and settings
running a job with Scrapy
collecting stats
advanced logging & error handling
full hubstorage support
custom scripts support
Requirements
Python 2/3
Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapinghub-entrypoint-scrapy-0.8.7.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 019679c4207ac380d7fdfed1cc5ccc703d6ea290ddac6730423f3fbc38dd88c8 |
|
MD5 | 44bbe838cc9847da3c62beb8ca600f0d |
|
BLAKE2b-256 | 29b5df1d7f974bcbaa5f47590eb1f57797ab966fa6296824af53baafe6feae93 |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.8.7-py2-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9a9b48925f50cdad599c82419fed5ad9332637309416fc07c84b2e8ef93b2dc7 |
|
MD5 | b48b60e148738ccab337e9c0fbea78c0 |
|
BLAKE2b-256 | 3979a704ec2c10e9d0f502420fea27fcd140a6e3e6111c1b5ffb2275981882ed |