Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
parsing job data from environment
processing job args and settings
running a job with Scrapy
collecting stats
advanced logging & error handling
full hubstorage support
custom scripts support
Requirements
Python 2/3
Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapinghub-entrypoint-scrapy-0.8.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 142313613f7377ac73eb8acfde80899b371adfbb90b5b8262d16ed2620e5fc58 |
|
MD5 | 508cc4ea0346768b92836be6da675b0f |
|
BLAKE2b-256 | 2de754ef1284726226468bb93f98b1b1c87d5b92f141f95b0464632b91a9c478 |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.8.1-py2-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 874f8385c1173c0febfe56dc703859432ec4e1555af180a8dd924501be30c12e |
|
MD5 | 15bed24e904863254132d2d36d06b1ad |
|
BLAKE2b-256 | 5a15a9775eff2f0272168bb4dc826d01d525ab37eb4e7874bca1cee58efe4dfb |