Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
parsing job data from environment
processing job args and settings
running a job with Scrapy
collecting stats
advanced logging & error handling
full hubstorage support
custom scripts support
Requirements
Python 2/3
Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapinghub-entrypoint-scrapy-0.8.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 609b3185e5709f69af4114b33b855bf66882b7370e4e3f58e27589c715d86fce |
|
MD5 | e8ffafd50103a3dac71de573efe183fb |
|
BLAKE2b-256 | 0e2ffe047705887824e023eb578627faacb04b256eb4138f8c8dbb8f7eb7ad13 |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.8.0-py2-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 41c272780b2ddf67db14c0f66f34a5873fa69a3c22ae91be90b16120aa6a58c2 |
|
MD5 | 1a1d275c2b73196e87ddd2cf2982b302 |
|
BLAKE2b-256 | 91da8026422066cc5fe07d81e962b2c433c5fe1f0d3c4914d1e08b8206c1d5e7 |