Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
parsing job data from environment
processing job args and settings
running a job with Scrapy
collecting stats
advanced logging & error handling
full hubstorage support
custom scripts support
Requirements
Python 2/3
Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapinghub-entrypoint-scrapy-0.8.10.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 00f1dd5041b587af3efe102f57cb5b30e442b1a9ff11fcca047ad6047ddc8339 |
|
MD5 | ee7272c60d9c5a3f569e509ca757661f |
|
BLAKE2b-256 | abe7876b2b81dd69e6271d35986232678bfcc05b8c5c423a23b1a451c4fb4ac9 |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.8.10-py2-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f8f908c794ac0b782bf5bffb907b40ddb1133636555405c543b0e706e91c4b83 |
|
MD5 | ed20a94fc5020444dade879d633f4b44 |
|
BLAKE2b-256 | 012ca71e250c981a7167f2ec55d193cfb069d3aa1aef9b704f392ed98929ae46 |