Scrapy worker based on buildout with JSON-RPC 2.0 API
Project description
This package provides a daemon which is able to run buildout based scrapy spider packages. (see s01.demo). The daemon processes pending crawl jobs stored in mongodb. The daemon also provides a WSGI server providing a JSON-RPC 2.0 API. The JSON-RPC proxy from the s01.client package can be used for install packages and add new crawl jobs to the mongodb which is the queue.
Changes
0.5.0 (2012-11-18)
initial implementation
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
s01.worker-0.5.0.zip
(73.1 kB
view details)
File details
Details for the file s01.worker-0.5.0.zip.
File metadata
- Download URL: s01.worker-0.5.0.zip
- Upload date:
- Size: 73.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | b8f9f72cc43bb2b3ee83563ca11275e147b4836fa0604b431a599a43e5fa086c |
|
| MD5 | a87a023ceac226116f8184b40ee1bcb3 |
|
| BLAKE2b-256 | 651d8efc244acccf442e08a6f7e60f2565e832b001444a2208598958f107c4d7 |
