Scrapy http proxy middleware that gets proxy parameters from settings
Project description
Scrapy middleware that reads proxy config from settings.
Install:
pip install proxy-middleware
Usage: add it to DOWNLOADER_MIDDLEWARES in scrapy settings:
DOWNLOADER_MIDDLEWARES = { 'proxy_middleware.ProxyFromSettingsMiddleware': 10, ...
Pass proxy config via HTTP_PROXY and HTTPS_PROXY settings variables. HTTPPROXY_AUTH_ENCODING is also respected:
scrapy crawl my-spider -s HTTP_PROXY=http://localhost:8118
License is MIT.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
proxy-middleware-0.1.1.tar.gz
(1.8 kB
view hashes)