Client library to process URLs through Zyte API
Project description
Installation
pip install scrapy-zyte-apiThis package requires Python 3.7+.
How to configure
Replace the default http and https in Scrapy’s DOWNLOAD_HANDLERS in the settings.py of your Scrapy project.
You also need to set the ZYTE_API_KEY.
DOWNLOAD_HANDLERS = {
"http": "scrapy_zyte_api.handler.ScrapyZyteAPIDownloadHandler",
"https": "scrapy_zyte_api.handler.ScrapyZyteAPIDownloadHandler"
}
# Having the following in the env var would also work.
ZYTE_API_KEY = "<your API key>"Also, make sure to install the asyncio-based Twisted reactor in the settings.py file as well:
TWISTED_REACTOR = "twisted.internet.asyncioreactor.AsyncioSelectorReactor"How to use
Set the zyte_api Request.meta key to download a request using Zyte API. Full list of parameters is provided in the Zyte API Specification.
yield scrapy.Request(
"http://books.toscrape.com/",
callback=self.parse,
meta={
"zyte_api": {
"browserHtml": True,
"geolocation": "US",
"javascript": True,
"echoData": {"something": True}
}
}
)Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapy-zyte-api-0.1.0.tar.gz
(4.1 kB
view details)
Built Distribution
File details
Details for the file scrapy-zyte-api-0.1.0.tar.gz.
File metadata
- Download URL: scrapy-zyte-api-0.1.0.tar.gz
- Upload date:
- Size: 4.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.10.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.10.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 41bf16610dea3eb28a1068a5c05773b3f2cff7d683efe2cbc737bd4bfe321267 |
|
| MD5 | 7af1c2e6158096f27005401b8cf7a785 |
|
| BLAKE2b-256 | bf2403908b2c47ff69f757943e3e2a92f25cd4202f51ed985b678646abd7c3dc |
File details
Details for the file scrapy_zyte_api-0.1.0-py3-none-any.whl.
File metadata
- Download URL: scrapy_zyte_api-0.1.0-py3-none-any.whl
- Upload date:
- Size: 4.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.8.0 pkginfo/1.8.2 readme-renderer/32.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.8 tqdm/4.62.3 importlib-metadata/4.10.1 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.10.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | f91c36c6243ab310e5e38e4a8b6b927da8cc975671be61a17a4b75c7d50e21cf |
|
| MD5 | 43b3f6b5cf162872d9b175145246dffd |
|
| BLAKE2b-256 | 453e7305cba1a5499fd59d545a95b291d66b4ef2dcbfd6354768e4c68e81b0ac |
