Skip to main content

A high-level Web Crawling and Web Scraping framework

Project description

PyPI Version Supported Python Versions Build Status Wheel Status Coverage report Conda Version

Overview

Scrapy is a fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

Check the Scrapy homepage at https://scrapy.org for more information, including a list of features.

Requirements

  • Python 3.5.2+

  • Works on Linux, Windows, macOS, BSD

Install

The quick way:

pip install scrapy

See the install section in the documentation at https://docs.scrapy.org/en/latest/intro/install.html for more details.

Documentation

Documentation is available online at https://docs.scrapy.org/ and in the docs directory.

Releases

You can check https://docs.scrapy.org/en/latest/news.html for the release notes.

Community (blog, twitter, mail list, IRC)

See https://scrapy.org/community/ for details.

Contributing

See https://docs.scrapy.org/en/master/contributing.html for details.

Code of Conduct

Please note that this project is released with a Contributor Code of Conduct (see https://github.com/scrapy/scrapy/blob/master/CODE_OF_CONDUCT.md).

By participating in this project you agree to abide by its terms. Please report unacceptable behavior to opensource@scrapinghub.com.

Companies using Scrapy

See https://scrapy.org/companies/ for a list.

Commercial Support

See https://scrapy.org/support/ for details.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Scrapy-2.2.0.tar.gz (1.0 MB view details)

Uploaded Source

Built Distribution

Scrapy-2.2.0-py2.py3-none-any.whl (241.4 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file Scrapy-2.2.0.tar.gz.

File metadata

  • Download URL: Scrapy-2.2.0.tar.gz
  • Upload date:
  • Size: 1.0 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.3.1 requests-toolbelt/0.9.1 tqdm/4.46.1 CPython/3.8.1

File hashes

Hashes for Scrapy-2.2.0.tar.gz
Algorithm Hash digest
SHA256 8ce036a113ef9bc575a65b463994244e4298145f70bcdcf40b8847a2f528a1fb
MD5 b1326737175d4c5617e9a674b7a221c1
BLAKE2b-256 8fd6014da03d8eb18094f4619edad4b561a45924bd093d55eb2674365b99d5da

See more details on using hashes here.

Provenance

File details

Details for the file Scrapy-2.2.0-py2.py3-none-any.whl.

File metadata

  • Download URL: Scrapy-2.2.0-py2.py3-none-any.whl
  • Upload date:
  • Size: 241.4 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.3.1 requests-toolbelt/0.9.1 tqdm/4.46.1 CPython/3.8.1

File hashes

Hashes for Scrapy-2.2.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 e6b724ef3d522bbce93bede7af18d1603b57cefcf589337b0f4c5b15ea0a5f29
MD5 3effab6190d0a7fa25183404c6f11f0e
BLAKE2b-256 d2b1105fe9a289e5bb64ec104076546f72060296d9989a0fc31a8b608c810868

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page