A high-level Web Crawling and Web Scraping framework

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for dangra from gravatar.com dangra Avatar for kmike from gravatar.com kmike Avatar for pablohoffman from gravatar.com pablohoffman Avatar for scrapy from gravatar.com scrapy

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: BSD License (BSD)
  • Author: Scrapy developers
  • Maintainer: Pablo Hoffman
  • Requires: Python >=3.6
Classifiers

Project description

PyPI Version Supported Python Versions Ubuntu macOS Windows Wheel Status Coverage report Conda Version

Overview

Scrapy is a fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

Scrapy is maintained by Zyte (formerly Scrapinghub) and many other contributors.

Check the Scrapy homepage at https://scrapy.org for more information, including a list of features.

Requirements

  • Python 3.6+

  • Works on Linux, Windows, macOS, BSD

Install

The quick way:

pip install scrapy

See the install section in the documentation at https://docs.scrapy.org/en/latest/intro/install.html for more details.

Documentation

Documentation is available online at https://docs.scrapy.org/ and in the docs directory.

Releases

You can check https://docs.scrapy.org/en/latest/news.html for the release notes.

Community (blog, twitter, mail list, IRC)

See https://scrapy.org/community/ for details.

Contributing

See https://docs.scrapy.org/en/master/contributing.html for details.

Code of Conduct

Please note that this project is released with a Contributor Code of Conduct (see https://github.com/scrapy/scrapy/blob/master/CODE_OF_CONDUCT.md).

By participating in this project you agree to abide by its terms. Please report unacceptable behavior to opensource@zyte.com.

Companies using Scrapy

See https://scrapy.org/companies/ for a list.

Commercial Support

See https://scrapy.org/support/ for details.

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for dangra from gravatar.com dangra Avatar for kmike from gravatar.com kmike Avatar for pablohoffman from gravatar.com pablohoffman Avatar for scrapy from gravatar.com scrapy

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: BSD License (BSD)
  • Author: Scrapy developers
  • Maintainer: Pablo Hoffman
  • Requires: Python >=3.6
Classifiers

Release history Release notifications | RSS feed

2.11.2

2.11.1

2.11.0

2.10.1

2.10.0

2.9.0

2.8.0

2.7.1

2.7.0

2.6.3

2.6.2

2.6.1

2.6.0

2.5.1

This version

2.5.0

2.4.1

2.4.0

2.3.0

2.2.1

2.2.0

2.1.0

2.0.1

2.0.0

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.0

1.5.2

1.5.1

1.5.0

1.4.0

1.3.3

1.3.2

1.3.1

1.3.0

1.2.3

1.2.2

1.2.1

1.2.0

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.1.0rc4 pre-release

1.1.0rc3 pre-release

1.1.0rc2 pre-release

1.1.0rc1 pre-release

1.0.7

1.0.6

1.0.5

1.0.4

1.0.3

1.0.2

1.0.1

1.0.0

1.0.0rc3 pre-release

1.0.0rc2 pre-release

1.0.0rc1 pre-release

0.24.6

0.24.5

0.24.4

0.24.3

0.24.2

0.24.1

0.24.0

0.22.2

0.22.1

0.22.0

0.20.2

0.20.1

0.20.0

0.18.4

0.18.3

0.18.2

0.18.1

0.18.0

0.16.5

0.16.4

0.16.3

0.16.2

0.16.1

0.16.0

0.14.4

0.14.3

0.14.2

0.14.1

0.12.0.2550

0.10.4.2364

0.9

0.8

0.7

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Scrapy-2.5.0.tar.gz (1.1 MB view details)

Uploaded Source

Built Distribution

Scrapy-2.5.0-py2.py3-none-any.whl (254.6 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file Scrapy-2.5.0.tar.gz.

File metadata

  • Download URL: Scrapy-2.5.0.tar.gz
  • Upload date:
  • Size: 1.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.2

File hashes

Hashes for Scrapy-2.5.0.tar.gz
Algorithm Hash digest
SHA256 0a68ed41f7173679f160c4cef2db05288548c21e7164170552adae8b13cefaab
MD5 effcb0d38434d950f442d45739e0b84e
BLAKE2b-256 e9dcb2f41d802f548fba35c2d7c443ab0ab3df358eebd261f4d30d19ad39658a

See more details on using hashes here.

Provenance

File details

Details for the file Scrapy-2.5.0-py2.py3-none-any.whl.

File metadata

  • Download URL: Scrapy-2.5.0-py2.py3-none-any.whl
  • Upload date:
  • Size: 254.6 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.2

File hashes

Hashes for Scrapy-2.5.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 5f590fdc84b496e5a4bb5ef99836b0aa688a07cfcb4bc3bb7290f66486f27424
MD5 0a9266f242f2299a4533af844f01e483
BLAKE2b-256 95680e84466f83ed7d57b0da402bef1bb33a030224a64523a744de2abb8595f4

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page