Scrapy

A high-level Web Crawling and Web Scraping framework

These details have been verified by PyPI

Maintainers

dangra kmike pablohoffman scrapy

These details have not been verified by PyPI

Project links

Homepage

Project description

Overview

Scrapy is a fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

For more information including a list of features check the Scrapy homepage at: http://scrapy.org

Requirements

Python 2.7 or Python 3.3+
Works on Linux, Windows, Mac OSX, BSD

Install

The quick way:

pip install scrapy

For more details see the install section in the documentation: http://doc.scrapy.org/en/latest/intro/install.html

Releases

You can download the latest stable and development releases from: http://scrapy.org/download/

Documentation

Documentation is available online at http://doc.scrapy.org/ and in the docs directory.

Community (blog, twitter, mail list, IRC)

See http://scrapy.org/community/

Contributing

See http://doc.scrapy.org/en/master/contributing.html

Code of Conduct

Please note that this project is released with a Contributor Code of Conduct (see https://github.com/scrapy/scrapy/blob/master/CODE_OF_CONDUCT.md).

By participating in this project you agree to abide by its terms. Please report unacceptable behavior to opensource@scrapinghub.com.

Companies using Scrapy

See http://scrapy.org/companies/

Commercial Support

See http://scrapy.org/support/

Project details

These details have been verified by PyPI

Maintainers

dangra kmike pablohoffman scrapy

These details have not been verified by PyPI

Project links

Homepage

Release history Release notifications | RSS feed

2.11.2

May 14, 2024

2.11.1

Feb 14, 2024

2.11.0

Sep 18, 2023

2.10.1

Aug 30, 2023

2.10.0

Aug 4, 2023

2.9.0

May 8, 2023

2.8.0

Feb 2, 2023

2.7.1

Nov 2, 2022

2.7.0

Oct 17, 2022

2.6.3

Sep 27, 2022

2.6.2

Jul 25, 2022

2.6.1

Mar 1, 2022

2.6.0

Mar 1, 2022

2.5.1

Oct 5, 2021

2.5.0

Apr 6, 2021

2.4.1

Nov 17, 2020

2.4.0

Oct 11, 2020

2.3.0

Aug 4, 2020

2.2.1

Jul 17, 2020

2.2.0

Jun 24, 2020

2.1.0

Apr 24, 2020

2.0.1

Mar 18, 2020

2.0.0

Mar 3, 2020

1.8.4

Feb 14, 2024

1.8.3

Jul 25, 2022

1.8.2

Mar 1, 2022

1.8.1

Oct 5, 2021

1.8.0

Oct 29, 2019

1.7.4

Oct 21, 2019

1.7.3

Aug 1, 2019

1.7.2

Jul 23, 2019

1.7.1

Jul 18, 2019

1.7.0

Jul 18, 2019

1.6.0

Jan 30, 2019

1.5.2

Jan 22, 2019

1.5.1

Jul 11, 2018

1.5.0

Dec 29, 2017

This version

1.4.0

May 18, 2017

1.3.3

Mar 10, 2017

1.3.2

Feb 13, 2017

1.3.1

Feb 8, 2017

1.3.0

Dec 21, 2016

1.2.3

Mar 3, 2017

1.2.2

Dec 6, 2016

1.2.1

Oct 21, 2016

1.2.0

Oct 3, 2016

1.1.4

Mar 3, 2017

1.1.3

Sep 22, 2016

1.1.2

Aug 18, 2016

1.1.1

Jul 13, 2016

1.1.0

May 11, 2016

1.1.0rc4 pre-release

Apr 28, 2016

1.1.0rc3 pre-release

Mar 1, 2016

1.1.0rc2 pre-release

Feb 29, 2016

1.1.0rc1 pre-release

Feb 4, 2016

1.0.7

Mar 3, 2017

1.0.6

May 4, 2016

1.0.5

Feb 3, 2016

1.0.4

Dec 30, 2015

1.0.3

Aug 11, 2015

1.0.2

Aug 6, 2015

1.0.1

Jul 1, 2015

1.0.0

Jun 19, 2015

1.0.0rc3 pre-release

Jun 12, 2015

1.0.0rc2 pre-release

Jun 5, 2015

1.0.0rc1 pre-release

May 22, 2015

0.24.6

Apr 20, 2015

0.24.5

Feb 25, 2015

0.24.4

Aug 9, 2014

0.24.3

Aug 9, 2014

0.24.2

Jul 8, 2014

0.24.1

Jun 27, 2014

0.24.0

Jun 26, 2014

0.22.2

Feb 14, 2014

0.22.1

Feb 8, 2014

0.22.0

Jan 17, 2014

0.20.2

Dec 9, 2013

0.20.1

Nov 28, 2013

0.20.0

Nov 8, 2013

0.18.4

Oct 10, 2013

0.18.3

Oct 3, 2013

0.18.2

Sep 4, 2013

0.18.1

Aug 27, 2013

0.18.0

Aug 10, 2013

0.16.5

May 30, 2013

0.16.4

Jan 23, 2013

0.16.3

Dec 7, 2012

0.16.2

Nov 9, 2012

0.16.1

Oct 26, 2012

0.16.0

Oct 18, 2012

0.14.4

May 15, 2012

0.14.3

Apr 20, 2012

0.14.2

Mar 11, 2012

0.14.1

Jan 2, 2012

0.12.0.2550

Oct 25, 2011

0.10.4.2364

Nov 16, 2010

0.9

Jun 28, 2010

0.8

Dec 12, 2009

0.7

Jan 20, 2010

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Scrapy-1.4.0.tar.gz (898.2 kB view details)

Uploaded May 18, 2017 Source

Built Distribution

Scrapy-1.4.0-py2.py3-none-any.whl (248.8 kB view details)

Uploaded May 18, 2017 Python 2 Python 3

File details

Details for the file Scrapy-1.4.0.tar.gz.

File metadata

Download URL: Scrapy-1.4.0.tar.gz
Upload date: May 18, 2017
Size: 898.2 kB
Tags: Source
Uploaded using Trusted Publishing? No

File hashes

Hashes for Scrapy-1.4.0.tar.gz
Algorithm	Hash digest
SHA256	`04a08f027eef5d271342a016439533c81ba46f14bfcf230fecf602e99beaf233`
MD5	`889a9639ab18f03d434963f3a64bf06c`
BLAKE2b-256	`c5510ae702719bdd01577fc1921777bb6f7af4b5fddbe0f7ca64fdd42721e019`

See more details on using hashes here.

Provenance

File details

Details for the file Scrapy-1.4.0-py2.py3-none-any.whl.

File metadata

Download URL: Scrapy-1.4.0-py2.py3-none-any.whl
Upload date: May 18, 2017
Size: 248.8 kB
Tags: Python 2, Python 3
Uploaded using Trusted Publishing? No

File hashes

Hashes for Scrapy-1.4.0-py2.py3-none-any.whl
Algorithm	Hash digest
SHA256	`8f14888a86cc94d51ec8cdfdbd8c8a76c8bea7d6244bc4e4cc89c1958c58ed5d`
MD5	`a14b1766ca0277a846ff4b01bb81ff9b`
BLAKE2b-256	`a8963affe11cf53a5d2105536919113d5b453479038bb486f7387f4ce4a3b83f`