Skip to main content

Library of web-related functions

Project description

https://secure.travis-ci.org/scrapy/w3lib.png?branch=master Coverage report

Overview

This is a Python library of web-related functions, such as:

  • remove comments, or tags from HTML snippets

  • extract base url from HTML snippets

  • translate entites on HTML strings

  • convert raw HTTP headers to dicts and vice-versa

  • construct HTTP auth header

  • converting HTML pages to unicode

  • sanitize urls (like browsers do)

  • extract arguments from urls

Requirements

Python 2.7 or Python 3.3+

Install

pip install w3lib

Documentation

See http://w3lib.readthedocs.org/

License

The w3lib library is licensed under the BSD license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

w3lib-1.14.1.tar.gz (41.0 kB view details)

Uploaded Source

Built Distribution

w3lib-1.14.1-py2.py3-none-any.whl (15.9 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file w3lib-1.14.1.tar.gz.

File metadata

  • Download URL: w3lib-1.14.1.tar.gz
  • Upload date:
  • Size: 41.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for w3lib-1.14.1.tar.gz
Algorithm Hash digest
SHA256 b8fbbaf4dfc7f03c8ac632d04c353570f5a1ccaaa24f34b5ec43a1f36872b065
MD5 f68496aa71a5144dc9e2e444097d596f
BLAKE2b-256 6bb3eecf5c0da4c632bea0be5d97bb0bb0365dafa9350e759f020336b404f883

See more details on using hashes here.

Provenance

File details

Details for the file w3lib-1.14.1-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for w3lib-1.14.1-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 49a7f653ce67fc36576d5e60dc2191a9543a910862f7c1427a5d48ad21c47b2e
MD5 2d73abc1b03330f1c4984d066294cc99
BLAKE2b-256 d1e5aadf1c3f3b4fe9cf16cf4f70dc7816acb3c7dd22f918b456123010ca281a

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page