Skip to main content

Library of web-related functions

Project description

https://secure.travis-ci.org/scrapy/w3lib.png?branch=master Coverage report

Overview

This is a Python library of web-related functions, such as:

  • remove comments, or tags from HTML snippets

  • extract base url from HTML snippets

  • translate entites on HTML strings

  • convert raw HTTP headers to dicts and vice-versa

  • construct HTTP auth header

  • converting HTML pages to unicode

  • sanitize urls (like browsers do)

  • extract arguments from urls

Requirements

Python 2.7 or Python 3.3+

Install

pip install w3lib

Documentation

See http://w3lib.readthedocs.org/

License

The w3lib library is licensed under the BSD license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

w3lib-1.16.0.tar.gz (48.8 kB view details)

Uploaded Source

Built Distribution

w3lib-1.16.0-py2.py3-none-any.whl (18.5 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file w3lib-1.16.0.tar.gz.

File metadata

  • Download URL: w3lib-1.16.0.tar.gz
  • Upload date:
  • Size: 48.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for w3lib-1.16.0.tar.gz
Algorithm Hash digest
SHA256 cbe45d4defe917562c1cc8ffd7ea6a2b9137a6ed33791df0170cd0bcd1db0052
MD5 09be7841a9f5c651bc9e759bed7c7dc5
BLAKE2b-256 c84d47d96235c171e456e711c0e5f14eb836e3215b838b064c1f2e5d336a7ca5

See more details on using hashes here.

Provenance

File details

Details for the file w3lib-1.16.0-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for w3lib-1.16.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 375dd57054c37a47f4c0220d7e7ee3e7f2f410bfb65759e461000fb682fe54c1
MD5 9558137ad92ff9765307de284261b963
BLAKE2b-256 55bfc27ca43ff457e2b476aa67d0623e7ca3b91de1521e8880bf2b20d7309f4c

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page