Skip to main content

Library of web-related functions

Project description

https://secure.travis-ci.org/scrapy/w3lib.png?branch=master

Overview

This is a Python library of web-related functions, such as:

  • remove comments, or tags from HTML snippets

  • extract base url from HTML snippets

  • translate entites on HTML strings

  • convert raw HTTP headers to dicts and vice-versa

  • construct HTTP auth header

  • converting HTML pages to unicode

  • sanitize urls (like browsers do)

  • extract arguments from urls

Requirements

Python 2.7 or Python 3.3+

Install

pip install w3lib

Documentation

See http://w3lib.readthedocs.org/

License

The w3lib library is licensed under the BSD license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

w3lib-1.7.0.tar.gz (12.0 kB view details)

Uploaded Source

Built Distribution

w3lib-1.7.0-py2-none-any.whl (14.7 kB view details)

Uploaded Python 2

File details

Details for the file w3lib-1.7.0.tar.gz.

File metadata

  • Download URL: w3lib-1.7.0.tar.gz
  • Upload date:
  • Size: 12.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for w3lib-1.7.0.tar.gz
Algorithm Hash digest
SHA256 76ecda47264ddfb22873e22ca34c0a90525f2aa0721e4d28234cdeb6fda1a177
MD5 31dc535de3684bc4e5f60a3b2558d1c5
BLAKE2b-256 83a99233f46fdf2378d6bc884fa2d8e01a60fb6313c5c6544bd4b80a24bd5190

See more details on using hashes here.

Provenance

File details

Details for the file w3lib-1.7.0-py2-none-any.whl.

File metadata

File hashes

Hashes for w3lib-1.7.0-py2-none-any.whl
Algorithm Hash digest
SHA256 1c75d1752703e17a4d7772fd56e63d2f1f236f3758f5ca3b93a9272e0beaf25d
MD5 fa1d6d1159e720f80a8b76d0c729f0ad
BLAKE2b-256 48849af1710553c760bee60125a7c9745ebc96981d788da75d56a84157bd804f

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page