Skip to main content

Library of web-related functions

Project description

https://secure.travis-ci.org/scrapy/w3lib.png?branch=master

Overview

This is a Python library of web-related functions, such as:

  • remove comments, or tags from HTML snippets

  • extract base url from HTML snippets

  • translate entites on HTML strings

  • convert raw HTTP headers to dicts and vice-versa

  • construct HTTP auth header

  • converting HTML pages to unicode

  • sanitize urls (like browsers do)

  • extract arguments from urls

Requirements

Python 2.7 or Python 3.3+

Install

pip install w3lib

Documentation

See http://w3lib.readthedocs.org/

License

The w3lib library is licensed under the BSD license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

w3lib-1.10.0.tar.gz (12.3 kB view details)

Uploaded Source

Built Distribution

w3lib-1.10.0-py2.py3-none-any.whl (14.8 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file w3lib-1.10.0.tar.gz.

File metadata

  • Download URL: w3lib-1.10.0.tar.gz
  • Upload date:
  • Size: 12.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for w3lib-1.10.0.tar.gz
Algorithm Hash digest
SHA256 670cd9647dfc9f5adbf973c300bc62f3ba6687acfed93b67527f4cca5459380b
MD5 4188d2dbc790dd07c6b3877dcd0ebd7b
BLAKE2b-256 3a135ad14fea89a10c373737f76fc078d04c0b56974b49b8afecb830d1ca3982

See more details on using hashes here.

Provenance

File details

Details for the file w3lib-1.10.0-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for w3lib-1.10.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 4c71b69160e26ded1cad2e0462622de30294c90df3a27420cbbc6530591e0678
MD5 971e8fff6286a697837c26c7b421d856
BLAKE2b-256 7f6885d31f2132d36bc89f19d80868c832adb7c4d029a8ba8dae6cba9c1f768e

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page