Skip to main content

Library of web-related functions

Project description

https://secure.travis-ci.org/scrapy/w3lib.png?branch=master

Overview

This is a Python library of web-related functions, such as:

  • remove comments, or tags from HTML snippets

  • extract base url from HTML snippets

  • translate entites on HTML strings

  • convert raw HTTP headers to dicts and vice-versa

  • construct HTTP auth header

  • converting HTML pages to unicode

  • sanitize urls (like browsers do)

  • extract arguments from urls

Requirements

Python 2.7 or Python 3.3+

Install

pip install w3lib

Documentation

See http://w3lib.readthedocs.org/

License

The w3lib library is licensed under the BSD license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

w3lib-1.9.0.tar.gz (12.3 kB view details)

Uploaded Source

Built Distribution

w3lib-1.9.0-py2.py3-none-any.whl (15.3 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file w3lib-1.9.0.tar.gz.

File metadata

  • Download URL: w3lib-1.9.0.tar.gz
  • Upload date:
  • Size: 12.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for w3lib-1.9.0.tar.gz
Algorithm Hash digest
SHA256 b124659467de0a161f17ade88d616c2270356c5eeea66aea20285d92efb515f3
MD5 91411a8b0b52279fd889c7ba12f2aad5
BLAKE2b-256 33940f0aef4fc65e0d3c1c21545bd635350389539397a893786e09ef7f8c8405

See more details on using hashes here.

Provenance

File details

Details for the file w3lib-1.9.0-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for w3lib-1.9.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 5332b7f36ae2e086536f7ea15aced881a34c69816e246755a259da0074b7878c
MD5 fbecbb660efe720efdf55a5b3903405d
BLAKE2b-256 cb87571b640bb0692c0d23fbf79335ff98545a68db204e67779d337f5c58b67c

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page