Skip to main content

Library of web-related functions

Project description

https://secure.travis-ci.org/scrapy/w3lib.png?branch=master

Overview

This is a Python library of web-related functions, such as:

  • remove comments, or tags from HTML snippets

  • extract base url from HTML snippets

  • translate entites on HTML strings

  • convert raw HTTP headers to dicts and vice-versa

  • construct HTTP auth header

  • converting HTML pages to unicode

  • sanitize urls (like browsers do)

  • extract arguments from urls

Requirements

Python 2.7 or Python 3.3+

Install

pip install w3lib

Documentation

See http://w3lib.readthedocs.org/

License

The w3lib library is licensed under the BSD license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

w3lib-1.7.1.tar.gz (12.3 kB view details)

Uploaded Source

Built Distribution

w3lib-1.7.1-py2.py3-none-any.whl (14.7 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file w3lib-1.7.1.tar.gz.

File metadata

  • Download URL: w3lib-1.7.1.tar.gz
  • Upload date:
  • Size: 12.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for w3lib-1.7.1.tar.gz
Algorithm Hash digest
SHA256 0d5084f3aa6f6be4f2be969d2e2c6d1bd38f410537583e0c1f3758fe49242669
MD5 2ecb72f40c3b0de2abdc3859c633d56b
BLAKE2b-256 34c294491be3c840f9f88517a7252b3f98544b9f22eb41360f5e2cd7fb0e908c

See more details on using hashes here.

Provenance

File details

Details for the file w3lib-1.7.1-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for w3lib-1.7.1-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 a86f530fdc284db86d838e8b11d0f4c08bf80b640cc567c70b06e497841e1a34
MD5 388dcfeaa58a1bf25b678dc3a13e039f
BLAKE2b-256 1818bec2e27b69def41b244c0017a8e70d5591c3b4f429fe142f5876d1be2177

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page