Skip to main content

A high-level cross-protocol url-grabber

Project description

A high-level cross-protocol url-grabber.

Using urlgrabber, data can be fetched in three basic ways:

urlgrab(url) copy the file to the local filesystem
urlopen(url) open the remote file and return a file object
(like urllib2.urlopen)
urlread(url) return the contents of the file as a string

When using these functions (or methods), urlgrabber supports the
following features:

* identical behavior for http://, ftp://, and file:// urls
* http keepalive - faster downloads of many files by using
only a single connection
* byte ranges - fetch only a portion of the file
* reget - for a urlgrab, resume a partial download
* progress meters - the ability to report download progress
automatically, even when using urlopen!
* throttling - restrict bandwidth usage
* retries - automatically retry a download if it fails. The
number of retries and failure types are configurable.
* authenticated server access for http and ftp
* proxy support - support for authenticated http and ftp proxies
* mirror groups - treat a list of mirrors as a single source,
automatically switching mirrors if there is a failure.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

urlgrabber-4.0.0.tar.gz (86.3 kB view details)

Uploaded Source

File details

Details for the file urlgrabber-4.0.0.tar.gz.

File metadata

  • Download URL: urlgrabber-4.0.0.tar.gz
  • Upload date:
  • Size: 86.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: Python-urllib/2.7

File hashes

Hashes for urlgrabber-4.0.0.tar.gz
Algorithm Hash digest
SHA256 79c5a01c5dd31906a7f38ef1f500030e137704804d585644693d3e474ed15f39
MD5 7c0011462b7225e7ce70615fd0e86581
BLAKE2b-256 ad9fcdd1619aa18f1bcf442032cdc0878196400270abc99035d6cf98901e51a1

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page