Skip to main content

A high-level cross-protocol url-grabber

Project description

A high-level cross-protocol url-grabber.

Using urlgrabber, data can be fetched in three basic ways:

urlgrab(url) copy the file to the local filesystem
urlopen(url) open the remote file and return a file object
(like urllib2.urlopen)
urlread(url) return the contents of the file as a string

When using these functions (or methods), urlgrabber supports the
following features:

* identical behavior for http://, ftp://, and file:// urls
* http keepalive - faster downloads of many files by using
only a single connection
* byte ranges - fetch only a portion of the file
* reget - for a urlgrab, resume a partial download
* progress meters - the ability to report download progress
automatically, even when using urlopen!
* throttling - restrict bandwidth usage
* retries - automatically retry a download if it fails. The
number of retries and failure types are configurable.
* authenticated server access for http and ftp
* proxy support - support for authenticated http and ftp proxies
* mirror groups - treat a list of mirrors as a single source,
automatically switching mirrors if there is a failure.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

urlgrabber-3.10.2.tar.gz (84.6 kB view details)

Uploaded Source

File details

Details for the file urlgrabber-3.10.2.tar.gz.

File metadata

  • Download URL: urlgrabber-3.10.2.tar.gz
  • Upload date:
  • Size: 84.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for urlgrabber-3.10.2.tar.gz
Algorithm Hash digest
SHA256 05b7164403d49b37fe00f7ac8401e56b00d0568ac45ee15d5f0610ac293c3070
MD5 20a83a415287e4ae232996ce12d95f8f
BLAKE2b-256 291af509987826e17369c52a80a07b257cc0de3d7864a303175f2634c8bcb3e3

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page