Skip to main content

A high-level cross-protocol url-grabber

Project description

A high-level cross-protocol url-grabber.

Using urlgrabber, data can be fetched in three basic ways:

urlgrab(url) copy the file to the local filesystem
urlopen(url) open the remote file and return a file object
(like urllib2.urlopen)
urlread(url) return the contents of the file as a string

When using these functions (or methods), urlgrabber supports the
following features:

* identical behavior for http://, ftp://, and file:// urls
* http keepalive - faster downloads of many files by using
only a single connection
* byte ranges - fetch only a portion of the file
* reget - for a urlgrab, resume a partial download
* progress meters - the ability to report download progress
automatically, even when using urlopen!
* throttling - restrict bandwidth usage
* retries - automatically retry a download if it fails. The
number of retries and failure types are configurable.
* authenticated server access for http and ftp
* proxy support - support for authenticated http and ftp proxies
* mirror groups - treat a list of mirrors as a single source,
automatically switching mirrors if there is a failure.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

urlgrabber-3.9.1.tar.gz (72.1 kB view details)

Uploaded Source

File details

Details for the file urlgrabber-3.9.1.tar.gz.

File metadata

  • Download URL: urlgrabber-3.9.1.tar.gz
  • Upload date:
  • Size: 72.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for urlgrabber-3.9.1.tar.gz
Algorithm Hash digest
SHA256 b4e276fa968c66671309a6d754c4b3b0cb2003dec8bca87a681378a22e0d3da7
MD5 63424f4e3675c8df474bd5bcac1f4790
BLAKE2b-256 3cfd710150d9647e32f1eafe9d60ff55553a8754e185c791781da0246c7d6b57

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page