Skip to main content

ASF Sentinel-1 Metadata Download tool

Project description

ASF Sentinel-1 Metadata Download tool

Small Python tool (asfsmd) that allows to download XML files containing Sentinel-1 products metadata from the ASF archive.

Sentinel-1 products are stored in the ASF arcive as ZIP files that are quite large because they contain both the products annotations and the binary image data.

The asfsmd tool is able to retrieve only the relatively samll annotation files (in XML format) without downloading the entire ZIP archive.

asfsmd exploits Python packages like fsspec or httpio for reading HTTP resources as random-access file-like objects. In order to do it the remote server must support the Range header.

This approach allows to open the ZIP archive remotely, inspects contents, and download only the pieces of data that are actually necessary to the user.

Performnces of this approach are quite poor but, in the specific case of Sentinel-1 products, the entire process results to be faster than downloading the entire ZIP archive and extracting only annotation files.

Command Line Interface

$ python3 -m asfsmd --help

usage: asfsmd [-h] [--version]
              [--loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
              [-q] [-v] [-d]
              [-f] [--urls] [-o OUTDIR] [-u USERNAME] [-p PASSWORD]
              [--block-size BLOCK_SIZE]
              [-b {s1,s2,s3,s4,s5,s6,iw1,iw2,iw3,ew1,ew2,ew3,ew4,ew5,wv1,wv2}]
              [--pol {vv,vh,hv,hh}] [-c] [-n] [-r] [--data]
              INPUT [INPUT ...]

ASF Sentinel-1 Metadata Download tool. Small Python tool (`asfsmd`) that
allows to download XML files containing Sentinel-1 products metadata from
the ASF archive. Sentinel-1 products are stored in the ASF arcive as ZIP
files that are quite large because they comntain both the products
annotations and the binary image data. The `asfsmd` tool is able to
retrieve only the relatively small annotation files (in XML format) without
downloading the entire ZIP archive.

positional arguments:
  INPUT                 Sentinel-1 product name(s). If the '-f' flag is set
                        then the argument is interpreted as the filename
                        containing the list of products. If the '--urls'
                        flag is set then the arguments are interpreted as
                        URLs pointing to product on the ASF server.
                        See '--file--list' and the '--urls' options for
                        more details.

options:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}
                        logging level (default: WARNING)
  -q, --quiet           suppress standard output messages, only errors are
                        printed to screen
  -v, --verbose         print verbose output messages
  -d, --debug           print debug messages
  -f, --file-list       read the list of products from a file. The file
                        can be a JSON file (with '.json' extension) or a
                        text file.The text file is expected to contain one
                        product name per line.The json file can contain a
                        list of products or a dictionary containing a list
                        of products for each key.In this case the key is
                        used as sub-folder name to store the corresponding
                        products.Example: <OUTDIR>/<KEY>/<PRODUCT>
  --urls                Indicate the inputs are a list of URLs from ASF.
  -o OUTDIR, --outdir OUTDIR
                        path of the output directory (default='.')
  -u USERNAME, --username USERNAME
                        username for ASF authentication. If not provided
                        the tool attempts to retrieve the authentication
                        parameters for the user's '.netrc' file looking
                        for the host 'urs.earthdata.nasa.gov'
  -p PASSWORD, --password PASSWORD
                        password for ASF authentication. If not provided
                        the tool attempts to retrieve the authentication
                        parameters for the user's '.netrc' file looking
                        for the host 'urs.earthdata.nasa.gov'
  --block-size BLOCK_SIZE
                        httpio block size in bytes (default: 1024)
  -b {s1,s2,s3,s4,s5,s6,iw1,iw2,iw3,ew1,ew2,ew3,ew4,ew5,wv1,wv2},
  --beam {s1,s2,s3,s4,s5,s6,iw1,iw2,iw3,ew1,ew2,ew3,ew4,ew5,wv1,wv2}
                        Choose only one beam to download. If not provided
                        all beams are downloaded.
  --pol {vv,vh,hv,hh}   Choose only one polarization to download. If not
                        provided both polarizations are downloaded.
  -c, --cal             Download calibration files.
  -n, --noise           Download noise calibration files.
  -r, --rfi             Download RFI files.
  --data                Download measurement files.

License

Copyright (c) 2021-2022 Antonio Valentino

The asfsmd package is distributed under the MIT License.

asfsmd release history

asfsmd v1.3.0 (18/12/2022)

asfsmd v1.2.0 (04/12/2022)

  • Refactoring to convert the asfsmd.py module into a package.

  • Support multiple backends for remote file access: httpio, fsspec and remotezip. The httpio based implementation seems to be slightly faster w.r.t. the other ones.

  • Fix issue with the management of default values for the make_patterns function.

  • Improve the management of the download of large files (chunking and progress).

asfsmd v1.1.0 (03/12/2022)

  • Now it is possible to customize the selection of files to be downloaded. Beyond the manifest and the annotation files, now it is also possible to download:

    • calibration annotations

    • noise annotations

    • rfi annotations

    • measurement files

    Moreover now it is possible to select a specific beams or polarizations. Patch developed by @scottstanie and @avalentino.

  • Restore compatibility with Python 3.6.

asfsmd v1.0.0 (09/01/2022)

Initial release.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

asfsmd-1.3.0.tar.gz (13.0 kB view details)

Uploaded Source

Built Distribution

asfsmd-1.3.0-py3-none-any.whl (13.8 kB view details)

Uploaded Python 3

File details

Details for the file asfsmd-1.3.0.tar.gz.

File metadata

  • Download URL: asfsmd-1.3.0.tar.gz
  • Upload date:
  • Size: 13.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.10.7

File hashes

Hashes for asfsmd-1.3.0.tar.gz
Algorithm Hash digest
SHA256 1c4deb4077e3520e4e574f6537156e4eeb941b3719077f060ba9bed564c6397d
MD5 8e7bd30ae71bd25a9c3ad9339a5b5733
BLAKE2b-256 6a4b434c52a5b36fd8257d720fc9d2522d5aad340b2b4c1ef49e86ab02571945

See more details on using hashes here.

File details

Details for the file asfsmd-1.3.0-py3-none-any.whl.

File metadata

  • Download URL: asfsmd-1.3.0-py3-none-any.whl
  • Upload date:
  • Size: 13.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.10.7

File hashes

Hashes for asfsmd-1.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 072f65b3e6fb5e0777e6f8443b7387f0d1bc8bb3b527a6de9b6f20cd2d8551d3
MD5 7ef2bb9dacbdabedd4070fd35ea44a4e
BLAKE2b-256 efd96fddcf86d1c3d66749b4c038ad2a4c5e43e35b0c7caedd7fba0f194e0bc3

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page