GEO-Bench: Toward Foundation Models for Earth Monitoring

GeoBench is a ServiceNow Research project.

GEO-Bench is a General Earth Observation benchmark for evaluating the performances of large pre-trained models on geospatial data. Read the full paper for usage details and evaluation of existing pre-trained vision models.

Installation

You can install GEO-Bench with pip:

pip install geo-benchmark

Downloading the data

Set $GEO_BENCH_DIR to your preferred location. If not set, it will be stored in $HOME/dataset/geobench.

Next, use the download script. This will automatically download from Zenodo

Run the command:

geobench-download

The current version of the benchmark is 0.9.1. It will soon be updated to incorporate minor changes

This will download all datasets in parallel. If some files are already downloaded, it will verify the md5 checksum. Feel free to restart the downloader if it is interrupted or if you get Error: TOO MANY REQUESTS. m-bigearthnet takes the longest time and Zenodo is a bit slow some days.

Test installation

Make sure the benchmark is downloaded before launching tests. Optionnaly run tests:

geobench-test

Loading Datasets

See example_load_dataset.py for how to iterate over datasets.

import geobench

for task in geobench.task_iterator(benchmark_name="classification_v0.9.1"):
    dataset = task.get_dataset(split="train")
    sample = dataset[0]
    for band in sample.bands:
        print(f"{band.band_info.name}: {band.data.shape}")

Visualizing Results

See the notebook baseline_results.ipynb for an example of how to visualize the results.