GEO-Bench is a ServiceNow Research project.

GEO-Bench is a General Earth Observation benchmark for evaluating the performances of large pre-trained models on geospatial data. Read the full paper for usage details and evaluation of existing pre-trained vision models.

You can install GEO-Bench with pip:

pip install geobench

Note: Python 3.9+ is required.

Set $GEO_BENCH_DIR to your preferred location. If not set, it will be stored in $HOME/dataset/geobench.

Next, use the download script. This will automatically download from Hugging Face

Run the command:

geobench-download

You need ~65 GB of free disk space for download and unzip (once all .zip are deleted it takes 57GB). If some files are already downloaded, it will verify the md5 checksum. Feel free to restart the downloader if it is interrupted.

You can run tests. Note: Make sure the benchmark is downloaded before launching tests.

pip install pytest

geobench-test

See example_load_dataset.py for how to iterate over datasets.

import geobench
for task in geobench.task_iterator(benchmark_name="classification_v1.0"):
    dataset = task.get_dataset(split="train")
    sample = dataset[0]
    for band in sample.bands:
        print(f"{band.band_info.name}: {band.data.shape}")

See the code for reproducing experiments as a starting point for fine-tuning:

geo-bench-experiments

See the notebook baseline_results.ipynb for an example of how to visualize the results.