🎉 This work has been accepted to ICCV 2025!

Egocentric Visual Query Localization (EgoVQL) aims to locate a target object in both space and time within first-person videos based on a given visual query. However, existing methods often struggle with significant appearance variations and cluttered backgrounds, leading to reduced localization accuracy.

To overcome these challenges, PRVQL introduces a progressive knowledge-guided refinement approach. By dynamically extracting and refining knowledge from the video itself, PRVQL continuously enhances query and video features across multiple stages, resulting in more accurate localization.

PRVQL employs appearance and spatial knowledge extraction modules at each stage to iteratively refine the query and video features. This progressive refinement leads to increasingly accurate localization results.

Set up your environment with the following commands:

conda create --name prvql python=3.8 -y
conda activate prvql
conda install pytorch==1.12.0 torchvision==0.13.0 torchaudio==0.12.0 cudatoolkit=11.6 -c pytorch -c conda-forge
pip install -r requirements.txt

Download the pretrained model weights from Google Drive and place them in:

./output/ego4d_vq2d/train/train

Follow the instructions in the VQLoC repository to process the dataset into video clips and images.

Ensure the dataset is structured as follows:

./your/dataset/path/
└── datav2
    ├── clips
    │   ├── 1.mp4
    │   └── ...
    ├── images
    │   ├── 1   
    │   │   ├── 1.mp4
    │   │   └── ...
    │   └── ...        
    ├── train_annot.json
    ├── val_annot.json
    ├── vq_test_unannotated.json
    ├── vq_train.json
    └── vq_val.json

Modify the dataset path in the following configuration files:

• config/eval.yaml
• config/train.yaml
• config/val.yaml

Update the dataset root path:

root: './your/dataset/path/'

We will release the model and code soon.

If you find this repository useful, please consider starring ⭐ it and citing our work:

@article{fan2025prvql,
  title={PRVQL: Progressive Knowledge-Guided Refinement for Robust Egocentric Visual Query Localization},
  author={Fan, Bing and Feng, Yunhe and Tian, Yapeng and Lin, Yuewei and Huang, Yan and Fan, Heng},
  journal={arXiv preprint arXiv:2502.07707},
  year={2025}
}