You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In this work we present HyperE2VID, a dynamic neural network architecture for event-based video reconstruction. Our approach extends existing static architectures by using hypernetworks and dynamic convolutions to generate per-pixel adaptive filters guided by a context fusion module that combines information from event voxel grids and previously reconstructed intensity images. We show that this dynamic architecture can generate higher-quality videos than previous state-of-the-art, while also reducing memory consumption and inference time.
The pretrained model of HyperE2VID can be found here.
For evaluation and analysis of HyperE2VID model, please use the codes in EVREAL repository.
Instructions to generate training data can be found in the datagen folder.
Training codes will be published soon.
Citations
If you use code in this repo in an academic context, please cite the following:
@article{ercan2024hypere2vid,
title={{HyperE2VID}: Improving Event-Based Video Reconstruction via Hypernetworks},
author={Ercan, Burak and Eker, Onur and Saglam, Canberk and Erdem, Aykut and Erdem, Erkut},
journal={IEEE Transactions on Image Processing},
year={2024},
volume={33},
pages={1826--1837},
doi={10.1109/TIP.2024.3372460},
publisher={IEEE}}
Acknowledgements
This work was supported in part by KUIS AI Center Research Award, TUBITAK-1001 Program Award No. 121E454, and BAGEP 2021 Award of the Science Academy to A. Erdem.
This code borrows from or is inspired by the following open-source repositories: