| CARVIEW |
From Sim-to-Real: Toward General Event-based Low-light Frame Interpolation with Per-scene Optimization
*Authors contributed equally. †Corresponding authors.
Abstract
Video Frame Interpolation (VFI) is important for video enhancement, frame rate up-conversion, and slow-motion generation. The introduction of event cameras, which capture per-pixel brightness changes asynchronously, has significantly enhanced VFI capabilities, particularly for high-speed, nonlinear motions. However, these event-based methods encounter challenges in low-light conditions, notably trailing artifacts and signal latency, which hinder their direct applicability and generalization. Addressing these issues, we propose a novel per-scene optimization strategy tailored for low-light conditions. This approach utilizes the internal statistics of a sequence to handle degraded event data under low-light conditions, improving the generalizability to different lighting and camera settings. To evaluate its robustness in low-light condition, we further introduce EVFI-LL, a unique RGB+Event dataset captured under low-light conditions. Our results demonstrate state-of-the-art performance in low-light environments. Both the dataset and the source code will be made publicly available upon publication.
Video
Sim2Real Presentation
Presentation Slides
The presentation is available for download. The slides contain detailed information about our methodology, experiments, and results.
Download SlidesBibTeX
@article{zhang2024sim,
title={From Sim-to-Real: Toward General Event-based Low-light Frame Interpolation with Per-scene Optimization},
author={Zhang, Ziran and Ma, Yongrui and Chen, Yueting and Zhang, Feng and Gu, Jinwei and Xue, Tianfan and Guo, Shi},
journal={arXiv preprint arXiv:2406.08090},
year={2024}
}