CARVIEW

MOTORHOMES

Select Language

HTTP/2 200 server: GitHub.com content-type: text/html; charset=utf-8 last-modified: Wed, 20 Aug 2025 17:23:39 GMT access-control-allow-origin: * strict-transport-security: max-age=31556952 etag: W/"68a6049b-7619" expires: Mon, 29 Dec 2025 04:01:44 GMT cache-control: max-age=600 content-encoding: gzip x-proxy-cache: MISS x-github-request-id: 2812:2118F1:8283D3:92DA95:6951FAD0 accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 03:51:45 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210021-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1766980305.881558,VS0,VE225 vary: Accept-Encoding x-fastly-request-id: 34a1c186892e837552d4ea26a8bec429cdfd15c7 content-length: 6241 BEAM2025

The First Workshop on Benchmarking and Expanding AI Multimodal Approaches at CVPR 2025

BEAM2025

Workshop Time, June 11 8:30am - 12:30pm

Workshop location, Music City Center and Online (Hybrid) | Room 202 C

The event will feature discussions highlighting the evolution and future of multimodal AI benchmarks, exploring methodologies and their real-world applications. It will conclude with a dynamic panel discussion, offering insights into the challenges and solutions for creating robust multimodal benchmarks and fostering interdisciplinary collaboration and innovation in AI benchmarking standards.

As a continuation of the 1st Workshop on Evaluation for Generative Foundation Models at CVPR 2024, the 1st Workshop on Benchmarking and Expanding AI Multimodal Approaches at CVPR 2025 aims to build a forum to discuss ongoing efforts in industry and academia, share best practices, and engage the community in working towards more comprehensive AI evaluation framework incorporating audio, visual, and textual inputs, addressing the limitations of current unimodal benchmarks.

Workshop Schedule

Opening Remarks
(Laszlo A. Jeni)

8:30 - 8:40 CDT

Keynote, Katerina Fragkiadaki
(Laszlo A. Jeni)

RobotArena: Towards Unlimited Robot Evaluation via Demonstration-to-Simulation Translation
Abstract 8:40 - 9:15 CDT

Keynote, Sherry Tongshuang Wu
(Mosam Dahbi)

Practical Evaluation of General-Purpose AI
Abstract 9:15 - 9:50 CDT

Oral Presentation
(Yang Zou)

TextInVision: Text and Prompt Complexity Driven Visual Text Generation Benchmark: Forouzan Fallah, Maitreya Patel, Agneet Chatterjee, Vlad Morariu, Chitta Baral, Yezhou Yang
Choosing 'Right' from Wrong: A Closer Look at Selection Bias in Spatial Multiple-Choice Questions in Large Multimodal Models: Giselle Zeno, Nour Jedidi, Steven Gomez

9:50 - 10:05 CDT

Poster Session, Coffee Break
(Ananya Bal)

10:05 - 10:45 CDT

Keynote, Andre Araujo
(Xu Zhang)

What's missing in multimodal AI? Towards spatial awareness, effective tool use and fine-grained understanding
Abstract 10:45 - 11:20 CDT

Invited Paper Presentation from the Main Conference
(Liuyue Xie)

Spatial457: A Diagnostic Benchmark for 6D Spatial Reasoning of Large Multimodal Models
- Poster Session and Location: Exhibit Hall D, Poster #348 @ Sun June 15th 10:30 -- 12:30 CDT (Poster Session 5)
- Paper
VISCO: Benchmarking Fine-Grained Critique and Correction Towards Self-Improvement in Visual Reasoning
- Poster Session and Location: Exhibit Hall D, Poster #396 @ Fri June 13th 16:00 -- 18:00 CDT (Poster Session 2)
- Paper
- Poster
Towards Scalable Human-aligned Benchmark for Text-guided Image Editing
- Poster Session and Location: Exhibit Hall D, Poster #238 @ Sat June 14th 17:00 -- 19:00 CDT (Poster Session 4)
- Paper
- Poster
- Slides

11:20 - 11:40 CDT

Benchmark Promotion
(Zhaowei Cai)

11:40 - 11:45 CDT

Keynote, Huaxiu Yao
(Ce Zheng)

From Generation to Feedback: Benchmark-Guided Reward Modeling and Closed-Loop Alignment for Multimodal Systems
Abstract 11:45 - 12:20 CDT

Closing Remarks
(Davide Modolo)

12:20 - 12:30 CDT

Invited Speakers

Huaxiu Yao

Assistant Professor at University of North Carolina at Chapel Hill

Andre Araujo

Senior Staff Research Scientist / Tech Lead Manager at Google DeepMind

Sherry Tongshuang Wu

Assistant Professor at Carnegie Mellon University

Katerina Fragkiadaki

JPMorgan Chase Associate Professor at Carnegie Mellon University

Accepted Papers

Behind the Magic, MERLIM: Multi-modal Evaluation Benchmark for Large Image-Language Models; Andrés Villa, Juan Léon, Alvaro Soto, Bernard Ghanem.

Beyond Raw Videos: Understanding Edited Videos with Large Multimodal Model; Lu Xu, Sijie Zhu, Chunyuan Li, Chia-Wen Kuo, Fan Chen, Xinyao Wang, Guang Chen, Dawei Du, Ye Yuan, Longyin Wen.

Revisiting Referring Expression Comprehension Evaluation in the Era of Large Multimodal Models; Jierun Chen, Fangyun Wei, Jinjing Zhao, Sizhe Song, Bohuai Wu, Zhuoxuan Peng, S.-H. Gary Chan, Hongyang Zhang.

TextInVision: Text and Prompt Complexity Driven Visual Text Generation Benchmark; Forouzan Fallah, Maitreya Patel, Agneet Chatterjee, Vlad Morariu , Chitta Baral, Yezhou Yang .

Choosing 'Right' from Wrong: A Closer Look at Selection Bias in Spatial Multiple-Choice Questions in Large Multimodal Models; Giselle Zeno, Nour Jedidi, Steven Gomez.

Quantum Federated Learning for Multimodal Data: A Modality-Agnostic Approach; Atit Pokharel, Ratun Rahman, Thomas Morris, Dinh C. Nguyen.

Revisiting Multi-Modal LLM Evaluation; Jian Lu, Junyu Chen, Robik Shrestha, Manoj Acharya, Kushal Kafle, Christopher Kanan.

MerCulture: A Comprehensive Benchmark to Evaluate Vision-Language Models on Cultural Understanding in Singapore; Pranav Tushar, Eshan Pandey, Lyka Diane Bala Austria, Loo Yin Yin, Jing Hao Lim, Indriyati Atmosukarto, Donny Soh Cheng Lock.

KOFFVQA: An Objectively Evaluated Free-form VQA Benchmark for Large Vision-Language Models in the Korean Language; Yoonshik Kim, Jaeyoon Jung.

Reviewers

Ananya Bal

CMU

Avik Kuthiala

Plus

Ce Zheng

CMU

George Z. Wei

CMU

Gozde Sahin

Amazon

Hossein Foroushani

Amazon

Jingru Yi

Amazon

Liuyue Xie

CMU

Morteza Ziyadi

Amazon

Mosam Dabhi

CMU

Nitesh Sekhar

Amazon

Rohan Choudhury

CMU

Saurabh Sahu

Amazon

Shiyu Zhao

Amazon

Songyao Jiang

Amazon

Xiang Yue

CMU

Yang Zou

Amazon

Yicheng Wang

Amazon

Zhikang Zhang

Amazon

Zhu Cheng

Amazon

Program Committee

George Z. Wei

Website Chair & Dataset Chair

CMU

Avik Kuthiala

Dataset Chair

Plus

CMU

We thank Jalpc for the jekyll template

Original Source | Taken Source

The First Workshop on Benchmarking and Expanding AI Multimodal Approaches at CVPR 2025

BEAM2025

Workshop Schedule

Opening Remarks (Laszlo A. Jeni)

Keynote, Katerina Fragkiadaki (Laszlo A. Jeni)

Keynote, Sherry Tongshuang Wu (Mosam Dahbi)

Oral Presentation (Yang Zou)

Poster Session, Coffee Break (Ananya Bal)

Keynote, Andre Araujo (Xu Zhang)

Invited Paper Presentation from the Main Conference (Liuyue Xie)

Benchmark Promotion (Zhaowei Cai)

Keynote, Huaxiu Yao (Ce Zheng)

Closing Remarks (Davide Modolo)

Invited Speakers

Huaxiu Yao

Andre Araujo

Sherry Tongshuang Wu

Katerina Fragkiadaki

Accepted Papers

Reviewers

Program Committee

George Z. Wei

Website Chair & Dataset Chair

Avik Kuthiala

Dataset Chair

Organizers

Laszlo A. Jeni

Morteza Ziyadi

Hao Yang

Xu Zhang

Yang Zou

Zhaowei Cai

Maria Zontak

Davide Modolo

Ashwin Swaminathan

Liuyue Xie

Mosam Dabhi

Xiang Yue

Ce Zheng

Rohan Choudhury

Ananya Bal

Opening Remarks
(Laszlo A. Jeni)

Keynote, Katerina Fragkiadaki
(Laszlo A. Jeni)

Keynote, Sherry Tongshuang Wu
(Mosam Dahbi)

Oral Presentation
(Yang Zou)

Poster Session, Coffee Break
(Ananya Bal)

Keynote, Andre Araujo
(Xu Zhang)

Invited Paper Presentation from the Main Conference
(Liuyue Xie)

Benchmark Promotion
(Zhaowei Cai)

Keynote, Huaxiu Yao
(Ce Zheng)

Closing Remarks
(Davide Modolo)