| CARVIEW |
Select Language
HTTP/2 200
server: GitHub.com
content-type: text/html; charset=utf-8
last-modified: Sun, 16 Feb 2025 14:40:33 GMT
access-control-allow-origin: *
strict-transport-security: max-age=31556952
etag: W/"67b1f8e1-59ce"
expires: Mon, 29 Dec 2025 11:59:54 GMT
cache-control: max-age=600
content-encoding: gzip
x-proxy-cache: MISS
x-github-request-id: 6B13:328FD3:8C7622:9DA37D:69526AE1
accept-ranges: bytes
age: 0
date: Mon, 29 Dec 2025 11:49:54 GMT
via: 1.1 varnish
x-served-by: cache-bom-vanm7210041-BOM
x-cache: MISS
x-cache-hits: 0
x-timer: S1767008994.963408,VS0,VE216
vary: Accept-Encoding
x-fastly-request-id: b1305e3e070c8e04262f6809128086d8ac714fd6
content-length: 7505
RSS RoboEval 2025
1st Workshop on Evaluating Robots for the Real World: Aligning Academia, Industry, and Policymakers (RoboEval)
Proposed Workshop @ Robotics Science and Systems Conference (RSS 2025) - June 25 - Los Angeles, USA
About
There has been immense progress from the scientific community in developing methods and systems that boast impressive performance on public benchmarks. However, systems still struggle with generalization, robustness, safety, and reliability when deployed to real-world settings (e.g., factories, construction sites, residential homes). In addition to the notion of optimising the algorithms and methods for deployment and transferability, this workshop raises such questions as "Should we also optimize our benchmarks to be more representative assessments of good real-world behavior?" and "Should we take more care in assessing the current level of robot capabilities, so that we know when significant and readily-deployable advancements are made?" We assert that these questions point to critical challenges in evaluating robots in the real world, which we organize into three categories:
There has been immense progress from the scientific community in developing methods and systems that boast impressive performance on public benchmarks. However, systems still struggle with generalization, robustness, safety, and reliability when deployed to real-world settings (e.g., factories, construction sites, residential homes). In addition to the notion of optimising the algorithms and methods for deployment and transferability, this workshop raises such questions as "Should we also optimize our benchmarks to be more representative assessments of good real-world behavior?" and "Should we take more care in assessing the current level of robot capabilities, so that we know when significant and readily-deployable advancements are made?" We assert that these questions point to critical challenges in evaluating robots in the real world, which we organize into three categories:
- Evaluations and Progress. How do we ensure evaluations are created to drive meaningful advancements in the field without causing too many barriers to stifle progress?
- Accessibility and Relevance. Should benchmarks emphasize the full complexity of real-world deployment challenges, or should they be simplified to encourage broader participation and replicability across diverse research contexts?
- Alignment Across Stakeholders. What are the needs of evaluations for different stakeholders? Should benchmarks address the needs of academia, industry, and policy simultaneously? Or developed more to fit individual needs?
Speakers and Panelists
