| CARVIEW |
Select Language
HTTP/2 200
server: GitHub.com
content-type: text/html; charset=utf-8
last-modified: Mon, 13 Mar 2023 08:38:50 GMT
access-control-allow-origin: *
strict-transport-security: max-age=31556952
etag: W/"640ee11a-39e3"
expires: Tue, 30 Dec 2025 14:22:10 GMT
cache-control: max-age=600
content-encoding: gzip
x-proxy-cache: MISS
x-github-request-id: 3D86:2118F1:A184F9:B57E4A:6953DDB9
accept-ranges: bytes
age: 0
date: Tue, 30 Dec 2025 14:12:10 GMT
via: 1.1 varnish
x-served-by: cache-bom-vanm7210032-BOM
x-cache: MISS
x-cache-hits: 0
x-timer: S1767103930.136797,VS0,VE232
vary: Accept-Encoding
x-fastly-request-id: 7b551bf0b2be5d485264a71251b8331fdad1a473
content-length: 5132
HRI 2023 Workshop on Test Methods and Metrics
Despite their availability, accessibility in HRI is still relatively limited. While costs continue to go down as the market grows, commercial robots are still expensive and do not accommodate or benefit all demographics equally. Similarly, manufacturers and potential customers enter the market without clear means of classifying, categorizing, or comparing robots of similar capabilities, making it difficult to make informed decisions. Finally, new researchers entering the field are at a disadvantage as they often lack access to institutional knowledge, best practices, or clear and consistent guidance for conducting directed research.
This year's Workshop on Test Methods and Metrics for Effective Human-Robot Interaction is focused on addressing various aspects of accessibility that act as barriers to entry into the field and market of HRI. In particular, this workshop seeks to identify and address the issues that impact accessibility in terms of human factors, product standardization and development, and growth of the research community. Topics to be addressed include:
This workshop is the fifth in a series of workshops leading toward formalized HRI performance standards. Previous workshops have been leveraged to target community and consensus building, and the IEEE Robotics and Automation Society has advanced two new standards development efforts to support this. The first is focused on developing consistent terminology for HRI technologies, and the second is working to establish best practices in human-subject studies. Discussions of these efforts will be included in the workshop, and related standards meetings will be held in conjunction with the workshop.
5th Annual Workshop on Emerging Test Methods & Metrics for Accessible HRI
@ HRI 2023 - Stockholm, Sweden (& online)
Monday, March 13, 2023 (full-day)
Updates
Join our slack for more information and discussions: https://join.slack.com/t/hri-standards/shared_invite/zt-wovcar3k-Lye8p71x1amdSYgR0MH5bg
1/10/2023: Due to delayed opening of conference registration, we have extended the rolling submission deadlines below.
Please contact shelly.bagchi@nist.gov with any questions about the workshop.
About the workshop
The evolution of human-centric robotics has led to considerable advances in HRI technologies, enabling the rapid growth of the market of "collaborative" robots. From industrial applications to crowd-sourced social robot platforms, robots are more available now than ever.Despite their availability, accessibility in HRI is still relatively limited. While costs continue to go down as the market grows, commercial robots are still expensive and do not accommodate or benefit all demographics equally. Similarly, manufacturers and potential customers enter the market without clear means of classifying, categorizing, or comparing robots of similar capabilities, making it difficult to make informed decisions. Finally, new researchers entering the field are at a disadvantage as they often lack access to institutional knowledge, best practices, or clear and consistent guidance for conducting directed research.
This year's Workshop on Test Methods and Metrics for Effective Human-Robot Interaction is focused on addressing various aspects of accessibility that act as barriers to entry into the field and market of HRI. In particular, this workshop seeks to identify and address the issues that impact accessibility in terms of human factors, product standardization and development, and growth of the research community. Topics to be addressed include:
- design considerations and behaviors of robotic systems to ensure a broad user base without regard to demographics, physical capabilities, or experience with robots;
- policies and procedures that reflect the diverse robotics market;
- lowering the barrier to entry to HRI research by providing clear and consistent guidance for new researchers in an inclusive and equitable community;
- specifications and terminology to consistently describe the capabilities and performance of human-centric robotic solutions; and
- the provision, development, and growth of publicly-available tools, data sets, and knowledge bases to increase the availability and quality of research findings.
This workshop is the fifth in a series of workshops leading toward formalized HRI performance standards. Previous workshops have been leveraged to target community and consensus building, and the IEEE Robotics and Automation Society has advanced two new standards development efforts to support this. The first is focused on developing consistent terminology for HRI technologies, and the second is working to establish best practices in human-subject studies. Discussions of these efforts will be included in the workshop, and related standards meetings will be held in conjunction with the workshop.
Schedule:
| Time (CEST) | Time (EST) | Subject | Details |
| 0900 | 0400 | Introduction | |
| 0920 | 0420 | Invited Speaker: Maartje de Graaf | Human-factors for accessibility |
| 1000 | 0500 | Questions and Answers | |
| 1020 | 0520 | Breakout Session | Accessibility of Robots |
| 1100 | 0600 | Coffee Break | |
| 1130 | 0630 | Invited Speaker: Paul Schmitt | Do You See Me? Challenges and Opportunities for Pedestrian and Autonomous Vehicle Research and Standards |
| 1210 | 0710 | Questions and Answers | |
| 1230 | 0730 | Breakout Session | |
| 1300 | 0800 | Lunch | |
| 1400 | 0900 | Jeremy Marvel | A Gentle Introduction to Standards |
| 1420 | 0920 | Tutorial | IEEE P3108 Reporting Form |
| 1500 | 1000 | Invited Speakers: Reuben Aronson and Elaine Short | Accessibility of Research |
| 1540 | 1040 | Questions and Answers | |
| 1600 | 1100 | Coffee Break | |
| 1630 | 1130 | Short Paper Talks |
Considerations for Minimizing Data Collection Biases for Eliciting Natural Behavior in HRI Snehesh Shrestha, Ge Gao, Cornelia Fermuller, Yiannis Aloimonos Metrics for Evaluation of Mixed Reality Technologies for Robotic Assembly of Manufacturing Tasks Medhavi Kamran, Shyam Sasi Pallissery, Shelly Bagchi, Jeremy Marvel, Megan Zimmerman, Vinh Nguyen |
| 1700 | 1200 | Breakout Session | |
| 1730 | 1230 | Wrap Up | |
| 1745 | 1245 | Adjourn |
Call for Papers
We invite submission of extended abstracts (1-2p, references excluded) for presentation at the workshop. Submissions may be made under the following main areas of discussion:
- human-centric design of robotic systems to accommodate different demographics;
- finding, accessing, and understanding standards and specifications for HRI-relevant topics; and
- best practices in HRI research, including how to conduct human subject studies, creating statistical designs of experiments, and assembling knowledge bases to capture information that is not generally reported in published papers.
Please follow the HRI23 short paper submission instructions regarding format and accessibility requirements. Papers do not need to be anonymized.
Submissions can be made at this form: https://forms.gle/xQDfBbUhwLvYezYS6.
For any questions, please send an email to shelly.bagchi@nist.gov. We do not intend these topics to be restrictive, so feel free to contact us if your paper is slightly outside our listed subject areas.
Important dates
We will accept submissions on a rolling basis this year. We aim to provide notifications about 1-2 weeks after submission.
- 27 January 2023 (Friday): Submission deadline for notification before early-bird registration deadline (Jan 31)
- 24 February 2023 (Friday): Final deadline for rolling submissions
- 3 March 2023: Notification of acceptance for workshop presentations
- 13 March 2023: Workshop takes place
Organizers
Dr. Jeremy A. Marvel, U.S. DOC, National Institute of Standards and Technology (NIST) - jeremy.marvel@nist.gov
Shelly Bagchi, U.S. DOC, NIST - shelly.bagchi@nist.gov
Megan Zimmerman, U.S. DOC, NIST - megan.zimmerman@nist.gov
Dr. Jennifer Case, U.S. DOC, NIST - jennifer.case@nist.gov
Murat Aksu, U.S. DOC, NIST - murat.aksu@nist.gov
Brian Antonishek, U.S. DOC, NIST - brian.antonishek@nist.gov
Dr. Terry Fong, NASA Ames Research Center - terry.fong@nasa.gov
Dr. Vinh Nguyen, Michigan Technical University - vinhn@mtu.edu
Adam Norton, UMass Lowell - adam_norton@uml.edu
Dr. Emmanuel Senft, Idiap Research Institute - esenft@idiap.ch
Dr. Yue Wang, Clemson University - yue6@clemson.edu