Robot Programming Methods & Tools

Verifying Social HRI

With RoVer, interaction designers can program interactions and receive feedback on social norms being violated.

Adapting Robot Programs

Robot end-users provide feedback to a robot, which then automatically updates its own interaction program.

Bodystorming HRI

Two users role-play, or bodystorm, an interaction, which is then synthesized into a human-robot interaction program.

The goal of this project is to investigate and create novel representations, methods, and tools that enable the programming of interactive robots by designers and end-user programmers with a wide range of backgrounds and experience in robotics. Our research involves three key components:

Capturing user intent

We create novel interfaces that translate designer and end-user-programmer input into executable robot programs, which can involve techniques such as Programming by Demonstration (PbD), embodied design methods, and natural language instruction

Iterative development and testing

We transform the pipeline for how these programs are tested and refined, studying and improving how non-experts may receive and integrate feedback into their programs

User studies and real-world deployment

We seek to evaluate the efficacy of our representations, methods, and tools from the perspectives of both experienced and inexperienced robot programmers, in addition to the individuals who interact with the resulting programs

This work has been supported by National Science Foundation awards 1426824, 1651129, and 1925043

Related Publications

Schoen, A., D. Sullivan, H. Zhang, D. Rakita, and B. Mutlu. Lively: Enabling Multimodal, Lifelike, and Extensible Real-Time Robot Motion. Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’23), 2023.

Read more Document
Porfirio, D., L. Stegner, M. Cakmak, A. Sauppé, A. Albarghouthi, and B. Mutlu. “"Sketching Robot Programs On the Fly"”. Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’23), Association for Computing Machinery, 2023, p. 584-93.

Read more Document
Schoen, A., N. White, C. Henrichs, A. Siebert-Evenstone, D. Shaffer, and B. Mutlu. “CoFrame: A System for Training Novice Cobot Programmers”. Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction, IEEE Press, 2022, pp. 185–194.

Read more Document
Hagenow, M., E. Senft, R. Radwin, M. Gleicher, B. Mutlu, and M. Zinn. “Corrective Shared Autonomy for Addressing Task Variability”. IEEE Robotics and Automation Letters, Vol. 6, no. 2, 2021, pp. 3720-7.

Read more Document
Porfirio, D., L. Stegner, M. Cakmak, A. Sauppé, A. Albarghouthi, and B. Mutlu. “"Figaro: A Tabletop Authoring Environment for Human-Robot Interaction"”. CHI ’21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Association for Computing Machinery, 2021, pp. 1-15.

Read more Document
Porfirio, D., A. Sauppé, A. Albarghouthi, and B. Mutlu. “Transforming Robot Programs Based on Social Context”. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 2020, pp. 1-12.
Schoen, A., C. Henrichs, M. Strohkirch, and B. Mutlu. “Authr: A Task Authoring Environment for Human-Robot Teams”. Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, 2020, pp. 1194-08.

Read more Document
Porfirio, D., E. Fisher, A. Sauppé, A. Albarghouthi, and B. Mutlu. “Bodystorming Human-Robot interactions”. Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, ACM, 2019, pp. 479-91.
Porfirio, D., A. Sauppé, A. Albarghouthi, and B. Mutlu. “Authoring and Verifying Human-Robot interactions”. Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, 2018, pp. 75-86.