{Seminar} @ CDS: 28th July: “Contact-Rich Manipulation by Humans and Robots”

Our daily manipulation activities are rich in contact with surfaces in the environment. Understanding this contact is the first step towards building contact-friendly robots. In the first part of this talk, I will discuss a series of projects aimed at accurately observing hand-object contact, synchronized with RGB-D images. The data from these methods has been used to train image-based hand-object contact prediction models for human-computer interaction and virtual reality applications. In the second part I will discuss contact-friendly robot manipulation systems, which are inspired by the contact-rich nature of our own daily interactions with the world. In particular, I will talk about robots that use gripper contact inferred from an on-board camera to perform precise grasping of tiny objects, and another robot that can insert already grasped objects in tight locations by leveraging force sensor feedback. Together, these works demonstrate a progression from systems that can perceive human hand-object contact to robot systems that can leverage contact feedback to perform difficult real-world tasks.