Dexterous telemanipulation aims to realize real-time mapping between a human operator’s hand and a distant robot hand to perform tasks that require precise, coordinated manipulation. Progress is fundamentally limited by three unresolved challenges:
(1) inaccurate and user-dependent hand tracking,
(2) haptic feedback that is either insufficient or cognitively distracting, and
(3) the lack of robust methods for achieving precise, stable manipulation under human–robot kinematic mismatch and contact uncertainty.
In this project, we develop an integrated telemanipulation framework that jointly improves hand tracking accuracy, haptic feedback design, and precise manipulation control.
The project aims to enable stable, high-fidelity dexterous telemanipulation across users, tasks, and robot hands, with an impact on reliable human-in-the-loop manipulation and scalable data collection for learning-based robotics.
Experimental validation involves teleoperated robots performing Jenga gameplay tasks, along with other tests in virtual reality environments, to evaluate precision, adaptability, and operator intent recognition under real-world conditions.
Member:
Haiyun Zhang, Job Ramirez, Stefano Dalla Gasperina
Collaboration:
This project is ReNeu Lab's collaboration with Dongho Kang and Aaron Kim from Human Centered Robotics Lab, led by Dr. Luis Sentis, and sponsored by SONY.
Related Works
- [Proprioceptive Studies for Enhanced Telemanipulation over Virtual Environment]
- [Maestro Hand Exoskeleton v4.0]