MIT wristband uses ultrasound to control robot hands

MIT researchers and collaborators at the University of Southern California have developed a wristband that uses an ultrasound “sticker” to capture how muscles, tendons, and ligaments move beneath the skin. The system targets one of robotics’ hardest dexterity problems: replicating hands that coordinate 34 muscles, 27 joints, and over 100 tendons and ligaments.

The device pairs a miniaturized ultrasound transducer with a skin-safe hydrogel, producing live images of the wrist as the wearer moves. An AI algorithm trained on human-labeled ultrasound images translates those images into the corresponding positions of the five fingers and the palm, allowing the wristband to track hand motion precisely in real time.

In demonstrations, users wirelessly controlled a robotic hand that mirrored their gestures, played a simple piano tune, and shot a mini basketball into a desktop hoop. The same setup also enabled screen-based interactions, such as pinching fingers to enlarge or minimize a virtual object.

The team plans to shrink hardware that is currently similar in size to a cell phone and expand AI training across more volunteers, hand sizes, finger shapes, and gestures. Longer term, the researchers envision a large hand-motion data set for training humanoid robots and enabling high-dexterity control in virtual reality, design applications, video games, and robotic systems.