Researchers at the Massachusetts Institute of Technology have developed a wearable ultrasound wristband that allows users to control robotic hands using their own natural movements, a breakthrough that could improve dexterity in both robotics and virtual reality systems.
The device captures detailed images of muscles and tendons in the wrist as a person moves their fingers and hand. Artificial intelligence software then translates those images into precise hand positions, enabling real-time control of robotic systems.
In demonstrations, users wearing the wristband were able to manipulate a robotic hand wirelessly, performing actions such as playing simple piano notes or shooting a miniature basketball into a hoop. The system also allowed users to control objects in a digital environment, pinching or rotating virtual items with natural gestures.
The research highlights a growing effort to bridge the gap between human motor control and robotic manipulation.
Seeing the Mechanics of the Human Hand
Replicating the dexterity of the human hand has long been one of the most difficult challenges in robotics.
A human hand contains dozens of muscles and joints working together to produce subtle and continuous movements. Capturing those movements accurately enough to control a robot has traditionally required complex camera setups or sensor-filled gloves.
The MIT team approached the problem differently by focusing on the wrist, where the muscles and tendons responsible for finger movement are located.
The wearable device incorporates a miniaturized ultrasound sensor similar to those used in medical imaging. Positioned on the wrist, the sensor continuously captures images of the internal structures that control the hand.
Each ultrasound image reflects the changing state of muscles and tendons as they contract and relax. Because these structures act like strings pulling the fingers, researchers realized the images could be used to infer the exact position of the hand.
AI Translates Motion into Robot Control
To interpret the ultrasound data, the team trained an artificial intelligence model to recognize patterns in the images and associate them with specific hand positions.
Human fingers have 22 degrees of freedom, meaning they can move in many subtle combinations. By analyzing different regions of the ultrasound images, the system can estimate how each finger is positioned and how the hand is oriented.
During training, volunteers performed various gestures while cameras recorded their hand movements. The researchers matched those recorded positions with the ultrasound images to create a dataset used to train the AI system.
Once trained, the algorithm could accurately predict hand movements directly from ultrasound data.
In testing, participants used the wristband to perform a range of gestures, including forming the letters of the American Sign Language alphabet and manipulating objects such as bottles, scissors and tennis balls.
Implications for Humanoid Robots
Beyond human-machine interfaces, the researchers believe the wristband could play a significant role in training future humanoid robots.
One challenge in robotics is collecting large datasets that capture how humans manipulate objects in real-world environments. By recording wrist ultrasound data from many people performing everyday tasks, researchers could build datasets that teach robots more natural manipulation skills.
Such data could eventually help robots learn delicate tasks such as assembling electronics, handling tools or even assisting with surgical procedures.
The technology could also replace existing hand-tracking methods used in virtual and augmented reality systems. Compared with camera-based systems, wearable ultrasound sensors are less affected by lighting conditions or visual obstructions.
The MIT team plans to further miniaturize the device and expand the training dataset with more users and gestures.
If successful, wearable ultrasound interfaces could provide a more intuitive way for humans to interact with machines – and potentially offer robots a better model for replicating the remarkable dexterity of the human hand.