MIT’s FLEX helps robots infer intent behind commands

Researchers at MIT and New York University have developed FLEX, an AI-assisted learning platform that helps robots interpret the underlying intent of natural language commands. Rather than responding only to explicit instructions, the system draws broader conclusions that guide motion planning. A warning not to spill coffee on a laptop, for example, can prompt a robot to avoid nearby electronics as well.

FLEX combines language-based corrections, a large language model and computer vision. Its “Add” stage identifies an object that matters to the task. “Generalize” uses GPT-4o to infer broader categories, such as fragile objects or electronics, from the user’s feedback. “Mask” removes objects no longer present from the robot’s feature list. The process takes about five seconds, helping keep its representation of the environment relevant as conditions change.

Testing took place in simulations involving a robotic arm serving coffee, moving a candle and carrying a bowling ball. Volunteers typed instructions, and the system extended those corrections into precautions around electronics, flammable materials and fragile items. The approach aims to reduce how much users must explain before a robot understands their preferences. The findings demonstrate simulated behavior, with household and assistive robots identified as potential applications.