MIT’s CSAIL department has unveiled RoboGrocery, a system that combines computer vision with a soft robotic gripper to bag various grocery items gently.

Researchers placed 10 unknown objects on a grocery conveyor belt to test the new system. The products ranged from more delicate items such as grapes, kale, muffins, bread, and crackers to sturdier items such as meal boxes, soup cans, ice cream containers, etc.

The machine’s vision system begins by detecting the objects and determining their size and orientation on the belt. As the grasper touches the items, pressure sensors in the fingers can determine whether the items are delicate or sturdy.

The soft robotic hand combines vision, motor-based proprioception, and soft tactile sensors to identify, sort, and pack unknown objects. The multimodal sensing approach allows the robotic manipulator to estimate an object’s size and stiffness.

This translates the concept of packing into attainable metrics, allowing the robotic system to correctly place the items in the bag to ensure they’re transported safely, with the more delicate items at the top and the sturdier items at the bottom.

“This is a significant first step towards having robots pack groceries and other items in real-world settings,” said Annan Zhang, one of the study’s lead authors. “Although we’re not quite ready for commercial deployment, our research demonstrates the power of integrating multiple sensing modalities in soft robotic systems.”

The team behind the new technology stated that they would like to continue to improve the system, including upgrades to the grasper and imaging system to allow the machine to determine the packing order better. They also believe that, as the technology improves accuracy, it could be used in more industrial environments such as recycling plants.