New Robot Model Delivers Accurate Pick and Place Solutions

Pick-and-place machines play a crucial role in various industries, enabling the automated placement of objects into methodically organized locations. These technologies find applications in numerous fields, from electronics assembly and packaging to bin picking and inspection. However, many current pick-and-place solutions face limitations, particularly in their ability to generalize efficiently while maintaining accuracy.

“In the manufacturing sector, it’s common for companies to create highly specialized solutions tailored to specific challenges, resulting in significant engineering efforts but limited adaptability,” explains Maria Bauza Villalonga, PhD ’22, a senior research scientist at Google DeepMind who specializes in robotics and manipulation. “SimPLE addresses this limitation, offering a pick-and-place solution that combines flexibility with the precision needed for industrial applications.”

A recent study by researchers from the Mechanical Engineering department published in the journal Science Robotics delves into advanced pick-and-place methodologies. This innovative approach—known as kitting—enables robots to transform an array of disorganized objects into a neat arrangement. The technique, named SimPLE (Simulation to Pick, Localize, and Place), leverages a computer-aided design (CAD) model of objects to learn how to pick, regrasp, and place items without needing prior experience or exposure to those specific objects.

“SimPLE offers the potential to tackle a variety of tasks using the same hardware and software, with simulations that adaptively learn models tailored to individual challenges,” states Alberto Rodriguez, an MIT visiting scientist and former faculty member in Mechanical Engineering, who serves as the associate director of manipulation research at Boston Dynamics. Developed by the Manipulation and Mechanisms Lab at MIT (MCube) under Rodriguez’s guidance, SimPLE demonstrates impressive adaptability.

“Our research reveals that it’s possible to achieve the required levels of positional accuracy for various industrial pick-and-place operations without needing additional specialization,” Rodriguez adds.

The SimPLE methodology, which utilizes a dual-arm robot equipped with visuotactile sensing capabilities, operates based on three essential components: task-aware grasping, visuotactile perception (the integration of sight and touch), and effective regrasp planning. By employing supervised learning, real observations are aligned with simulated scenarios, enabling the robot to accurately estimate potential object poses and execute placements successfully.

In practical tests, SimPLE demonstrated remarkable proficiency, achieving successful placements over 90% of the time with six different objects, and over 80% success with eleven objects.

“There has been a shared understanding within the robotics community of the combined advantages of vision and touch; however, systematic demonstrations of their collaborative effectiveness for complex tasks have been scarce—until now,” notes mechanical engineering doctoral candidate Antonia Delores Bronars SM ’22. Currently working alongside Pulkit Agrawal, an assistant professor in Electrical Engineering and Computer Science (EECS), Bronars is advancing her research in the integration of tactile capabilities within robotic systems.

“Most research concerning grasping overlooks subsequent tasks,” comments Matt Mason, chief scientist at Berkshire Grey and emeritus professor at Carnegie Mellon University, who was not involved in this study. “This paper moves beyond the mere mimicry of human actions, providing a functional perspective on the advantages of combining tactile sensing and vision with dual-arm operation.”

Ken Goldberg, the William S. Floyd Jr. Distinguished Chair in Engineering at the University of California, Berkeley, who also did not participate in the study, remarked that the described robot manipulation methodology offers a valuable alternative to the current trend toward AI and machine learning. “The authors effectively blend well-established geometric algorithms with high precision for particular object shapes, significantly enhancing performance compared to traditional AI approaches,” says Goldberg, who is also co-founder and chief scientist at Ambi Robotics and Jacobi Robotics. “This has immediate implications for industrial applications and exemplifies what I term ‘good old-fashioned engineering’ (GOFE).”

Bauza and Bronars stress that their research benefited from extensive collaboration across multiple labs and generations. “Establishing a comprehensive robotic system that effectively illustrates the synergy of vision and touch poses significant challenges for an individual over a limited timeframe,” says Bronars. “Working collaboratively with peers such as Nikhil [Chavan-Dafle PhD ‘20] and Yifan [Hou PhD ’21, CMU] , along with the support of various labs, proved essential in constructing this end-to-end system.”