Collect Blocks! Learning to assemble with massively structured reinforcement

The robotic assembly of objects from a given number of parts is an intriguing task in artificial intelligence.

Robot assembly line in car factory.

Robot assembly line in car factory. Image credits: Fiat Chrysler Automobiles, CC BY-NC-ND 2.0 via Flickr

To investigate this task, a recent paper published on proposes an assembly domain that allows the controlled study of generalizations in reinforcement learning (RL). It consists of a natural environment with differently sized blocks that can be magnetically attached to each other.

Instead of grabbing robotic arms, agents can move the desired blocks directly. The work requires abilities such as multi-step planning, physical reasoning and bimonthly coordination. It is demonstrated that a single agent can simultaneously solve all observed assembly tasks, generalize to unseen tasks and operate in a reset-free manner even when trained in an episodic fashion. could.

The proposed solution requires a combination of large-scale RL, structured policies and multitasking training.

The assembly of multi-part physical structures is a valuable end product for autonomous robotics, as well as a valuable clinical task for open-end training of embodied intelligent agents. We feature a natural physics-based environment with a set of connectable magnet blocks inspired by children’s toy kits. The objective is to assemble the blocks in the order of the target blueprint. Despite the simplicity of the objective, the creative nature of creating diverse blueprints from a set of blocks creates an explosion of complexity in the structures exposed to agents. In addition, the assembly emphasizes multi-step planning of agents, physical reasoning and bi-manual coordination. We find that the combination of large-scale reinforcement learning and graph-based policies – surprisingly with no added complexity – is an effective recipe for training agents that not only generalize to complex unseen blueprints in a zero-shot manner, Rather it even works in a reset-free setting without being trained to do so. Through extensive experiments, we shed light on the effects of large-scale training, structured representation, the contribution of multi-task versus single-task learning, as well as the effects of curriculum, and discuss the qualitative behavior of trained agents.

Research Paper: Kamyar Syed Ghasmipur, S., “BLOCK ASSEMBLY! Learning to assemble with large-scale structured reinforcement learning”, 2022. Link of Paper:
Link to project page:

Leave a Reply

Your email address will not be published.