Collect Blocks! Learning to assemble with massively structured reinforcement

The robotic assembly of objects from a given number of parts is an intriguing task in artificial intelligence.

To investigate this task, a recent paper published on arXiv.org proposes an assembly domain that allows the controlled study of generalizations in reinforcement learning (RL). It consists of a natural environment with differently sized blocks that can be magnetically attached to each other.

Instead of grabbing robotic arms, agents can move the desired blocks directly. The work requires abilities such as multi-step planning, physical reasoning and bimonthly coordination. It is demonstrated that a single agent can simultaneously solve all observed assembly tasks, generalize to unseen tasks and operate in a reset-free manner even when trained in an episodic fashion. could.

The proposed solution requires a combination of large-scale RL, structured policies and multitasking training.

The assembly of multi-part physical structures is a valuable end product for autonomous robotics, as well as a valuable clinical task for open-end training of embodied intelligent agents. We feature a natural physics-based environment with a set of connectable magnet blocks inspired by children’s toy kits. The objective is to assemble the blocks in the order of the target blueprint. Despite the simplicity of the objective, the creative nature of creating diverse blueprints from a set of blocks creates an explosion of complexity in the structures exposed to agents. In addition, the assembly emphasizes multi-step planning of agents, physical reasoning and bi-manual coordination. We find that the combination of large-scale reinforcement learning and graph-based policies – surprisingly with no added complexity – is an effective recipe for training agents that not only generalize to complex unseen blueprints in a zero-shot manner, Rather it even works in a reset-free setting without being trained to do so. Through extensive experiments, we shed light on the effects of large-scale training, structured representation, the contribution of multi-task versus single-task learning, as well as the effects of curriculum, and discuss the qualitative behavior of trained agents.

Research Paper: Kamyar Syed Ghasmipur, S., “BLOCK ASSEMBLY! Learning to assemble with large-scale structured reinforcement learning”, 2022. Link of Paper: https://arxiv.org/abs/2203.13733
Link to project page: https://sites.google.com/view/learning-direct-assembly

You can submit your link to a page that is relevant to the topic of this post.