SCANet: Correcting LEGO Assembly Errors with Self-Correct Assembly Network

Autonomous assembly in robotics and 3D vision presents significant challenges, particularly in ensuring assembly correctness. Presently, predominant methods such as MEPNet focus on assembling components based on manually provided images.

However, these approaches often fall short in achieving satisfactory results for tasks requiring long-term planning. Concurrently, we observe that integrating a self correction module can partially alleviate such issues. Motivated by this concern, we introduce the single-step assembly error correction task, which involves identifying and rectifying misassembled components.

To support research in this area, we present the LEGO Error Correction Assembly Dataset (LEGO-ECA), comprising manual images for assembly steps and instances of assembly failures. Additionally, we propose the Self-Correct Assembly Network (SCANet), a novel method to address this task. SCANet treats assembled components as queries, determining their correctness in manual images and providing corrections when necessary. Finally, we utilize SCANet to correct the assembly results of MEPNet. Experimental results demonstrate that SCANet can identify and correct MEPNet’s misassembled results, significantly improving the correctness of assembly.

Framing as a Markov process:

Reduced Complexity: By only considering the last step, the model becomes more efficient.

Most Relevant Information: More distant steps typically contribute less useful information.

Scalability: Prevent model becoming overwhelmed by a long sequence of past steps.

Core Idea: Treat components as queries to check whether they are correct or not.

Self-Correct Assembly Network (SCANet):

Neural Network Backbone: extract differences.

Correction Module: check and correct the error.

Convolutional Neural Network Backbone:

Two Input Branches: manual-Shape branch and Assembly-Shape branch.

Difference Extractor: compares feature differences between the two input branches.

Assembly Correction Module (Core Module):

Component Pose Encoder: Encodes the assembled components.

Transformer Network: Queries components, analyzing assembly correctness from feature differences.

Component Pose Corrector: Identifies errors and adjusts component poses.

It comprises 1,429 LEGO assembly manuals.

Dataset containing approximately 120,000 instances of incorrectly assembled examples.