Schematics Comparator

With ConvNeXt model based Siamese Network

Background

During computer hardware development, circuit schematics documentation with hundreds of pages is updated on a regular basis. It is very difficult for hardware engineers to find the difference between two versions of schematics documentation.

Challenge

The idea is to distinguish the difference between two versions of schematics documentation that are originally

However, methods such as classical Mean Squared Error and Image Matching algorithms can not solve this problem due to the complex nature of large-scale circuit schematics. For example, a schematic layout can be changed but still represent the same circuit. Thus, the traditional image comparison method is not compatible with such task.

Solution

Because of the adaptivity and ability to identify similarities between images, machine learning is the most suitable method for solving this problem. Machine learning models can learn the schematics patterns to correctly distinguish whether two schematics represent the same logic or not.

Specific Machine Learning Method

A Siamese Network is designed to identify the similarity between two images. It is a neural network that consists of two identical subnetworks meaning that they contain the same parameters and weights. Each subnetwork can be any neural network designed for images like a Convolutional Neural Network.

The network’s input is a pair of images that are either similar (positive example) or not similar (negative example). The network's output is the two feature vectors, one for each image.

If the input pairs are similar, we want these two vectors to be as close as possible and vice versa. To achieve this, we use the contrastive loss function that takes a pair of vectors (x_i, x_j) and minimizes their Euclidean distance when they come from similar images while maximizing the distance otherwise.

Siamese Network

Model Used in the Siamese Networks

The pre-trained ConvNeXt model is chosen to construct the Siamese Networks. And the pre-trained Resnet18 model is used to construct the same Siamese Networks as a comparison with the ConvNeXt model.

The ConvNeXt model was proposed in A ConvNet for the 2020s by Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, Saining Xie. ConvNeXT is a pure convolutional model (ConvNet), inspired by the design of Vision Transformers, that claims to outperform them.

Datasets for Model Fine-tuning

A custom dataset is created with the actual circuit schematics documentation that are in the current computer development.

There are 21 different classes prepared. The same class has different variations of schematics that represent the same logic, the different classes have schematics that are either on the same page of the PDF but represent different logics or from a different page of the PDF.

ConvNeXt and Resnet18 Fine-tune Rusults

By comparing with the Resnet18 model, the ConvNeXt model can achieve a stable out-of-sample accuracy of more than 90% in 10 epochs, and its training loss can reach less than 2.5% at 15 epochs. The Resnet18 model, on the other hand, has out-of-sample accuracy fluctuating, and can not maintain a stable high out-of-sample accuracy, and its training loss is still around 6% after 25 epochs.

Best Results Settings for both Models

ConvNeXt	Resnet18

Conclusion

This ConvNeXt model-based Siamese Model has proven its ability to accurately disdinguish the similarity between two complex schematics documents, and it has been deployed to the actual computer hardware development team.

Siamese Networks with ConvNeXt Implementation

With_ConvNeXt.py

class SiameseNetwork(nn.Module):
    def __init__(self, freeze_parameter):
        super(SiameseNetwork, self).__init__()

        model_name = "convnext_xlarge.fb_in22k"
        self.model = create_model(model_name, pretrained=True)

        # fix pretrained parameters or not
        # True = not freeze
        # False = freeze
        for param in self.model.parameters():
            param.requires_grad = freeze_parameter

        # # add linear layers to compare between the features of the two images
        self.model.head.fc = nn.Sequential(
            nn.Linear(self.model.num_features, 256),
            nn.GELU(),
            nn.Linear(256, 2),
        )

    def forward_once(self, x):
        output = self.model(x)
        output = output.view(output.size()[0], -1)
        return output

    def forward(self, input1, input2):
        # get two images' features
        output1 = self.forward_once(input1)
        output2 = self.forward_once(input2)
        
        return output1, output2

Catalog

Description	Links
Siamese Networks with ConvNeXt model	Link
Siamese Networks with Resnet18 model	Link
Fine-tune and validation results	Link

Reference

A ConvNet for the 2020s Paper

ConvNeXt Github by Facebook Research

ConvNeXt Hugging Face

Citation

@Article{liu2022convnet,
  author  = {Zhuang Liu and Hanzi Mao and Chao-Yuan Wu and Christoph Feichtenhofer and Trevor Darrell and Saining Xie},
  title   = {A ConvNet for the 2020s},
  journal = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year    = {2022},
}