[Bug]: mask input for DeformableConv2D is ignored when ONNX model is converted into OpenVINO IR format using Python openvino.convert_model.

[murase-masamitsu]

OpenVINO Version

master

Operating System

Ubuntu 20.04 (LTS)

Device used for inference

CPU

Framework

ONNX

Model used

No response

Issue description

DeformableConvolution-8 supports mask input as the the 4th parameter according to the document.
However, openvino.convert_model in Python module ignores mask input.

The following image shown in Netron is an ONNX model to be converted into OpenVINO IR format.

The following image is a converted model in OpenVINO IR format. mask should exists in the converted model, but it does not.

The following is the expected image.

Step-by-step reproduction

1. Generate a sample ONNX model including a DeformableConv2D node with mask input.
2. Convert the model into OpenVINO IR format using openvino.convert_model in Python module.
3. The converted OpenVINO model does not include mask input.

The following Python script reproduces this issue:

import onnx
from onnx import helper
from onnx import TensorProto
import openvino as ov


def save_deform_conv2d_onnx_model(filename):
    batch_size = 4
    in_channels = 3
    in_height = 10
    in_width = 10
    offset_groups = 1
    kernel_height = 3
    kernel_width = 3
    out_height = 8
    out_width = 8
    out_channels = 5
    groups = 1

    arg = helper.make_tensor_value_info(
        'arg', TensorProto.FLOAT, [
            batch_size, in_channels, in_height, in_width,
        ])
    offsets = helper.make_tensor_value_info(
        'offsets', TensorProto.FLOAT, [
            batch_size, 2 * offset_groups * kernel_height * kernel_width,
            out_height, out_width,
        ])
    filters = helper.make_tensor_value_info(
        'filters', TensorProto.FLOAT, [
            out_channels, in_channels // groups, kernel_height, kernel_width,
        ])
    mask = helper.make_tensor_value_info(
        'mask', TensorProto.FLOAT, [
            batch_size, offset_groups * kernel_height * kernel_width,
            out_height, out_width,
        ])
    output = helper.make_tensor_value_info(
        'output', TensorProto.FLOAT, [
            batch_size, out_channels, out_height, out_width,
        ])

    input_name_list = ['arg', 'offsets', 'filters', 'mask']
    input_list = [arg, offsets, filters, mask]

    deform_conv2d_node = helper.make_node('DeformableConv2D', input_name_list,
                                          ['output'])

    graph = helper.make_graph(
        [deform_conv2d_node],
        'deform_conv2d_model',
        input_list,
        [output]
    )

    model = helper.make_model(graph)
    onnx.save(model, filename)


def convert_onnx_to_openvino(onnx_filename, ov_filename):
    ov_model = ov.convert_model(onnx_filename)
    ov.save_model(ov_model, ov_filename)


def main():
    onnx_filename = 'deform_conv2d_model_with_mask.onnx'
    ov_filename = 'deform_conv2d_model_with_mask.xml'
    save_deform_conv2d_onnx_model(onnx_filename)
    convert_onnx_to_openvino(onnx_filename, ov_filename)


if __name__ == "__main__":
    main()

Relevant log output

No response

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.

[andrei-kochin]

resolved by #24347