findHomography returns bad homography and ouliers in the inlier mask

[cdeln]

System Information

OpenCV python version: 4.9.0.80
Operating System / Platform: Ubuntu 23.10
Python version: 3.10.14

Detailed description

findHomography returns a bad homography and an inlier mask with outliers.
The RANSAC reprojection threshold is set to 5 pixels, but checking the reprojection error manually afterwards using the returned homography and inlier mask gives a maximum error around 410 pixels (i.e. the inlier mask contains outliers).

Steps to reproduce

Put the source_points.json and the target_points.json files in your current working directory and run the following script

import json

import cv2 as cv
import numpy as np


# Load source and target points from JSON
with open('source_points.json', 'r') as f:
    source_points = np.array(json.load(f))

with open('target_points.json', 'r') as f:
    target_points = np.array(json.load(f))

# Estimate homography from source to target using RANSAC with a specified point to point distance threshold
# Use the "index with None trick" to unsqueeze one axis to fit OpenCV shape requirements
point_to_point_distance_threshold = 5
homography, mask = cv.findHomography(source_points[:, None, :], target_points[:, None, :], cv.RANSAC, point_to_point_distance_threshold)

# Extract the points that passed in the RANSAC step using the mask
# These points should be within the point to point distance threshold specified earlier
mask = mask.astype(bool).squeeze()
predicted_points = cv.perspectiveTransform(source_points[:, None, :], homography).squeeze(1)
point_to_point_distances = np.linalg.norm(predicted_points[mask] - target_points[mask], axis=-1)

# The maximum error is around 410 pixels, which is way bigger than the specified threshold of 5 pixels
assert point_to_point_distances.max() <= point_to_point_distance_threshold, point_to_point_distances.max()

Issue submission checklist

• I report the issue, it's not a question
• I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
• I updated to the latest OpenCV version and the issue is still there
• There is reproducer code and related data files (videos, images, onnx, etc)

[asmorkalov]

Reproduced the issue with current OpenCV 4.x (f2d6527). As workaround I could recommend USAC algorithm. See cv.USAC_DEFAULT option as example. It converges well for your case.