How to get description from giving point(x,y)

[phamkhactu]

Thanks for great work!

I want to get description from point(x,y).

For example in cv2 I can get:

gray_image = cv2.imread(path,0)

sift = cv2.SIFT_create()
keypoint = cv2.KeyPoint(x, y, 20)
keypoints, descriptors = sift.compute(gray_image, [keypoint])

but I don't know the way to get it in LightGlue
I'm happy for your help! Thank you.

[Phil26AT]

Hey @phamkhactu, LightGlue actually takes the description of each point(x,y) as input to find correspondences between 2 images. However, you can get the description from the extractors, something like this:

from lightglue import LightGlue, SuperPoint
from lightglue.utils import load_image, rbd

# SuperPoint+LightGlue
extractor = SuperPoint(max_num_keypoints=2048).eval().cuda()  # load the extractor
matcher = LightGlue(features='superpoint').eval().cuda()  # load the matcher

# load image as a torch.Tensor on GPU with shape (3,H,W), normalized in [0,1]
image0 = load_image('path/to/image_0.jpg').cuda()

# extract local features
feats0 = extractor.extract(image0)  # auto-resize the image, disable with resize=None
keypoints, descriptors = feats0["keypoints"][0], feats0["descriptors"][0] 

[phamkhactu]

Hey @phamkhactu, LightGlue actually takes the description of each point(x,y) as input to find correspondences between 2 images. However, you can get the description from the extractors, something like this:

from lightglue import LightGlue, SuperPoint
from lightglue.utils import load_image, rbd

# SuperPoint+LightGlue
extractor = SuperPoint(max_num_keypoints=2048).eval().cuda()  # load the extractor
matcher = LightGlue(features='superpoint').eval().cuda()  # load the matcher

# load image as a torch.Tensor on GPU with shape (3,H,W), normalized in [0,1]
image0 = load_image('path/to/image_0.jpg').cuda()

# extract local features
feats0 = extractor.extract(image0)  # auto-resize the image, disable with resize=None
keypoints, descriptors = feats0["keypoints"][0], feats0["descriptors"][0] 

Hi @Phil26AT

As your describe above, each point has description. But I see that model limited by max_num_keypoints, so with image has 1280x720, some pixel doesn't has description.

I am so sorry if I misunderstand your response.
Thank you.