Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[KeyPoints] - add from_mediapipe connector allowing Supervision to be used with Google MediaPipe models #1174

Open
SkalskiP opened this issue May 6, 2024 · 4 comments
Assignees
Labels
api:keypoints enhancement New feature or request help wanted Extra attention is needed

Comments

@SkalskiP
Copy link
Collaborator

SkalskiP commented May 6, 2024

Description

  • Add Skeleton.GHUM to the Skeleton enum. This is the skeleton format used by Mediapipe.
  • Extend the KeyPoints object and add the from_keypoints connector.
pose_landmarks_index

Links

Additional

  • Note: Please share a Google Colab with minimal code to test the new feature. We know it's additional work, but it will speed up the review process. The reviewer must test each change. Setting up a local environment to do this is time-consuming. Please ensure that Google Colab can be accessed without any issues (make it public). Thank you! 🙏🏻
@SkalskiP SkalskiP added enhancement New feature or request api:keypoints labels May 6, 2024
@LinasKo
Copy link
Collaborator

LinasKo commented May 9, 2024

Last time I worked on it, Mediapipe provided visiblity and not confidence. Should the implementer treat this as confidence for now?
Docs on: https://developers.google.com/mediapipe

There's cases when we have high confidence of invisible points (e.g. left or right shoulder when person is facing sideways).

(I'm not claiming the issue - just asking for details)

@LinasKo LinasKo added the help wanted Extra attention is needed label May 17, 2024
@LinasKo
Copy link
Collaborator

LinasKo commented May 17, 2024

Let's not get bogged down by details. I propose we implement visibility as confidence for now and adjust later if required. Other than that, this is one of the easier features - get frames+keypoints from mediapipe, and implement something similar to from_ultralytics 🙂

@LinasKo
Copy link
Collaborator

LinasKo commented May 17, 2024

Hey @David-rn,

I saw your comment on the RLE ticket. If you wish, this one is up for grabs. Should be pretty straightforward too.

Is this something you'd be interest in helping out with?

@David-rn
Copy link

Hi @LinasKo! I haven't worked with MediaPipe but I'll give it a try! 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api:keypoints enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants