eyegaze

[atv2016]

How would one implement a axis that shows the direction of the eyes are looking at ?

[trishume]

You need to track a stable reference point to measure the offset of the pupil from. Like the eye corners.
I haven't managed to do this, it's quite difficult, especially if you want high accuracy.

[atv2016]

Hi Tristan, sorry for my late reply.

Could i not just draw a line instead of a circle for the eye? I'm doing this currently (in main.cpp)
line(debugFace,rightPupil,rightPupil+cv::Point(-15,-2),CV_RGB(255,5,255),2,8,0,0.5);
line(debugFace,leftPupil,leftPupil+cv::Point(-15,-2),CV_RGB(255,5,255),2,8,0);

The only problem i have is that it moves the wrong end of the line. It moves the pupil, not the other end . Ideally i would have the centre in the eye, and then the other end of the line would show the pupil movements.

Would appreciate your help!

[thomper]

Hey atv2016, I've looked into this as well so thought I'd share what I found.

eyeLike doesn't really track it at the moment, I think that's what's meant by "I haven't managed to do this".

Gaze tracking would be very useful though. Last time I checked (several years ago) there were a few teams claiming working proprietary implementations but nothing freely available. I think there are some vague papers written seemingly intentionally in broken Engrish available on Google Scholar.

I think you're gonna have to (re)invent the wheel if you want gaze tracking. Please share it with us if you do :)

[atv2016]

Yes, but currently i put a line instead of a circle. That line moves with the pupil. All i need is the other end of the line to move with the pupil? That shouldn't be to hard?

Or am i mistaking how to implement this? All i need is a line in which way the user is looking, or maybe a cone later on.

Alternatively, how about detecting the eye again with a cascade and taking the centre of that for a reference point?

[thomper]

It is hard, that's the thing. Not saying pupil detection is easy though.

Pupil detection and gaze tracking are two different problems.

Tristan has worked hard to provide this free software library which includes pupil detection but is yet to do gaze tracking (the disclaimer in the readme says this clearly). Tristan did not invent this method, it's based on others' research.

There are papers on gaze detection, but I don't believe there's an open source implementation for average quality video yet. Maybe you'll be the one to do it? :)

1. https://scholar.google.com/scholar?as_ylo=2016&q=gaze+detection+opencv&hl=en&as_sdt=0,5
2. http://sci-hub.cc
3. Tell us about it once you've done it

If you've got the skills and the time to implement this and are willing to release it back to the community, I'm sure you'll find willing help. The OpenCV community would be a good place to look for collaborators.

[DesignsWithDavid]

Have you considered integrating the work found here? The project, aitrack, would be able to provide all of the reference points necessary to accurately determine gaze.

[trishume]

I'm not really working on this project anymore, but one issue is that the reference point and pupil tracking both need to be very accurate to get reasonable gaze point estimation. I'm not sure eyeLike is even accurate enough at pupil location to be that useful, and most 6dof head pose systems aren't accurate enough for this use case either. It would also take a bunch of work to integrate.