from random pose parameters to a plausible pose

[JacobLiu-S]

Dear PoseNDF team,
Thanks for your great work!
Recently I met a problem, I run the experiments/sample_poses.py trying to get the closest plausible pose from random pose parameters, by the code here;
And I increase to interations from 100 to 1000, here is what I get:


the first is the initial pose and the second is after run the code.
My problems are:

1. if the posendf is a set of all plausible poses, why the optimization can't reach a plausible pose (although it has been improved a lot with respect to the initial one)?
2. from my understanding, the optimization is try to find a closest point on manifold w.r.t the given pose if the given pose not on manifold, however, since I get such results, does it mean that, the points on the manifold (denoting the underlying pose) are not all plausible?.
3. And is it possible to directly sample a plausible pose from the manifold without proving a motion file?
   Look forward to your reply!
   Thanks in advance!

[garvita-tiwari]

if the posendf is a set of all plausible poses, why the optimization can't reach a plausible pose (although it has been improved a lot with respect to the initial one)?

PoseNDF learns a manifold of plausible poses. So ideally given a random pose when you do the projection steps multiple times, you are likely to generate a realistic pose. In the case of completely random pose, this problem is more difficult, especially if such bad poses are not seen during training. The model uploaded in version2 is not trained with such poses. Please follow this random pose projection

from my understanding, the optimization is try to find a closest point on manifold w.r.t the given pose if the given pose not on manifold, however, since I get such results, does it mean that, the points on the manifold (denoting the underlying pose) are not all plausible?.

The learned distance field is not exact, but approximated. e.g. if you predict the distance value of AMASS poses, you will find non-zero small distance, which ideally should be 0. This comes from the fact, that network learns a smoother version of desired manifold. So it might happen aht the project poses, although is better than the input post, is still not realistic.

And is it possible to directly sample a plausible pose from the manifold without proving a motion file?

Yes you can generate pose, by creating a random pose

noisy_pose = torch.rand((batch_size,21,4))
noisy_pose = torch.nn.functional.normalize(noisy_pose,dim=2).to(device=device)

However, this might also result in some bad poses. This is the kind of poses you will get, by random pose projection.

Left: initial pose, RIght: Projected pose(100 steps)

[JacobLiu-S]

if the posendf is a set of all plausible poses, why the optimization can't reach a plausible pose (although it has been improved a lot with respect to the initial one)?

PoseNDF learns a manifold of plausible poses. So ideally given a random pose when you do the projection steps multiple times, you are likely to generate a realistic pose. In the case of completely random pose, this problem is more difficult, especially if such bad poses are not seen during training. The model uploaded in version2 is not trained with such poses. Please follow this random pose projection

from my understanding, the optimization is try to find a closest point on manifold w.r.t the given pose if the given pose not on manifold, however, since I get such results, does it mean that, the points on the manifold (denoting the underlying pose) are not all plausible?.

The learned distance field is not exact, but approximated. e.g. if you predict the distance value of AMASS poses, you will find non-zero small distance, which ideally should be 0. This comes from the fact, that network learns a smoother version of desired manifold. So it might happen aht the project poses, although is better than the input post, is still not realistic.

And is it possible to directly sample a plausible pose from the manifold without proving a motion file?

Yes you can generate pose, by creating a random pose

noisy_pose = torch.rand((batch_size,21,4)) noisy_pose = torch.nn.functional.normalize(noisy_pose,dim=2).to(device=device)

However, this might also result in some bad poses. This is the kind of poses you will get, by random pose projection.

Left: initial pose, RIght: Projected pose(100 steps)

Thanks for your clarification! Really appreciate it!