Doesn't handle camera rotations properly

[kingsman142]

I'm currently using the Realsense D415 camera to gather RGBD data of an object, specifically a bottle. This camera is attached to the end-effector of a 6DOF robot. I use the robot to gather real-world pose coordinates of the camera. I use this kinematics data as the pose of the camera for both translation and rotation. This program works if I perform translation across one axis. However, it doesn't work when I perform a rotation.

Reading the papers linked in the README, I can't seem to figure out if their pose data is relative to the world's origin or if they're absolute physical locations, such as ones received from a robot's kinematics data. Do you know if I should perform any additional transformations on my pose data, similar to how the C++ programs calculate the inverse of the base frame and multiply that with each subsequent frame?

The first image is with the camera just performing a translation across one axis. The second image is the camera performing a translation across the same axis, but also performing a 0 to 360 degree rotation along the way.

EDIT: By the way, I have double-checked the camera intrinsics, and have used two different realsense depth cameras and I'm encountering similar issues in both. Also, I have verified the positions of the end-effector (which is carrying the camera) are correct.

[YJonmo]

any solution for this?

[thebirdgr]

@kingsman142 Hello, did you manage to solve this?

[kingsman142]

Hi @thebirdgr I did not solve this issue

[thebirdgr]

@kingsman142 Thanks for the prompt reply. If i may also ask, could you share your repo that used to create this volume? My version is not even half as good as yours. My repo is also publicly available if you want to check it out.

[kingsman142]

Sorry @thebirdgr , I worked on this project years ago as part of a private research project. It's not open to the public and I've lost track of the logic of the code so I wouldn't be able to help read through the code anymore.

[thebirdgr]

@kingsman142 It's alright. I appreciate your time and effort to reply. =)

[YJonmo]

Just keep in mind that the coordinate directions of this repo should be the same as your data. For example, as I remember, the z axis means movement towards the front, x and y are horizontal and vertical, or the other way around. It took me a while to find the correct axis and polarities.

[thebirdgr]

@YJonmo Oh nice. Thanks for sharing that. I used ICP to get the pose of the camera in the world coordinate system. It worked well for a real camera in the real world. A camera in gazebo was not so good. We feel that ICP did not have enough features.