Human centered ROS package for
Human Robot Interaction

General Dependencies

• ROS (tested on Ubuntu1804/Melodic)
• catkin
• python/cpp
• pybullet
• GPU (tested on GeForce 1050 for only one perception module)
• RGB-D camera (in order to compute 3D body poses)

ROS Topics

• /humans/poses/body/2D
• /humans/poses/body/3D
• /humans/poses/body/3D/markers
• /humans/poses/gaze/3D
• /humans/poses/gaze/3D/markers
• /humans/visual_attention
• /humans/memory

Vocabulary

• Body: parts of the body
• Gaze: gaze estimation
• Complete body: parts of the body + gaze estimation
• Center of attention: perfect projection along the gaze estimation (Point XYZ)
• Visual attention: current attention of one human
• Memory: cumulated visual attention over the time

Human Perception Package

• rt-gene integration
• openpose integration
• 2D to 3D body poses, using depth
• fusion of head and body

Human Visual Attention Package

• dynamic world
• human head in bullet
• human body in bullet
• PR2 in bullet
• Pepper in bullet
• change sigma regarding head speed
• compute center of attention first
• get element of attention
• compute intensity based on distance with center
• define an attention structure for each element
• store time of attention
• clean cpp code with headers
• publish human attention cumulated
• publish discrete human attention cumulated (WIP)
• publish human memory (WIP)
• test object = 0 if new and no points

UWDS Human Clients Package

• gaze to uwds
• body to uwds
• PR2 to uwds
• Pepper to uwds
• create a new World for each Human

Human Experiments Package

• full gaze provider from PR2 Kinect
• full gaze provider from webcam
• full gaze provider from Pepper
• full body provider from PR2 Kinect
• full body provider from Pepper
• full human provider from PR2 Kinect (not tested recently)
• full human provider from Pepper
• full visual attention (WIP)

Human Viz Package

• viz of body
• viz of head
• viz of human (body+head)
• refactoring for common code
• viz of human attention
• viz of human memory