Skip to content

weigao95/kPAM

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

31 Commits
 
 
 
 
 
 

Repository files navigation

kPAM: Generalizable Robotic Manipulation

This project aims at pose-aware robotic manipulation for a category of objects. In contrast to most existing methods that must contain an explicit pose estimation, we define the object target configuration on top of semantic keypoints. In this way, the proposed pipeline can handle potentially unknown objects with substantial variations on shape and size, as shown in the demo below.

Demo

Publication

Lucas Manuelli*, Wei Gao*, Pete Florence, and Russ Tedrake, "kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation", International Symposium on Robotics Research (ISRR) 2019 [Project][Paper][Video]

Code Organization

The code is distributed into several modularized packages

In our experiment setup, these packages are used as the submodules of spartan, which provides the interface to our robots. Although spartan contains private submodules and is not supported for external use, all packages above are not specified to particular environments and can run independently.

Run Instruction and Test Data

The test data, trained model and run instruction for each package are provided in its own repository. Following the manipulation pipeline, the instructions are:

After these operations, the remaining task is to apply the planned action to the grasped object. In kPAM, the action is essentially a rigid transformation on the robot end-effector. This part is platform-specific and you might implement it according to your own robot interface.

Training

The instruction to setup the dataset is available here. The code for training the instance segmentation and keypoint detection network are available in their own repo. Please follow the instruction accordingly.

Releases

No releases published

Packages

No packages published