Mobile Robot Grasping

Here, we consider the problem of real manipulation in a realistic environment - a living room. Similarly to the previous example, we assume that a number of predefined grasps is given and suitable grasp is generated depending on the current pose of the object. The experiment shows a XR400 platform with a hand-mounted camera. The task is to approach the dinner table including several objects. The robot is instructed to pick up a package of rice having an arbitrary placement. Here, Distributed Control Architecture [16] is used for integration of the different methods into a fully operational system. The system includes i) recognition, ii) image based tracking, iii) initial pose estimation, iv) hand alignment for grasping, v) grasp execution and vi) delivery of object. The details of the system implementation are unfortunately beyond this paper.

The sequence in Fig.  shows the robot starting at the far end of the living room, moving towards a point where a good view of the dinner table can be obtained. After the robot is instructed to pick up the rice package, it recognizes it and locates in the scene. After that, the robot moves closer to the table keeping the rice package centered in the image. Finally, the gripper is aligned with the object and grasping is performed.

Figure: The robot moves up to the table, recognizes the rice box, approaches it, picks it up and hands it over to the user.