Visual Perception and Robotic Manipulation: 3d Object Recognition, Tracking and Hand-Eye Coordination

Geoffrey Taylor, Lindsay Kleeman9783540334545, 3-540-33454-8

As autonomous robots expand into the service domain, new solutions to the challenge of operating in domestic environments must be developed. Widespread adoption of service robots demands high robustness to environmental change and operational wear, and minimal reliance on application specific knowledge. As such, rich sensing modalities such as vision will play a central role in their success. This book takes steps towards the realization of domestic robots by presenting an integrated systems view of computer vision and robotics, covering fundamental topics including optimal sensor design, visual servoing, 3D object modelling and recognition, and multi-cue tracking, with a solid emphasis on robustness throughout. With in-depth treatment of both theory and implementation, extensive experimental results and comprehensive multimedia support including video clips, VRML data, C++ code and lecture slides, this book has wide appeal to both theoretical and practical roboticists and stands as a valuable teaching resource.

---

Table of contents :
front-matter.pdf……Page 1
001-010.pdf……Page 20
1.1 Motivation and Challenges……Page 23
1.2 Chapter Outline……Page 25
2.1.1 Homogeneous and Inhomogeneous Vectors……Page 30
2.1.2 Coordinate Frames and Homogeneous Transformations……Page 31
2.1.3 Representation of 3D Orientation……Page 32
2.1.4 Pose and Velocity Screw……Page 34
2.2.1 Pin-Hole Camera Model……Page 35
2.2.2 Radial Distortion……Page 36
2.2.3 Active Stereo Camera Head……Page 37
2.2.4 Rectilinear Stereo and Projective Recti.cation……Page 40
2.3.1 Digital Image Processing……Page 41
2.3.2 The Role of Abstraction……Page 42
2.4 From Perception to Control……Page 44
2.4.2 Visual Servoing……Page 45
2.4.3 Summary……Page 49
031-056.pdf……Page 50
3.1 Conventional Light Stripe Ranging and Related Work……Page 51
3.2.1 Problem Statement……Page 53
3.2.2 System Model……Page 56
3.2.3 Generalized Image Plane Error Function……Page 57
3.2.4 Special Case: Rectilinear Stereo and Pin-Hole Cameras……Page 59
3.2.5 Laser Plane Error……Page 61
3.2.6 Additional Constraints……Page 62
3.3 Active Calibration of System Parameters……Page 63
3.4 Implementation……Page 65
3.4.1 Light Stripe Measurement……Page 66
3.4.2 Range Data Post-processing……Page 67
3.5.1 Robustness……Page 68
3.5.2 Error Analysis……Page 71
3.6 Discussion……Page 73
3.7 Summary and Conclusions……Page 75
057-083.pdf……Page 76
4.1 Motivation……Page 77
4.2 The Gaussian Image in Computer Vision……Page 78
4.3 Segmentation Algorithm……Page 80
4.4 Non-parametric Surface Type Classi.cation……Page 81
4.4.1 Principal Curvatures……Page 82
4.4.2 Convexity……Page 85
4.5 Fitting Geometric Primitives……Page 86
4.5.2 Spheres……Page 87
4.5.3 Cylinders……Page 88
4.5.4 Cones……Page 89
4.6 Object Modelling and Classi.cation……Page 90
4.6.1 Modelling a Box……Page 92
4.6.2 Modelling a Cup/Bowl/Can……Page 93
4.7.1 Segmentation and Object Modelling……Page 94
4.7.2 Curvature-Based Versus Non-parametric Surface Type Classi.ers……Page 99
4.8 Discussion and Conclusions……Page 101
085-113.pdf……Page 103
5.1 Introduction……Page 104
5.2 System Overview……Page 107
5.3 Kalman Filter Framework……Page 109
5.4.1 Colour……Page 110
5.4.2 Edges……Page 112
5.4.3 Texture……Page 115
5.5 Implementation and Experimental Results……Page 119
5.5.1 Sequence 1: Poor Visual Conditions……Page 120
5.5.2 Sequence 2: Occlusions……Page 123
5.5.4 Sequence 4: Ego-Motion Compensation……Page 127
5.6 Discussion and Conclusions……Page 129
115-144.pdf……Page 132
6.1 Introduction……Page 133
6.2 Visual Servo Controller Overview……Page 135
6.3.1 Gripper Model with Active Visual Cues……Page 137
6.3.2 Pose Estimation……Page 138
6.3.3 Compensation of Calibration Errors……Page 140
6.4 Kinematic Feedback……Page 143
6.5 Fusion of Visual and Kinematic Feedback with Self-Calibration……Page 144
6.6 Implementation……Page 147
6.6.1 Image Processing……Page 148
6.6.2 Initialization and Calibration……Page 151
6.7 Experimental Results……Page 152
6.7.1 Positioning Accuracy……Page 153
6.7.2 Tracking Robustness……Page 155
6.7.3 Effect of Camera Calibration Errors……Page 156
6.8 Discussion and Conclusions……Page 159
145-170.pdf……Page 162
7.1.1 Hardware Con.guration……Page 163
7.1.2 Software Architecture……Page 166
7.2 Task Speci.cation……Page 167
7.3 Task Planning……Page 168
7.4 Experiment 1: Grasping an Unknown Object……Page 169
7.4.1 Implementation……Page 170
7.4.2 Results……Page 171
7.5 Experiment 2: Pouring Task……Page 172
7.5.1 Implementation……Page 174
7.5.2 Results……Page 175
7.6 Experiment 3: Multi-sensor Synergy……Page 178
7.6.1 Implementation……Page 179
7.6.2 Results……Page 182
7.7 Discussion and Conclusions……Page 186
8.1 Answering the Challenge of Service Robotics……Page 188
8.2 A Foundation for the Future……Page 191
177-178.pdf……Page 193
B.1 Optimization of Light Plane Error Function……Page 195
B.2 Optimal Reconstruction for Rectilinear Stereo and Pin-Hole Cameras……Page 196
185-187.pdf……Page 200
D.1 Optimal Reconstruction of a Single Point……Page 203
D.2 Error Model for Reconstruction of a Single Point……Page 205
D.3 Error Model for Pose Estimation……Page 207
195-197.pdf……Page 209
F.1 Grasping a Box……Page 212
F.2 Grasping a Cup……Page 214
F.3 Trajectory Planning……Page 216
F.4 Puma Kinematic Model……Page 217
back-matter.pdf……Page 219