One of the key components of a robotic system that operates in a dynamic, unstructured environment is robust perception. Our current research considers the problem of mobile manipulation in domestic settings where, in order for the robot to be able to detect and manipulate objects in the environment, robust visual feedback is of key importance. Humans use visual feedback extensively to plan and execute actions. However, planning and execution is not a well-defined one-way stream: how we plan and execute actions depends on what we already know about the environment we operate in, what we are about to do, and what we think our actions will result in. Complex coordination between the eye and the hand is used during execution of everyday activities such as pointing, grasping, reaching or catching. Each of these activities or actions requires attention to different attributes in the environment - while pointing requires only an approximate location of the object in the visual field, a reaching or grasping movement requires more exact information about the object's pose.

Related Publications