Adaptive Methods For Realistic And Intuitive Human-robot Interaction

Due to the advancement in robotics and computer technology, having access to sophisticated robot hardware is becoming common. An increase in the number of robots allows one to achieve different kind of tasks by exploiting cooperation, such as: robot swarms, object manipulation using multiple degrees of freedom of different robots. It is quite challenging for the human operator to coordinate multiple robots to achieve a task in an efficient manner. The channels available to receive feedback information from the robot system can vary from simple encoder reading to video stream acquisition of the robot system performing tasks. It is important for us to make use of and coordinate as many sensing modalities as it is required to perform necessary tasks. On the other hand, we would want to keep the number of modalities utilized optimal just to maintain usage intuitiveness for human operators and efficiency.In this dissertation, we have investigated 3 different modalities of interaction with robots. Physical sensing: we present a novel approach to enhance human-robot interactivity through the use of artificial skin and the Extended Kalman filter. Visual sensing: we present a novel approach to enhance interaction of humanoid robot actor through the use of pose estimation and visual servoing. Interface devices: we present a work on combining dynamic gestures based commands from an interface device to improve the intuitiveness of control / planning of multiple degrees of freedom robot system through reinforcement learning.We propose an efficient way to coordinate multiple modalities sensing as generalized interface for multiple robots manipulation / planning. In addition, performance metrics are proposed so that we have the quantitative way to identify our interaction efficiency. Eventually, the outcome of this research is a reconfigurable multiple interfaces system that can be used with different multiple robots system in a way that it is easy and intuitive for the human operator to learn to operate.