The Institute of Systems and Robotics has amassed a solid body of work over the years in the research of probabilistic approaches to perception, namely in the context of socially-relevant active perception [FC BD12, FLD11], behaviour analysis [SD11, QKA+11], and novelty detection [DNR+10]. We propose to use this know-how in researching an integrated probabilistic framework to deal with the coordinated control of multisensory attention. This coordination will be both endogenous and exogenous, in the sense that it will involve the management of internal pathways relaying goal-directed and stimulus-based influences, and also of the external communication channels established with other agents in the interaction enacted during joint attention.


This research will have the ultimate purpose of addressing the following scientific objectives (Fig. 1): (SO 1) active perception for social interaction; (SO 2) behaviour generation for social interaction. Within the scope of these scientific objectives, we will also undertake the following technical and technological objectives: (TO 1) hierarchical multimodal perception; (TO 2) anticipatory automatic orientation and vergence; (TO 3) prediction and anticipation in joint attention; (TO 4) goal-dependent action selection and turn-taking in social interaction.


The algorithms resulting from this research will be developed mainly to address the application of robot-human interaction in a social context, tested using a robotic active perception platform, fitted with the appropriate visual, auditory and proprioceptive sensing systems, designed to evoke the sense of intentionality. However, the proposed framework will be developed in a modular fashion -- this will allow its components to be reused in other contexts, such as cutting-edge applications for which there is a need for assisting and training operators of mission-critical control systems through attention guidance.


Therefore, with this research we expect to contribute with a significant breakthrough, not only specifically in social robotics, but also in human-machine interaction or automatic surveillance systems, for which assessing intent or attention control might be crucial factors - see Fig. 2. On the other hand, since robots offer the possibility of studying the processes underlying joint attention in a repeatable and separable fashion, we also expect this research to shed further light on this important human social skill (Fig. 2).



[DNR+10] Paulo Drews Jr, Pedro Núñez, Rui Rocha, M. Campos, and Jorge Dias. Novelty Detection and 3D Shape Retrieval using Superquadrics and Multi-Scale Sampling for Autonomous Mobile Robots. In Proc. of 2010 IEEE Int. Conf. on Robotics and Automation (ICRA'2010), Anchorage, Alaska, USA, May 38 2010.


[FCBD12] João Filipe Ferreira, Miguel Castelo-Branco, and Jorge Dias. A hierarchical Bayesian framework for multimodal active perception. Adaptive Behavior, Adaptive Behavior, 20(3):172-190, June 2012.


[FLD11] João Filipe Ferreira, Jorge Lobo, and Jorge Dias. Bayesian realtime perception algorithms on GPU  Real-time implementation of Bayesian models for multimodal perception using CUDA. Journal of Real-Time Image Processing, 6(3):171186, September 2011.


[QKA+11] João Quintas, Kamrad Khoshhal, Hadi Aliakbarpour, Martin Hofmann, and Jorge Dias. Using Concurrent Hidden Markov Models To Analyse Human Behaviours In A Smart Home Environment. In Wiamis 2011, 12th international Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), 1315 April 2011.


[SD11] Luís Santos and Jorge Dias. Hierarchy and Reversibility in Human Motion Modelling: A Bayesian Approach. In 1st Workshop on Recognition and Action for Scene Understanding (REACTS) 2011, 2011.