EACare: Embodied Agent to support elderly mental wellbeing

The main goal of this multidisciplinary project is to develop a robot head with communicative skills capable of interacting with elderly people at their convenience.
The robot will be able to analyze their mental and psychological status via powerful audiovisual sensing and assessing their mental abilities to identify subjects in high risk or possibly at the first stages of depressive or dementing disorders, who would benefit from proper medical consultation. The framework will also provide tools for dementia preventive training.
We propose an ambitious, innovative and integrated approach that combines:
  • an easy-to-use patient-friendly and caregiver-friendly embodied agent to interact with patients via both verbal and non-verbal communication, including dialogue especially designed by psychiatrists to probe, measure, test, and monitor the patients mental state,
  • pioneering technological solutions at both the algorithmic and system level for multimodal information processing to achieve audiovisual human-computer interaction including action recognition, aural and facial expressions, and speech understanding,
  • systematic evaluation of the developed systems in both large and focused social groups, using feedback from each evaluation to guide further scientific research and design.


Department of Robotics, Perception and Learning (RPL)
KTH Royal Institute of Technology, Sweden
Project Director: Hedvig Kjellström
Department of Speech, Music and Hearing (TMH)
KTH Royal Institute of Technology, Sweden
Principal Investigator: Joakim Gustafson
Principal Investigator: Jonas Beskow
Department of Neurobiology, Care Sciences and Society (NVS)
Karolinska Institute, Sweden
Principal Investigator: Miia Kivipelto


Sverige behöver satsa på AI, IVA Aktuellt, 2:18-23, April, 2018.

Sweden: 'Social Robot' designed to recognise very early stages of Alzheimers, Ruptly TV, August, 2016.

Social robot ska kunna upptäcka demens tidigare, Dagens Nyheter, July, 2016.


T Kucherenko. Data driven non-verbal behavior generation for humanoid robots. In International Conference on Multimodal Interfaces Doctoral Symposium, 2018.
Non-verbal behavior production

T Ngandu, F Mangialasche, and M Kivipelto. Lifestyle interventions to prevent cognitive impairment and dementia. Nature Neurology, (in press).
Clinical geriatrics

K Håkansson, M Kivipelto, and T Ngandu. The patient with cognitive impairment. In V Hachinsky (Ed), Treatable and Potentially Preventable Dementias, Cambridge University Press, 2018.
Clinical geriatrics

P Jonell, M Bystedt, P Fallgren, D Kontogiorgos, J Lopes, Z Malisz, S Mascarenhas, C Oertel, E Raveh, and T Shore. ARMI: An architecture for recording multimodal interactions. In Language Resources and Evaluation Conference, 2018.
Dialogue system design

P Jonell, C Oertel, D Kontogiorgos, J Beskow, and J Gustafson. Crowdsourced multimodal corpora collection tool. In Language Resources and Evaluation Conference, 2018.
Dialogue system design

O Mikheeva, C H Ek, and H Kjellström. Perceptual facial expression representation. In IEEE International Conference on Automatic Face and Gesture Recognition, 2018.
Machine learning

M Kivipelto and K Håkansson. A rare success against Alzheimer's. Scientific American (April):32-37, 2017.
Clinical geriatrics

P Jonell, J Mendelson, T Storskog, G Hagman, P Östberg, I Leite, T Kucherenko, O Mikheeva, U Akenine, V Jelic, A Solomon, J Beskow, J Gustafson, M Kivipelto, and H Kjellström. Machine Learning and Social Robotics for Detecting Early Signs of Dementia, http://arxiv.org/abs/1709.01613, 2017.
Position paper

P Jonell, C Oertel, D Kontogiorgos, J Beskow, and J Gustafson. Crowd-powered design of virtual attentive listeners. In International Conference on Intelligent Virtual Agents, 2017.
Dialogue system design

C Oertel, P Jonell, D Kontogiorgos, J Mendelson, J Beskow, and J Gustafson. Crowd-sourced design of artificial attentive listeners. In INTERSPEECH: Situated Interaction, 2017.
Dialogue system design

C Oertel, P Jonell, K El Addad, E Szekely, and J Gustafson. Using crowd-sourcing for the design of listening agents: Challenges and opportunities. In ACM SIGCHI International Workshop on Investigating Social Interactions with Artificial Agents, 2017.
Dialogue system design

K Håkansson, A Ledreux, K Daffner, Y Terjestam, P Bergman, R Carlsson, M Kivipelto, B Winblad, A-C Granholm, A K H Mohammed. BDNF responses in healthy older persons to 35 minutes of physical exercise, cognitive training, and mindfulness: Associations with working memory function. Journal of Alzheimer's Disease. 55(2):645-657, 2016.
Clinical geriatrics