Sidney D'Mello, Tanner Jackson, Scotty Craig, Brent Morgan, Patrick Chipman, Holly White, Natalie Person, Barry Kort, Rana el Kaliouby, Rosalind Picard, Art Graesser

Abstract

This paper provides a synthesis of our research towards the development of an affect-sensitive Intelligent Tutoring System called AutoTutor. The affect-sensitive AutoTutor detects the emotions (boredom, flow/engagement, confusion, frustration) of a learner by monitoring conversational cues, gross body language, and facial features. It is also mindful of the learners’ affective and cognitive states in selecting its pedagogical and motivational dialogue moves. Finally, the AutoTutor embodied pedagogical agent synthesizes affective responses through animated facial expressions and modulated speech. The paper provides an overview of our theoretical framework, methodology, implementation details, and results.