Investigating distinctiveness and individual variation in the expression of visual prosodic attitudes

Abstract

A talker can communicate different attitudes simply by changing how an utterance is expressed rather than by what is said. Typically, such changes in prosody have been investigated by measuring vocal properties; here we examined the expression of different attitudes by measuring changes in visual ones (Facial Action Units and head motion). Using multinomial logistic regression and a recognition experiment, we determined the extent to which different attitudes can be discriminated, and the variability of expressions within and across production sessions. Ten talkers expressed six attitudes, “warning”, “criticism”, “doubt”, “suggestion”, “longing”, “neutral” in four within session trials across four different day sessions. Face/head motion was tracked using a Constrained Local Neural Field model on 2D movies. The regression models and recognition experiment showed that attitudes were discriminable; with some better discriminated than others and some talkers much clearer than others. Within-talker productions were more consistent, both within and across sessions.