Grounded Situation Models for Situated Conversational Assistants

A Situated Conversational Assistant (SCA) is any system with sensing, acting and speech abilities, which engages in physically situated natural language conversation with human partners and assists them in tasks. Towards such assistants, a computational model of embodied agents is presented, which produces systems that are capable of a core set of situated natural language skills, and which provides concrete leverage for numerous extensions. The central idea is to endow agents with a sensor-updated set of structures and processes called a Grounded Situation Model (GSM), which is closely related to the cognitive psychology notion of situation models. The GSM contains descriptions of physical & mental aspects of past, current, or imagined situations, enabling bidirectional translation between linguistic descriptions and perceptual data/expectations. The power of the GSM proposal is demonstrated through the real-world example of a manipulator robot with speech and vision, with abilities comparable to those required by a normally- developing child in order to pass the Token Test, a standard psychological test for three-year old children.