Reinforcement Learning in Spoken Dialogue Systems

Humans communicate by talking, so wouldn't it be great if they could talk with machines? The goal of Spoken Dialogue System (SDS) designers is to make this vision a reality, but there are big challenges - for example, dialogue strategy design. A dialogue strategy specifies what the SDS will say/do next, and designing it by hand can be difficult. It involves anticipating how users will behave, and repeated testing and refining. This book describes a successful machine learning approach where a strategy is modelled as a sequential decision problem called a Markov Decision Process (MDP), and reinforcement learning applied to training dialogues. The dialogues are generated with a probabilistic user simulation derived from real user data. The reader is taken through theory, relevant previous research, original experiments, and likely future research directions. A complete, learned strategy is shown to perform well with real users, (better than a hand-crafted strategy), and adding linguistically-motivated information to the MDP is shown to improve repair strategies. Repair strategies try to get the dialogue back-on-track following SDS understanding errors, and are hence very important.

Autorentext

Matthew Frampton has a BA in linguistics from the University of Cambridge, and an MSc and PhD in computational linguistics from the University of Edinburgh. He now works in the Center for the Study of Language and Information (CSLI) at Stanford University, where his main research interests are in computational semantics and pragmatics.

Klappentext

Humans communicate by talking, so wouldn't it be great if they could talk with machines? The goal of Spoken Dialogue System (SDS) designers is to make this vision a reality, but there are big challenges - for example, dialogue strategy design. A dialogue strategy specifies what the SDS will say/do next, and designing it by hand can be difficult. It involves anticipating how users will behave, and repeated testing and refining. This book describes a successful machine learning approach where a strategy is modelled as a sequential decision problem called a Markov Decision Process (MDP), and reinforcement learning applied to training dialogues. The dialogues are generated with a probabilistic user simulation derived from real user data. The reader is taken through theory, relevant previous research, original experiments, and likely future research directions. A complete, learned strategy is shown to perform well with real users, (better than a hand-crafted strategy), and adding linguistically-motivated information to the MDP is shown to improve repair strategies. Repair strategies try to get the dialogue back-on-track following SDS understanding errors, and are hence very important.