Semantic frame induction in an assistive vocal interface using hierarchical HMMs

TitleSemantic frame induction in an assistive vocal interface using hierarchical HMMs
Publication TypeTalks
Authorsvan de Loo, J., Gemmeke J. F., De Pauw G., Van hamme H., & Daelemans W.
Place PresentedPresented at the 23rd Meeting of Computational Linguistics in the Netherlands (CLIN2013), Enschede, The Netherlands
Year of Publication2013
Date Presented18/01/2013

In the ALADIN project, we are developing a self-learning assistive vocal interface for people with physical impairments. This system should enable them to operate devices at home by giving vocal commands in an intuitive way; the system automatically learns the user-specific command structures and speech characteristics and adapts itself accordingly. The vocal interface is trained with a small number of training examples: spoken commands and their associated actions, of which the latter are represented as semantic frames with slots and fillers (slot values). The learning task of the system is to find meaningful units and structures in the spoken commands and relate them to slots and slot values in the semantic frames. After training, it should then be able to induce the semantic frame descriptions of commands uttered by the user.

We developed a semantic frame induction framework in which the spoken commands and their underlying semantic structures are modelled in hierarchical hidden Markov models (HHMMs). We present results of semantic frame induction experiments on command-and-control data which have been recorded in a voice-controlled card game setup. In these experiments, orthographic transcriptions of the spoken commands – both manual transcriptions and automatic transcriptions produced by a speech recognizer – are used as input for the semantic frame induction system. In future experiments, audio input will be used as well.