SLS RESEARCH

Language Learning

It is widely recognized that one of the best ways to learn a foreign language is through spoken dialogue with a native speaker. However, this is not a practical method in the classroom due to the one-to-one student/teacher ratio it implies. A potential solution to this problem is to rely on computer spoken dialogue systems to role play a conversational partner. Ongoing research in SLS, spanning the past several years, has involved developing spoken dialogue systems specifically designed to address this need. Students can engage in dialogue with the computer either over the telephone or through audio/typed input at a Web page, where multimodal (mouse plus speech) interaction enhances the experience. Thus far we have focused on the language pair English/Mandarin. Several different topic domains are being developed, in which a student's conversational interaction with a software dialogue partner is assisted by an additional software agent functioning as a "tutor." This tutor can both suggest possible dialogue moves and provide translation assistance. The Web-based remote server model allows easy access to students from around the world. The framework also offers an effective testbed for research in speech-to-speech translation, dialogue planning, and speech recognition of accented speech. In the future, we plan to develop assessment technology that can pinpoint specific problem areas in the student's pronunciation, prosodics, and grammar usage.

Click here if you would like to play customizable web-based games that help students practice speaking Mandarin. The video below gives an overview of the system called Word War.

Further Reading

S. Seneff, "Web-based Dialogue and Translation Games for Spoken Language Learning," Keynote Speech, Proc. SIGSLaTe, Farmington, PA, 2007. (PDF)

I. McGraw and S. Seneff, "Immersive Second Language Acquisition in Narrow Domains: A Prototype ISLAND Dialogue System," Proc. SIGSLaTe Workshop, Farmington, PA, October 2007. (PDF)