We investigate language in different contexts: from how it is learned, to how it is grounded in visual perception, all the way to how machines can readily interact with humans. To this end we utilize and integrate tools from various disciplines, including Natural Language Processing, Information Retrieval and Computer Vision. We have developed START, a general-purpose question answering system and parser, an early version of which inspired today's Q&A systems such as Siri and Google Assistant. As part of our quest to understand human language we model language learning, the development of language, and how knowledge is transferred between languages. To enable communication with machines, we investigate how perception interacts with language and how robots can be controlled.