Current Projects

START: Question Answering

Boris Katz, Sue Felshin
The START Natural Language System is a software system designed to answer questions that are posed to it in natural language. START parses incoming questions, matches the queries created from the parse trees against its knowledge base and presents the appropriate information segments to the user. In this way, START provides untrained users with speedy access to knowledge that in many cases would take an expert some time to find.

Language Learning from a Computational Perspective

Yevgeni Berzak
Language learning is a fundamental aspect of human intelligence. Our project takes an interdisciplinary approach to this topic, integrating methodologies and theories from linguistics and psychology with modern computational modelling tools from Natural Language Processing (NLP) and computational linguistics. We conduct empirical studies of language production (writing) as well as language comprehension (reading), with particular focus on the role of a first language in second language acquisition and processing. We harness insights from such studies to develop NLP applications for learner language.

Inferring “Theory of Mind” with vision and language

Yen-Ling Kuo
Human agents are known for having "Theory of Mind (ToM)" ability to infer others’ mental states, e.g. intention and belief. Some early studies suggest language development help ToM reasoning.In this project, we aim to build a vision and language system, which learns and understands the world like a child. The way it understands other agents’ intent is from its interpretation of visual perceptions. Here, the interpretation is the other agent’s plan to achieve her goal (i.e. ground perception in planning) and the system can describe and evaluate this interpretation in natural language.We know humans formulate complex goals and we know those goals are influenced by language. So, this project is both an effort to shed light on the fundamentals of human planning and how we conceive about plans and at the same time an effort to understand videos at a higher, more human-like, level.

Generating Plans for Agents using Video and Language

Candace Ross
Human cognition is extremely flexible. We can easily adapt to new tasks, learn from our environment, and generalize about the world around us. Artificial intelligence is in stark contrast, where many models cannot generalize to novel tasks and perform in constrained environments. In this project, I work with the classical AI problem of planning where an agent is finding an approach to optimally achieve a goal. I work in a new realm of planning based around natural language processing and grounded in perception to both tackle classical planning problems and novel robotic and visually grounded planning problems. The work aims to simulataneously track agents and generate plans for agents to optimally accomplish goals.