We believe human intelligence differs from that of other primates, past and present, because we are able to understand, tell, and compose stories, and because we are able to direct our perceptual faculties to describe, and answer questions about, real and imagined situations. It follows that if we are to develop a full account of human intelligence, we must develop computational models of our story and directed perception faculties.
Our vision is explained in more detail in a trilogy of papers and white paper on story understanding.
Believing that system building can be both rigor enforcing and powerfully motivating, we have built and continue to empower the Genesis System, a system that understands stories using common sense rules and higher level concept patterns, tells stories using listener models, and composes new stories by recombining existing stories, as well as producing event descriptions using visual sensors.
We approach our building in five steps reminiscent of the methodological approach championed by David Marr: we identify the competence to be understood; formulate computational problems; propose computational solutions; develop an exploratory implementation; and crystalize emergent principles.
Our first building goal is to bring our Genesis System to a level where many aspects of human story processing and directed perception are illustrated, such as listener-aware story telling, precedent-based story composition, story summarizing, question answering, attention to personality and mood, attention to cultural differences in interpretation, question-driven processing, anticipation of harm, and understanding on multiple levels.
We hammer out our ideas on an anvil of manually prepared stories, common sense rules, and concept patterns, all expressed in simple English, along with annotated video, with annotations also expressed in simple English.
Our second building goal is to scale up our illustrations-of-concept Genesis System such that the common sense rules and concept patterns are learned, the stories are real, albeit simple, and the video is not annotated, albeit also simple.
We believe that scaling up will make our Genesis System sufficiently powerful that we can perform experiments that determine how well our models explain and predict human behavior. We further believe that scaling up will lay the foundations for systems of practical value in, for example, education and intercultural understanding.
Because we believe that our work will help us educate ourselves better, understand ourselves better, and understand each other better, we are determined to ensure that our work is broadly disseminated and brought to bear on important 21st century challenges, thereby serving everyone. Dissemination mechanisms include publications, web video, and downloadable demonstration software.
See our story demonstration videos for videos of our story understanding work.
See our vision demonstration videos for videos of our vision understanding work.
You can also have a look at a movie prepared by Sajit Rao and Deniz Yuret showing their vision system commenting on the activities of two men with a ball.
Some of the story-understanding functionality of Genesis is now available via the Java WebStart mechanism, which comes with the Java Runtime Environment, the so-called JRE. So, if you don't have the Java Runtime Enviornment installed, you should install it first. Then, you can run Genesis. Web Start will ask if you trust us. Indicate that you do. Then Web Start will download Genesis onto your computer. You will see something like this:
Click on the Demonstrations menu item to run experiments. When you run the first one, you will see this:
|Note that this is an early release, tested only on Windows 7. Documentation will appear eventually.|
See Papers and theses.
The roots of the Genesis Group lie in the thinking that led to the thoughts of a group that styled itself as The Human Intelligence Enterprise.
Check out the documentation button at the top of the page.