We believe human intelligence differs from that of other primates, past and present, because we humans understand, tell, and compose stories, and because we are able to direct our perceptual faculties to describe, and answer questions about, real and imagined situations. It follows that if we are to develop a full account of human intelligence, we must develop computational models of our human story competence and directed perception faculties.
Our vision is explained in more detail in a trilogy of papers, a white paper on story understanding, and a paper on story summarization.
Believing that system building forces rigor and motivates, we of CSAIL's Genesis Group have built and continue to empower the Genesis System, a system that understands, tells, and composes stories using common sense rules and higher level concept patterns.
Genesis has been brought to a level where Genesis exhibits many aspects of human story processing and directed perception, such as listener-aware instructive story telling, persuasive story telling, precedent-based story composition, story summarizing, question answering, attention to personality and mood, attention to culturally grounded differences in interpretation, question-driven reflection, anticipation of potential harm, and understanding on multiple levels.
Our approach is reminiscent of the methodological approach championed by David Marr: we identify the competence to be understood; formulate computational problems; propose computational solutions; develop an exploratory implementation; and crystalize emergent principles.
We hammer out our ideas on an anvil of manually prepared stories, common sense rules, and concept patterns, all expressed in simple English, along with annotated video, with annotations also expressed in simple English. Genesis reads stories ranging from 100 sentence summaries of Shakespeare's plots to conflict in cyberspace to Crow creation myths to stories used in psychological studies.
Among our current projects are many that pose particularly interesting challenges along with great potential. One such project involves retelling a story from the perspective of one of the agents in the story, keeping track of what that agent is likely to know about what is going on. What, for example, would the Macbeth plot look like from the perspective of Lady Macbeth?
Another current project focuses on hypothetical reasoning and determining the consequences of a subtracted or added characteristic or event. In the Russia-Estonia cyberwar of 2007, for example, what would have happened if Estonia had not moved a Soviet-era war memorial from Tallinn to a suburban location?
Our biggest challenge is to scale up our illustrations-of-concept Genesis System such that the common sense rules and concept patterns are learned, the stories are real, albeit simple, and the video is not annotated, albeit also simple. We believe that scaling up will make our Genesis System sufficiently powerful that we will be able to perform experiments that determine how well our models explain and predict human behavior.
We believe that when we scale up and do the science required to understand human-level story competence, many undreamed-of applications will emerge. One of many exciting possibilities we have dreamed of already is the construction of tools for policy analysts that will be as useful to them as spreadsheets are to financial analysts.
See our story demonstration videos for videos of our story understanding work.
See our vision demonstration videos for videos of our vision understanding work.
You can also have a look at a movie prepared by Sajit Rao and Deniz Yuret showing their vision system commenting on the activities of two men with a ball.
Some of the story-understanding functionality of Genesis is now available via the Java WebStart mechanism, which comes with the Java Runtime Environment, the so-called JRE. So, if you don't have the Java 8 Runtime Enviornment installed, you should install it first.
Next, because Java security becomes steadily greater, you will need to do some preparation.
Then, you can run Genesis. It will likely take a while to download, as it is about 30MB, because it contains some big dictionary files. Web Start will ask if you trust us. Indicate that you do. Then Web Start will download Genesis onto your computer. You will see something like the following:
Click on Use this screen in the navigation pane on the left for instructions.
Alternatively, just click on the Demonstrations menu item to run experiments.
See Papers and theses.
The roots of the Genesis Group lie in the thinking that led to the thoughts of a group that styled itself as The Human Intelligence Enterprise.
Check out the documentation button at the top of the page.