Haystack Blog

While Situational Awareness (SA) theory [1] has primarily been applied to analyzing how people act in highly time-critical military and industrial roles, such as how fighter pilots assess their aerial and tactical situations to decide what maneuvers to make, it may also serve as a useful theoretical model to inform the design of future personal information management tools.

While the connection between the tactical decision making of a fighter pilot and the every-day decision-making of an ordinary person browsing the web may seem tenuous, many of the considerations used to design systems for the former may be relevant to the latter. A fighter pilot has a number of things competing for her attention at any moment, all of which she must maintain a certain level of awareness at all times. Similarly, faced with a constant stream of incoming information and interruptions from the Web, e-mail, and phone calls, people in every day information environments are constantly critically cognitively loaded, often failing to remain aware of everything needed to fulfill their responsibilities. Slight disturbances such as distractions can quickly upset one’s attentional balance, causing one to get sidetracked, neglect priorities and responsibilities, and to forget to do things. [7]

The consequences of informational and awareness failure in everyday tasks if often less immediate (and usually less life-threatening) that that of a fighter pilot, therefore its impact is less visible. Effects, however, are nonetheless present, and are thought to contribute significantly to stress, the perception of being overloaded, and a failure to manage responsibities. For example, Carton et al. documented correlations between the number of social interruptions a person experiences while trying to perform a task and decreased (objective) task performance, as well as increased stress levels while performing those tasks [8]. Mark et al. showed that an individual who is interrupted in the middle of a task spends more time on that particular work sphere than those who were not, and that interrupted individuals took an average of 25 minutes to return to an original task [6]. Recognizing these failures as forms of SA failures may make it easier for us to devise better-designed PIM systems based on well-known human factors design principles.

Another consideration surrounds the fact that the very ways we consume information are changing — instead of reading and contemplating books, we read text in small fragments, “bouncing” from one piece of text to the next, collecting only what we need as we go. Kevin Kelly calls this high-velocity information foraging a form of human-based pattern recognition, akin to the the kind of real-time instantaneous assessment that pilots and real time plant operators perform [2]. This may mean that the rapid sensemaking tactics used in in such real-time theatres of operation may be appropriate to handle the immediate information demands of modern information work as well.

How might SA theory be used to inform the design of PIM tools? Defining the scope of a “situation” in the world of every-day decision making may be more difficult than that in the fighter pilot domain, which has clearly defined objectives, risks, dangers and time-demands. The tactical situation for a fighter pilot can be approximately characterised by a relatively small number of well-defined variables: the location, altitude, velocity/heading, and engine state of her plane and the identity and position of nearby aircraft. By comparison, everyday life decision-making incorporates dozens or hundreds of more vague “variables” to consider. A married, working spouse with children, for example, may consider aspects of the health and happiness of her spouse and children, her own health and happiness, her professional work commitments, social commitments to her friends and relatives, her finances, her residency, and even the state of her automobile(s), neighbors, or front lawn. Each of these “states” could have a multitude of complex factors that need to be considered; and any disturbances to any of the states of his situation might cause a person to have to re-prioritize and take appropriate action.

Thus, instead of trying to reproduce the characteristics of the paper based information tools (paper and pen, daily planners, sticky notes) which were most suitable for a time when information arrival volume was low and consumption patterns were less rapid and more contemplative, PIM tools might be made to better support informing users in dynamic environments, accounting for our innate sparsity of time, attention and working memory. Specifically, PIM tools could be designed to support users’ situational awareness in two ways:

1. Perception / Information acquisition – Keeping people informed about appropriate aspects of the world around them.
   Needs: Attention management, low cost information acquisition, filtering, and keeping (for later).

2. Interpretation and sensemaking- Letting people recognize situations quickly based on perceived information, knowledge and past experience – arrival at predictive mental models for what is going to happen next.
   Needs: contextual sensemaking, long-term memory support.

While SA-based design criteria differ significantly from previously proposed designs for PIM tools [3], a number of “PIM prototypes” have been essentially designed around these criteria. For example, work from the MIT Media Lab in the mid-90′s surrounding “mobile perceptual intelligence” [4] and JITIR (Just-In-Time Info. Retrieval) [5] with wearable computers, fundamentally surrounded giving people relevant contextual information to help people recall important information pertaining to the people and objects they were interacting with people and objects in their environment. Similarly, systems such as Implicit Query [9] offered people relevant information in the midst of their main task.

Despite the groundbreaking nature of these research PIM systems, few have made it into the reach of end-users. We wished to continue our line of research with SA-oriented PIM research efforts embodied as fully-functional, open-source projects that could be evaluated with many real users out of the laboratory, and then improved upon or incorporated into other systems. We see the SA-aspects of the following systems:

• Atomate- is a front-end perceptual processing framework that treats web streams about changes in world state (e.g., users moving, events occurring, etc) as sensor inputs to a rule engine that can automatically perform simple tasks specified by the user.
• Poyozo – is a lifestream visualiser which users re-view a chronological history of information that arrived during their day, contextualized according to where they were, and what they were doing at the time.
• Notes that Float – Just-in-time adaptive notes retrieval algorithm for list-it, our micro note taking tool that uses features extracted from the user’s notes and his or her context to determine relevance.

Although these are initial simple sketches of SA-oriented PIM systems we hope this will spark further exploration along these directions.

References

[1] – Endsley, Mica R. “Toward a Theory of Situation Awareness in Dynamic Systems” Human Factors: The Journal of the Human Factors and Ergonomics Society, Volume 37, Number 1, March 1995 , pp. 32-64(33)

[2] – Kelly, K. “Reading in a Whole New Way”, 40th Anniversary Special Issue, Smithsonian, July 2010 http://www.smithsonianmag.com/specialsections/40th-anniversary/Reading-in-a-Whole-New-Way.html.

[3] – William Jones. Keeping Found Things Found: The Study and Practice of Personal Information Management (Interactive Technologies) (Interactive Technologies). Morgan Kaufmann, November 2007.

[4] – Pentland, A. 1999. Perceptual Intelligence. In Proceedings of the 1st international Symposium on Handheld and Ubiquitous Computing (Karlsruhe, Germany, September 27 – 29, 1999). H. Gellersen, Ed. Lecture Notes In Computer Science, vol. 1707. Springer-Verlag, London, 74-88.

[5] – Rhodes, B. 2003. Using Physical Context for Just-in-Time Information Retrieval. IEEE Trans. Comput. 52, 8 (Aug. 2003), 1011-1014. DOI= http://dx.doi.org/10.1109/TC.2003.1223636

[6] – Mark, G., Gonzalez, V. M., and Harris, J. 2005. No task left behind?: examining the nature of fragmented work. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Portland, Oregon, USA, April 02 – 07, 2005). CHI ’05. ACM, New York, NY, 321-330. DOI=http://doi.acm.org/10.1145/1054972.1055017

[7] – Mark, G., Gudith, D., and Klocke, U. 2008. The cost of interrupted work: more speed and stress. In Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy, April 05 – 10, 2008). CHI ’08. ACM, New York, NY, 107-110. DOI= http://doi.acm.org/10.1145/1357054.1357072

[8] – Carton, Andrew M., Aiello, J. “Control and Anticipation of Social Interruptions: Reduced Stress and Improved Task Performance” Journal of Applied Social Psychology, Volume 39, Number 1, January 2009 , pp. 169-185(17)

[9] – Dumais, S., Cutrell, E., Sarin, R., and Horvitz, E. 2004. Implicit queries (IQ) for contextualized search. In Proceedings of the 27th Annual international ACM SIGIR Conference on Research and Development in information Retrieval (Sheffield, United Kingdom, July 25 – 29, 2004). SIGIR ’04. ACM, New York, NY, 594-594. DOI= http://doi.acm.org/10.1145/1008992.1009137