I proposed researching some tools that, instead of relying on luck to determine which tweets you see, instead figure out the most valuable ones to show you.  I don’t believe that filtering by person (a big mix of different interests) and hashtag (unreliable, often nonexistent) is the best way to locate the tweets that are most useful to me.  But KP poured some cold water on this idea, arguing that tools that improved your ability to filter tweets would just lead to people following more users, such that they got swamped with tweets again.  More generally, that regardless of what information filtering tools we get, we will always push them to the limit of delivering too much information.

I realized I’ve experienced this myself—I used to visit various web sites to gather information. As I began to find it burdensome to keep up with all these web sites, I ultimately switched to an RSS reader to make it easier for me.  But that has simply allowed me to subscribe to more sources than I was following manually, such that I am again feeling swamped by my information feeds.

Does this mean that any assault on the cliched “information overload” problem is doomed, since whenever we fix it people will load up more?  It seems the only hope is to convince people that they don’t actually need the information they are gathering.

This idea actually relates to another line of our research, on note-taking.   People like to write down all sorts of little scraps of information.  But according to data we’ve logged from our list.it notetaking plugin for firefox (13,000 users—you should give it a try), a lot of those notes are never retrieved.  So why are they written down?  Perhaps it’s because people worry they might need them later, even though they never do.   Something similar seems to be going on with information streams—once they exist, people start to worry they might miss something important, even though they never worried about it before.   If we could somehow convince people that they could find anything that really mattered, they might become less gluttonous followers of information.

And this leads to another project of ours, FeedMe, described in a recent post on this blog.   I’d love to stop following a bunch of my newsfeeds, if only I could be confident that the really good bits would be brought to my attention.  There are collaborative filtering tools like Digg, but I don’t trust them to know what I like.  FeedMe is instead based on having friends forward interesting content to me.   I trust my friends more than any algorithm; if enough of them read a given blog, I can stop on the assumption that they’ll forward interesting content to me.   But right now I don’t have good feedback on how many of my friends are reading what blogs.   That might be an interesting feature to add to FeedMe.  An alternative might be for a group of friends to “divvy up” a blog, each reading a subset of the content and deciding which to forward to which friends.  This would need some supporting interfaces, of course.

• Memory triggers (reminders) are important – from a randomly selected subset of the notes we examined, 58% were primarily memory triggers, a general term that refers to notes like to-dos and reminders that serve a primary purpose of calling your attention to something.
• Memory triggers are easy to identify but not to interpret – when people other than a note’s author are shown a memory trigger note, there’s strong agreement (consensus) identifying it as such (k=0.77, N=2500). However, they’re difficult to interpret than other note types because of their brevity and ambiguity. For example, the note “lincoln park” might be a memory trigger, but it is not clear its interpretation: does the author have to go to Lincoln Park, in Chicago? Is it a reference requiring context, such as a name of a book to read? Is it a reminder to listen to the band Linkin Park?
• Shorter notes are deleted more quickly than longer notes – This is likely because short notes often represent memory-triggers which are often discarded after use, whereas longer notes tend to be reference items and items kept for posterity.
• Bookmarking – Notes containing URLs contain little text modifying them – 36% contained only the URL; the median number of words per-note with URL was 2.
• Copy and paste from the web – A significant number of individuals regularly copy and paste entire fragments of web pages, e-mails, and even spreadsheets into their notes for safekeeping.  These kinds of notes account for the majority of notes which are over 200 words long and which are kept for more than a month.
• There seem to be are two types of people – keepers and deleters.  Although the statistics here aren’t extremely significant, the distribution of percentage of notes people delete has peaks at around 30% and above 90%.  This implies that most people delete a small fraction of their notes, while others delete most of their notes.  An initial inspection reveals that this latter category are people who use List-It primarily as a to-do manager (and delete notes when they’re done).
• Searches to personal note collections were very infrequent but repeated – The median number of searches people performed over a one month period of usage was 5, (min: 0: max: 122).  Among the searches, 60.2% constituted repeated searches (searches that had been done at least once before on a separate occasion).  Interestingly and not surprisingly, the most frequently searched term was “todo”.

Based on interviews and surveys with participants, we also arrived with the following conclusions:

• List-It was used to permanently keep track of a number of different kinds of information.  Instead of being merely a temporary resting place, interviewees mentioned that they kept most (if not all) of certain kinds of information permanently in List-It.  The kinds of information that participants most often cited keeping entirely in list-it were: 1) Ideas and brainstorming. 2) Text copied and pasted from the web. 3) Short-term to-do reminders. 4) Things to look up some day.
• Ease of creating and accessing notes in List-It were its most valued attributes. A majority of users rated  “Much easier” (1) to write information down in than other tools they used (on a 7-point scale), and “Somewhat easier” to look things up in than their other tools.

We are currently in the process of preparing a paper describing these results in detail; please stay tuned and we’ll announce when this is available.  But we think that these results immediately suggest ways that we think we can improve how well List-It works for people.  First, since speed and ease of use were important, we plan to work on making List-It more easily accessible directly from places outside the browser, and to let users directly add links to their List-Its by pressing a single hotkey/button.  Second, due to the importance of reminding, we plan to investigate ways to add reminding to List-It such that it will effectively show people notes at appropriate times. More on this later!

1) Strongly typed worksheets with “advisory” error checking. For instance, the user can designate a particular column to hold numbers only, and maybe proceed to enter a date, but would then see an Excel-style warning dot in the cell in question.

2) Transparent many-to-many or one-to-many relationships between worksheets in a workbook (think foreign key relationships in database-speak). The user can designate a particular column to hold references to rows in another worksheet, or lists of such. The other worksheet will then automatically have a corresponding column added containing references going in the other direction.  (E.g. if each row in the “Departments” worksheet has a column referencing “Courses”, then “Courses” has a column referencing the corresponding rows in the “Departments” worksheet.

3) Hierarchical presentation of relationships between worksheets in the workbook. Columns that reference other worksheets may be configured to show any subset of columns from the referenced worksheet, and so on.

User testing with multiple prototypes showed that the user interface needed to be very similar to that of a traditional spreadsheet (e.g. Excel) to be usable by most users in the target population. Significant features hypothesized to make the interface more efficient (e.g. automatic report layout management) proved only to confuse the users and make it harder to design consistent editing affordances. Nevertheless, we did manage to integrate the key high-level features of the application (relationships between worksheets and the presentation of resulting hierarchical data on screen) into a prototype bearing very much of a resemblance to Excel.

(This project was done by Paul Grogan, Yod Watanaprakornkul, and me.)

I argued in favor of letting individuals make their own RDF collections (using, for example, our Exhibit framework) and worry about merging them with other people’s data later.  Stefano’s response accused me of using “RDF” and “structured data” interchangeably, asserting Exhibit is really just a nice UI over spreadsheet (tabular) data—that although it can export RDF, it is “not properly using RDF” because it has “lost the notion of globally unique identifiers (and in that regard, is much more similar to Excel than to Tabulator)”.  Tim Berners Lee has made similar complaints to me about Exhibit not using RDF.

This argument highlights for me yet an important ambiguity about what RDF is.   I occasionally have to help people understand that RDF is a model, not a syntax.  That some data can be RDF even if it isn’t serialized to RDF/XML.  That the key is to have items named by URIs, connected by relations named by URIs.  Stefano’s argument suggests a different blurring: between the model and its intended use.  Stefano’s “not properly using” phrase implies that if you don’t intend to merge your data into the global namespace, then even if you implement the model  and wrote it down as RDF/XML to boot, you won’t be “properly using RDF”.

I want to address both these claims: that Exhibit is just a UI over spreadsheets, and that using RDF this way isn’t proper.

RDF and spreadsheets

Regarding the spreadsheet claim, I’ll begin by admitting that Stefano is absolutely right:  Exhibit is a visualization tool for tabular (spreadsheet) data.  But notice that all RDF is spreadsheet data—I can take all the RDF in the world and throw it into one spreadsheet.  In fact, I only need three columns to contain the subject (tail), object (head), and predicate (link) for each RDF statement.  Admittedly none of today’s spreadsheets would have enough rows, but that’s an engineering detail.  So, the spreadsheet model isn’t the problem.   And we also agree that Exhibit’s interface is nothing like spreadsheets’, and far better for the collection visualization tasks it is designed for.

I think instead that what Stefano is objecting to is a usage characteristic of spreadsheets versus RDF.  When I open a spreadsheet, the data it shows me is right there, in a file on my own system.  Global identifiers don’t matter because the data is all there (and presumably self-consistent) in the one spreadsheet.   In contrast, in Stefano’s image of RDF (and in Tim’s, as one can see from the Tabulator project) the data about a particular entity is spread all over the web, and it is the globally unique identifier that lets you go out, gather all that data together, and know that it is all about the same entity.

This is certainly an appealing vision.  But I want to argue that a focus on globally unique identifiers neglects two benefits of RDF that I consider equally important: data portability and schema flexibility.

Spreadsheets suffice

To illustrate this argument, I’ll hark back to a previous post where I discussed a data integration problem that should have been easy but wasn’t.   I keep an  Exhibit of folk dance videos on the web.   Recently, Nissim Ben Ami posted a collection of 511 new dance videos on Youtube.  I wanted to incorporate it into my site.  But it quickly became apparent that said incorporation would basically require my entering all 511 video descriptions manually into my system, and I still haven’t gotten around to it.

The major barriers were twofold.  The first was syntactic:, the structured descriptions of the videos were delivered as XML.   That meant that in order to get at the data, I was going to have to learn XSLT—something I’ve been putting off for years.   The second hurdle is semantic: Youtube has the wrong schema for my folkdance videos.  I care about choreographer, dance type, and year choreographed; YouTube only offers slots for submitter and submission date of the video.  So, as you can see from this example, the contributor takes the usual approach: he takes his nice structure data and shoves it into the generic comment (info) field as free text.  All that structure is instantly lost.

Suppose instead that spreadsheets (or, in a pinch, RDF) were the accepted framework for publishing information on the web.  The YouTube “spreadsheet” would contain submitter and submission date information, but Nissim could just add “artist” and “composition-date” columns to hold the data he wanted to enter.   I would then be in a great position to download his data and incorporate it into my own catalog (spreadsheet).  What would I have to do?  After opening his spreadsheet and mine, I’d have to match columns—perhaps he called his “artist” and “composition date” while mine are “choreographer” and “year”.  But a simple copy and paste fixes that discrepancy.  Merging entities is not much harder than merging properties: a simple global replace will convert his choreographer “Israel Ya’akovi” to my “Israel Yakovee”.  The local consistency of his data and mine means that I only have to work once per choreographer (and in most cases I won’t have to: there’s a standard spelling for almost every choreographer’s name, which serves as a unique identifier in this context even if it isn’t a URL).

Overall, my work has reduced by order of magnitude.  Instead of laboriously entering 511 new records, I just download a spreadsheet and match up a handful of properties (columns) and a few tens of choreographer names.

Stepping back, observe that I’ve relied on two things.   First, on data portability—my being able to download the data in a convenient form: not XML, which is a programmer’s friend but an end-user’s enemy; rather, something I can just look at and understand.  Second, on schema flexibility—on Nissim’s being able to add whatever columns/properties he decides are important, instead of being limited to those used on the hosting web application.

I’m also relying on some features of this particular scenario, but I believe they often hold.   I am relying on Nissim’s data having only a small number of properties so that I can map them manually to mine.   I also rely on there being a small number of choreographers, and hope to take advantage of most of them having matching names in his data and mine—these names certainly aren’t globally unique identifers, but they are “unique enough” when considering just my data and his.  Critically, I am not thinking of pulling all data about a given dance from a multitude of different web sites—this would demand global unique identifiers to link data since I would never have the patience.  Rather, I am considering a pairwise data acquistion: taking data I want from one internally consistent site.

Such pairwise acquisition is commonplace: any time a scientists wants to pull a data set from some other scientist’s lab, or a consumer wants to download product information about several cameras from a review site, or a student wants to include a Wikipedia data set in a report they are writing, there is an obvious single source and target for a data merger.   And there’s a human being who has the incentive, and with the right tools the capability, to do the limited amount of work needed to accomplish that merger.

This is a simple low-hanging fruit argument.  It would be wonderful to be able to automatically merge data from thousands of different sources into a coherent whole.  And this is a problem Freebase will need to solve, if they want to become the hub for aggregation of structured data.  But right now we can’t even manually merge data from two sites without doing a ridiculous amount of grunt work—so perhaps we should give some attention to that easier problem on our way to solving the hard one.

Don’t skip the wild west

I’d like to so these efforts proceed in parallel, but I’m worried about enthusiasm for the more ambitious goal blocking movement toward the low-hanging fruit.  I recently submitted a proposal to NIH on the topic of data integration that reflected my perspective above.  I argued that the current efforts in the Biology community to force everyone to adopt a common ontology (and sometimes repository) for their experimental data are being resisted by biologists who think they know best how to present their data.  I suggested as an alternative that we give biologists tools, such as Exhibit, that would encourage them to publish their data in a common structured syntax, and worry about integrating all that data after it has become available in structured form.  The proposal rejection was accompanied by a review that said, on the one hand, “The benefit of the proposed approach is that it is very different from some multi-institutional data sharing projects (like caBIG), which have used a very rigid, top-down approach to creating semantics. Even if this project is unsuccessful it could bring to light new ideas and strategies that might make those large-scale projects more responsive to investigators and more successful.”  At the same time, it argued for rejection because “The absence of any control over the information models and ontologies – truly a semantic wild west – is daring and may ultimately be the downfall of this project.”

I’m fascinated to see, in the same review, a recognition of the problems that the current centralized approach is bringing (lack of buy-in to common ontologies by individual scientists who think they know better and probably do), and an unwillingness to tolerate the contrary (anarchic) solution.  I also love the metaphor of the “semantic wild west” because I think it supports my argument.  Would anyone have suggested establishing a city of several million people just after the west was opened for settlement?  The west’s early wildness was an unavoidable phase of its evolution towards the thickly settled and uniformly governed area it is now.    In the same vein, I think that our semantic web is best grown by encouraging individual semantic-web settlers to create their own data homesteads and begin looking for the trails that connect them to neighboring collections.  We need to get the data into plain view first.   Later we can send in the data sheriffs and place all those data sets under uniform governance.

Being argumentative by nature, I wanted to highlight some of the benefits of the looser, sloppier approach to data sharing that we took for SIMILE.   Obviously, being able to link data from multiple sources, and feed it into a search engine as Stefano describes, is a great thing.  But there are some tremendous advantages that accrue when even a single individual decides to create a blob of structured data with no reference to anyone else’s.

The first is interaction.  As shown with our Exhibit framework (created by David Huynh, now also at Metaweb), structured data enables rich visualization.  If my data objects have coordinates, I can plot them on a map.  If they have dates, I can put them on a timeline.  If they have colors, I can filter or sort by color.  It doesn’t matter if I call those properties latitude, longitude, date and color, or northSouth, eastWest, sinceTheCreation and elementOfTheRainbow, and whether I decide that my city is Chicago or the Windy City—as long as I have my own internally consistent names for them, I can use them to hook my data into interesting visualizations and interactions.

The second benefit is portability.  If I publish some interesting data as part of an HTML document, then anyone who wants to use that data for something else—to rebut my argument, to mash it up with some other data, to put it some use I never thought of—has the unpleasant job of scraping said data out of the HTML into a usable form.  This generally requires a programmer, and even for them it’s a tedious task that distracts them, and may deter them, from what they really want to do with the data.  But if that data is published as data—even in something old fashioned as a spreadsheet—it becomes way easier to grab it and reuse it.  Look at how much of the blogosphere is made up of cross-references, trackbacks, and responses to other blog postings.  If you’re going to argue about something involving data—for example, whether a single payer system is going to end up saving or costing money, or whether today’s perfect game is all that unusual—you probably want to publish that data to support your argument.  At which point, someone who wants to refute your argument is going to want to use that same data.  That’s going to be a lot easier if they can get that data from your posting.  That’s the theory behind our Datapress project, which aims to let you post data sets (and visualizations of them) in your Wordpress blog, and lets other people refer to and reuse that data.  In that sort of one-on-one debate over data, it really doesn’t matter whether I use the same identifiers as Freebase—you can take my identifiers and use them to build your rebuttal.

Uniformity does start to matter when someone wants to mash up data from multiple sources.  If those sources haven’t agreed on identifiers beforehand, then the masher has some work ahead—this is a case where a centralized vocabulary is really helpful.  But again, getting the data at all is such a big jump over the current state of affairs—I imagine how grateful mashup makers would be if all they had to do was merge some identifiers instead of retyping a whole spreadsheet from scratch.  The point here is that unlike Stefano’s hypothetical search engine, that wants to issue a query against all the world’s data at once, your typical mashup author just needs to deal with a couple of (probably small) data sets.  His or her particular data integration problem is quite manageable a posteriori.

I’ll also dust off an argument David Huynh once made to me, even if it might get him in trouble with his current employer.   Unification is not an absolute, but contextual.  Whether two things are the same may change depending on what you are doing with them.   Continuing my never-before attempted forays into sports analogies, are the Brooklyn Dodgers the same as the L.A. Dodgers?  If you want to talk about the team that moved from Brooklyn to LA, the answer must be yes!  But in a different context you might be interested in comparing the lifetime records of these two distinct teams.  (In fact, Freebase tries to have it both ways: it asserts that the Brooklyn Dodgers were “later known as” the Los Angeles Dodgers (implying they are the same team with a name change) but asserts that Los Angeles Dodgers were founded in 1958, which clearly isn’t true of the Brooklyn Dodgers that folded in 57.)

This is obviously one of those half-empty half-full debates:  We both recognize the value of both approaches, but are compelled by different aspects.  Stefano looks at the amazing things that could be done with a single consistent data universe, and worries about how to create it.  I look at the amazing things that can already be done with a host of disjoint but internally-consistent data microverses, and find that compelling enough to allay any worry about whether we’ll ever need http://www.freebase.com/view/en/lion to unify with http://en.wikipedia.org/wiki/Lamb .

We received lots of positive feedback after the talk, and received a number of questions and suggestions.  We wished to briefly re-address them here in case others were also wondering:

• Can I try it? Yes, find it here: List.it – a Quick note taking tool for Firefox
• Is the source code available?  Yes, via our List.it Google Code Respository
• Can I use it to run my own study on note taking?  Yes, that’s a major reason why we’re releasing the source under an MIT license.   We have not yet released our server code because it’s being cleaned up and refactored; expect this soon.
• Are you releasing a notes corpus to the public for PIM research use?  We hope to, eventually. We believe that a corpus of notes-to-self will help us to develop smarter tools.  But, we are currently trying to figure out how to help users who volunteer to share their notes to easily scrub note contents to prevent the release of sensitive info, and yet to retain enough of the original characteristics of the notes so as to be useful for research.  We will keep you updated.
• Are there plans to produce a mobile (iPhone/Android/Symbian) version? Yes. We have one new student who has joined the project to build a mobile list.it.  In the meantime we recommend the following alternatives: Evernote for iPhone and AK Notepad for Android.
• Are you planning on adding feature X?  Please see our Google code Feature Request List.

Once again, we are looking for contributors who are interested in making list.it more useful and have some coding experience. Also we are always looking for your comments, opinions and insight about features that would make list.it more useful.  Join our Google Group if you are interested in contributing your insight and/or code.

They built a system called Giornata (UIST 2008) that extends the desktop with ABC.   It’s a generalization of virtual desktops, offering a virtual desktop PER TASK.  But it’s more than just a virtual desktop—besides windows, it flips colleagues, files on the desktop, etc.  It offers persistent display of tags associated with an activity.  You can add whatever tags you like.  Giornata applies the currently active tags to everything you touch now, and can also apply them to whatever you’ve touched in this activity in the past as well.  Tags on activities are transfered to individual documents you are working with, so can use them as search terms.  There are collaboration tools—listing of people and groups you associate with the activity.  And activity-specific awareness tools—eg, how many unread emails you have from each of these relevant people.   This tool was described at UIST 08.

They gave Giornata to 5 people (2 faculty, 2 grad students, one industrial designer); in CHI they report on its use over time.  People used it for a couple of months, and reported ways it helped.  Users averaged 7 activities with 28 activity switches/day.  Very few activities ever got “closed”.  One user kept many activities as a “todo list” but worked only on a few at a time.  Others had few activities but one was a “hub” activity they switched back to often.   There were an average  of 1.8 tag words per activity.  Almost all were set at activity creation—very few changes later.  Project or event names.  Many activities with no tags.

Subjects said it helped (vs typical virtual desktops) to explicitly bind specific activities.  Storing on the desktop by activity instead of filing was the biggest win.  Tagging was perceived as less important.

Unfortunately, with such a small sample they couldn’t get statistically significant results.  But it is nonetheless thought provoking.  I wish they’d explained why the subjects stopped using the tool after a while—I didn’t get a good sense of its limitations/drawbacks.

The list.it team is proud to announce the release of list.it version 0.4.0. We’ve been working hard on it and we’re ready to release it to you!

Most of these features were strongly influenced from feedback from our users. If you have any feedback of your own (feature suggestions, bug reports, questions, comments) please contact us through our new Google Group at  http://groups.google.com/group/list-it or email us at  listit@csail.mit.edu.

Remember, list.it is open source and our code is hosted here, so please take a look if you are interested. We also have a list of features we’re currently working on in the Issues section of the site, and we’re always looking for more.

For help in understanding some of our more useful features, we have created a short tutorial movie you can find on the list.it project site or by clicking “About” at the top of the list.it window.

This version also contains quite a few new and exciting features:

• Notes can be re-ordered! (drag them by their dot)
• Faster and lighter code.
• Improved synchronization.
• Quick re-searches using the re-search tab
• Undelete (dustbin)
• Enabled click-and-hold-to-select notes
• Changed sidebar (note) icon: Left-click icon to open listit, right-click now opens input panel

We hope list.it v0.4.0 will make for easier, better and happier note-taking!

Expect much more coming very soon.

We have 10,400 registered users!!

We’re always very excited to see list.it get mentioned in the press and we were elated when we saw that Le Monde’s website was featuring list.it as its top Technology story last week. I’ve been told the article was quite good.

This might help explain why we currently have 10,400 registered users of list.it. I would like to congratulate the list.it team (electronic Max, Michael Bernstein, Katrina Panovich and Mason Tang) for this achievement.

Although this is personally the first major open source project I’ve helped to launch, we are following the advice of our friends with the Simile Project, in particular recent Haystack alumnus David Hyunh who led the creation of the Simile Widgets google code project early last year.   And, of course, we live in the birthplace of the Free Software Foundation so we have a lot of people to lean on.   But we welcome comments; we’re newbies to this and want your input!

Over the next few days, you will see our project page slowly take shape; we’re working to rapidly create some tutorials and documentation both for end-users and developers.  We’re going to fork multiple branches of our client to support more experimental development and listit extensions.  And finally we will populate our “Issues” page with our “Want.It” plans for features we are currently working towards for future list.it releases.

We are hoping that people will find our project even more useful, and welcome those interested to take part in it’s development.  We need your help!

My co-investigators Michael Bernstein, Greg Vargas and Katrina Panovich and advisors David Karger and mc schraefel and I are very pleased at this nomination, and are looking forward to presenting it in April here in Boston.  We are meanwhile also working vigorously to design a follow-up study surrounding specific aspects of note-taking, mobility and retrieval.   We now have over 4,000 list.it users and many have volunteered to let us study their note taking habits.

We would like to thank all of our study participants and those who have downloaded and use list.it on a daily basis.  We also thank everyone who has used list.it and given us feedback about ideas and areas for improvement.   As PIM tools are the most used tools in the everyday activities of our lives, we believe that this area is a fertile and important area for formal study and innovation.  We would also like to reach out to the numerous other researchers in HCI studying note taking and related areas in PIM to identify potential ways to run more effective studies and design future tools.

Our submission for our paper can be found here:
M. Van Kleek, M. Bernstein, K. Panovich, G. Vargas, D. Karger and mc schraefel. Note to Self: Examining Personal Information Keeping in a Lightweight Note-Taking Tool. Note, CHI 2009.

Please contact us if you have any comments or questions at listit at csail dot mit dot edu .