<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
		>
<channel>
	<title>Comments on: Blogs and the Dissemination of Scientific Research</title>
	<atom:link href="http://groups.csail.mit.edu/haystack/blog/2009/11/04/blogs-and-the-dissemination-of-scientific-research/feed/" rel="self" type="application/rss+xml" />
	<link>http://groups.csail.mit.edu/haystack/blog/2009/11/04/blogs-and-the-dissemination-of-scientific-research/</link>
	<description>MIT CSAIL Research</description>
	<lastBuildDate>Tue, 24 Nov 2009 22:48:44 -0500</lastBuildDate>
	<generator>http://wordpress.org/?v=2.8.6</generator>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
		<item>
		<title>By: Sanjay Kairam</title>
		<link>http://groups.csail.mit.edu/haystack/blog/2009/11/04/blogs-and-the-dissemination-of-scientific-research/comment-page-1/#comment-1102</link>
		<dc:creator>Sanjay Kairam</dc:creator>
		<pubDate>Thu, 05 Nov 2009 00:03:27 +0000</pubDate>
		<guid isPermaLink="false">http://groups.csail.mit.edu/haystack/blog/?p=687#comment-1102</guid>
		<description>I think this is a great idea!  Two important but simple questions are:

1) I think that the audience will likely need to be either for other HCI researchers or for a general audience.  Focused effort on explaining results in an accessible way to a general audience might go a long way towards getting people interested in HCI topics, the way that NASA seems to do for space exploration, which is a great goal.
2) Who would do the vetting?  I am relatively new to the whole CHI process, but I do know that it seems like reviewers are often already stretched just with reviewing papers and may not be eager to take on blog post reviews as well.  I think that just forcing authors to vet their own posts would encourage them to think critically about how to explain results and would probably make for better CHI presentations anyway - I wouldn&#039;t worry about posting the results all at once, but you could invite people to post in waves throughout December and January.

In general, I look forward to more blogging and more idea-sharing from HCI writers - it&#039;s nice reading papers for depth, but blogs would be a great way to understand someone&#039;s research interests and capture some interesting findings.</description>
		<content:encoded><![CDATA[<p>I think this is a great idea!  Two important but simple questions are:</p>
<p>1) I think that the audience will likely need to be either for other HCI researchers or for a general audience.  Focused effort on explaining results in an accessible way to a general audience might go a long way towards getting people interested in HCI topics, the way that NASA seems to do for space exploration, which is a great goal.<br />
2) Who would do the vetting?  I am relatively new to the whole CHI process, but I do know that it seems like reviewers are often already stretched just with reviewing papers and may not be eager to take on blog post reviews as well.  I think that just forcing authors to vet their own posts would encourage them to think critically about how to explain results and would probably make for better CHI presentations anyway &#8211; I wouldn&#8217;t worry about posting the results all at once, but you could invite people to post in waves throughout December and January.</p>
<p>In general, I look forward to more blogging and more idea-sharing from HCI writers &#8211; it&#8217;s nice reading papers for depth, but blogs would be a great way to understand someone&#8217;s research interests and capture some interesting findings.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: David Huynh</title>
		<link>http://groups.csail.mit.edu/haystack/blog/2009/11/04/blogs-and-the-dissemination-of-scientific-research/comment-page-1/#comment-1100</link>
		<dc:creator>David Huynh</dc:creator>
		<pubDate>Wed, 04 Nov 2009 19:48:31 +0000</pubDate>
		<guid isPermaLink="false">http://groups.csail.mit.edu/haystack/blog/?p=687#comment-1100</guid>
		<description>&quot;Interfaces are replicable by observing the design and the description.&quot; - I have not seen this done by HCI researchers before.

My point is that the reward system may be the more important thing to fix than the publication time lag.</description>
		<content:encoded><![CDATA[<p>&#8220;Interfaces are replicable by observing the design and the description.&#8221; &#8211; I have not seen this done by HCI researchers before.</p>
<p>My point is that the reward system may be the more important thing to fix than the publication time lag.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Paul André</title>
		<link>http://groups.csail.mit.edu/haystack/blog/2009/11/04/blogs-and-the-dissemination-of-scientific-research/comment-page-1/#comment-1099</link>
		<dc:creator>Paul André</dc:creator>
		<pubDate>Wed, 04 Nov 2009 19:39:38 +0000</pubDate>
		<guid isPermaLink="false">http://groups.csail.mit.edu/haystack/blog/?p=687#comment-1099</guid>
		<description>It&#039;s a great idea, and something I consider problematic in HCI. I read a lot of economics, psychology, neuroscience etc. blogs that all discuss recent and interesting research, but there&#039;s very little from HCI.

I wonder why HCI doesn&#039;t do it. To throw a provocative statement out, perhaps researchers would do more interesting research if they thought they had to blog about it and get others interested :).

DK&#039;s point about the work being well known by the time of the conference may also a) conference organisers to rethink the format (which is already being discussed), and b) present the work in a new or interesting manner. As you say, it may even serve as a good indicator of what to go see.</description>
		<content:encoded><![CDATA[<p>It&#8217;s a great idea, and something I consider problematic in HCI. I read a lot of economics, psychology, neuroscience etc. blogs that all discuss recent and interesting research, but there&#8217;s very little from HCI.</p>
<p>I wonder why HCI doesn&#8217;t do it. To throw a provocative statement out, perhaps researchers would do more interesting research if they thought they had to blog about it and get others interested <img src='http://groups.csail.mit.edu/haystack/blog/wordpress/wp-includes/images/smilies/icon_smile.gif' alt=':)' class='wp-smiley' /> .</p>
<p>DK&#8217;s point about the work being well known by the time of the conference may also a) conference organisers to rethink the format (which is already being discussed), and b) present the work in a new or interesting manner. As you say, it may even serve as a good indicator of what to go see.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Michael Bernstein</title>
		<link>http://groups.csail.mit.edu/haystack/blog/2009/11/04/blogs-and-the-dissemination-of-scientific-research/comment-page-1/#comment-1098</link>
		<dc:creator>Michael Bernstein</dc:creator>
		<pubDate>Wed, 04 Nov 2009 19:20:24 +0000</pubDate>
		<guid isPermaLink="false">http://groups.csail.mit.edu/haystack/blog/?p=687#comment-1098</guid>
		<description>@David K--
It&#039;s a legitimate worry about the work becoming broadly known.  I&#039;d personally worry that people who read the post wouldn&#039;t come to my talk at the conference.  But, if you do a good job as an advertisement, it could also potentially increase turnout too.

@David H--
Unlike in psychology, there is very little reward in HCI for reproducing previous results.  I think this is partially because HCI sometimes has tenuous claims to being &quot;a science&quot;.  But maybe levying the criticism at systems papers is unwarranted; their core contribution is rarely about the evaluation anyway. Interfaces are replicable by observing the design and the description.

But I don&#039;t think a results blog would affect this issue either way.  The paper still had to get through peer review, which required its evaluation to be strong enough to be believable. But I wouldn&#039;t be surprised if blog posts suppressed the evaluation parts of systems papers -- if you built Air Guitar Hero, few people on the web care about the results of your ANOVA.</description>
		<content:encoded><![CDATA[<p>@David K&#8211;<br />
It&#8217;s a legitimate worry about the work becoming broadly known.  I&#8217;d personally worry that people who read the post wouldn&#8217;t come to my talk at the conference.  But, if you do a good job as an advertisement, it could also potentially increase turnout too.</p>
<p>@David H&#8211;<br />
Unlike in psychology, there is very little reward in HCI for reproducing previous results.  I think this is partially because HCI sometimes has tenuous claims to being &#8220;a science&#8221;.  But maybe levying the criticism at systems papers is unwarranted; their core contribution is rarely about the evaluation anyway. Interfaces are replicable by observing the design and the description.</p>
<p>But I don&#8217;t think a results blog would affect this issue either way.  The paper still had to get through peer review, which required its evaluation to be strong enough to be believable. But I wouldn&#8217;t be surprised if blog posts suppressed the evaluation parts of systems papers &#8212; if you built Air Guitar Hero, few people on the web care about the results of your ANOVA.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: David Huynh</title>
		<link>http://groups.csail.mit.edu/haystack/blog/2009/11/04/blogs-and-the-dissemination-of-scientific-research/comment-page-1/#comment-1097</link>
		<dc:creator>David Huynh</dc:creator>
		<pubDate>Wed, 04 Nov 2009 18:45:41 +0000</pubDate>
		<guid isPermaLink="false">http://groups.csail.mit.edu/haystack/blog/?p=687#comment-1097</guid>
		<description>When I read about psychology studies, I often read that they tend to get re-run by several independent labs in the world in order to see whether the results are actually reproducible. From my experience with HCI research, each user study tends to get run only once, maybe 2 weeks before a deadline. The results are then taken as bible and quoted by subsequent papers without any attempt to re-run the study to reproduce its result. Is it possible that we might be building on a shaky foundation of irreproducible results?

One reason why studies are not re-run is because research tools are not reusable. They fall apart as soon as the last usability subject leaves the lab. And no researcher would be willing to re-implement a competitive tool to do a proper comparison, let alone re-run a study that earns him/her no novelty point.

So the publication process is not just problematic for the time delay, but also for the reward scheme. So, fixing the time delay might just get us bogus papers faster.

A friend of mine claims that most if not all things you use a lot on your computer and on the Web today have not come out of HCI academic research proper in the last 2 decades. I am having trouble arguing with him :)</description>
		<content:encoded><![CDATA[<p>When I read about psychology studies, I often read that they tend to get re-run by several independent labs in the world in order to see whether the results are actually reproducible. From my experience with HCI research, each user study tends to get run only once, maybe 2 weeks before a deadline. The results are then taken as bible and quoted by subsequent papers without any attempt to re-run the study to reproduce its result. Is it possible that we might be building on a shaky foundation of irreproducible results?</p>
<p>One reason why studies are not re-run is because research tools are not reusable. They fall apart as soon as the last usability subject leaves the lab. And no researcher would be willing to re-implement a competitive tool to do a proper comparison, let alone re-run a study that earns him/her no novelty point.</p>
<p>So the publication process is not just problematic for the time delay, but also for the reward scheme. So, fixing the time delay might just get us bogus papers faster.</p>
<p>A friend of mine claims that most if not all things you use a lot on your computer and on the Web today have not come out of HCI academic research proper in the last 2 decades. I am having trouble arguing with him <img src='http://groups.csail.mit.edu/haystack/blog/wordpress/wp-includes/images/smilies/icon_smile.gif' alt=':)' class='wp-smiley' /> </p>
]]></content:encoded>
	</item>
	<item>
		<title>By: David Karger</title>
		<link>http://groups.csail.mit.edu/haystack/blog/2009/11/04/blogs-and-the-dissemination-of-scientific-research/comment-page-1/#comment-1095</link>
		<dc:creator>David Karger</dc:creator>
		<pubDate>Wed, 04 Nov 2009 17:50:15 +0000</pubDate>
		<guid isPermaLink="false">http://groups.csail.mit.edu/haystack/blog/?p=687#comment-1095</guid>
		<description>This is a great idea; I&#039;d definitely post work there.  It might be nice if the blog supported crossposting from specific research groups&#039; blogs, which is where they might actually want to announce their results.

Such a blog would exacerbate one big problem with CHI: as you say, presentations there are for 7-month-old work.  With the blog, it would become 7-month-old and _broadly known_ work---a recipe for boredom.  The right fix there, of course, is to shorten the submission-to-publication timeline.  I&#039;ve just finished chairing ISWC 2009 where we had quite a bit of success with a tighter schedule: from submission, 2 weeks for reviews (nobody does them before the last 2 weeks anyway), 1 week for metareviews, 1 week for rebuttals and final decision, 1 week for camera-ready copy.   If you abandon paper proceedings, that camera ready can arrive the day before the conference, giving you a 5-week timeline instead of 7 months.  We ended up stretching it a bit farther to give attendees time to get visas and book flights, but still had a June submission deadline for our October conference.</description>
		<content:encoded><![CDATA[<p>This is a great idea; I&#8217;d definitely post work there.  It might be nice if the blog supported crossposting from specific research groups&#8217; blogs, which is where they might actually want to announce their results.</p>
<p>Such a blog would exacerbate one big problem with CHI: as you say, presentations there are for 7-month-old work.  With the blog, it would become 7-month-old and _broadly known_ work&#8212;a recipe for boredom.  The right fix there, of course, is to shorten the submission-to-publication timeline.  I&#8217;ve just finished chairing ISWC 2009 where we had quite a bit of success with a tighter schedule: from submission, 2 weeks for reviews (nobody does them before the last 2 weeks anyway), 1 week for metareviews, 1 week for rebuttals and final decision, 1 week for camera-ready copy.   If you abandon paper proceedings, that camera ready can arrive the day before the conference, giving you a 5-week timeline instead of 7 months.  We ended up stretching it a bit farther to give attendees time to get visas and book flights, but still had a June submission deadline for our October conference.</p>
]]></content:encoded>
	</item>
</channel>
</rss>
