Where’s the duct tape?

In this experiment, we show turkers an image, and ask them to locate an object within the image (see HIT figure below).

One potential application of such a task is to help turkers assist blind people locating an object. A blind person who had misplaced an object (in this case their duct tape) could take a picture of a room, upload it to MT, and get back information about where the object is in the photo. Other tools could then help them use this information to locate the object. It would be great if this could be done in nearly realtime; therefore, we are concerned with speed as well as accuracy.

HIT

HIT

Results:

All the assignments were completed within 3.1 minutes. The average time for each assignment to be accepted was 77.4 seconds. The average time spent on each assignment was 59.2 seconds. The first assignment completed was submitted 90 seconds after the HIT was posted.

The following image shows where each of the 10 turkers clicked on the image:
Click locations of turkers

Discussion:

Getting a result in 90 seconds for $0.01 is encouraging. We’ll need to run more experiments to see how fast we can get results at different times of day.

The accuracy also seems pretty good — in this round, no turker tried to game the system.

You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.

1 Comment »