Research with Mechanical Turk

Opportunities and Challenges with Amazon's Crowdsourcing Marketplace

Create a free account

Amazon’s Mechanical Turk crowdsourcing task marketplace has changed social science research over the past 10 years. Originally designed as a distributed human workforce for small tasks that cannot be automated, such as complex categorization or labeling, the platform is now creatively used for a variety of tasks, including survey research.

Some sources give Mechanical Turk credit for creating a golden age in survey research. Others cite major drawbacks and ways to mitigate some of those problems.

Due to financial constraints, academics have traditionally relied on convenience samples, e.g. college students surveyed via clipboard on campus. Mechanical Turk represents a major step up over this methodology - reaching farther into the general population for $0.10 per survey complete.

“We show that respondents recruited in this manner are often more representative of the U.S. population than in-person convenience samples—the modal sample in published experimental political science—but less representative than subjects in Internet-based panels or national probability samples”

- www.jstor.org

Despite the adoption trend, Mechanical Turk was not designed for research.

Not enough respondents

Mechanical turk is estimated to have no more than 25k participants online in any given day. 50% of these are on the platform daily.

No demographic targeting

Tasks are labeled with the survey audience target and purpose and answered on a first come, first serve basis.

Sample distribution

Major differences between the MTurk population and the general population are shown in samples. e.g. over-representation of higher education.

Alternatives to Mechanical Turk

Traditional internet-based market research panels and probability samples (e.g. phone polling) provide some answers to these problems, but at much greater cost than Mechanical turk.

TapResearch provides the best of both worlds by reaching respondents on their mobile devices, in the apps they use every day. Learn more about the TapResearch audience network.

Our unique methodology allows us to reach millions of new respondents every month and deliver surveys at extremely low cost.

Sampling Approaches Compared

TapResearch brings together the benefits of professional research platforms and the reach of mobile.

Mechanical Turk Internet Panels Phone Polling TapResearch
Low price
DIY interface
Demographic profiling & targeting
Entry census balancing
Feasibility on small targets
(e.g. minority teens)
Throttling / pacing
Fresh respondents every day

Create a Free Account

  • Create a project in seconds
  • Choose your target audience
  • Purchase responses for as little as $0.04 per response
  • No subscription. Pay as you go!

Questions? We're here to help!
projectmanagement@tapresearch.com

By clicking register, you agree to our Service Agreement

TapResearch helps leading schools and brands answer questions every day.