Mechanical Turk: The New Face of Behavioral Science?
Geege Schuman stashed this in Psychology
Stashed in: Freakonomics
If you give the vase a good shake before taking your sample, then you’ll have randomized it, eliminating the sampling bias.
Similarly, if you’re doing a study of human psychology or behavior, and sample only consists of American undergraduate students who are either: (a) need beer money, or worse yet (b) are required by the same few professors to volunteer as subjects; you might come away with the mistaken impression that all humans are like western undergraduates. In these fields they’ve become the standard subject for the species at large, which is a status they might not deserve.
In a study titled, “The Weirdest People in the World?” researchers conducted a kind of audit of studies that exclusively sample US college students -- who, among other similarities, tend to hail from societies that are “Western, Educated, Industrialized, Rich, and Democratic (WEIRD)”. They found that American undergraduates in particular were vastly over-represented:
“67% of the American samples [in the Journal of Personality and Social Psychology in 2008] were composed solely of undergraduates in psychology courses. [...] A randomly selected American undergraduate is more than 4,000 times more likely to be a research participant than is a randomly selected person from outside the West.”
They then compared the results of WEIRD-biased studies to studies that researched the same effect, but sampled subjects from non-WEIRD populations.
College students are fun to study though because so many of them are narcissistic.