...Of Random Samples

Several years ago a journal paper was published entitled "The Weirdest People In the World" detailing how UNrepresentative the sample groups often employed in statistical (particularly psychological and behavioral) studies were. If I had to boil it all down (and I agree with the impetus of the paper) it simply says that good research studies, with generalizable results, ought employ random samples… and, guess what… there's no such thing as a truly random sample…

Anyway, if you enjoyed William Briggs' thoughts in a prior recent post, here's his take on this topic:

 http://wmbriggs.com/blog/?p=5566

A couple of quick excerpts:

"Over-confidence and over-certainty abounds in many areas, the closer the subject is to mankind the greater the false surety. This is why this new paper is such a delight: it is a rare admission that all might not be as solid as hoped."

"the authors list some recommendations, such as that “Journal editors and reviewers should press authors to both explicitly discuss and defend the generalizability of their findings. Claims and confidence regarding generalizability must scale with the strength of the empirical defense.” They also say to use people from more areas of the world."