Let's Get Real! (I'm a tad peeved)

"...there are real issues at stake here, and there’s nothing wrong—nothing wrong at all—with people arguing about the details while at the same time being aware of the big picture."   -- Andrew Gelman

Another great follow-up from Andrew Gelman today on the whole statistics debate I referenced yesterday:
http://tinyurl.com/hlqp5b4

Do read Gelman's more formal presentation, but here, my own further informal comment:

Some of those defensive about the "crisis in replication" in psychology are arguing that the so-called "replications" (that failed) weren't actually precise replications at all, but merely rough and poor approximations... well, DUHHHH, of course there are NO TRUE REPLICATIONS in psychology... get real!: when dealing with human behavior, if you use different samples at different times on different days in different places, under varying conditions, it is not going to be an exact replication (EVEN IF you were otherwise able to duplicate the same methodological steps).  And as others have noted, IF your finding requires absolute precise duplication in order to replicate, than you are NOT doing science, which seeks to find results that generalize more widely.

Moreover, Gilbert et.al. (and other psychologists) HAVE NEVER and WILL NEVER do an adequately controlled, well-defined, unbiased, highly-generalizable experiment in social psychology using a truly random sample... it IS NOT possible!... uncontrolled independent variables are far too many, complex, poorly-defined, and likely synergistic, to permit understanding of, or definitive conclusions about, any dependent variable(s) under study, AND no human sample is ever really random (...but, that won 't prevent them from publishing such studies). Why can't we just admit out-loud to such imperfections...

Nassim Taleb is fond of dismissing such studies as BS and moving on... I'm not that harsh -- social studies can lack rigor, yet still contain glints of value and interest for further exploration, SO LONG as their results aren't presumed reliable, valid, and widely-applicable from the get-go. GOOD (and meaningful) studies in psychology are very, very, very difficult to do, and so too, good replications.