Statistics, Intuition, and The Big Picture

(via Wikipedia)

For someone who isn't directly involved in either the education wars or the statistics wars, it's embarrassing how often I end up posting about these two areas. Anyway, a new piece in Nature,  (properly) worries once again over the use of statistics in research:

http://tinyurl.com/oofcafa

The writers agree that "P values are an easy target: being widely used, they are widely abused," but also note that banning p-values "...will in fact have scant effect on the quality of published science."
As they further stress:
"There are many stages to the design and analysis of a successful study... The last of these steps is the calculation of an inferential statistic such as a P value, and the application of a 'decision rule' to it (for example, P < 0.05). In practice, decisions that are made earlier in data analysis have a much greater impact on results..."

What is interesting to me is that 40 years ago when I took statistics in grad school I felt a definite wariness and distrust of statistics, on simply an intuitive basis (but didn't have the mathematical sophistication to clarify my concerns), that seems to have now become technically mainstream 40 years later. Statistics haven't changed much in that time, but the consideration of the field sure has.

The older I get, the more I trust my strongest intuitions (even as iffy and fragile as they may be), and the less I trust the assertions of 'experts.'

"The only real valuable thing is intuition." -- Albert Einstein

"Intuition becomes increasingly valuable in the new information society precisely because there is so much data." -- John Naisbitt

"Intuition is a conceptual bird's-eye view that allows humans to draw inferences from high-level abstractions without having to systematically trace out each step. Intuition is a wormhole. Intuition allows us get from here to there given limited computational resources." -- from "Less Wrong" blog

and lastly, Terence Tao:
"The point of rigour is not to destroy all intuition; instead, it should be used to destroy bad intuition while clarifying and elevating good intuition. It is only with a combination of both rigorous formalism and good intuition that one can tackle complex mathematical problems; one needs the former to correctly deal with the fine details, and the latter to correctly deal with the big picture."

 

Measurement

WikimediaCommons

Good, succinct post from Andrew Gelman today on statistics and "measurement," noting at start that "Statistics does not require randomness. The three essential elements of statistics are measurement, comparison, and variation." And of those, Gelman believes that "measurement" is the most slighted or "neglected" element:
http://andrewgelman.com/2015/04/28/whats-important-thing-statistics-thats-not-textbooks/

He says when it comes to measurement in research or statistics textbooks "...there’s silence, just an implicit assumption that the measurement is what it is, that it’s valid and that it’s as reliable as it needs to be."
Of course we witness this all across research... from Government economic statistics that get routinely "revised" or recomputed on a near-monthly basis, to epidemiology where statistics change sometimes with each new study sample, to even high level physics where new findings too often have to be altered or abandoned when measurements are re-taken or newly-analyzed. And perhaps psychology takes the greatest brunt of criticism as Gelman writes, "A common thread in these [psychological] studies is sloppy, noisy, biased measurement." Indeed, 'behavior' is one of the most difficult things to measure and generalize about empirically. ...But, no one ever said it was easy.

I don't think Gelman even goes far enough here. Part of the intrinsic problem of measurement is the necessity to recognize and precisely define all pertinent variables that are to be measured... in most fields, a close-to-impossible task; so measurement is relegated to a rough (and sometimes VERY rough) approximation, while still being discussed as if exact.
I suspect one of the reasons there is so much anti-science sentiment/distrust in this country is because of how often the public sees a scientific "measurement" go awry, after it had been presented as "certain" (I realize this is often more the fault of press or other science-writer coverage, than due to the scientists themselves).

Miniature Worlds

"Mathematicians are explorers of many miniature mathematical worlds. Explorers often find the objects or phenomena that they discover novel and surprising and they do not always describe them accurately. Indeed, just because they are novel and surprising, early explorers may mis-describe them, misunderstand them, and give most misleading reports.
"It is only after much further study that the 'true nature' of the kangaroo, or the manatee, or carnivorous plants are determined. The same is true of mathematicians exploring their miniature worlds."
-- David Wells


[p.s.: over at MathTango this morning a new book review.]

More Bookish Notes

Recently finished Jim Henle's brief, quirky, new book, "The Proof and the Pudding"... HIGHLY recommend it if math playfulness is your thang (will likely have a review of it up soon, perhaps Sunday or Monday at MathTango).
Meanwhile on Twitter, Steve Strogatz calls attention to an upcoming Eugenia Cheng book, "How To Bake Pi," which coincidentally appears to take a bit of the same approach as the Henle book:
http://www.amazon.com/How-Bake-Pi-Exploration-Mathematics/dp/0465051715

May as well also mention that prolific Alfred Posamentier has a new one on the way, "Problem-Solving Strategies in Mathematics: From Common Approaches to Exemplary Strategies":
http://tinyurl.com/oqb2hwu
and, if that sounds too dry for you, David Spiegelhalter has the antidote with "Sex By Numbers: What Statistics Can Tell Us About Sexual Behaviour":
http://tinyurl.com/ovkophy

Lastly, will just note again that the re-issue (by Liberalis Press) of Matthew Watkins' wonderful trilogy on prime numbers is just a couple of weeks away (although as a British volume I'm not sure it will have very good American distribution, other than online):
http://tinyurl.com/oa6oct5

Of Turing Tests and CAPTCHAs

(via WikimediaCommons)

 I've never been a fan of the so-called 'Turing Test.' This io9 article from George Dvorsky explains some of the problems with it, while offering up some possible alternatives:
http://io9.com/8-possible-alternatives-to-the-turing-test-1697983985

It also interestingly notes that we already have today a sort of 'reverse Turing test' wherein a machine must detect if a person is a real human or not via those ubiquitous "CAPTCHA" verification challenges -- all of which results in an 'arms-race' of bot-makers trying to defeat the CAPTCHAs, and other humans trying to stay a step ahead of the bot-makers.

On a side-note, if you've never heard this explanation of CAPTCHAs (and the positive, creative use they're put to), from their creator, it's definitely worth a listen:
http://www.npr.org/2013/10/04/191620023/can-you-crowdsource-without-even-knowing-it

Loving... Mathematics

For today, just a bit of humor lifted verbatim from Jim Henle's short, delightful, new volume, "The Proof and the Pudding," a fun book which, rather to my surprise, I'm loving (...a review sometime in the future --- now up HERE at MathTango):

"A mathematician was trying to decide: Should I get married? Or should I take a lover? The mathematician consulted a lawyer.
'By all means take a lover. The legal complications of marriage are immense. You're much better off with a simpler affair.'
The mathematician then consulted a doctor.
'By all means get married. Marriage is much healthier. Married people live longer. Don't distress yourself with the uncertainties of affairs.'
Finally, the mathematician consulted another mathematician.
'Do both. Your spouse will think you're with your lover, your lover will think you're with your spouse, and you can do mathematics.'"

Geometry and Physical Space

Today's 'Sunday reflection' from physicist Carl Rovelli (via John Brockman's "This Idea Must Die")... in which I get to learn the word, "pullulating"!:

"We will continue to use geometry as a useful branch of mathematics, but it's time to abandon the longstanding idea of geometry as the description of physical space. The idea that geometry is the description of physical space is ingrained in us and might seem hard to get rid of, but getting rid of it is unavoidable and just a matter of time. Might as well get rid of it soon....
"Einstein discovered that the Newtonian space described by geometry is in fact a field, like the electromagnetic field, and fields are nicely continuous and smooth only if measured at large scales. In reality, they're quantum entities that are discrete and fluctuating. Therefore, the physical space in which we're immersed is in reality a quantum-dynamical entity that has very little in common with what we call 'geometry.' It's a pullulating process of finite interacting quanta. We can still use expressions like 'quantum geometry' to describe it, but the reality is that a quantum geometry is not much of a geometry anymore."

[…If you have a favorite math-related passage that might make a nice Sunday morning reflection here let me know (SheckyR@gmail.com). If I use one submitted by a reader, I'll cite the contributor.]

Shuffling and Smooshing

There are so many fascinating individuals among mathematicians; sometimes fascinating in quirky, idiosyncratic, even neurotic ways; sometimes fascinating just as human individuals. Persi Diaconis falls into that latter category. A great teacher, great magician, great mathematician, with a somewhat storied background... a child-runaway who became a MacArthur Fellow. Erica Klarreich (who I just interviewed last Sunday) has posted a fine piece on him and his current work this week at Quanta:

https://www.quantamagazine.org/20150414-for-persi-diaconis-next-magic-trick/

Just Some Fun or Practical Tidbits Today

a)  Do you ever need some free mathematical clip-art? ...this blogger has a site-recommendation:
http://mathequalslove.blogspot.com/2015/04/stuff-worth-sharing-mathematical-clipart.html

b)  An interesting historical math tidbit from Open Culture (via Cliff Pickover):
http://www.openculture.com/2015/04/shortest-known-paper-in-a-serious-math-journal.html

c)  AMS blogs recommends a few mathematical 'online toolboxes':
http://blogs.ams.org/blogonmathblogs/2015/04/13/online-toolbox-for-number-theorists/#sthash.7ETz1Ulm.dpbs

d)  And lastly, Presh Talwalkar explains that brain-rattling Singapore logic problem that went viral:
http://mindyourdecisions.com/blog/2015/04/14/when-is-cheryls-birthday-answer-to-viral-math-puzzle/#.VSzrwWb5Vo4

Questions... Answers

For this morning's 'Sunday reflection,' an oldie-but-goodie:

"A joke is told that Epimenides got interested in eastern philosophy and made a pilgrimage to meet Buddha. He said to Buddha: 'I have come to ask you what is the best question that can be asked and what is the best answer that can be given.' Buddha replied: 'The best question that can be asked is the question you are asking and the best answer that can be given is the answer I am giving.'"

-- Raymond Smullyan (in "A Spiritual Journey")


p.s.:  be sure to also catch my interview with Erica Klarreich, one of the finest math journalists around, new at MathTango this morning.

Raymond Smullyan's Balls... again

Every couple of years I re-run a favorite old Raymond Smullyan puzzle (that actually goes back to "Annals of the New York Academy of Sciences," 1979, Vol. 321, although my version is an adaptation from Martin Gardner's presentation in his Colossal Book of Mathematics). Apologies to those of you who hate this puzzle (or just tired of me re-running it), but it's my blog and I get to indulge! ;-) -- actually, am re-playing it now in honor of the individual I'm interviewing this coming Sunday morning at MathTango, who is also a Raymond Smullyan fan. Here goes...:

Imagine you have access to an infinite supply of ping pong balls, each of which bears a positive integer label on it, which is its 'rank.' And for EVERY integer there are an INFINITE number of such balls available; i.e. an infinite no. of "#1" balls, an infinite no. of "#523" balls, an infinite no. of "#1,356,729" balls, etc. etc. etc. You also have a box that contains some FINITE number of these very same-type balls. You have as a goal to empty out that box, given the following procedure:

You get to remove one ball at a time from the finite box, but once you remove it, you must replace it with any finite no. of your choice of balls of 'lesser' rank (from the infinite supply box). Thus you can take out a ball labelled (or ranked) #768, and you could replace it with 27 million balls labelled, say #563 or #767 or #5 if you so desired, just as a few examples. The sole exceptions are the #1 balls, because obviously there are no 'ranks' below one, so there are NO replacements for a #1 ball.

Is it possible to empty out the box in a finite no. of steps??? OR, posing the question in reverse, as Martin Gardner does: "Can you not prolong the emptying of the box forever?" And then his answer: "Incredible as it seems at first, there is NO WAY to avoid completing the task." [bold added]
Although completion of the task is "unbounded" (there is no way to predict the number of steps needed to complete it, and indeed it could be a VERRRY large number), the box MUST empty out within a finite number of steps!
This amazing result only requires logical induction to see the general reasoning involved:

Once there are only #1 balls left in the box you simply discard them one by one (no replacement allowed) until the box is empty -- that's a given. In the simplest case we can start with only #2 and #1 balls in the box. Every time you remove a #2 ball, you can ONLY replace it with a #1, thus at some point (it could take a long time, but it must come) ONLY #1 balls will remain, and then essentially the task is over.
S'pose we start with just #1, #2, and #3 balls in the box... Every time a #3 ball is tossed, it can only be replaced with  #1 or #2 balls. Eventually, inevitably, we will be back to the #1 and #2 only scenario (all #3 balls having been removed), and we already know that situation must then terminate.
The same logic applies no matter how high up you go (you will always at some point run out of the very 'highest-ranked' balls and then be working on the next rank until they run out, and then the next, and then the next...); eventually you will of necessity work your way back to the state of just #1 and #2 balls, which then convert to just #1 balls and game over (even if you remove ALL the #1 and #2 balls first, you will eventually work back and be using them as replacements).

Of course no human being could live long enough to actually carry out such a procedure, but the process must nonetheless, amazingly, conclude after some mathematically finite no. of steps. Incredible! (a pity Cantor isn't around to appreciate this intuition-defying problem).

Mind… blown….

Die Statistics Die

There's been plenty of thrashing-about of late over the proper use of statistics in research and mathematical thinking, and interestingly toward the end of John Brockman's 2015 volume, "This Idea Must Die" (his yearly compendium of responses to an annual Edge question) several writers suggest statistical notions that are "ready for retirement":

Victoria Stodden on "Reproducibility":  https://edge.org/response-detail/25340

Emanuel Derman on "The Power of Statistics":  https://edge.org/response-detail/25349

Charles Seife on "Statistical Significance":  https://edge.org/response-detail/25414 

Gerd Gigerenzer on "Scientific Inference via Statistical Rituals":   
https://edge.org/response-detail/25462 

and perhaps my favorite:
Bart Kosko on "Statistical Independence":  https://edge.org/response-detail/25492

(For some reason Nassim Taleb's essay on "Standard Deviation" is missing from the online version of the volume, though included in the hard copy???)

The entire volume is an interesting read, with a wide range of opinions; most of which are included online below (including some other math-related essays as well):

https://edge.org/contributors/what-scientific-idea-is-ready-for-retirement

Recognizing Langlands

H/T to Colm Mulcahy for pointing out this longish and wonderful piece on Robert Langlands:

http://projects.thestar.com/math-the-canadian-who-reinvented-mathematics/

And one blogger's attempt to explain briefly/simply what the Langland's Program is all about:

http://thatsmaths.com/2014/03/13/the-langlands-program/

Chaotic Math…

"The mathematics of chaotic systems produces the same effect at every scale. Tell me how precise you want to be, and I can introduce my little germ of instability one decimal place further along; it may take a few more repetitions before the whole system's state becomes unpredictable, but the inevitability of chaos remains. The conventional image has the flap of a butterfly's wings in Brazil causing a storm in China, but even this is a needlessly gross impetus. The physicist David Ruelle, a major figure in chaos theory, gives a convincing demonstration that suspending the gravitational effect on our atmosphere of one electron at the limit of the observable universe would take no more than two weeks to make a difference in Earth's weather equivalent to having rain rather than sun during a romantic picnic."

-- From Michael and Ellen Kaplan's book "Chances Are…"


[…If you have a favorite math-related passage that might make a nice Sunday morning reflection here let me know (SheckyR@gmail.com). If I use one submitted by a reader, I'll cite the contributor.]

Incomparable Conway

Fa-a-a-antastic post yesterday from Colm Mulcahy on John Conway, about whom, it turns out, there is a biography, "Genius At Play," coming out in July!... Wooo-hooo, 400+ delicious pages on one of the most interesting maths guys around. Anyway, short videos, lotsa good links, and 'tales' to be had below:

http://tinyurl.com/lgtexw6

Also, a much older, longer (and very interesting) Charles Seife piece on Conway here:

http://www.users.cloud9.net/~cgseife/conway.html

Quoting Conway from the Seife piece:

"I'm not so much a mathematician as a teacher. In America, kids aren't supposed to like mathematics. It's so sad... Most people think that mathematics is cold. But it's not at all! For me, the whole damn thing is sensual and exciting. I like what it looks like, and I get a hell of a lot more pleasure out of math than most people do out of art!... I feel like an artist. I like beautiful things -- they're there already; man doesn't have to create it. I don't believe in God, but I believe that nature is unbelievably subtle and clever. In physics, for instance, the real answer to a problem is usually so subtle and surprising that it wasn't even considered in the first place. That the speed of light is a constant -- impossible! Nobody even thought about it. And quantum mechanics is even worse, but it's so beautiful, and it works!... I really do enjoy the beauty of nature -- and math is natural. Nobody could have invented the mathematical universe. It was there, waiting to be discovered, and it's crazy, it's bizarre."

"Everybody Loves Riemann" (new from BBC)

In conjunction with their long-running Horizon science broadcasts, the wonderful folks at the BBC begin a new 3-part series this evening, on the life and work of seminal German mathematician Bernhard Riemann, entitled, "Everybody Loves Riemann."  The first hour (Part One), narrated by Ray Romano, is a biographical profile of Riemann's brief 39-year lifetime, focusing on early mathematical work and achievements. Thursday evening's Part Two will be devoted entirely to the 150+ year-old, still-unproven Riemann Hypothesis for which he is most famous. And Part Three, we are told, will culminate on Friday with a special, and highly-anticipated, 1-hour live presentation of some sort from eminent British mathematician Sir Andrew Wiles.
Read more about it here:

http://www.bbc.uk.org/programming/Horizon/RiemannSpecial/af4115