"Benny's Rules"

image from Janzak via WikimediaCommons

No need to read this post if you already know all about "Benny's Rules"…. Keith Devlin, fount of wisdom in all things mathematical(!) :-),  recently directed me to this topic, which I was largely unfamiliar with.
Interestingly (to me), even though I was in college in 1973 and quite interested in "learning theory," I don't recall ever encountering the 1973 Stanley Erlwanger paper that introduced the notion of Benny's rules (though Devlin notes it "rapidly became one of the most famous and heavily studied papers in the mathematics education research literature")… perhaps it was more familiar to students specifically in the field of education or even social psychology, but not in the areas of cognitive psychology that I frequented.

It all has to do with the inherent weakness of 'mechanical' means of education, no matter how well-formulated or intentioned (in the instance of Benny, IPI or "individually prescribed instruction" is involved)… i.e. the inability to insure that a student is indeed acquiring a true understanding of the subject matter, even if they are scoring well on automated testing.
The specific focus of the paper is a young student, "Benny," who seems to perform well, and whose teacher believes he is progressing well in his math learning, only to discover upon closer, more personal examination, that he has merely created his own set of rules or patterns that seem to lead to many correct answers, despite a completely false understanding of the actual mathematical process involved. It harkens back to the bottom-line that mathematics is the study of patterns, and the brain naturally searches for patterns… BUT, if not adequately guided or without adequate feedback, a learner may recognize or internalize a very WRONG set of patterns within a given knowledge field.

Read this longish piece first if you're unfamiliar with the Benny case -- especially significant is the final paragraph warning of the potential analogy between Benny's "learning" and that accomplished by a digital consumer of, say, Khan Academy; the author notes "we need to be critical (but not necessarily dismissive) of Khan Academy":

http://blog.mathed.net/2011/07/rysk-erlwangers-bennys-conception-of.html

And you can follow that up with another longish post from Keith Devlin from earlier this year:

http://profkeithdevlin.org/tag/bennys-rules/

Excerpts:

"...In what rapidly became one of the most famous and heavily studied papers in the mathematics education research literature, Stanley Erlwanger exposed the crippling limitations of what at the time was thought to be a major step forward in mathematics education: Individually Prescribed Instruction (IPI)...

"The subject of Erlwanger’s study was a twelve-year-old boy called Benny, chosen because he was doing particularly well on the program, moving rapidly from level to level, scoring highly at each stage. As Erlwanger states in his paper, Benny’s teacher, who was administering the program for Benny, felt sure that his pupil could not have progressed so far without having a good understanding of previous work.

"Erlwanger’s research methodology was essentially the same as the approach Marilyn Burns used. He interviewed Benny to see what the boy understood. And when he did, a large can of worms spilled out. Though he got high scores on all the question sheets, Benny had almost no understanding of any mathematics, and a totally warped view of what mathematics is, to boot.
"Being bright, Benny had quickly worked out a strategy for tacking the IPI question sheets. His strategy was based in part on pattern recognition, and in part on developing a theory about how the game was constructed – yes, he viewed it as a game! And he did what any smart kid would do, he figured out how to game the game...

"Only when you understand the nature of mathematics does Benny’s strategy seem crazy. Without such understanding, his approach is perfectly sensible. He does not know about math, but he already knows a lot about people and about playing games of different kinds. And when this particular game keeps telling him he is doing well, and making progress, he has no reason to change his basic assumptions."