The Talented Dr. Fox and the Role of Charisma in Teaching, Part 1

Can a good con artist dazzle even a sophisticated audience into mistaking a load of bull for a substantive lesson? We all suspect that it's true, and that we've seen it happen. But what would we have to do to prove it? And if it is true, should we always meet charisma with skepticism?

I recently came across a 35-year-old paper that has become a minor classic in the education literature. It's called The Dr. Fox Lecture: A Paradigm of Educational Seduction. The authors, Donald Naftulin, John Ware, and Frank Donnelly, sought to investigate the hypothesis that

"Given a sufficiently impressive lecture paradigm, an experienced group of educators participating in a new learning situation can feel satisfied that they have learned despite irrelevant, conflicting, and meaningless content conveyed by the lecturer."

They tested this conjecture by hiring an accomplished character actor named Michael Fox.

(This was not Michael J. Fox. Michael Fox worked a generation before Michael J. Fox, and was the reason the younger actor had to use his middle initial in his professional name.)

Michael Fox had, so far as I know, no scientific or medical training. But—I'm sure you see it coming—he had played a doctor on TV. Several doctors, actually. If you're willing to sit through a couple of commercials, you can see Fox in action as Dr. Hoxie, deputy coroner, 33 minutes and 45 seconds into this episode of the courtroom drama Perry Mason. He can project the Voice of Authority as well as anyone. And that's relevant to our story.

Naftulin and colleagues prepped Fox to give a lecture on a topic the actor knew nothing about: how game theory can be applied to medical education. The lecture was loosely based on a real article from Scientific American, but was intentially larded with made-up terms, non sequiturs, contraditory statments, and other nonsense. And then garnished with jokes and meaningless asides.

The researchers introduced the actor as Dr. Myron L. Fox, expert on the mathematics of human behavior, and turned him loose on an unwitting audience of psychiatrists, psychologists, and social workers, all of whom were also educators. Fox blathered vacuosly, but charismatically, for an hour. He then took questions for another 30 minutes. At the end of it all, the researchers asked the members of the audience to report their subjective impressions on a fairly standard course evaluation form.

If you're anything like me, you'll take shameful delight in learning that a vast majority of the listeners gave Dr. Fox high marks. They said he seemed interested in his subject, that his presentation was well-organized, that he had stimulated their thinking, and on and on.

A number of conclusions spring to mind, many of which Naftulin and colleagues drew themselves. The results confirm the hypothesis. Students' subjective evaluations don't tell you much about what they actually learned. There's more to teaching than making the students happy. And, of course, the audience members were gullible—surprisingly so, given their level of academic achievement.

As Naftulin and colleagues put it, in what I'm sure is the least charitable assessment of their own research subjects I've ever seen authors offer in print, "...the learners in this study obviously failed as competent crap detectors... it is striking that none of them detected the lecture for what it was."

Ouch.

It's undoubtedly true that some of the (non-volunteer) subjects who provided the data deserved to be taken down a notch. Particularly the one who claimed to have read some of Dr. Fox's publications prior to attending his lecture. But the more I think about it, the less certain I am that any of these conclusions are justified. At least not on the evidence the authors present.

The authors' experiment was, in fact, poorly designed. For one thing, there was no control lecture. Perhaps the members of the audience were just supportive people, who would have given a positive evaluation to anyone. Furthermore, since the students weren't given a pre-test and post-test, we don't actually know that they didn't learn anything. This may seem like a ludicrous claim, given that Fox's lecture was bogus by design. But if part of teaching is encouraging students to think for themselves, and Fox's presentation did that, it might have inadvertantly taught the students something after all. Unless we collect better data, we simply cannot say.

In other words, The Dr. Fox Lecture might be an example of the very phenonmenon it purports to document: a charming and therefore convincing presentation with little actual stubstance.

But wait; there's more.

Less widely-known than the original Paradigm of Educational Seduction is the follow-up study that one of the authors, John Ware, ran with another colleague, Reed Williams. This time, the study was better designed. And the results are illuminating. I'll tell you about it in my next post, so please tune in again.

Sources:

The full citation for the paper by Naftulin and colleagues is: Naftulin, D. H., J. E. Ware, Jr., and F. A. Donnelly. 1973. The Doctor Fox Lecture: A Paradigm of Educational Seduction. Journal of Medical Education 48: 630–35.

Michael Fox's contribution to the research is noted and acknowledged in: Williams, R. G. and J. E. Ware, Jr. 1976. Validity of student ratings of instruction under different incentive conditions: A further study of the Dr. Fox effect. Journal of Educational Psychology 68: 48–56.

The follow-up study that I'll discuss in my next post is: Ware, J. E., Jr. and R. G. Williams. 1975. The Dr. Fox effect: A study of lecturer effectiveness and ratings of instruction. Journal of Medical Education 50: 149–156.