We all know that making someone a drawing is a great way to explain scientific ideas. Reviews of textbooks will talk about the quality of the diagrams, and our own virtual biology labs and interactive ecology chapters get high praise for how good the graphics are. Proselytizers of e-learning assume that animations and videos are even better, perhaps worth 1,000 pictures. But is all that true?
That’s not as trivial a question as it seems. As instructors, it turns out we often are not as good at judging what our students understand, and what we’re teaching them, as we think. In our own research at SimBiotic we’ve seen this several times when we’ve found misconceptions around evolutionary biology that very experienced, talented teachers have not been aware of. So although I would have agreed that pictures and animations are an unabashed good thing, I should have known better. A friend recently pointed me towards a couple studies from Davenport and colleagues at the Open Learning Initiative at Carnegie Mellon University which did controlled studies looking at exactly how much effect a good diagram can have.
The authors did studies on learning undergraduate chemistry, and found that adding diagrams did not necessarily aid student learning. In one study, they took a text-only tutorial on acid-base chemistry and added relevant diagrams that they thought would help the students. There was no difference in learning between the text only and the text-plus-diagram version. In a second study, the researchers built diagrams based on how expert chemists solved problems involving chemical equilibrium systems. They then filmed lectures that either showed only worked problems without diagrams, vs. lectures that included the diagrams. Again, they found very little difference in learning between those treatments. The one place they did find a difference was among students who had performed at a lower level on other topics in the class. Those students did better when shown the diagrams.
Interestingly, I heard a similar story at a poster session at NSF’s Vision and Change conference last summer. I unfortunately don’t remember who presented the poster, but she compared student learning with animations she designed of cell biology processes versus student learning without the animations. Her preliminary results were that there was no difference between the groups.
So are pictures worthless and animations even more so? That would be too broad a conclusion to make from these studies. The authors themselves point to research from other groups describing guidelines for how to make useful diagrams and other multimedia elements. In our own work, we have found diagrams to be very illuminating for assessing students understanding of topics in biology, including chemistry related topics like diffusion and osmosis (for instance, around our OsmoBeaker labs). I think there are two lessons from these studies. The first is that even teaching tools that seem self-evidently good need assessment rather than just assumptions about their benefit. The second is that learning will happen best when students engage with the material rather than simply viewing it. Pictures and animations which encourage students to think for themselves will be effective and those that don’t, won’t. That intuition is why at SimBiotic we continue to focus on simulations rather than animations in our learning materials, because with simulations we can force students to engage through conducting experiments. But of course, as with pictures and animations, I can only make that claim to the extent I have assessment data to back it up.
