• HOME
  • GETTING STARTED
    • Quick Start Guide
    • Evaluation Software
    • Webinars 2023
    • Demo Videos
  • OUR PRODUCTS
    • Browse All Content
    • Molecular Biology Series
    • Introductory Biology
    • SimUText Ecology
    • Get Evaluation Software
  • DELIVERY SYSTEM
    • What is SimUText?
    • Accessibility
  • PRICING & ORDERING
  • ABOUT
    • Overview
    • Research & Publications
    • Testimonials
    • Jobs
    • Accessibility
  • TECH SUPPORT
    • Knowledge Base
    • Help for Students
    • FAQ – Instructors
  • CONTACT
  • BLOG

Mobile Menu

  • Menu
  • Skip to right header navigation
  • Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer

SimBio logo

Simulated Biology. Real Learning.

  • Evaluation Software
  • Contact
  • Blog
  • HOME
  • GETTING STARTED
    • Quick Start Guide
    • Evaluation Software
    • Webinars 2023
    • Demo Videos
  • OUR PRODUCTS
    • Browse All Content
    • Molecular Biology Series
    • Introductory Biology
    • SimUText Ecology
    • Get Evaluation Software
  • DELIVERY SYSTEM
    • What is SimUText?
    • Accessibility
  • PRICING & ORDERING
  • ABOUT
    • Overview
    • Employee Ownership
    • Research & Publications
    • Blog
    • News & Events
    • Testimonials
    • Jobs
    • Accessibility
  • TECH SUPPORT
    • Knowledge Base
    • Help for Students
    • FAQ – Instructors
SimBio Blog

Assessing Misconceptions

Home » All Things SimBio » Assessing Misconceptions

August 22, 2008 //  by Eli Meir

Last week I was at a meeting sponsored by NSF that brought together people who have worked on project funded by their CCLI program (Course, Curriculum, and Laboratory Improvement). They fund an impressively large number of projects through that program, there must have been 300 people at the meeting, and it turned out to be a really fun meeting as everyone there was doing something cool involving science education at the college level. I’ll make a separate post about some of the interesting projects I saw while there, but the session that applied most directly to our current work at SimBiotic was on concept inventories and assessing misconceptions.

We want to test the labs we make to see if they are educationally effective – in other words, do students learn from doing our labs. It turns out, we discovered (and we’re not alone), this is a deceptively hard question to answer. All of us involved in teaching make test questions all the time, and we think we do an OK job. So to test a lab on, say, the evidence for evolution (a new lab we’re working on now), one approach would be to simply make some standard test questions on evolution and fossils and so on, and see if students answer those questions better after doing our lab than before. The problem is that standard questions that you just write down off the top of your head tend to test content knowledge – can students remember what they read and repeat it back. They don’t tend to test conceptual knowledge – do students have a working understanding of the concepts that they can apply in a novel situation. Students tend to have “misconceptions”, conceptual ideas that are wrong in general but nevertheless explain parts of their world, so they are hard for the student to change to a correct conception of how the world works.

So we think, along with many others, that one of the things a good teaching tool should do is overcome students misconceptions on a topic and move them towards having more scientifically accurate concepts that they use. The problem is that its not easy to write test questions that really get at conceptual knowledge. And often, even very experienced teachers don’t know where their students misconceptions are (at least, among our group, our initial predictions of where students will have trouble seem to always be very incomplete). As we’ve worked on our labs in OsmoBeaker and EvoBeaker, we developed something of a protocol for finding student misconceptions and developing test questions on those. At the CCLI session, it was nice to hear that what we’ve developed seems in line with what people with a more formal training in educational research recommend.

As part of a session on concept inventories (listing those concepts that are important to understanding a topic, and the misconceptions that students have around each), Julie Libarkin from Michigan State gave a nice concrete presentation on how to make a good test. She talked about two basic principles behind test design – validity and reliability. Validity is whether the test is really measuring what you think it is. Reliability is whether it measures that consistently, both on different questions in the test addressing the same topic, and when you give the test to different groups of students. To get validity, she talked about using think-aloud interviews with students to find out why they answer questions in a certain way, and giving essay questions to students and using those essay answers to tease out the misconceptions and develop the test questions. She also talked about having experts critique the test questions (both scientists and education experts). And she mentioned using focus groups to probe for misconceptions. We’ve been doing most of those other than the focus groups with our tests. We come up with some essay or drawing questions that we think will hit on concepts students have trouble with, and then ask students to explain their answers and we probe their explanations in an interview. We then redesign the questions and add questions where we think we missed something with the first set. We continue doing that iteratively until our questions seem to be capturing most of the common thoughts we hear from the students, and the written answers seem to match the explanations. Then we select student-written answers to each question that hit on each misconception we’ve seen, and use those in making the question multiple choice. Finally, we give these to some outside experts to critique.

We haven’t thought as much about reliability, so that’s something we should do. And there are some statistical analyses we should be using as well, along with accumulated wisdom on how to write a good test question. So lots to learn. But still, it was interesting to hear that the process we came up with from a little reading in the literature and just trying things out is not that far from accepted practice. It’s encouraging to know that somewhat independent groups coming from different disciplines converge on a similar process. This whole process takes a seriously large effort, though. We’ve spent months writing a test, almost as much time as we spend writing the lab we’re trying to test. It would be great if there was some organized repository of misconceptions in various areas of biology, and tests that have been validated and reliability verified that we could just grab and run. Of course then our labs (and others) would be teaching to those tests, but if the tests are good, I don’t think that’s a bad thing. At the meeting, I heard snippets about various groups that are trying to make concept inventories and conceptual tests for different areas in biology and have these available, I’m hoping those efforts really take off as it would be such a time-saver for anyone trying to make a really good teaching tool.

Author

  • Eli Meir
    Eli Meir

    SimBio founder Eli Meir brings together a PhD in biology, a long history of developing sophisticated software for both biology research and education markets, and over 20 years of experience running software companies. He has a passion for using these skills to improve education in both college and secondary schools through innovative products that let students learn by doing and playing.

    View all posts

Category: SimBio

Related Posts

An Undergraduate Intern’s Summer at SimBio

Mitosis Explored – SimBio’s new tutorial

Male Antelope Lie To Get Sex

Octopus Aces Physics Homework

Survival of the Fittest? Not Always.

E-Textbooks and the Blind

Language, Lying, and Evolution

Is a picture worth 1,000 words?

Genomic Analysis of Identical Twins Finds Few Differences

Better Gene Discovery Through Evolutionary Reasoning

SimUText Ecology

The Talented Dr. Fox, Part 2

Next Post: Cool Projects at NSF’s CCLI Meeting »

Footer

Testimonials

“[SimBio’s modules] provide a powerful environment for hands-on, active learning in Ecology and Evolution. The depth, design, and pedagogical effectiveness of these packages is unequaled. They have greatly enriched student interest and understanding in all of my courses, from Introductory Biology to Advanced Population Biology”

— Dr. Brad Lister
Rensselaer Polytechnic Institute

“The lab simulations have been very helpful to reinforce the lecture content and encourage students to use critical thinking skills to solve problems.”

— Patti Fallest-Strobl
Neumann University

“I was amazed how quickly and effortlessly the simulation taught them a dynamic system.”

— Paula Philbrick
University of Connecticut

“The students like [it] so much that they are working ahead of what I have assigned. They just think it’s fun. And from the quiz results I have so far, they’re really getting it. You guys are geniuses.”

— Paulette Bierzychudek
Lewis & Clark College

“Since we began using Simbio’s Ecology chapters our students consistently come to class better prepared. This is a key to the success of our hybrid flipped classroom approach.”

— Dr. Peter Curtis
Ohio State University

“That was the most fun I’ve had teaching ecology and, I would venture to guess, the most fun my students have had taking ecology from me.”

— Matt Orr
Oregon State University, Cascades

“Our students are more prepared and our class discussions are more sophisticated because SimUText is such a great out of class “instructor”. Since our students have SimUText preparation we can spend class time applying concepts and evaluating new information.”

— Emily Bernhardt
Duke University

“I LOVE that every lab is based on a real study. I LOVE that all of the labs offer an open-ended inquiry. I LOVE that the labs gradually teach the concepts and build up a repertoire of data collection techniques. Thank you, for creating them.”

— Jeanette Williams
Community College of Vermont

“I just wanted to say how great simutext has been. I could have memorized facts about the electron transport chain and passed the test, but would not have had any solid concept of what was happening inside. I hope that we continue to use it often even when things return to normal.”

— Student
Rochester Community and Technical College

“Congratulations for developing such a high quality chapter. I was very impressed by its comprehensiveness, accuracy, and thoughtful design. It really is superb.”

— Richard Boone
Humboldt State University

“I recommend your modules to anyone I can, because of all the online materials I’ve found, SimBio is really the best in content and best managed. I am definitely a SimBio fan!”

— Valerie Anderson
Marymount California University

All Things SimBio

  • John RoachMaking Climate Change Real While Giving Students Hope
    By John Roach
    January 19, 2023
  • CC CarsonA Systems Thinking Framework for Biology Classes
    By CC Carson
    November 29, 2022
  • John RoachSix Degrees of Biology
    By John Roach
    October 27, 2022
More Blog Posts →

Contact

Shipping: 1280 S. Third St W., Missoula, MT 59801
Billing: P.O. Box 7158, Missoula, MT 59807
Phone: (833) 314-7701
Fax: (617) 279-0055
Questions?

SimBio Order Form
VPAT / Accessibility Statement

Site Footer

  • Facebook

Copyright © 2023 SimBio. All Rights Reserved. Trademarks.