Developing and Implementing Instruments for Measuring Scientific Reasoning Abilities using Online Intelligent Tutoring Systems

Lev Horodyskyj, Arizona State University, Center for Education Through eXploration, School of Earth and Space Exploration, Tempe, AZ, United States, Sanlyn Buxner, Planetary Science Institute Tucson, Tucson, AZ, United States, Steven Semken, Arizona State University, School of Earth and Space Exploration and EarthScope National Office, Tempe, AZ, United States and Ariel D Anbar, Arizona State University, Department of Chemistry, Tempe, AZ, United States

Contact First Author: Lev Horodyskyj; levh@sciencevoices.org

Abstract ID#: 35832

 

English Abstract:
Evaluation of educational activities tends to focus on content knowledge, which is often relatively easy to evaluate. However, this is only one component of scientific reasoning and literacy. A student's inability to reason scientifically can be a result of high-level failures (linking concepts and building mental models), but can also be a result of low-level failures (basic literacy and numeracy). Different student difficulties require different interventions to help students succeed and ultimately improve scientific literacy in general education science courses, whether they are online, in-person, or hybrid.

Habitable Worlds is an introductory level online-only astrobiology lab course that has been offered at Arizona State University since Fall 2011. The course is built and offered through an intelligent tutoring system, Smart Sparrow's Adaptive eLearning Platform, which provides in-depth analytics that allow the instructor to investigate detailed student behavior, from time spent on question to number of attempts to patterns of answers.

Analysis of multiple semesters of course data has revealed deficiencies in lesson design, which have informed subsequent redesign efforts. Yet despite significant gains in content knowledge, we have failed to make gains in students' high-level concept comprehension. There are many points of failure between basic information and true concept comprehension, which we are currently not tracking. We are aiming to develop and will discuss our formal definition of scientific reasoning and literacy, its components, and metrics we can use to measure component competencies. This will allow us to develop a suite of generalized tools and activities that can be deployed in any context that can score a student's abilities in various core competencies before, during, and after a course to determine course effectiveness as well as provide instructors with deep insight into problematic skills for individual students.

Back to: General Education Posters