Data-Driven Design: Learning from Student Experiences and Behaviors

Abstract:

Good instructors know that lessons and courses change over time. Limitations in time and data often prevent instructors from making changes that will most benefit their students. For example, in traditional in-person classrooms an instructor may only have access to the final product of a student's thought processes (such as a term paper, homework assignment, or exam). The thought processes that lead to a given answer are opaque to the instructor, making future modifications to course content an exercise in trial-and-error and instinct. Modern online intelligent tutoring systems can provide insight into a student's behavior, providing transparency to a previously opaque process and providing the instructor with better information for course modification.

Habitable Worlds is an introductory level online-only astrobiology lab course that has been offered at Arizona State University since Fall 2011. The course is built and offered through an intelligent tutoring system, Smart Sparrow's Adaptive eLearning Platform, which provides in-depth analytics that allow the instructor to investigate detailed student behavior, from time spent on question to number of attempts to patterns of answers.

We will detail the process we employ of informed modification of course content, including time and trial comparisons between semesters, analysis of submitted answers, analysis of alternative learning pathways taken, and A/B testing.