A determination and classification of student errors in lower-level calculus through computer-aided assessment and analysis

Abstract

This thesis describes the determination of student errors through paper-based assessment through to computer-aided assessment. The focus on identification of errors is through low-level calculus questions; on polynomial differentiation through to product rule and chain rule usage in integration.

A major objective of this work is the design of suitable computer-aided assessment to fulfil learning objectives and usage as a diagnostic tool. The limitations of such diagnostic tools (whether paper-based or computer-aided) are explored in some depth, and the role of question design is brought to the fore through careful analysis of student errors on summative as well as formative assessments. It is from the data gained through these assessments that we can begin to classify mistakes into usable taxonomies. Firstly I have chosen to use the ResultsPlus data files from summative assessments via Edexcel, secondly the results from several years of paper-based formative assessment on a multiple-choice diagnostic test, and thirdly formative testing scores from Brunel University first year Economics students. The basis for forming an over-arching taxonomy for mistakes is built up using the SOLO model for classification and the discussion turns very much back towards question design as the nature of student errors changes as question structure changes. The data generated through computer-aided assessment is firstly unpacked to allow comparison between difficulty levels and cognitive levels. I go on to look at temporal comparisons between cohorts over time to discover weaker skill areas and question discrimination that will yield improved diagnostic assessments as well as selectively difficult assessments for the top-end of a cohort. The thesis looks carefully at the limitations of classification from these data sets and explores further distracter design in computer-aided assessment questions.