Effective computer-aided assessment of mathematics; principles, practice and results

Abstract

This article outlines some key issues for writing effective computer-aided assessment (CAA) questions in subjects with substantial mathematical or statistical content, especially the importance of control of random parameters and the encoding of wrong methods of solution (mal-rules) commonly used by students. The pros and cons of using CAA and different question types are discussed. Issues surrounding the selection and encoding of mal-rules are highlighted, especially for multi-choice and responsive numerical input questions. These generate mal-rule-specific feedback, the mal-rule used being deduced 15 from the student’s selection or input. Student answer file data from the use of over 800 questions and their embedding within an overall assessment regime is analysed and presented to show that this has had a very beneficial effect on the examination performance of a large cohort of first-year economics students in their mathematics module over the last 6 years. Question analysis of over 270,000 question attempts, identifying the most 20 difficult/discriminating questions, shows that the questions are robust, valid and span an appropriate range of difficulties. The idea of underlying mal-rules is examined to see how far this explains this range.