Measuring situation awareness in complex systems: Comparison of measures study

Abstract

Situation Awareness (SA) is a distinct critical commodity for teams working in complex industrial systems and its measurement is a key provision in system, procedural and training design efforts. This article describes a study that was undertaken in order to compare three different SA measures (a freeze probe recall approach, a post trial subjective rating approach and a critical incident interview technique) when used to assess participant SA during a military planning task. The results indicate that only the freeze probe recall method produced a statistically significant correlation with performance on the planning task and also that there was no significant correlation between the three methods, which suggests that they were effectively measuring different things during the trials. In conclusion, the findings, whilst raising doubts over the validity of post trial subjective rating and interview-based approaches, offer validation evidence for the use of freeze probe recall approaches to measure SA. The findings are subsequently discussed with regard to their implications for the future measurement of SA in complex collaborative systems.