Document Type

Conference Proceeding

Publication Date



Mathematics, Statistics, and Computer Science


educational tests and measurements, higher education, statistics, validity


Few tools exist to assess students’ conceptual understanding in post-secondary, introductory statistics courses. The CAOS test is widely considered to be the gold standard, but was first published in 2007 and does not necessarily reflect some of the changes in student learning at the secondary level. Furthermore, it may not be sensitive enough to measure student conceptual understanding in modern post-secondary statistics courses (e.g., simulation-based inference). In this paper we will describe the process of developing a new instrument which uses some CAOS items, as well as additional new items to improve validity and reliability. We will share the validity and reliability results across n=3,833 students at 49 institutions, as well as information about external factors associated with student performance (e.g., test setting, question order).


Proceedings of the Tenth International Conference on Teaching Statistics (ICOTS10, July, 2018), Kyoto, Japan.

Source Publication Title

Looking Back Looking Forward Proceedings


International Statistical Institute