Measuring Computational Thinking - Developing A Short Performance Test For Higher Education

Technological advancements, particularly in artificial intelligence, significantly transform our society and work practices. Computational thinking (CT) has emerged as a crucial 21st-century skill, enabling individuals to solve problems more effectively through an automation-oriented perspective and fundamental concepts of computer science. To ensure the effective integration of CT into educational curricula, it is crucial to develop efficient assessment frameworks that allow teachers to measure and promote student CT proficiency. Therefore, our aim is to develop a short test to measure CT among undergraduate students. To this end, we consider two performance tests: the Computational Thinking test (CTt) and the Algorithmic Thinking Test for Adults (ATTA). We use items from both instruments to compile a short test. Based on a sample of 290 second-year non-computer science undergraduate students, we provide evidence on the quality of our test. Besides classical test theory, we apply item response theory, namely Rasch modeling, and confirmatory factor analysis. Our test shows favorable properties, e.g., Cronbach's alpha > .75, and may be suitable for the efficient assessment of CT across higher education programs.