THE ASSESSMENT OF HIGHER-ORDER THINKING SKILLS IN HIGH STAKES TESTING: AN UPDATED VISION ON HOW TO DEVELOP VALID AND RELIABLE ASSESSMENT INSTRUMENTS IN 21ST CENTURY CONTEXTS

Abstract

Valid and reliable assessment of high-order thinking skills (HOTS) in high-stakes tests demands for a careful process of item writing. It starts with a sensible and well-argumented choice of a taxonomy or framework, in combination with and based on a common understanding of the concept of higher-order thinking. Several taxonomies and a framework are presented, as being options to choose from. Each classification has its merits and should be seen as a guideline, not a law. The combination of the use of a taxonomy with the model of Cognitive Load Theory can be relevant for better understanding cognitive processes as operated by students when answering HOTS tasks in exams.

Assessing HOTS in 21^st century demands for the use of real-life contexts in exam tasks. Standards and criteria for the right choice and preparation of contexts are given and illustrated by some examples from high-stakes tests from the Netherlands and Kenya. Both open and closed item formats can be used for assessing HOTS in high-stakes tests. For all formats it is important to pay attention to and preparation time for the construction of valid keys and distractors (closed) or valid and operational marking schemes (open).