Computerized Adaptive and Multistage Testing with R

1. Front Matter

   Pages i-xx

   PDF

2. Overview of Adaptive Testing

   • David Magis, Duanli Yan, Alina A. von Davier

   Pages 1-5

3. An Overview of Item Response Theory

   • David Magis, Duanli Yan, Alina A. von Davier

   Pages 7-31

4. Item-Level Computerized Adaptive Testing

   1. Front Matter

      Pages 33-33

      PDF

   2. An Overview of Computerized Adaptive Testing

      • David Magis, Duanli Yan, Alina A. von Davier

      Pages 35-51

   3. Simulations of Computerized Adaptive Tests

      • David Magis, Duanli Yan, Alina A. von Davier

      Pages 53-85

   4. Examples of Simulations Using catR

      • David Magis, Duanli Yan, Alina A. von Davier

      Pages 87-110

5. Computerized Multistage Testing

   1. Front Matter

      Pages 111-111

      PDF

   2. An Overview of Computerized Multistage Testing

      • David Magis, Duanli Yan, Alina A. von Davier

      Pages 113-122

   3. Simulations of Computerized Multistage Tests

      • David Magis, Duanli Yan, Alina A. von Davier

      Pages 123-140

   4. Examples of Simulations Using mstR

      • David Magis, Duanli Yan, Alina A. von Davier

      Pages 141-160

6. Back Matter

   Pages 161-171

   PDF

The goal of this guide and manual is to provide a practical and brief overview of the theory on computerized adaptive testing (CAT) and multistage testing (MST) and to illustrate the methodologies and applications using R open source language and several data examples.  Implementation relies on the R packages catR and mstR that have been already or are being developed by the first author (with the team) and that include some of the newest research algorithms on the topic.



The book covers many topics along with the R-code: the basics of R, theoretical overview of CAT and MST, CAT designs, CAT assembly methodologies, CAT simulations, catR package, CAT applications, MST designs, IRT-based MST methodologies, tree-based MST methodologies, mstR package, and MST applications.  CAT has been used in many large-scale assessments over recent decades, and MST has become very popular in recent years.  R open source language also has become one of the most useful tools for applications in almost all fields, including business and education. 


Though very useful and popular, R is a difficult language to learn, with a steep learning curve.  Given the obvious need for but with the complex implementation of CAT and MST, it is very difficult for users to simulate or implement CAT and MST.  Until this manual, there has been no book for users to design and use CAT and MST easily and without expense; i.e., by using the free R software.  All examples and illustrations are generated using predefined scripts in R language, available for free download from the book's website.

• pyschometrics
• educational measurement
• assessment
• teaching and evaluation
• quality control
• R package
• computerized adaptive testing
• simulation
• multistage testing
• model evaluation
• testing

• Department of Education, University of Liege, Liege, Belgium

  David Magis

• Educational Testing Service, Princeton, USA

  Duanli Yan

• ACTNext by ACT, Iowa City, USA

  Alina A. von Davier

David Magis, PhD, is Research Associate of the “Fonds de la Recherche Scientifique – FNRS” at the Department of Psychology, University of Liège, Belgium. His specialization is statistical methods in psychometrics, with special interest in item response theory, differential item functioning and computerized adaptive testing. His research interests include both theoretical and methodological development as well as open source implementation and dissemination in R. He is the main developer and maintainer of the packages catR and mstR, among others.

 

Duanli Yan, PhD, is Manager of Data Analysis and Computational Research for Automated Scoring group in the Research and Development division at the Educational Testing Service (ETS).  She is also an Adjunct Professor at Rutgers University.  At ETS, Dr. Yan’s responsibilities include the EXADEP™ test, the TOEIC® Institutional programs, and automated scoring engines upgrade and scoring. She has been a statistical coordinator and a Psychometrician for several operational programs and a Development Scientist for innovative research applications.  Dr. Yan received many awards including the 2011 ETS Presidential Award, the 2013 NCME Brenda Lyod award, the 2015 IACAT Early Career Award, and 2016 AERA Significant Contribution to Educational Measurement and Research Methodology Award.  She is a co-author for Bayesian Networks in Educational Assessment and a co-editor for Computerized Multistage Testing: Theory and Applications. 

 

Alina A. von Davier, PhD, is Vice-President at ACTNext and an Adjunct Professor at Fordham University. She was also Senior Research Director of the Computational Psychometrics Research Center at Educational Testing Service (ETS), where she was responsible for developing a team of experts and a psychometric research agenda in support of next generation assessments.  Computational psychometrics, which include machine learning and data mining techniques, Bayesian inference methods, stochastic processes and psychometric models are the main set of tools employed in her current work.  She also works with psychometric models applied to educational testing: test score equating methods, item response theory models, and adaptive testing.

Policies and ethics