|
Foreword |
7 |
|
|
References |
9 |
|
|
Preface |
10 |
|
|
References |
12 |
|
|
Contents |
13 |
|
|
Acronyms |
18 |
|
|
List of Symbols |
20 |
|
|
1 General Equating Theory Background |
24 |
|
|
1.1 Introduction |
24 |
|
|
1.1.1 A Conceptual Description of Equating |
25 |
|
|
1.1.2 A Statistical Model View of Equating |
25 |
|
|
1.2 Statistical Models |
26 |
|
|
1.2.1 General Definition, Notation, and Examples |
26 |
|
|
1.2.2 Types of Statistical Models |
27 |
|
|
1.2.3 Mathematical Statistics Formulation of the Equating Problem |
29 |
|
|
1.2.4 Mathematical Form of the Equating Transformation |
30 |
|
|
1.2.5 Continuization |
31 |
|
|
1.2.6 Requirements for Comparability of Scores |
32 |
|
|
1.2.7 Assessing the Uncertainty of Equating Results |
32 |
|
|
1.3 Collecting Data in Equating |
33 |
|
|
1.3.1 Data Collection Designs in Equating |
34 |
|
|
1.3.1.1 Single Group Design |
34 |
|
|
1.3.1.2 Equivalent Groups Design |
34 |
|
|
1.3.1.3 Counterbalanced Design |
34 |
|
|
1.3.1.4 Non Equivalent Groups with Anchor Test Design |
35 |
|
|
1.3.1.5 Non Equivalent Groups with Covariates Design |
35 |
|
|
1.4 Some Examples of Equating Transformations |
36 |
|
|
1.4.1 The Equipercentile Equating Function |
36 |
|
|
1.4.2 The Linear Equating Function |
37 |
|
|
1.4.3 The Kernel Equating Function |
37 |
|
|
1.5 R Packages That Are Used in This Book |
38 |
|
|
1.6 Summary and Overview of the Book |
38 |
|
|
References |
39 |
|
|
2 Preparing Score Distributions |
42 |
|
|
2.1 Data |
42 |
|
|
2.1.1 Data from Ch2:kolenbrennan2014 |
42 |
|
|
2.1.2 Data from Ch2:vondavieretal2004 |
43 |
|
|
2.1.3 The ADM Admissions Test Data |
43 |
|
|
2.1.4 The SEPA Test Data |
44 |
|
|
2.2 Preparing the Score Data |
44 |
|
|
2.2.1 Functions to Create Score Frequency Distributions |
45 |
|
|
2.2.2 Score Data in the EG Design |
45 |
|
|
2.2.3 Score Data in the SG Design |
50 |
|
|
2.2.4 Score Data in the NEAT Design |
53 |
|
|
2.3 Presmoothing the Score Distributions |
56 |
|
|
2.3.1 Polynomial Log-Linear Models for Presmoothing |
56 |
|
|
2.3.2 Polynomial Log-Linear Smoothing in equate |
58 |
|
|
2.3.3 Examples |
59 |
|
|
2.3.3.1 Smoothing Univariate Distributions |
59 |
|
|
2.3.3.2 Smoothing a Bivariate Distribution |
60 |
|
|
2.3.4 Choosing the Best Log-Linear Model |
61 |
|
|
2.4 Using Other Arguments, Packages and Functions |
64 |
|
|
2.5 Summary |
65 |
|
|
References |
65 |
|
|
3 Traditional Equating Methods |
66 |
|
|
3.1 Equipercentile, Linear, and Mean Equating Transformations |
66 |
|
|
3.2 Assumptions in the Different Designs |
67 |
|
|
3.2.1 Assumptions in EG, SG, and CB Designs |
67 |
|
|
3.2.2 Assumptions in the NEAT Design |
68 |
|
|
3.3 Traditional Equating Methods for the EG, SG and CB Designs |
69 |
|
|
3.4 Traditional Equating Methods for the NEAT Design |
69 |
|
|
3.4.1 Linear Equating Methods for the NEAT Design |
70 |
|
|
3.4.1.1 Tucker Equating |
70 |
|
|
3.4.1.2 Nominal Weights Equating |
71 |
|
|
3.4.1.3 Levine Observed-Score Equating |
71 |
|
|
3.4.1.4 Levine True-Score Equating |
72 |
|
|
3.4.1.5 Chained Linear Equating |
73 |
|
|
3.4.2 Equipercentile Equating Methods for the NEAT Design |
73 |
|
|
3.4.2.1 Frequency Estimation |
73 |
|
|
3.4.2.2 Chained Equipercentile Equating |
74 |
|
|
3.4.2.3 Braun-Holland Equating |
74 |
|
|
3.5 Examples with the equate Function |
75 |
|
|
3.5.1 The equate Function |
75 |
|
|
3.5.2 Examples Under the EG and SG Designs |
76 |
|
|
3.5.3 Examples Under the NEAT Design |
83 |
|
|
3.5.3.1 Linear Methods |
83 |
|
|
3.5.3.2 Equipercentile Methods |
83 |
|
|
3.5.3.3 Comparison Between Linear and Equipercentile Methods |
84 |
|
|
3.5.4 Examples Using the ADM Data Under the NEAT Design |
86 |
|
|
3.6 Additional Features in equate |
86 |
|
|
3.7 Performing Traditional Equating Methods with SNSequate |
87 |
|
|
3.8 Comparing Traditional Test Equating Methods |
88 |
|
|
3.8.1 Bootstrap Standard Errors of Equating |
88 |
|
|
3.8.2 Bias and RMSE |
89 |
|
|
3.8.3 Examples Using equate |
90 |
|
|
3.8.4 Additional Example: A Comparison of Traditional Equating Methods |
91 |
|
|
3.9 Summary |
94 |
|
|
References |
94 |
|
|
4 Kernel Equating |
96 |
|
|
4.1 A Quick Overview of Kernel Equating |
96 |
|
|
4.2 Step 1: Presmoothing |
97 |
|
|
4.2.1 Presmoothing with SNSequate |
97 |
|
|
4.2.1.1 Presmoothing Under the EG Design |
98 |
|
|
4.2.1.2 Presmoothing Under the SG Design |
99 |
|
|
4.2.1.3 Presmoothing Under the CB Design |
101 |
|
|
4.2.1.4 Presmoothing Under the NEAT Design |
101 |
|
|
4.2.1.5 Modeling Complexities in the Data |
102 |
|
|
4.2.2 Presmoothing with kequate |
104 |
|
|
4.2.2.1 Presmoothing Under the EG Design |
104 |
|
|
4.2.2.2 Presmoothing Under the SG Design |
105 |
|
|
4.2.2.3 Presmoothing Under the CB Design |
106 |
|
|
4.2.2.4 Presmoothing Under the NEAT Design |
106 |
|
|
4.2.2.5 Modeling Complexities in the Data |
107 |
|
|
4.2.2.6 Presmoothing Under the NEC Design |
108 |
|
|
4.2.3 Assessing Log-Linear Model Fit |
109 |
|
|
4.2.3.1 Assessing Log-Linear Model Fit in SNSequate |
110 |
|
|
4.2.3.2 Assessing Log-Linear Model Fit in kequate |
111 |
|
|
4.3 Step 2: Estimation of Score Probabilities |
113 |
|
|
4.3.1 Estimation of Score Probabilities with SNSequate |
113 |
|
|
4.3.2 Estimation of Score Probabilities with kequate |
114 |
|
|
4.4 Step 3: Continuization |
115 |
|
|
4.4.1 Bandwidth Selection |
116 |
|
|
4.4.2 Choosing the Kernel |
116 |
|
|
4.4.3 Continuization Choices in SNSequate |
117 |
|
|
4.4.4 Continuization Choices in kequate |
117 |
|
|
4.5 Step 4: Equating |
118 |
|
|
4.5.1 Equating in SNSequate |
118 |
|
|
4.5.2 Equating in kequate |
122 |
|
|
4.6 Step 5: Computation of Accuracy Measures |
125 |
|
|
4.6.1 Calculating the Standard Error of Equating |
126 |
|
|
4.6.2 Standard Error of Equating Difference |
126 |
|
|
4.6.3 Percent Relative Error |
126 |
|
|
4.6.4 Obtaining SEE, SEED, and PRE in SNSequate |
127 |
|
|
4.6.5 Obtaining SEE, SEED, and PRE in kequate |
129 |
|
|
4.7 Different Features in kequate and SNSequate |
132 |
|
|
4.8 Summary |
132 |
|
|
References |
132 |
|
|
5 Item Response Theory Equating |
134 |
|
|
5.1 IRT Models |
134 |
|
|
5.1.1 Scoring Using IRT Models |
135 |
|
|
5.2 Equating IRT Scores |
136 |
|
|
5.2.1 Parameter Linking |
136 |
|
|
5.2.1.1 Moments Methods to Estimate Equating Coefficients |
137 |
|
|
5.2.1.2 Characteristic Curves Methods to Estimate Equating Coefficients |
138 |
|
|
5.2.1.3 IRT Parameter Linking Using SNSequate |
138 |
|
|
5.2.1.4 IRT Parameter Linking Using equateIRT |
139 |
|
|
5.3 Equating Observed Scores Under the IRT Framework |
142 |
|
|
5.3.1 IRT True-Score Equating |
143 |
|
|
5.3.2 IRT Observed-Score Equating |
143 |
|
|
5.3.3 IRT True-Score and Observed-Score Equating Using SNSequate |
144 |
|
|
5.3.4 IRT True-Score and Observed-Score Equating Using equateIRT |
149 |
|
|
5.4 Other Equating Methods for IRT Scores |
151 |
|
|
5.4.1 Concurrent Calibration |
151 |
|
|
5.4.1.1 Concurrent Calibration Using ltm |
152 |
|
|
5.4.2 Fixed Item Parameter Calibration |
154 |
|
|
5.4.2.1 Fixed Item Parameter Calibration Using mirt |
154 |
|
|
5.5 Other R Packages for IRT Analysis |
156 |
|
|
5.6 Summary |
157 |
|
|
References |
157 |
|
|
6 Local Equating |
160 |
|
|
6.1 The Concept of Local Equating |
160 |
|
|
6.1.1 True Equating Transformation |
161 |
|
|
6.2 Performing Local Equating |
162 |
|
|
6.3 Local Linear Equating Transformations |
162 |
|
|
6.3.1 Local Linear Equating Conditioning on Anchor Test Scores: NEAT Design |
163 |
|
|
6.3.2 Local Linear Equating Method of Conditional Means: SG Design |
163 |
|
|
6.3.3 Local Linear Equating Examples in R |
163 |
|
|
6.3.3.1 Implementing Local Linear Equating Conditioning on Anchor Test Scores |
164 |
|
|
6.3.3.2 Implementing the Local Linear Equating Method of Conditional Means |
167 |
|
|
6.4 Local Equipercentile Equating Transformations |
167 |
|
|
6.4.1 Local IRT Observed-Score Equating |
168 |
|
|
6.4.2 Local Observed-Score Kernel Equating Conditioning on Anchor Test Scores |
169 |
|
|
6.4.3 Local IRT Observed-Score Kernel Equating |
169 |
|
|
6.4.4 Local Equipercentile Equating Examples in R |
170 |
|
|
6.4.4.1 Local IRT Observed-Score Equating Using SNSequate |
170 |
|
|
6.4.4.2 Local Observed-Score Kernel Equating Using kequate |
172 |
|
|
6.4.4.3 Local IRT Observed-Score Kernel Equating Using kequate |
174 |
|
|
6.5 Other Local Equating Methods |
177 |
|
|
6.6 Summary |
177 |
|
|
References |
177 |
|
|
7 Recent Developments in Equating |
179 |
|
|
7.1 Alternative Kernel Equating Transformations |
179 |
|
|
7.1.1 Epanechnikov Kernel |
179 |
|
|
7.1.2 Adaptive Kernels |
180 |
|
|
7.1.3 Examples of Epanechnikov and Adaptive Kernel Equating in SNSequate |
181 |
|
|
7.2 Bandwidth Selection in Kernel Equating |
183 |
|
|
7.2.1 Rule-Based Bandwidth Selection |
183 |
|
|
7.2.2 Bandwidth Selection with Double Smoothing |
184 |
|
|
7.2.3 Examples of the Rule-Based and Double Smoothing Bandwidth Selection Methods Using kequate |
184 |
|
|
7.3 Item Response Theory Kernel Equating |
185 |
|
|
7.3.1 Two Polytomous IRT Models |
185 |
|
|
7.3.2 Performing IRT Kernel Equating with kequate |
186 |
|
|
7.3.3 Examples of IRT Kernel Equating for Binary Scored Items Using kequate |
187 |
|
|
7.3.4 Examples of IRT Kernel Equating for Polytomous Scored Items Using kequate |
189 |
|
|
7.4 Bayesian Nonparametric Approach to Equating |
190 |
|
|
7.4.1 Bayesian Nonparametric Modeling |
190 |
|
|
7.4.2 BNP Model for Equating |
191 |
|
|
7.4.3 An Illustration of the BNP Model for Equating in SNSequate |
192 |
|
|
7.5 Assessing the Equating Transformation |
194 |
|
|
7.5.1 An Illustration of Assessing (x) in Kernel Equating Using SNSequate |
196 |
|
|
7.6 Summary |
199 |
|
|
References |
199 |
|
|
Appendix A Installing and Reading Data in R |
201 |
|
|
A.1 Installing R |
201 |
|
|
A.1.1 R Studio |
201 |
|
|
A.2 Installing and Loading R Packages |
202 |
|
|
A.3 Working Directory and Accessing Data |
202 |
|
|
A.4 Loading Data of Different File Formats |
203 |
|
|
Reference |
204 |
|
|
Appendix B Additional Material |
205 |
|
|
B.1 Design Functions |
205 |
|
|
B.2 C C C C Matrices |
207 |
|
|
B.3 Calculation of the SEE |
207 |
|
|
B.4 Score Distributions Under the NEAT Design |
208 |
|
|
B.5 The Lord-Wingersky Algorithm |
209 |
|
|
B.6 Other Justifications for Local Equating |
210 |
|
|
B.7 Epanechnikov Kernel Density Estimate and Derivatives |
211 |
|
|
B.8 The Double Smoothing Bandwidth Selection Method in Kernel Equating |
212 |
|
|
B.9 The DBPP Model |
213 |
|
|
B.10 Measures of Statistical Assessment When Equating Test Scores |
213 |
|
|
References |
214 |
|
|
Index |
216 |
|