Comparison of the reliability of scoring methods of multiple-response items, matching items, and sequencing items

Titolo Rivista CADMO
Autori/Curatori Theo J.H.M. Eggen, Tecla T.M. Lampe
Anno di pubblicazione 2012 Fascicolo 2011/2
Lingua Inglese Numero pagine 20 P. 85-104 Dimensione file 257 KB
DOI 10.3280/CAD2011-002008
Il DOI è il codice a barre della proprietà intellettuale: per saperne di più clicca qui

Qui sotto puoi vedere in anteprima la prima pagina di questo articolo.

Se questo articolo ti interessa, lo puoi acquistare (e scaricare in formato pdf) seguendo le facili indicazioni per acquistare il download credit. Acquista Download Credits per scaricare questo Articolo in formato PDF

Anteprima articolo

FrancoAngeli è membro della Publishers International Linking Association, Inc (PILA)associazione indipendente e non profit per facilitare (attraverso i servizi tecnologici implementati da CrossRef.org) l’accesso degli studiosi ai contenuti digitali nelle pubblicazioni professionali e scientifiche

Multiple-response items, sequencing items, and matching items are three innovative item types often included in systems for computer-based assessment that offer the benefit of polytomous scoring and the possibility to measure partial knowledge. In the present study, different scoring methods of these three item types were compared. Based on the assumption that different response patterns to these item types represent different knowledge levels, these knowledge levels are described. Features of different scoring methods were studied to select the scoring methods included in this study. Subsequently, a probability distribution of scoring results for each knowledge level was derived and computed. Based on classical test theory, a measure for the reliability of the different scoring methods on the level of a single item was derived. To compare the results of the scoring methods selected, reliabilities were computed for several distributions of knowledge levels in a population. For a multiple-response item, when an examinee must select all the right options, the dichotomous scoring method resulted in higher reliabilities than scoring the response patterns polytomously. For matching items and for multiple-response items, when an examinee is asked to select fewer options than the total number of right options given, polytomous scoring methods gave higher reliabilities than the dichotomous scoring method. Simple polytomous scoring by counting the selected right options or relations is recommended instead of more complex polytomous scoring methods, for instance, using a correction for wrong answers or a so-called "floor". The results of scoring sequencing items were not as conclusive as for the other two item types explored.

Keywords:Innovating item types, reliability, multiple response items, matching items, sequencing items

  • Evaluating Different Scoring Methods for Multiple Response Items Providing Partial Credit Joe Betts, William Muntean, Doyoung Kim, Shu-chuan Kao, in Educational and Psychological Measurement /2022 pp.151
    DOI: 10.1177/0013164421994636
  • Relation between examinees’ true knowledge and examination scores: systematic review and exemplary calculations on Pick-N items Dennis Schmidt, Tobias Raupach, Annette Wiegand, Manfred Herrmann, Philipp Kanzow, in Educational Research Review 100483/2022 pp.100483
    DOI: 10.1016/j.edurev.2022.100483
  • Technology Enhanced Assessment Silvester Draaijer, Sally Jordan, Helen Ogden, pp.210 (ISBN:978-3-319-97806-2)

Theo J.H.M. Eggen, Tecla T.M. Lampe, Comparison of the reliability of scoring methods of multiple-response items, matching items, and sequencing items in "CADMO" 2/2011, pp 85-104, DOI: 10.3280/CAD2011-002008