Home About us Contact
|
Home About us Contact | ||
|
|
||
Multiple-choice Items (multiple-choice + item)
Selected AbstractsThree Options Are Optimal for Multiple-Choice Items: A Meta-Analysis of 80 Years of ResearchEDUCATIONAL MEASUREMENT: ISSUES AND PRACTICE, Issue 2 2005Michael C. Rodriguez Multiple-choice items are a mainstay of achievement testing. The need to adequately cover the content domain to certify achievement proficiency by producing meaningful precise scores requires many high-quality items. More 3-option items can be administered than 4- or 5-option items per testing time while improving content coverage, without detrimental effects on psychometric quality of test scores. Researchers have endorsed 3-option items for over 80 years with empirical evidence,the results of which have been synthesized in an effort to unify this endorsement and encourage its adoption. [source] Using the Distractor Categories of Multiple-Choice Items to Improve IRT LinkingJOURNAL OF EDUCATIONAL MEASUREMENT, Issue 3 2006Jee-Seon Kim Simulation and real data studies are used to investigate the value of modeling multiple-choice distractors on item response theory linking. Using the characteristic curve linking procedure for Bock's (1972) nominal response model presented by Kim and Hanson (2002), all-category linking (i.e., a linking based on all category characteristic curves of the linking items) is compared against correct-only (CO) linking (i.e., linking based on the correct category characteristic curves only) using a common-item nonequivalent groups design. The CO linking is shown to represent an approximation to what occurs when using a traditional correct/incorrect item response model for linking. Results suggest that the number of linking items needed to achieve an equivalent level of linking precision declines substantially when incorporating the distractor categories. [source] Three Options Are Optimal for Multiple-Choice Items: A Meta-Analysis of 80 Years of ResearchEDUCATIONAL MEASUREMENT: ISSUES AND PRACTICE, Issue 2 2005Michael C. Rodriguez Multiple-choice items are a mainstay of achievement testing. The need to adequately cover the content domain to certify achievement proficiency by producing meaningful precise scores requires many high-quality items. More 3-option items can be administered than 4- or 5-option items per testing time while improving content coverage, without detrimental effects on psychometric quality of test scores. Researchers have endorsed 3-option items for over 80 years with empirical evidence,the results of which have been synthesized in an effort to unify this endorsement and encourage its adoption. [source] An Odds Ratio Approach for Assessing Differential Distractor Functioning Effects under the Nominal Response ModelJOURNAL OF EDUCATIONAL MEASUREMENT, Issue 3 2008Randall D. Penfield Investigations of differential distractor functioning (DDF) can provide valuable information concerning the location and possible causes of measurement invariance within a multiple-choice item. In this article, I propose an odds ratio estimator of the DDF effect as modeled under the nominal response model. In addition, I propose a simultaneous distractor-level (SDL) test of invariance based on the results of the distractor-level tests of DDF. The results of a simulation study indicated that the DDF effect estimator maintained good statistical properties under a variety of conditions, and the SDL test displayed substantially higher power than the traditional Mantel-Haenszel test of no DIF when the DDF effect varied in magnitude and/or size across the distractors. [source] Judges' Use of Examinee Performance Data in an Angoff Standard-Setting Exercise for a Medical Licensing Examination: An Experimental StudyJOURNAL OF EDUCATIONAL MEASUREMENT, Issue 4 2009Brian E. Clauser Although the Angoff procedure is among the most widely used standard setting procedures for tests comprising multiple-choice items, research has shown that subject matter experts have considerable difficulty accurately making the required judgments in the absence of examinee performance data. Some authors have viewed the need to provide performance data as a fatal flaw for the procedure; others have considered it appropriate for experts to integrate performance data into their judgments but have been concerned that experts may rely too heavily on the data. There have, however, been relatively few studies examining how experts use the data. This article reports on two studies that examine how experts modify their judgments after reviewing data. In both studies, data for some items were accurate and data for other items had been manipulated. Judges in both studies substantially modified their judgments whether the data were accurate or not. [source] A SIBTEST Approach to Testing DIF Hypotheses Using Experimentally Designed Test ItemsJOURNAL OF EDUCATIONAL MEASUREMENT, Issue 4 2000Daniel M. Bolt This paper considers a modification of the DIF procedure SIBTEST for investigating the causes of differential item functioning (DIF). One way in which factors believed to be responsible for DIF can be investigated is by systematically manipulating them across multiple versions of an item using a randomized DIF study (Schmitt, Holland, & Dorans, 1993). In this paper: it is shown that the additivity of the index used for testing DIF in SIBTEST motivates a new extension of the method for statistically testing the effects of DIF factors. Because an important consideration is whether or not a studied DIF factor is consistent in its effects across items, a methodology for testing item x factor interactions is also presented. Using data from the mathematical sections of the Scholastic Assessment Test (SAT), the effects of two potential DIF factors,item format (multiple-choice versus open-ended) and problem type (abstract versus concrete),are investigated for gender Results suggest a small but statistically significant and consistent effect of item format (favoring males for multiple-choice items) across items, and a larger but less consistent effect due to problem type. [source] Effects of Response Format on Difficulty of SAT-Mathematics Items: It's Not the StrategyJOURNAL OF EDUCATIONAL MEASUREMENT, Issue 1 2000Irvin R. Katz Problem-solving strategy is frequently cited as mediating the effects of response format (multiple-choice, constructed response) on item difficulty, yet there are few direct investigations of examinee solution procedures. Fifty-five high school students solved parallel constructed response and multiple-choice items that differed only in the presence of response options. Student performance was videotaped to assess solution strategies. Strategies were categorized as "traditional",those associated with constructed response problem solving (e.g., writing and solving algebraic equations),or "nontraditional",those associated with multiple-choice problem solving (e.g., estimating a potential solution). Surprisingly, participants sometimes adopted nontraditional strategies to solve constructed response items. Furthermore, differences in difficulty between response formats did not correspond to differences in strategy choice: some items showed a format effect on strategy but no effect on difficulty; other items showed the reverse. We interpret these results in light of the relative comprehension challenges posed by the two groups of items. [source] | ||||||||||