			Home About us Contact

Item Generation (item + generation)

Distribution by Scientific Domains

Medical Sciences	42%

Selected Abstracts

Manipulating Processing Difficulty of Reading Comprehension Questions: The Feasibility of Verbal Item Generation

JOURNAL OF EDUCATIONAL MEASUREMENT, Issue 4 2005
Joanna S. Gorin
Based on a previously validated cognitive processing model of reading comprehension, this study experimentally examines potential generative components of text-based multiple-choice reading comprehension test questions. Previous research (Embretson & Wetzel, 1987; Gorin & Embretson, 2005; Sheehan & Ginther, 2001) shows text encoding and decision processes account for significant proportions of variance in item difficulties. In the current study, Linear Logistic Latent Trait Model (LLTM; Fischer, 1973) parameter estimates of experimentally manipulated items are examined to further verify the impact of encoding and decision processes on item difficulty. Results show that manipulation of some passage features, such as increased use of negative wording, significantly increases item difficulty in some cases, whereas others, such as altering the order of information presentation in a passage, did not significantly affect item difficulty, but did affect reaction time. These results suggest that reliable changes in difficulty and response time through algorithmic manipulation of certain task features is feasible. However, non-significant results for several manipulations highlight potential challenges to item generation in establishing direct links between theoretically relevant item features and individual item processing. Further examination of these relationships will be informative to item writers as well as test developers interested in the feasibility of item generation as an assessment tool. [source]

Ensuring Quality Information for Patients: development and preliminary validation of a new instrument to improve the quality of written health care information

HEALTH EXPECTATIONS, Issue 2 2004
Beki Moult BA(Hons) MSc
Abstract Background, Despite the recent focus on improving the quality of patient information, there is no rigorous method of assessing quality of written patient information that is applicable to all information types and that prescribes the action that is required following evaluation. Objective, The aims of this project were to develop a practical measure of the presentation quality for all types of written health care information and to provide preliminary validity and reliability of the measure in a paediatric setting. Methods, The Ensuring Quality Information for Patients (EQIP) tool was developed through a process of item generation, testing for concurrent validity, inter-rater reliability and utility. Patient information managers and health care professionals tested EQIP in three annual audits of health care leaflets produced by a children's hospital. Results, The final tool comprised 20 items. Kendall's , B rank correlation between EQIP and DISCERN was 0.56 (P = 0.001). There was strong agreement between intuitive rating and the EQIP score (Kendall's , B = 0.78, P = 0.009). Internal consistency using Cronbach's , was 0.80. There was good agreement between pairs of raters (mean , = 0.60; SD = 0.18) with no differences based on types of leaflets. Audits showed significant improvement in the number of leaflets achieving a higher quality EQIP rating over a 3-year period. Conclusions, EQIP demonstrated good preliminary validity, reliability and utility when used by patient information managers and healthcare professionals for a wide variety of written health care information. EQIP uniquely identifies actions to be taken as a result of the quality assessment. Use of EQIP improved the quality of written health care information in a children's hospital. Wider evaluation of EQIP with written information for other populations and settings is recommended. [source]

Manipulating Processing Difficulty of Reading Comprehension Questions: The Feasibility of Verbal Item Generation

Improving Construct Validity With Cognitive Psychology Principles

JOURNAL OF EDUCATIONAL MEASUREMENT, Issue 4 2001
Susan Embretson
Cognitive psychology principles have been heralded as possibly central to construct validity. In this paper, testing practices are examined in three stages: (a) the past, in which the traditional testing research paradigm left little role for cognitive psychology principles, (b) the present, in which testing research is enhanced by cognitive psychology principles, and (c) the future, for which we predict that cognitive psychology's potential will be fully realized through item design. An extended example of item design by cognitive theory is given to illustrate the principles. A spatial ability test that consists of an object assembly task highlights how cognitive design principles can lead to item generation. [source]

Multi-item outcome measures for lateral ligament injury of the ankle: a structured review

JOURNAL OF EVALUATION IN CLINICAL PRACTICE, Issue 2 2004
K.L. Haywood BSc(Hons) DPhil MCSP
Abstract Objective, To identify and review evidence relating to the measurement properties of published multi-item outcome measures for the conservative management of lateral ligament injuries of the ankle. Methods, Systematic literature searches were used to identify measures, which were then assessed against pre-defined criteria relating to development, item content, reliability, validity and responsiveness. Results, Seven disease-specific measures of ankle status [Ankle Joint Functional Assessment Tool, Clinical Trauma Severity Score, Composite Inversion Injury Scale, Kaikkonen Functional Scale (KFS), Karlsson Ankle Function Score (KAFS), Olerud and Molander Ankle Score (OMAS), and the Point System] and two generic measures of health (McGill Pain Questionnaire, Sickness Impact Profile) met the review inclusion criteria. While all measures had been used in acute injuries, only two had also been applied during later stages of recovery (>6 months). The studies covered a comprehensive range of graded ligament injuries. Expert opinion dominated item generation for all measures. All measures lack evidence of test-retest or internal consistency reliability in patients with ankle sprain. Several measures were assessed for validity through comparison with other measures, but there was limited evidence of construct validity and no formal assessment of responsiveness for any measure. Conclusion, The disappointing lack of evidence for measurement properties suggests that any measure should be used with caution until appropriate evidence is provided. On the basis of limited evidence, the KFS offers the most promising approach to a combined clinician- and patient-assessment of ankle function, and the KAFS or OMAS if a patient-assessed evaluation of function is required. [source]

Evaluating Content Validity and Test,Retest Reliability of the Children's Health Risk Behavior Scale

PUBLIC HEALTH NURSING, Issue 4 2006
Susan K. Riesch
ABSTRACT Objectives: Describe the instrument development process and report the validity and reliability of the Children's Health Risk Behavior Scale (CHRBS), a scale designed to screen for health risk behaviors among youth aged 10,13 Years. Methods: Domain identification and item generation using the Youth Risk Behavior Surveillance Survey from the Centers for Disease Control and Prevention and testing relevance and test,retest reliability among a target audience sample of 77 fifth graders in their classrooms in two separate public school districts. Results: Youth performed their tasks as expert item reviewers effectively. Twenty-one items comprise the CHRBS with a reading level determined to be at the third grade. Conclusions: We have developed a reliable and valid measure to assess late elementary youth's participation in health risk behavior. [source]

Quality of life measurement in paediatric and adolescent populations with HIV: a review of the literature

CHILD: CARE, HEALTH AND DEVELOPMENT, Issue 4 2009
P. A. Garvie
Abstract Purpose To review the quality and utility of currently available self-report generic quality of life (QOL) and health-related quality of life (HRQOL) measures for use with children and adolescents with human immunodeficiency virus and/or acquired immunodeficiency syndrome (HIV/AIDS). Methods Literature searches were conducted to identify QOL and HRQOL measures developed for, adapted for, or otherwise used with paediatric and adolescent populations with HIV/AIDS. The quality of measures (i.e. item generation techniques, instrument properties including reliability, validity and responsiveness) were compared and critically evaluated. Results Nineteen QOL/HRQOL measures were identified. Item content was generated from the respondent (adult) population in only eight (42%) measures. Seventeen (90%) measures reported internal reliability in the accepted range between 0.70 and 0.90 and four (21%) reported reproducibility statistics in this range. Although validity was reported for 19 (100%) measures, only six (32%) showed evidence for three or more properties, with construct validity being the most commonly reported aspect. The authors of eight (42%) measures reported evidence for responsiveness. Conclusions While almost all measures reviewed demonstrated adequate psychometric properties, only one-third demonstrated all aspects of validity, and less than half demonstrated responsiveness. None included paediatric or adolescent populations with HIV/AIDS in their development, neglecting to obtain input from target respondents in item generation to determine what health-related and daily functioning factors are of importance to them. Despite noted limitations, the AUQUEI or the SWED-QUAL appear the best currently available generic measure, and the MQOL-HIV the preferred disease-specific measure, at least for use with older adolescents/young adults. [source]