			Home About us Contact

Interrater Agreement (interrater + agreement)

Distribution by Scientific Domains

Medical Sciences	73%

Selected Abstracts

Interrater Agreement between Nurses for the Pediatric Canadian Triage and Acuity Scale in a Tertiary Care Center

ACADEMIC EMERGENCY MEDICINE, Issue 12 2008
FRCPC, Jocelyn Gravel MD
Abstract Objectives:, The objective was to measure the interrater agreement between nurses assigning triage levels to children visiting a pediatric emergency departments (EDs) assisted by a computerized version of the Pediatric Canadian Triage and Acuity Scale (PedCTAS). Methods:, This was a prospective cohort study evaluating children triaged from Level 2 (emergent) to Level 5 (nonurgent). A convenience sample of patients triaged during 38 shifts from April to September 2007 in a tertiary care pediatric ED was evaluated. All patients were initially triaged by regular triage nurses using a computerized version of the PedCTAS. Research nurses performed a second evaluation blinded to the first evaluation using the same triage tool. These research nurses were regular ED nurses performing extra hours for research purposes exclusively. The primary outcome measure was the interrater agreement between the two nurses as measured by the linear weighted kappa score. Secondary outcomes included the proportion of patient for which nurses did not apply the triage level suggested by Staturg (override) and agreement for these overrides. Results:, A total of 499 patients were recruited. The overall interrater agreement was moderate (linear weighted kappa score of 0.55 [95% confidence interval {CI} = 0.48 to 0.61] and quadratic weighted kappa score of 0.61 [95% CI = 0.42 to 0.80]). There was a discrepancy of more than one level in only 10 patients (2% of the study population). Overrides occurred in 23.2 and 21.8% for regular and research triage nurses, respectively. These overrides were equally distributed between increase and decrease in triage level. Conclusions:, Nurses using Staturg, which is a computerized version of the PedCTAS, demonstrated moderate interrater agreement for assignment of triage level to children presenting to a pediatric ED. [source]

Quantification of upper extremity function and range of motion in children with cerebral palsy

DEVELOPMENTAL MEDICINE & CHILD NEUROLOGY, Issue 12 2008
L Andrew Koman MD
This study evaluated the hypothesis that upper extremity function and range of motion can be quantified reliably in children with cerebral palsy (CP) in a busy clinical setting. The specific aim was to determine the inter- and intrarater reliability of a modified House Functional Classification (MHC) system to evaluate upper extremity function and a standardized instrument to document upper extremity range of motion (Upper Extremity Rating Scale [UERS]). Sixty-five children with CP (43 males, 22 females, mean age 9y 2mo, SD 4y 1mo) with spasticity involving the upper extremity (quadriplegia n=22; hemiplegia n=36; diplegia n=7; Gross Motor Functional Classification System Levels I n=41, II n=6, III n=3, IV n=5, V n=10) were evaluated independently by occupational therapists and orthopedic surgeons using both instruments at several visits. Inter- and intrarater reliability were determined for both instruments by calculating measures of agreement (weighted kappa values and intraclass correlation coefficients [ICCs]). Interrater agreement (ICC=0.94) and intrarater agreement (ICC=0.96) on the MHC were good to excellent. Similarly, inter-rater agreement (kappa 0.66,0.81) and intrarater agreement (kappa 0.64,0.88) on the UERS was either good or excellent. The MHC and the UERS provide standardized, reliable, reproducible, and efficient instruments that can be used by occupational therapists and orthopedic surgeons to evaluate the upper extremities of children with CP. [source]

Variability in agreement between physicians and nurses when measuring the Glasgow Coma Scale in the emergency department limits its clinical usefulness

EMERGENCY MEDICINE AUSTRALASIA, Issue 4 2006
Anna Holdgate
Abstract Objective:, To assess the interrater reliability of the Glasgow Coma Scale (GCS) between nurses and senior doctors in the ED. Methods:, This was a prospective observational study with a convenience sample of patients aged 18 or above who presented with a decreased level of consciousness to a tertiary hospital ED. A senior ED doctor (emergency physicians and trainees) and registered nurse each independently scored the patient's GCS in blinded fashion within 15 min of each other. The data were then analysed to determine interrater reliability using the weighted kappa statistic and the size and directions of differences between paired scores were examined. Results:, A total of 108 eligible patients were enrolled, with GCS scores ranging from 3 to 14. Interrater agreement was excellent (weighted kappa > 0.75) for verbal scores and total GCS scores, and intermediate (weighted kappa 0.4,0.75) for motor and eye scores. Total GCS scores differed by more than two points in 10 of the 108 patients. Interrater agreement did not vary substantially across the range of actual numeric GCS scores. Conclusions:, Although the level of agreement for GCS scores was generally high, a significant proportion of patients had GCS scores which differed by two or more points. This degree of disagreement indicates that clinical decisions should not be based solely on single GCS scores. [source]

Specific Epileptic Syndromes Are Rare Even in Tertiary Epilepsy Centers: A Patient-oriented Approach to Epilepsy Classification

EPILEPSIA, Issue 3 2004
Christoph Kellinghaus
Summary: Purpose: To assess the practicability and reliability of a five-dimensional patient-oriented epilepsy classification and to compare it with the International League Against Epilepsy (ILAE) classification of epilepsy and epileptic syndromes. The dimensions consist of the epileptogenic zone, semiologic seizure type(s), etiology, related medical conditions, and seizure frequency. Methods: The 185 epilepsy patients (94 adults, 91 children, aged 18 years or younger) were randomly selected from the database of a tertiary epilepsy center and the general neurological department of a metropolitan hospital (28 adults). The charts were reviewed independently by two investigators and classified according to both the ILAE and the patient-oriented classification. Interrater reliability was assessed, and a final consensus among all investigators was established. Results: Only four (4%) adults and 19 (21%) children were diagnosed with a specific epilepsy syndrome of the ILAE classification. All other patients were in unspecific categories. The patient-oriented classification revealed that 64 adults and 56 children had focal epilepsy. In an additional 34 adults and 45 children, the epileptogenic zone could be localized to a certain brain region, and in 14 adults and five children, the epileptogenic zone could be lateralized. Fourteen adults and 21 children had generalized epilepsy. In 16 adults and 14 children, it remained unclear whether the epilepsy was focal or generalized. Generalized simple motor seizures were found in 66 adults and 52 children, representing the most frequent seizure type. Etiology could be determined in 40 adults and 45 children. Hippocampal sclerosis was the most frequent etiology in adults (10%), and cortical dysplasia (9%), in children. Seven adults and 31 children had at least daily seizures. Seventeen adults and 26 children had rare or no seizures at their last documented contact. The most frequent related medical conditions were psychiatric disorders and mental retardation. Interrater agreement was high (kappa values of 0.8 to 0.9) for both the patient-oriented and the ILAE classification. Conclusions: Specific epilepsy syndromes included in the current ILAE classification are rare even in a tertiary epilepsy center. Most patients are included in unspecific categories that provide only incomplete information. In contrast, all of the patients could be classified by the five-dimensional patient-oriented classification, providing all essential information for the management of the patients with a high degree of interrater reliability. [source]

The intermediate effect and the diagnostic accuracy in clinical case recall of students and experts in dental medicine

EUROPEAN JOURNAL OF DENTAL EDUCATION, Issue 3 2009
J. Eberhard
Abstract Introduction:, The extensive knowledge of experts facilitates the solving of domain-specific problems. In general, this is due to the fact that experts recall more detailed information than do novices or even advanced students. However, if physicians of different expertise levels are asked to write down the details of a given case, advanced medical students recall more information than experts. This phenomenon was called the ,intermediate effect' and is considered to be a specific feature of medical expertise. The aim of the here presented study was to examine this observation in the domain of dental medicine. Materials and methods:, Sixty-one students and 20 specialised dentists participated in this study. Three clinical case descriptions were presented and afterwards the participants were told to write down all concrete information they remembered. Finally, they had to come up with a diagnosis. Interrater agreement, diagnostic accuracy and the recall explanation protocols were analysed statistically in comparison to state-of-the-art (canonical) explanations of the clinical cases. Results:, The mean interrater agreement was 96.2 ± 3.37%. It was shown statistically that the more experienced the participants, the more accurate their diagnoses were (P < 0.001). The statistical analysis using the Games-Howell test demonstrated significant more written recall of the 5th-year students compared with 3rd- and 4th-year students and experts (P < 0.05). Conclusion:, The results of this study suggest the existence of the intermediate effect in clinical case recall in dental medicine and thereby corroborate its importance and general applicability for different medical domains. [source]

Interrater agreement for discriminant classifications for the adjustment scales for children and adolescents

PSYCHOLOGY IN THE SCHOOLS, Issue 4 2002
Gary L. Canivez
Investigation of interrater agreement for the Adjustment Scales for Children and Adolescents (ASCA) discriminant classifications is reported. Two teaching professionals or paraprofessionals working in the same classroom for a minimum of 1 hour per day provided independent ratings of the same child using the ASCA. A total of 119 students ranging in age from 7 to 18 years were independently rated on the ASCA. Results indicated significant and moderate to substantial interrater agreement for the discriminant classifications. © 2002 Wiley Periodicals, Inc. [source]

Variability in pain response to a non-pharmacological intervention across repeated routine pain exposure in preterm infants: a feasibility study

ACTA PAEDIATRICA, Issue 5 2009
E Cignacco
Abstract Aim: To explore the variability in pain response in preterm infants across time who received sucrose during routine heel stick. Method: Single group, exploratory repeated measures design. Setting: Two tertiary level neonatal intensive care units (NICU) in Switzerland. Subjects: Nine preterm infants born between 28 2/7 and 31 4/7 weeks of gestation during their first 14 days of life. Measurements: Pain was assessed by the Bernese Pain Scale for Neonates (BPSN), the Premature Infant Pain Profile (PIPP) and the Visual Analogue Scale (VAS). Salivary cortisol was analysed. Results: 72,94% of the variability was within-subject variability, indicating inconsistency of pain responses across the 5 heel sticks. Interrater agreement was highest during heel sticks 1,3 and decreased during heel stick 4 and 5, indicating a possible alteration of pain patterns. No significant differences in the amount of cortisol could be detected before and after the heel sticks (p = 0.55), indicating no stress-induced peak after the painful intervention. However, a general gradual decrease of cortisol levels across time could be detected. Conclusion: A high variability in pain response among preterm neonates across time could be described. Consistency of cortisol levels before and after the heel sticks could indicate the effectiveness of sucrose across time. [source]

Reliability of Computerized Emergency Triage

ACADEMIC EMERGENCY MEDICINE, Issue 3 2006
Sandy L. Dong MD
Objectives: Emergency department (ED) triage prioritizes patients based on urgency of care. This study compared agreement between two blinded, independent users of a Web-based triage tool (eTRIAGE) and examined the effects of ED crowding on triage reliability. Methods: Consecutive patients presenting to a large, urban, tertiary care ED were assessed by the duty triage nurse and an independent study nurse, both using eTRIAGE. Triage score distribution and agreement are reported. The study nurse collected data on ED activity, and agreement during different levels of ED crowding is reported. Two methods of interrater agreement were used: the linear-weighted , and quadratic-weighted ,. Results: A total of 575 patients were assessed over nine weeks, and complete data were available for 569 patients (99.0%). Agreement between the two nurses was moderate if using linear , (weighted ,= 0.52; 95% confidence interval = 0.46 to 0.57) and good if using quadratic , (weighted ,= 0.66; 95% confidence interval = 0.60 to 0.71). ED overcrowding data were available for 353 patients (62.0%). Agreement did not significantly differ with respect to periods of ambulance diversion, number of admitted inpatients occupying stretchers, number of patients in the waiting room, number of patients registered in two hours, or nurse perception of busyness. Conclusions: This study demonstrated different agreement depending on the method used to calculate interrater reliability. Using the standard methods, it found good agreement between two independent users of a computerized triage tool. The level of agreement was not affected by various measures of ED crowding. [source]

Adolescent Foley Catheter Technique for Visualizing Hymenal Injuries in Adolescent Sexual Assault

ACADEMIC EMERGENCY MEDICINE, Issue 9 2003
Jeffrey S. Jones MD
Abstract Objectives: To determine the usefulness of the Foley catheter balloon technique for visualizing injuries of the estrogenized hymen in adolescent sexual assault victims compared with supine labial traction. Methods: A prospective clinical trial of 20 adolescent (age 13,16 years old) victims of sexual assault evaluated at a free-standing Nurse Examiner Clinic was conducted over a four-month study period. The clinic, affiliated with an emergency medicine residency program, is staffed by registered nurses who have been specially trained to perform medicolegal examinations using colposcopy with digital imaging. The Foley catheter technique uses an inflated balloon in the distal vaginal vault to expand the estrogenized hymen to its full capacity so that the edge may be readily visualized for signs of trauma. The Foley technique was compared with gross inspection, using supine labial traction, to photodocument hymenal abnormalities. Photographs of the hymen were obtained using the labial traction technique and then with the Foley technique. Three emergency physicians independently examined each pair of photographs with high interrater agreement for the presence of injury (,= 0.88). Results: Twenty adolescent sexual assault victims volunteered for the study; mean age was 14.8 years. Gross inspection of the hymen using supine labial traction identified hymenal injuries in three patients (15%). Use of the Foley catheter balloon technique allowed identification of hymenal abnormalities in nine additional cases (60%). The common injuries to the hymen included lacerations (30%), followed by ecchymosis and abrasions. One patient (5%) voiced discomfort (mild pressure sensation) during inflation of the balloon. Conclusions: The Foley catheter balloon technique is a simple method allowing improved photodocumentation of hymenal trauma in adolescent sexual assault victims compared with supine labial traction. [source]

Farmers' perceptions of soil erosion and its consequences in India's semiarid tropics

LAND DEGRADATION AND DEVELOPMENT, Issue 3 2005
J. Kerr
Abstract This paper investigates farmers' perceptions of soil erosion and how it affects crop yields, land values, and private conservation investments in India's semiarid tropics. It is based on three types of data: (1) a survey of farmers in three study villages; (2) a plot survey by a professional soil surveyor in the same villages; and (3) experimental and simulated data from nearby research stations with similar conditions. Farmers' perceptions of erosion are compared to the surveyor's using kappa, a statistical measure of interrater agreement. Perceived erosion,yield relationships are estimated econometrically and compared to experimental and simulated data. Effects on land values and conservation investments are estimated econometrically. Findings suggest that farmers are keenly aware of rill erosion but less aware of sheet erosion; kappa values ranging from 0 to 0·28 suggest low agreement with the soil surveyor. They anticipate annual yield losses of 5·8,11,per,cent due to rill erosion; these figures are reasonably consistent with those from nearby research stations. They anticipate yield increases of 3·8,14·5,per,cent due to installation of soil conservation bunds, largely because they can harvest soil from up the slope and capture organic matter. Perceived erosion has some effect on land values and soil conservation investments, but other factors such as irrigation and soil type have a much greater effect. These findings suggest that promoting soil conservation requires capitalizing on farmers' interest in short-term gains, such as from water and nutrient management. Copyright © 2005 John Wiley & Sons, Ltd. [source]

Reliability and validity of the direct observation clinical encounter examination (DOCEE)

MEDICAL EDUCATION, Issue 3 2003
Hossam Hamdy
Context, The College of Medicine and Medical Sciences at the Arabian Gulf University, Bahrain, replaced the traditional long case/short case clinical examination on the final MD examination with a direct observation clinical encounter examination (DOCEE). Each student encountered four real patients. Two pairs of examiners from different disciplines observed the students taking history and conducting physical examinations and jointly assessed their clinical competence. Objectives, To determine the reliability and validity of the DOCEE by investigating whether examiners agree when scoring, ranking and classifying students; to determine the number of cases and examiners necessary to produce a reliable examination, and to establish whether the examination has content and concurrent validity. Subjects, Fifty-six final year medical students and 22 examiners (in pairs) participated in the DOCEE in 2001. Methods, Generalisability theory, intraclass correlation, Pearson correlation and kappa were used to study reliability and agreement between the examiners. Case content and Pearson correlation between DOCEE and other examination components were used to study validity. Results, Cronbach's alpha for DOCEE was 0·85. The intraclass and Pearson correlation of scores given by specialists and non-specialists ranged from 0·82 to 0·93. Kappa scores ranged from 0·56 to 1·00. The overall intraclass correlation of students' scores was 0·86. The generalisability coefficient with four cases and two raters was 0·84. Decision studies showed that increasing the cases from one to four improved reliability to above 0·8. However, increasing the number of raters had little impact on reliability. The use of a pre-examination blueprint for selecting the cases improved the content validity. The disattenuated Pearson correlations between DOCEE and other performance measures as a measure of concurrent validity ranged from 0·67 to 0·79. Conclusions, The DOCEE was shown to have good reliability and interrater agreement between two independent specialist and non-specialist examiners on the scoring, ranking and pass/fail classification of student performance. It has adequate content and concurrent validity and provides unique information about students' clinical competence. [source]