			Home About us Contact

Weighted Kappa Coefficient (weighted + kappa_coefficient)

Distribution by Scientific Domains

Medical Sciences	50%

Selected Abstracts

Measuring Agreement of Multivariate Discrete Survival Times Using a Modified Weighted Kappa Coefficient

BIOMETRICS, Issue 1 2009
Ying Guo
Summary Assessing agreement is often of interest in clinical studies to evaluate the similarity of measurements produced by different raters or methods on the same subjects. We present a modified weighted kappa coefficient to measure agreement between bivariate discrete survival times. The proposed kappa coefficient accommodates censoring by redistributing the mass of censored observations within the grid where the unobserved events may potentially happen. A generalized modified weighted kappa is proposed for multivariate discrete survival times. We estimate the modified kappa coefficients nonparametrically through a multivariate survival function estimator. The asymptotic properties of the kappa estimators are established and the performance of the estimators are examined through simulation studies of bivariate and trivariate survival times. We illustrate the application of the modified kappa coefficient in the presence of censored observations with data from a prostate cancer study. [source]

Lack of agreement between rheumatologists in defining digital ulceration in systemic sclerosis

ARTHRITIS & RHEUMATISM, Issue 3 2009
Ariane L. Herrick
Objective To test the intra- and interobserver variability, among clinicians with an interest in systemic sclerosis (SSc), in defining digital ulcers. Methods Thirty-five images of finger lesions, incorporating a wide range of abnormalities at different sites, were duplicated, yielding a data set of 70 images. Physicians with an interest in SSc were invited to take part in the Web-based study, which involved looking through the images in a random sequence. The sequence differed for individual participants and prevented cross-checking with previous images. Participants were asked to grade each image as depicting "ulcer" or "no ulcer," and if "ulcer," then either "inactive" or "active." Images of a range of exemplar lesions were available for reference purposes while participants viewed the test images. Intrarater reliability was assessed using a weighted kappa coefficient with quadratic weights. Interrater reliability was estimated using a multirater weighted kappa coefficient. Results Fifty individuals (most of them rheumatologists) from 15 countries participated in the study. There was a high level of intrarater reliability, with a mean weighted kappa value of 0.81 (95% confidence interval [95% CI] 0.77, 0.84). Interrater reliability was poorer (weighted , = 0.46 [95% CI 0.35, 0.57]). Conclusion The poor interrater reliability suggests that if digital ulceration is to be used as an end point in multicenter clinical trials of SSc, then strict definitions must be developed. The present investigation also demonstrates the feasibility of Web-based studies, for which large numbers of participants can be recruited over a short time frame. [source]

Measuring Agreement of Multivariate Discrete Survival Times Using a Modified Weighted Kappa Coefficient

Optometric glaucoma referrals , measures of effectiveness and implications for screening strategy

OPHTHALMIC AND PHYSIOLOGICAL OPTICS, Issue 6 2000
Jim Gilchrist
Summary The effectiveness of disease screening is conventionally evaluated using the epidemiological indices of sensitivity and specificity, which measure the association between screening test results and final diagnoses of all the patients screened. The effectiveness of optometric glaucoma referrals cannot be measured using such indices because diagnoses are obtained only on patients who are referred, while the true disease status of those not referred remains unknown. Instead, glaucoma referral effectiveness has been evaluated using measures of ,detection rate', the proportion of those screened who are correctly referred, and ,referral accuracy', the proportion of those referred who are correctly referred occurrence. Examination of these operational measures shows that their obtainable values and, hence, their interpretation are influenced by the total proportions of diseased and referred patients, one or both of which will generally be unavailable in evaluating samples of referrals. On the other hand, if valid estimates of these proportions can be obtained from other sources, it is possible to rescale detection rate and referral accuracy to take account of them. This rescaling produces a pair of weighted kappa coefficients, chance-corrected measures of association between referral and diagnosis, which provide a better indication of true referral effectiveness than other measures. An important consequence of this approach is that it provides a clear quantitative illustration of the need for a dual strategy to improve the overall quality of optometric glaucoma screening; widespread adoption of more comprehensive modes of screening to improve accuracy, together with a significant increase in the total numbers of patients screened to improve detection. In order for detection rates to reach desirable levels, the total number of referrals in any sub-population of patients must match or exceed the number of patients with disease. This analysis confirms quantitatively that which is intuitively obvious; not only that glaucoma awareness and uptake of screening opportunities must be encouraged in all patients over 40 years of age, but also that the older and/or more at risk patients are, the greater is their need to take advantage of glaucoma screening. [source]