Home About us Contact
|
Home About us Contact | ||
|
|
||
Skill Scores (skill + score)
Kinds of Skill Scores Selected AbstractsProbabilistic temperature forecast by using ground station measurements and ECMWF ensemble prediction systemMETEOROLOGICAL APPLICATIONS, Issue 4 2004P. Boi The ECMWF Ensemble Prediction System 2-metre temperature forecasts are affected by systematic errors due mainly to resolution inadequacies. Moreover, other errors sources are present: differences in height above sea level between the station and the corresponding grid point, boundary layer parameterisation, and description of the land surface. These errors are more marked in regions of complex orography. A recursive statistical procedure to adapt ECMWF EPS-2metre temperature fields to 58 meteorological stations on the Mediterranean island of Sardinia is presented. The correction has been made in three steps: (1) bias correction of systematic errors; (2) calibration to adapt the EPS temperature distribution to the station temperature distribution; and (3) doubling the ensemble size with the aim of taking into account the analysis errors. Two years of probabilistic forecasts of freezing are tested by Brier Score, reliability diagram, rank histogram and Brier Skill Score with respect to the climatological forecast. The score analysis shows much better performance in comparison with the climatological forecast and direct model output, for all forecast timse, even after the first step (bias correction). Further gains in skill are obtained by calibration and by doubling the ensemble size. Copyright © 2004 Royal Meteorological Society. [source] Limited-area ensemble predictions at the Norwegian Meteorological InstituteTHE QUARTERLY JOURNAL OF THE ROYAL METEOROLOGICAL SOCIETY, Issue 621 2006Inger-Lise Frogner Abstract This study aims at improving 0,3 day probabilistic forecasts of precipitation events in Norway. For this purpose a limited-area ensemble prediction system (LAMEPS) is tested. The horizontal resolution of LAMEPS is 28 km, and there are 31 levels in the vertical. The state variables provided as initial and lateral boundary conditions for the limited-area forecasts are perturbed using a dedicated version of the European Centre for Medium-Range Weather Forecasts (ECMWF) global ensemble prediction system, TEPS. These are constructed by combining initial and evolved singular vectors that at final time (48 h) are targeted to maximize the total energy in a domain containing northern Europe and adjacent sea areas. The resolution of TEPS is T255 with 40 levels. The test period includes 45 cases with 21 ensemble members in each case. We focus on 24 h accumulated precipitation rates with special emphasis on intense events. We also investigate a combination of TEPS and LAMEPS resulting in a system (NORLAMEPS) with 42 ensemble members. NORLAMEPS is compared with the 21-member LAMEPS and TEPS as well as the regular 51-member EPS run at ECMWF. The benefit of using targeted singular vectors is seen by comparing the 21-member TEPS with the 51-member operational EPS, as TEPS has considerably larger spread between ensemble members. For other measures, such as Brier Skill Score (BSS) and Relative Operating Characteristic (ROC) curves, the scores of the two systems are for most cases comparable, despite the difference in ensemble size. NORLAMEPS has the largest ensemble spread of all four ensemble systems studied in this paper, while EPS has the smallest spread. Nevertheless, EPS has higher BSS with NORLAMEPS approaching for the highest precipitation thresholds. For the area under the ROC curve, NORLAMEPS is comparable with or better than EPS for medium to large thresholds. Copyright © 2006 Royal Meteorological Society [source] A ground-level ozone forecasting model for Santiago, ChileJOURNAL OF FORECASTING, Issue 6 2002Héctor Jorquera Abstract A physically based model for ground-level ozone forecasting is evaluated for Santiago, Chile. The model predicts the daily peak ozone concentration, with the daily rise of air temperature as input variable; weekends and rainy days appear as interventions. This model was used to analyse historical data, using the Linear Transfer Function/Finite Impulse Response (LTF/FIR) formalism; the Simultaneous Transfer Function (STF) method was used to analyse several monitoring stations together. Model evaluation showed a good forecasting performance across stations,for low and high ozone impacts,with power of detection (POD) values between 70 and 100%, Heidke's Skill Scores between 40% and 70% and low false alarm rates (FAR). The model consistently outperforms a pure persistence forecast. Model performance was not sensitive to different implementation options. The model performance degrades for two- and three-days ahead forecast, but is still acceptable for the purpose of developing an environmental warning system at Santiago. Copyright © 2002 John Wiley & Sons, Ltd. [source] High-resolution limited-area ensemble predictions based on low-resolution targeted singular vectorsTHE QUARTERLY JOURNAL OF THE ROYAL METEOROLOGICAL SOCIETY, Issue 582 2002Inger-Lise Frogner Abstract The operational limited-area model, HIRLAM, at the Norwegian Meteorological Institute is used at 0.25° latitude/longitude resolution for ensemble weather prediction over Northern Europe and adjacent parts of the North Atlantic Ocean; this system is called LAMEPS. Initial and lateral boundary perturbations are taken from coarse-resolution European Centre for Medium-Range Weather Forecasts global ensemble members based on targeted singular vectors (TEPS). Five winter and five summer cases in 1997 consisting of 20 ensemble members plus one control forecast are integrated. Two sets of ensembles are generated, one for which both initial and lateral boundary conditions are perturbed, and another with only the initial fields perturbed. The LAMEPS results are compared to those of TEPS using the following measures: r.m.s. ensemble spread of 500 hPa geopotential height; r.m.s. ensemble spread of mean-sea-level pressure; Brier Skill Scores (BSS); Relative Operating Characteristic (ROC) curves; and cost/loss analyses. For forecasts longer than 12 hours, all measures show that perturbing the boundary fields is crucial for the performance of LAMEPS. For the winter cases TEPS has slightly larger ensemble spread than LAMEPS, but this is reversed for the summer cases. Results from BSS, ROC and cost/loss analyses show that LAMEPS performed considerably better than TEPS for precipitation, a result that is promising for forecasting extreme precipitation amounts. We believe this result to be linked to the high predictability of mesoscale flows controlled by complex topography. For two-metre temperature, however, TEPS frequently performed better than LAMEPS. Copyright © 2002 Royal Meteorological Society [source] Comparing the scores of hydrological ensemble forecasts issued by two different hydrological modelsATMOSPHERIC SCIENCE LETTERS, Issue 2 2010A. Randrianasolo Abstract A comparative analysis is conducted to assess the quality of streamflow forecasts issued by two different modeling conceptualizations of catchment response, both driven by the same weather ensemble prediction system (PEARP Météo-France). The two hydrological modeling approaches are the physically based and distributed hydrometeorological model SIM (Météo-France) and the lumped soil-moisture-accounting type rainfall-runoff model GRP (Cemagref). Discharges are simulated at 211 catchments in France over 17 months. Skill scores are computed for the first 2 days of forecast range. The results suggest good performance of both hydrological models and illustrate the benefit of streamflow data assimilation for ensemble short-term forecasting. Copyright © 2010 Royal Meteorological Society [source] On the use of the intensity-scale verification technique to assess operational precipitation forecastsMETEOROLOGICAL APPLICATIONS, Issue 1 2008Gabriella Csima Abstract The article describes the attempt to include the intensity-scale technique introduced by Casati et al. (2004) into a set of standardized verifications used in operational centres. The intensity-scale verification approach accounts for the spatial structure of the forecast field and allows the skill to be diagnosed as a function of the scale of the forecast error and intensity of the precipitation events. The intensity-scale method has been used to verify two different resolutions of the European Centre for Medium-Range Weather Forecasts (ECMWF) operational quantitative precipitation forecast (QPF) over France, and to compare the performance of the ECMWF and the Hungarian Meteorological Service operational model (ALADIN) forecasts, run over Hungary. Two case studies have been introduced, which show some interesting insight into the spatial scale of the error. The distribution of daily skill score for an extended period of time is also presented. The intensity-scale technique shows that the forecasts in general exhibit better skill for large-scale events, and lower skill for small-scale and intense events. In the paper, it is mentioned how some of the stringent assumptions on the domain over which the method can be applied, and the availability of the matched forecasts and observations, can limit its usability in an operational environment. Copyright © 2008 Royal Meteorological Society [source] How much does simplification of probability forecasts reduce forecast quality?METEOROLOGICAL APPLICATIONS, Issue 1 2008F. J. Doblas-Reyes Abstract Probability forecasts from an ensemble are often discretized into a small set of categories before being distributed to the users. This study investigates how such simplification can affect the forecast quality of probabilistic predictions as measured by the Brier score (BS). An example from the European Centre for Medium-Range Weather Forecasts (ECMWF) operational seasonal ensemble forecast system is used to show that the simplification of the forecast probabilities reduces the Brier skill score (BSS) by as much as 57% with respect to the skill score obtained with the full set of probabilities issued from the ensemble. This is more obvious for a small number of probability categories and is mainly due to a decrease in forecast resolution of up to 36%. The impact of the simplification as a function of the ensemble size is also discussed. The results suggest that forecast quality should be made available for the set of probabilities that the forecast user has access to as well as for the complete set of probabilities issued by the ensemble forecasting system. Copyright © 2008 Royal Meteorological Society [source] Assessing the spatial and temporal variation in the skill of precipitation forecasts from an NWP modelMETEOROLOGICAL APPLICATIONS, Issue 1 2008Nigel Roberts Abstract It is becoming increasingly important to be able to verify the spatial accuracy of precipitation forecasts, especially with the advent of high-resolution numerical weather prediction (NWP) models. In this article, the fractions skill score (FSS) approach has been used to perform a scale-selective evaluation of precipitation forecasts during 2003 from the Met Office mesoscale model (12 km grid length). The investigation shows how skill varies with spatial scale, the scales over which the data assimilation (DA) adds most skill, and how the loss of that skill is dependent on both the spatial scale and the rainfall coverage being examined. Although these results come from a specific model, they demonstrate how this verification approach can provide a quantitative assessment of the spatial behaviour of new finer-resolution models and DA techniques. Copyright © 2008 Royal Meteorological Society [source] Measuring forecast skill: is it real skill or is it the varying climatology?THE QUARTERLY JOURNAL OF THE ROYAL METEOROLOGICAL SOCIETY, Issue 621C 2006Thomas M. Hamill Abstract It is common practice to summarize the skill of weather forecasts from an accumulation of samples spanning many locations and dates. In calculating many of these scores, there is an implicit assumption that the climatological frequency of event occurrence is approximately invariant over all samples. If the event frequency actually varies among the samples, the metrics may report a skill that is different from that expected. Many common deterministic verification metrics, such as threat scores, are prone to mis-reporting skill, and probabilistic forecast metrics such as the Brier skill score and relative operating characteristic skill score can also be affected. Three examples are provided that demonstrate unexpected skill, two from synthetic data and one with actual forecast data. In the first example, positive skill was reported in a situation where metrics were calculated from a composite of forecasts that were comprised of random draws from the climatology of two distinct locations. As the difference in climatological event frequency between the two locations was increased, the reported skill also increased. A second example demonstrates that when the climatological event frequency varies among samples, the metrics may excessively weight samples with the greatest observational uncertainty. A final example demonstrates unexpectedly large skill in the equitable threat score of deterministic precipitation forecasts. Guidelines are suggested for how to adjust skill computations to minimize these effects. Copyright © 2006 Royal Meteorological Society [source] Measures of skill and value of ensemble prediction systems, their interrelationship and the effect of ensemble sizeTHE QUARTERLY JOURNAL OF THE ROYAL METEOROLOGICAL SOCIETY, Issue 577 2001David S. Richardson Abstract Ensemble forecasts provide probabilistic predictions for the future state of the atmosphere. Usually the probability of a given event E is determined from the fraction of ensemble members which predict the event. Hence there is a degree of sampling error inherent in the predictions. In this paper a theoretical study is made of the effect of ensemble size on forecast performance as measured by a reliability diagram and Brier (skill) score, and on users by using a simple cost-loss decision model. The relationship between skill and value, and a generalized skill score, dependent on the distribution of users, are discussed. The Brier skill score is reduced from its potential level for all finite-sized ensembles. The impact is most significant for small ensembles, especially when the variance of forecast probabilities is also small. The Brier score for a set of deterministic forecasts is a measure of potential predictability, assuming the forecasts are representative selections from a reliable ensemble prediction system (EPS). There is a consistent effect of finite ensemble size on the reliability diagram. Even if the underlying distribution is perfectly reliable, sampling this using only a small number of ensemble members introduces considerable unreliability. There is a consistent over-forecasting which appears as a clockwise tilt of the reliability diagram. It is important to be aware of the expected effect of ensemble size to avoid misinterpreting results. An ensemble of ten or so members should not be expected to provide reliable probability forecasts. Equally, when comparing the performance of different ensemble systems, any difference in ensemble size should be considered before attributing performance differences to other differences between the systems. The usefulness of an EPS to individual users cannot be deduced from the Brier skill score (nor even directly from the reliability diagram). An EPS with minimal Brier skill may nevertheless be of substantial value to some users, while small differences in skill may hide substantial variation in value. Using a simple cost-loss decision model, the sensitivity of users to differences in ensemble size is shown to depend on the predictability and frequency of the event and on the cost-loss ratio of the user. For an extreme event with low predictability, users with low cost-loss ratio will gain significant benefits from increasing ensemble size from 50 to 100 members, with potential for substantial additional value from further increases in number of members. This sensitivity to large ensemble size is not evident in the Brier skill score. A generalized skill score, dependent on the distribution of users, allows a summary performance measure to be tuned to a particular aspect of EPS performance. [source] Assimilating humidity pseudo-observations derived from the cloud profiling radar aboard CloudSat in ALADIN 3D-VarMETEOROLOGICAL APPLICATIONS, Issue 4 2009Andrea Storto Abstract This paper describes an experimental procedure for assimilating CloudSat Cloud Profiling Radar (CPR) observations in ALADIN 3D-Var through the use of humidity pseudo-observations derived from a one-dimensional Bayesian analysis. Cloud data are considered as binary occurrences (,cloud' vs ,no-cloud'), which makes the approach feasible to be extended to other cloudiness observations, and to any other binary observation in general. A simple large-scale condensation scheme is used for projecting the prior information from a Numerical Weather Prediction model into cloud fraction space. Verification over a 1 month assimilation test period indicates a clear benefit of the pseudo-observation assimilation scheme for the limited CloudSat CPR data set, especially in terms of improved skill scores for dynamical parameters such as geopotential and wind. Copyright © 2008 Royal Meteorological Society [source] Communicating the value of probabilistic forecasts with weather rouletteMETEOROLOGICAL APPLICATIONS, Issue 2 2009Renate Hagedorn Abstract In times of ever increasing financial constraints on public weather services it is of growing importance to communicate the value of their forecasts and products. While many diagnostic tools exist to evaluate forecast systems, intuitive diagnostics for communicating the skill of probabilistic forecasts are few. When the goal is communication with a non-expert audience it can be helpful to compare performance in more everyday terms than ,bits of information'. Ideally, of course, the method of presentation will be directly related to specific skill scores with known strengths and weaknesses. This paper introduces Weather Roulette, a conceptual framework for evaluating probabilistic predictions where skill is quantified using an effective daily interest rate; it is straightforward to deploy, comes with a simple storyline and importantly is comprehensible and plausible for a non-expert audience. Two variants of Weather Roulette are presented, one of which directly reflects proper local skill scores. Weather Roulette contrasts the performance of two forecasting systems, one of which may be climatology. Several examples of its application to ECMWF forecasts are discussed illustrating this new tool as useful addition to the suite of available probabilistic scoring metrics. Copyright © 2008 Royal Meteorological Society [source] Changing Skill Intensity in Australian IndustryTHE AUSTRALIAN ECONOMIC REVIEW, Issue 1 2007Ross Kelly This article examines changes in industry skill scores for Australia for the period 1991 to 2001 using indices of cognitive skill for industry based on Census employment data. Changes in mean industry cognitive skill levels are analysed, as are the relative contributions of changes to the occupational structure within industry and changes to the industry structure of employment. The main findings are that the drivers of skill change differed substantially between the two Census periods. Prior to 1996 the majority of the change was due to shifts towards industries with a more highly skilled workforce. After 1996 changes in the economy-wide skill level were dominated by within-industry changes in occupational composition. This coincided with a sharp pickup in the rate of capital expenditure on information and communication technologies. The increasing use of part-time employment overall had a deskilling effect. [source] Simulation of the Madden, Julian Oscillation and its teleconnections in the ECMWF forecast systemTHE QUARTERLY JOURNAL OF THE ROYAL METEOROLOGICAL SOCIETY, Issue 649 2010Frédéric Vitart Abstract A series of 46-day ensemble integrations starting on the 15th of each month from 1989 to 2008 has been completed with the European Centre for Medium-Range Weather Forecasts (ECMWF) forecast system. The Madden, Julian Oscillation (MJO) simulated by the hindcasts is diagnosed using an index based on combined empirical orthogonal functions (EOFs) of zonal winds at 200 and 850 hPa and outgoing long-wave radiation. Results indicate that the dynamical model is able to maintain the amplitude of the MJO during the 46 days of integrations and the model displays skill for up to about 20 days to predict the evolution of the MJO. However, the MJO simulated by the model has a too slow eastward propagation and has difficulties crossing the Maritime Continent. The MJO teleconnections simulated by the ECMWF forecast system have been compared to reanalyses. In the Tropics, the impact of the MJO on precipitation is generally consistent with reanalysis. In the Northern Extratropics, the MJO simulated by the model has an impact on North Atlantic weather regimes, but with a smaller amplitude than in reanalysis which can be partly explained by the too slow eastward propagation of the simulated MJO events. The impact of the MJO on the probabilistic skill scores has been assessed. Results indicate that the MJO simulated by the model has a statistically significant impact on weekly mean probabilistic skill scores in the Northern Extratropics, particularly at the time range 19, 25 days. At this time range, the reliability of the probabilistic forecasts over Europe depends strongly on the presence of an MJO event in the initial conditions. This result confirms that the MJO is a major source of predictability in the Extratropics in the sub-seasonal time-scale. Copyright © 2010 Royal Meteorological Society [source] Assimilation of satellite-derived soil moisture from ASCAT in a limited-area NWP modelTHE QUARTERLY JOURNAL OF THE ROYAL METEOROLOGICAL SOCIETY, Issue 648 2010Jean-François Mahfouf Abstract A simplified Extended Kalman Filter is developed for the assimilation of satellite-derived surface soil moisture from the Advanced Scatterometer (ASCAT) instrument (on board the polar-orbiting satellite METOP) in a limited-area NWP model where soil water vertical transfers are described by a force,restore method. An analytic formulation of the land surface scheme Jacobians is derived to simplify the coupling between land surface and atmospheric data assimilation systems. Various steps necessary before the assimilation of ASCAT products are defined: projection of satellite data on the model grid, screening based on various criteria, bias correction using a CDF matching technique, and specification of model and observation errors. Three-dimensional variational data assimilation experiments are then performed during a four-week period in May 2009 over western Europe. A control assimilation is also run where the soil moisture evolves freely. Forecasts from these analyses show that the assimilation of ASCAT data slightly reduces the daytime low-level relative humidity positive bias of the control run. Forecast skill scores with respect to other variables are rather neutral. A comparison of the control run with the operational system where soil moisture is corrected from short-range forecast errors of screen-level observations show similar improvements but are more pronounced. These differences come from the fact that the number of screen-level observations from the surface network over Europe is significantly larger than those provided by a polar-orbiting satellite. These results are consistent with those obtained at ECMWF using soil moisture products derived from other satellite instruments (X-band radiometer TMI and C-band scatterometer ERS). Several avenues for improving this preliminary methodology are proposed. Copyright © 2010 Royal Meteorological Society [source] Can multi-model combination really enhance the prediction skill of probabilistic ensemble forecasts?THE QUARTERLY JOURNAL OF THE ROYAL METEOROLOGICAL SOCIETY, Issue 630 2008A. P. Weigel Abstract The success of multi-model ensemble combination has been demonstrated in many studies. Given that a multi-model contains information from all participating models, including the less skilful ones, the question remains as to why, and under what conditions, a multi-model can outperform the best participating single model. It is the aim of this paper to resolve this apparent paradox. The study is based on a synthetic forecast generator, allowing the generation of perfectly-calibrated single-model ensembles of any size and skill. Additionally, the degree of ensemble under-dispersion (or overconfidence) can be prescribed. Multi-model ensembles are then constructed from both weighted and unweighted averages of these single-model ensembles. Applying this toy model, we carry out systematic model-combination experiments. We evaluate how multi-model performance depends on the skill and overconfidence of the participating single models. It turns out that multi-model ensembles can indeed locally outperform a ,best-model' approach, but only if the single-model ensembles are overconfident. The reason is that multi-model combination reduces overconfidence, i.e. ensemble spread is widened while average ensemble-mean error is reduced. This implies a net gain in prediction skill, because probabilistic skill scores penalize overconfidence. Under these conditions, even the addition of an objectively-poor model can improve multi-model skill. It seems that simple ensemble inflation methods cannot yield the same skill improvement. Using seasonal near-surface temperature forecasts from the DEMETER dataset, we show that the conclusions drawn from the toy-model experiments hold equally in a real multi-model ensemble prediction system. Copyright © 2008 Royal Meteorological Society [source] The economic value of ensemble forecasts as a tool for risk assessment: From days to decadesTHE QUARTERLY JOURNAL OF THE ROYAL METEOROLOGICAL SOCIETY, Issue 581 2002T. N. Palmer Abstract Despite the revolutionary development of numerical weather and climate prediction (NWCP) in the second half of the last century, quantitative interaction between model developers and forecast customers has been rather limited. This is apparent in the diverse ways in which weather forecasts are assessed by these two groups: root-mean-square error of 500 hPa height on the one hand; pounds, euros or dollars saved on the other. These differences of approach are changing with the development of ensemble forecasting. Ensemble forecasts provide a qualitative tool for the assessment of weather and climate risk for a range of user applications, and on a range of time-scales, from days to decades. Examples of the commercial application of ensemble forecasting, from electricity generation, ship routeing, pollution modelling, weather-risk finance, disease prediction and crop yield modelling, are shown from all these time-scales. A generic user decision model is described that allows one to assess the potential economic value of numerical weather and climate forecasts for a range of customers. Using this, it is possible to relate analytically, potential economic value to conventional meteorological skill scores. A generalized meteorological measure of forecast skill is proposed which takes the distribution of customers into account. It is suggested that when customers' exposure to weather or climate risk can be quantified, such more generalized measures of skill should be used in assessing the performance of an operational NWCP system. Copyright © 2002 Royal Meteorological Society. [source] A probability and decision-model analysis of PROVOST seasonal multi-model ensemble integrationsTHE QUARTERLY JOURNAL OF THE ROYAL METEOROLOGICAL SOCIETY, Issue 567 2000T. N. Palmer Abstract A probabilistic analysis is made of seasonal ensemble integrations from the PROVOST project (PRediction Of climate Variations On Seasonal to interannual Time-scales), with emphasis on the Brier score and related Murphy decomposition, and the relative operating characteristic. To illustrate the significance of these results to potential users, results from the analysis of the relative operating characteristic are input to a simple decision model. The decision-model analysis is used to define a user-specific objective measure of the economic value of seasonal forecasts. The analysis is made for two simple meteorological forecast conditions or ,events', E, based on 850 hPa temperature. The ensemble integrations result from integrating four different models over the period 1979,93. For each model a set of 9-member ensembles is generated by running from consecutive analyses. Results from the Brier skill score analysis taken over all northern hemisphere grid points indicate that, whilst the skill of individual-model ensembles is only marginally higher than a probabilistic forecast of climatological frequencies, the multi-model ensemble is substantially more skilful than climatology. Both reliability and resolution are better for the multi-model ensemble than for the individual-model ensembles. This improvement arises both from the use of different models in the ensemble, and from the enhanced ensemble size obtained by combining individual-model ensembles; the latter reason was found to be the more important. Brier skill scores are higher for years in which there were moderate or strong El Niño Southern Oscillation (ENSO) events. Over Europe, only the multi-model ensembles showed skill over climatology. Similar conclusions are reached from an analysis of the relative operating characteristic. Results from the decision-model analysis show that the economic value of seasonal forecasts is strongly dependent on the cost, C, to the user of taking precautionary action against E, in relation to the potential loss, L, if precautionary action is not taken and E occurs. However, based on the multi-model ensemble data, the economic value can be as much as 50% of the value of a hypothetical perfect deterministic forecast. For the hemisphere as a whole, value is enhanced by restriction to ENSO years. It is shown that there is potential economic value in seasonal forecasts for European users. However, the impact of ENSO on economic value over Europe is mixed; value is enhanced by El Niño only for some potential users with specific C/L. The techniques developed are applicable to complex E for arbitrary regions. Hence these techniques are proposed as the basis of an objective probabilistic and decision-model evaluation of operational seasonal ensemble forecasts. [source] Hydrological ensemble prediction and verification for the Meuse and Scheldt basinsATMOSPHERIC SCIENCE LETTERS, Issue 2 2010Joris Van den Bergh Abstract We present the hydrological ensemble prediction system developed at the Royal Meteorological Institute (RMI) of Belgium to study the Meuse and Scheldt basins. An overview is presented of the hydrological model and the operational setup of the forecasting system. We present some results of a 3-year hindcast that was performed to verify the quality of the probabilistic forecasting system. The raw precipitation forecasts and streamflow forecasts are considered: we provide skill scores and relative economic value for various subcatchments of the Meuse and Scheldt basins. Copyright © 2010 Royal Meteorological Society [source] Assessment of surgical competence at carotid endarterectomy under local anaesthesia in a simulated operating theatre,BRITISH JOURNAL OF SURGERY (NOW INCLUDES EUROPEAN JOURNAL OF SURGERY), Issue 4 2010S. A. Black Background: Methods of surgical training that do not put patients at risk are desirable. A high-fidelity simulation of carotid endarterectomy under local anaesthesia was tested as a tool for assessment of vascular surgical competence, as an adjunct to training. Methods: Sixty procedures were performed by 30 vascular surgeons (ten junior trainees, ten senior trainees and ten consultants) in a simulated operating theatre. Each performed in a non-crisis scenario followed by a crisis scenario. Performance was assessed live by means of rating scales for technical and non-technical skills. Results: There was a significant difference in technical skills with ascending grade for both generic and procedure-specific technical skill scores in both scenarios (P < 0·001 for all comparisons). Similarly, there was also a significant difference in non-technical skill with ascending grade for both scenarios (P < 0·001). There was a highly significant correlation between technical and non-technical performance in both scenarios (non-crisis: rs = 0·80, P < 0·001; crisis: rs = 0·85, P < 0·001). Inter-rater reliability was high (,, 0·80 for all scales). Conclusion: High-fidelity simulation offers competency-based assessment for all grades and may provide a useful training environment for junior trainees and more experienced surgeons. Copyright © 2010 British Journal of Surgery Society Ltd. Published by John Wiley & Sons, Ltd. [source] Handicaps and the development of skills between childhood and early adolescence in young people with severe intellectual disabilitiesJOURNAL OF INTELLECTUAL DISABILITY RESEARCH, Issue 12 2005O. Chadwick Abstract Background While a number of studies have examined the development of skills in children with intellectual disabilities (ID), most have been cross-sectional, most have been concerned with particular syndromes such as Down's syndrome or autism and few have attempted to identify factors associated with improvements in skills. Methods From a sample of 111 children with severe ID who had been identified from the registers of six special schools at 4,11 years of age, 82 were traced and reassessed 5 years later at the age of 11,17 years. On both occasions, information on the children's handicaps and skills was collected by interviewing their main carers using a shortened version of the Vineland Adaptive Behaviour Scales and the Disability Assessment Schedule. Results and conclusions There were small but statistically significant improvements in Vineland age-equivalent communication and daily living skills scores, but not in Vineland Socialization scores, over the 5-year period of follow-up. This pattern of improvement was observed in most aetiological subgroups. Improvement in skills was greatest in younger children, and was associated with reductions in behaviour problems and in levels of parental stress. In spite of the improvements in age-equivalent scores, Vineland standard scores showed significant declines over the same period of time, indicating that the improvements observed were smaller than would be expected in a general population sample of children of the same age. The dangers of using standard scores or quotients to quantify the level of functioning of children with severe ID are highlighted. [source] Trained lay observers can reliably assess medical students' communication skillsMEDICAL EDUCATION, Issue 7 2009George R Bergus Context, Our project investigated whether trained lay observers can reliably assess the communication skills of medical students by observing their patient encounters in an out-patient clinic. Methods, During a paediatrics clerkship, trained lay observers (standardised observers [SOs]) assessed the communication skills of Year 3 medical students while the students interviewed patients. These observers accompanied students into examination rooms in an out-patient clinic and completed a 15-item communication skills checklist during the encounter. The reliability of the communication skills scores was calculated using generalisability analysis. Students rated the experience and the validity of the assessment. The communication skills scores recorded by the SOs in the clinic were correlated with communication skills scores on a paediatrics objective structured clinical examination (OSCE). Results, Standardised observers accompanied a total of 51 medical students and watched 199 of their encounters with paediatric patients. The reliability of the communication skills scores from nine observed patient encounters was calculated to be 0.80. There was substantial correlation between the communication skills scores awarded by the clinic observers and students' communication skills scores on their OSCE cases (r = 0.53, P < 0.001). Following 83.8% of the encounters, students strongly agreed that the observer had not interfered with their interaction with the patient. After 95.8% of the encounters, students agreed or strongly agreed that the observers' scoring of their communication skills was valid. Conclusions, Standardised observers can reliably assess the communication skills of medical students during clinical encounters with patients and are well accepted by students. [source] Faculty training in evidence-based medicine: Improving evidence acquisition and critical appraisalTHE JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS, Issue 1 2007Laura J. Nicholson MD Abstract Introduction: Evidence-based medicine (EBM) integrates published clinical evidence with patient values and clinical expertise, the output of which is informed medical decision making. Key skills for evidence-based practice include acquisition and appraisal of clinical information. Faculty clinicians often lack expertise in these skills and are therefore unable to demonstrate this process for students and residents. Methods: We conducted a yearlong case-based EBM workshop for 28 clinician educators, with precourse and postcourse evaluations of EBM resource use and literature appraisal skills. Results: Of the original 28 participants, 26 completed the course. Self-assessed EBM resource use improved significantly. Self-reported EBM knowledge correlated with measured skill (r = 0.45), and both improved with the intervention (both p < .001). Higher EBM skills scores correlated with time logged on the course's EBM Web sites (r = 0.56; p < .05), workshop attendance rates (r = 0.55; p = .003), and fewer years since medical school graduation (r = ,0.56; p < .005). Discussion: An interactive, longitudinal, EBM course derived from a needs assessment can improve 2 skills important for evidence-based practice: online literature retrieval and critical appraisal skills. [source] Effectiveness of problem-based learning on academic performance in geneticsBIOCHEMISTRY AND MOLECULAR BIOLOGY EDUCATION, Issue 6 2007Gülsüm Araz Abstract This study aimed at comparing the effectiveness of problem-based learning (PBL)1 and traditional lecture-based instruction on elementary school students' academic achievement and performance skills in a science unit on genetics while controlling for their reasoning ability. For the specified purpose, twoinstructional methods were randomly assigned to intact classes of two different teachers. Each teacher had both PBL classes and traditional classes. Although students in PBL classes (n = 126) worked on ill-structured problems cooperatively with the guidance of the teacher, students in traditional classes (n = 91) received instruction based on teacher's explanations, discussions, and textbooks. Genetics Achievement Test was developed by researchers to measure the academic achievement and performance skills. Multivariate Analysis of Covariance results showed that the PBL students had higher academic achievement and performance skills scores (M = 11.44 and M = 2.67, respectively) when compared with those in traditional classes (M = 10.91 and M = 2.20, respectively). This indicated that the PBL students tend to better acquire scientific conceptions related to genetics and integrate and organize the knowledge. Moreover, it was found that the reasoning ability explained a significant portion of variance in the scores of academic achievement and performance skills. [source] | ||||||||||||||||