			Home About us Contact

Formative Evaluation (formative + evaluation)

Distribution by Scientific Domains

Medical Sciences	35%
Education	17%

Selected Abstracts

Formative Evaluation of Computer-Based Training For a University Financial System

PERFORMANCE IMPROVEMENT QUARTERLY, Issue 1 2002
Kathryn M. Fischer
ABSTRACT This article describes a formative evaluation of a one-day introductory computer-based training (CBT) course for a new on-line financial and purchasing system at a large public university. The purposes of the evaluation were to evaluate the effectiveness of the training and to identify appropriate revisions and incorporate them into the training program. Participants were 78 university employees who were likely future users of the financial and purchasing system. The mean score on an on-line performance posttest that simulated real-work tasks was 94%, and the mean on a 40-item knowledge posttest covering the CBT content was 74%. Learner attitudes toward the course were positive, averaging 4.4 on a 5-point scale. Formative evaluation of the program resulted in revisions that had the potential to improve its effectiveness and provided evidence of the value of ongoing formative evaluation of workplace training. [source]

Strengthening Prevention Performance Using Technology: A Formative Evaluation of Interactive Getting To Outcomes®

AMERICAN JOURNAL OF ORTHOPSYCHIATRY, Issue 4 2009
Matthew Chinman PhD
Communities face challenges implementing evidence-based prevention programs. To help, policymakers are exploring how to build community-level capacity for prevention for thousands of organizations or communities across the United States. This article reports on a formative evaluation within 2 states' prevention systems of an Internet system designed to build capacity on a large scale, interactive Getting To Outcomes® (iGTO). In Tennessee, 30 coalitions were randomly assigned to receive either the iGTO system or nothing. In Missouri, 18 coalitions receiving iGTO were compared with 8 like coalitions who did not receive iGTO. The primary outcome was iGTO's impact on the performance of the coalitions' programs, assessed through interviews at baseline and after a year of iGTO implementation. Analyses suggest that iGTO-programs improved their performance of prevention practices over non-iGTO programs. Semi-structured interviews of iGTO users and state-level stakeholders showed that iGTO was adopted by most iGTO-assigned coalitions, albeit in mostly an elementary fashion. Perceptions of the iGTO system were mixed. The findings suggest that more comprehensive integration requires that state leadership also use iGTO and provide more support for its use at the local level. [source]

Formative Evaluation of Computer-Based Training For a University Financial System

The frontline and the ivory tower: A case study of service and professional-driven curriculum

AUSTRALIAN JOURNAL OF RURAL HEALTH, Issue 3 2009
Sue Lenthall
Abstract Objective:,To describe the development of a postgraduate, multidisciplinary program designed to meet the needs of remote health professionals, present formative evaluation findings and to offer an analysis of the difficulties and lessons learnt. Design:,Case study. Setting:,University Department of Rural Health in a remote region. Participants:, University staff, students and stakeholders involved in the development of the remote health practice program. Results:,Formative evaluation suggests that a curriculum driven by service and professional groups, such as the Flinders University Remote Health Practice program, is able to better prepare remote health practitioners and improve their effectiveness. Difficulties in development included a lack of recognition by some university academics of the value of practitioner knowledge and a reluctance to accept a clinical component in a masters program. Lessons learnt included the importance of: (i) respect for practitioner knowledge; (ii) explicit and appropriate values; (iii) high-quality academics with strong service links; (iv) appropriate length of lead time; (v) institutional links between university and both relevant professional organisations and health services; (vi) a receptive university; (vii) location; and (viii) ongoing engagement with services and professional responsive development. Conclusion:,The success of the program was due in large part to the relationship with professional bodies and close links with remote health services. We have described a number of lessons learnt from this experience that can be useful to other educational groups developing or revising their educational programs. [source]

Development and evaluation of a cd-rom to support student learning in dentist,patient communication

EUROPEAN JOURNAL OF DENTAL EDUCATION, Issue 2 2003
J. T. Newton
The interaction between dentist and patient is central to the provision of effective healthcare. A multidisciplinary team containing expertise in the social and behavioural sciences, medicine, dentistry, education and information technology was formed to design a CD-ROM to support more traditional teaching and learning in this area. The final version of the program consists of a ,virtual' patient from whom students are asked to take a medical history. The ,patient' interacts with the student and responds to the style of their questioning by changes in anxiety, or degree of annoyance. To date, the CD-ROM has been evaluated in two stages. The first stage was a predictive formative evaluation by the research/development team, and the second stage was a formative evaluation by 144 dental students of the first full prototype. A third stage, to be conducted next year, will involve measuring the longer term impact of the CD-ROM through the assessment of students' abilities to communicate with patients. The evaluation had two components: students made rates of how useful and easy to use they found the program, and students' attitudes towards interacting with patients were evaluated adopting constructs from the theory of Planned Behaviour. Students were positive about the use of the CD-ROM, though they requested some changes in the functionality of the program. Compared to a sample of students who had not used the CD-ROM, the students who used the CD-ROM felt more positive about communicating with their patients, at the end of the course, and expressed stronger intentions to communicate well with patients. We conclude that the CD-ROM we have developed in a useful adjunct to learning in dentist,patient communication. [source]

Adapting and personalizing the communication in a synchronous communication tool

JOURNAL OF COMPUTER ASSISTED LEARNING, Issue 3 2008
A. Gogoulou
Abstract In this paper, we present a synchronous text-based communication tool, referred to as Adaptive Communication Tool (ACT), which provides capabilities for adaptation and personalization. ACT supports both the free and the structured form of dialogue. The structured dialogue is implemented by two types of Scaffolding Sentence Templates (SST); i.e. sentence openers or communicative acts. The capability of adaptation is considered in the sense of making suggestions for the supported form of dialogue and SST type and providing the most meaningful and complete set of SST with respect to the learning outcomes addressed by the collaborative learning activity and the model of collaboration followed by the group members. Also, ACT enables learners to have control on the adaptation by selecting the form of dialogue and the SST type they prefer to use and enriching the provided SST set with their own ones in order to cover their communication needs. The results from the formative evaluation of the tool showed that (i) the proposed dialogue form, SST type and the provided set of SST cover students' communication needs, (ii) the capability of personalizing the communication by selecting the desired communication means as well as by enriching the provided SST set satisfied students, and (iii) students used adequately both types of SST resulting into on-task and coherent dialogues. [source]

A model for quantitative evaluation of an end-to-end question-answering system

JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, Issue 8 2007
Nina Wacholder
We describe a procedure for quantitative evaluation of interactive question-answering systems and illustrate it with application to the High-Quality Interactive Question-Answering (HITIQA) system. Our objectives were (a) to design a method to realistically and reliably assess interactive question-answering systems by comparing the quality of reports produced using different systems, (b) to conduct a pilot test of this method, and (c) to perform a formative evaluation of the HITIQA system. Far more important than the specific information gathered from this pilot evaluation is the development of (a) a protocol for evaluating an emerging technology, (b) reusable assessment instruments, and (c) the knowledge gained in conducting the evaluation. We conclude that this method, which uses a surprisingly small number of subjects and does not rely on predetermined relevance judgments, measures the impact of system change on work produced by users. Therefore this method can be used to compare the product of interactive systems that use different underlying technologies. [source]

Flashpoint: An Innovative Media Literacy Intervention For High-Risk Adolescents

JUVENILE AND FAMILY COURT JOURNAL, Issue 2 2000
JANE MOORE MSW
ABSTRACT This paper describes the development and formative evaluation of a media literacy, media based intervention for high-risk adolescents. The program described, Flashpoint, was developed to (1) moderate the influence of media presentations of violence, substance abuse and prejudice on adolescents; and t (2) teach participants cognitive skills which would enable them to resist impulses to engage in behavior involving violence, substance abuse or prejudice. The evaluation described studied the pilot testing of the program with three groups of adolescents (N=33) involved in the juvenile justice system: adolescents in a diversion program (first time, nonviolent offenders); adolescents on probation; and adolescents in residential custody of the Department of Youth Services. Qualitative findings are reviewed in detail. [source]

The impact of a new educational strategy on acquiring neonatology skills

MEDICAL EDUCATION, Issue 5 2002
I Treadwell
Overview A shortage of staff for teaching neonatology skills to large numbers of students, in small groups and following a new curriculum, necessitated an innovative educational strategy. This entailed the development and implementation of an interactive multimedia program (CD-ROM) to deliver information about skills and to demonstrate them. Methods Students had to study a specific skill using the CD-ROM and then practise in the Skills Laboratory, supported by lecturers who provided formative evaluation. Objectives The aims of this study were to assess the students' perspectives on the new strategy, and to compare the skills of students following the new curriculum to those of students following the traditional curriculum, who do not follow structured programmes on practical skills but experience a practical neonatology rotation. Results The evaluation of the CD-ROM program was very favourable. The majority of students still preferred live demonstrations but found the CD-ROM useful for revision purposes. With the exception of one skill, endotracheal intubation, the new curriculum students were found to be as competent as the students following the traditional curriculum and performed mask ventilation and cardiac massage significantly better than them. [source]

Falling forward: Lessons learned from critical reflection on an evaluation process with a prisoner reentry program

NEW DIRECTIONS FOR EVALUATION, Issue 127 2010
Barbara Hooper
A formative evaluation of life-skills learning modules with men and women in a residential prisoner reentry program where careful attention was given to voice, power, and engagement, is described and analyzed. The author reflects on the evaluation process through the critical theory lens of "being self-critical can illuminate how practices maintain oppressive conditions." Questions about whose voices remained dominant and whose power was suppressed are discussed. © Wiley Periodicals, Inc., and the American Evaluation Association. [source]

Real-time evaluation in humanitarian emergencies

NEW DIRECTIONS FOR EVALUATION, Issue 126 2010
Emery Brusset
The authors describe real-time evaluation (RTE) as a specific tool in disaster management and within the literature on formative evaluation, monitoring, and impact assessment. RTE offers the possibility of exploring innovative ways to empower frontline disaster response staff, possibly even beneficiaries of assistance. The authors describe conditions for the success of RTE, including field credibility, organization, and rapid analysis. © Wiley Periodicals, Inc., and the American Evaluation Association. [source]

Instructional Efficiency of Performance Analysis Training for Learners at Different Levels of Competency in Using a Web-Based EPSS

PERFORMANCE IMPROVEMENT QUARTERLY, Issue 4 2004
A. Aubteen Darabi
ABSTRACT The measure of performance improvement potential (Gilbert, 1978) in human performance technology uses an exemplary performance as a criterion against which to measure the potential improvement in the performance of a workforce. The measure is calculated based on the performance efficiency which compares expended resources to productivity. The same notion is used to measure the efficiency of instructional conditions, based on learners' mental effort invested in a learning task compared to their performance. This article compares the efficiency of an instructional condition for three groups of students differentiated by their use of an electronic performance support system (EPSS) to conduct a performance analysis project. The results indicated that the instruction was most efficient for those learners who showed greater competency in using the EPSS. Implications for using this technique as a means of formative evaluation of a course are discussed. [source]

Formative Evaluation of Computer-Based Training For a University Financial System

Evaluating Peer Review in an Introductory Instructional Design Course

PERFORMANCE IMPROVEMENT QUARTERLY, Issue 3 2001
Nicholas H. Woolf
ABSTRACT A peer review process, in which students reviewed other students' projects in a graduate introductory instructional design course, was evaluated. Peer review was experienced by these students as a learning activity about the process of instructional design (ID). The role of traditional ID models in representing ID as overly procedural-ized was mitigated, and the value, inter-personal processes, and affective aspects of formative evaluation were recognized. The effectiveness of peer review was influenced by the culture of the course in which it was embedded and by the structure of the process itself. Peer review is proposed as an authentic and efficient means to introduce graduate students to the strategic knowledge needed to apply ID skills. Recommendations are made to increase the effectiveness of peer review. [source]