Document JNm7qM5QeQonRe7MyevYN6RXa

TM Workplace Safety and Health IC 9463 INFORMATION CIRCULAR/2002 Strategies For Improving Miners' Training Department of Health and Human Services Centers for Disease Control and Prevention National Institute for Occupational Safety and Health SoSH Information Circular 9463 Strategies For Improving Miners' Training Robert H. Peters, Editor U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES Centers for Disease Control and Prevention National Institute for Occupational Safety and Health Pittsburgh, PA and Spokane, WA Research Laboratories September 2002 ORDERING INFORMATION Copies of National Institute for Occupational Safety and Health (NIOSH) documents and information about occupational safety and health are available from NIOSH-Publications Dissemination 4676 Columbia Parkway Cincinnati, OH 45226-1998 FAX: Telephone: E-mail: Web site: 513-533-8573 1-800-35-NIOSH (1-800-356-4674) pubstaft@cdc.gov www.cdc.gov/niosh This document is the public domain and may be freely copied or reprinted. Disclaimer: Mention of any company or product does not constitute endorsement by NIOSH. DHHS (NIOSH) Publication No. 2002-156 CONTENTS Page Introduction ............................................................................................................................................................................................ Principles of Adult Learning: Application for Mine Trainers..................................................................................................... Getting Through to Greenhorns: Do Old Training Styles Work With New Miners? ........................................................... An Overview of the Evaluation Process for Mine Trainers......................................................................................................... Innovative Alternatives to Traditional Classroom Health and Safety Training ....................................................................... Considerations in Training On-the-Job Trainers ............................................................................................................................ Releasing the Energy of Workers To Create a Safer Workplace: The Value of Using Mentors to Enhance Safety Training.............................................................................................................................................................................................. Developing Toolbox Training Materials for Mining ..................................................................................................................... Communicating the Same Message with Different Media: An Example from Hearing Loss Prevention......................... 1 3 9 13 19 27 35 39 45 1 INTRODUCTION By Robert H. Peters1 This Information Circular from the National Institute for Oc cupational Safety and Health (NIOSH) documents and supple ments the information presented in a series of workshops held during 2002 and 2003. The primary intended audience consists of all who are involved in developing and conducting miners' training. According to the Mine Safety and Health Administration (MSHA), mine operators reported 240,000 full-time equivalent workers and independent contractors reported 42,000 full-time equivalent workers working on mine property during the year 2000. Unfortunately, these workers have a relatively high risk of suffering serious work-related injuries and illnesses. The mining industry has the highest rate of occupational fatalities among all U.S. industries. The fatality rate is 30 deaths per 100,000 workers compared to 4.6 for all private industry (Morbidity and Mortality Week Report, 2001; NIOSH, 2002). Compared to workers in other industries, miners also have a relatively high rate of nonfatal lost-time injuries, and their injuries tend to be more severe (Bureau of Labor Statistics, 1999). Many miners are also exposed to significant health hazards, including coal and silica dust, diesel exhaust, and noise. More than 1000 U.S. miners die of lung disease each year (NIOSH, 1999). Mine safety and health professionals have long recognized training as a critical element of an effective safety and health program. Federal regulations (30 CFR, Parts 46 and 48) require mine operators to provide initial safety and health training to all new miners, as well as a minimum of 8 hours of refresher train ing each year. The time and money being spent to train U.S. miners is substantial, and so there is a strong and steady demand for new and better mine training materials and methods. A growing concern among mine safety professionals regards the training of new workers. A major change in the mining workforce is anticipated within the next decade. In major seg ments of mining, especially coal, relatively few workers have been hired since the 1970's. Thus, as an entire cohort of miners in the current workforce nears retirement, the replacement of these employees will require an influx of new workers. New miners may be young people who lack the ability to recognize and respond to mining hazards in an appropriate manner. They may also have had different educational experiences than their older counterparts. Many safety professionals believe that these two cohorts require different approaches to training. The papers in this report should help prepare mine trainers for the changes about to occur in the workforce and acquaint them with strate gies they can use to enhance the effectiveness of their training. The first three papers present basic principles for teaching adults. The five remaining papers are intended to illustrate how these principles can be applied to the development and imple mentation of effective training for miners. Below is an overview of these papers. 1 Supervisory social scientist, Pittsburgh Research Laboratory, National Institute for Occupational Safety and Health, Pittsburgh, PA. 1. Kowalski and Vaught review the process and principles of adult learning. The learning model they present includes a discussion of goals, content, delivery, assessment, and reme diation. Adults are viewed as active learners, experiencedbased, expert in their own right in specific areas, independent, real-life centered, task-centered, problem-centered, solutiondriven, skill-seeking, self-directing, and internally and externally motivated. Basic aspects of curriculum development are briefly reviewed. For further information about adult learning, see Camm and Cullen's paper. 2. Mallett and Reinke's first paper discusses issues related to training new miners who have recently or will soon be entering the mining workforce. These new generations ofminers have different learning style preferences and training needs than Baby Boomers and other older miners. Even trainers who have been highly effective in the past should reassess their training styles and their classroom materials to determine if they are prepared to meet the needs of these young new workers. This paper provides information that will help mine trainers communicate across the generation gap. 3. Mallett and Reinke's second paper provide an overview of training evaluation. Trainers and decision-makers are given a framework for planning and assessing training evaluation strategies. The authors present Kirkpatrick's (2001) model of evaluation categories and discuss how to start an evaluation plan and various ways to collect data. They do not provide detailed instruction in evaluation methodologies, but give trainers a review of the things they need to consider when developing an evaluation plan. A good evaluation plan can inform revisions to a training course, assess trainee learning, and/or answer managers' questions about program effectiveness. 4. Brnich, Derick, Mallett, and Vaught discuss a technique for incorporating worker participation into fire prevention and safe equipment operation training. This technique involves development of short (5 to 7 minutes) videotapes coupled with toolbox talks that ground the content of the videos within the context of a miner's workplace. An example of one of these training modules is presented, along with the results of an evaluation performed on its effectiveness at improving miners' understanding of what types of information they should be sure to communicate during a mine emergency. 5. Wiehagen, Conrad, Friend, and Rethi discuss on-the-job training (OJT) as a method for teaching miners safety and production skills. Much of the training is done by experienced workers. This paper describes how small investments can help improve the effectiveness of on-the-job training. Effective onthe-job training involves some structure and planning in the transfer of responsibility for task performance from the trainer to the novice. Assisting the trainer could involve helping develop up-to-date job analyses and offering strategies for teaching and evaluating job skills. This paper addresses organizational considerations supporting structured on-the-job 2 training, general strategies, a typical approach for conducting on-the-j ob training, characteristics and duties oftrainers, and the limitations of on-the-job training as a teaching method. Health, safety, and operational skills can be connected through training. Haul truck operators are used to illustrate some of the concepts discussed. 6. Camm and Cullen discuss the mentor-protege model for teaching miners. Miners know when they see excellence and have a deep respect for experts in their field. By using expert miners as mentors to other workers, training programs can be developed that will have a legitimacy and credibility that reso nates with those being trained. Building upon concepts in cur rent theories of adult education, these authors highlight the unique advantages of using mentoring as a teaching method that can make educational experiences both interesting and effective. 7. Varley and Boldt discuss how mine trainers can develop their own tailgate training. Tailgate training--short (usually 10 to 15 minutes) weekly sessions conducted on-site prior to work shifts and involving work crews--is a popular mode of worker occupational safety and health training employed by many fieldbased companies. Too often the trainer attempts to use generic information, cover too broad a subject, or teach a new skill during the short period available for training. Toolbox training should be used to refresh knowledge and skills and to commun icate new hazards specific to a given minesite. This paper pre sents methods trainers can use to prepare toolbox training materials and make it relevant to miners and specific minesites. 8. Randolph, Kohler, and Byrne discuss why multiple ver sions of an educational message can reach a diverse population more effectively than a single version. For instance, some workers are trained in formal classrooms, while others are self-taught. NIOSH has developed multiple versions of a hearing loss simulation to show how a single set of information can be readily adapted to different delivery methods. Three versions of the simulator--an interactive software package, a web-based module, and an electronic slide show--were developed with a minimum of effort and expense when compared to a single, less-flexible version. Interactive software is best for training sessions led by a relative expert in the field (audiological testing) while Web pages are best for an individual worker, and slide shows are best for small, more general training classes. This paper describes additional advantages and disadvantages of different delivery systems and shows what considerations are helpful when designing content that can be readily adapted to alternate presentations. REFERENCES Bureau of Labor Statistics. 1999. Lost-worktime injuries and illnesses: Characteristics and resulting time away from work. U. S. Department of Labor news release 99-102, April 22. Kirkpatrick, D. 2001. The four-level evaluation process. Ch. 12 in What Smart Trainers Know: The Secrets of Success from the World's Foremost Experts, L.L. Ukens, ed. San Francisco, CA: Jossey-Bass/Pfeiffer. Pp. 122-132. Morbidity and Mortality Week Report. Fatal occupational injuries-United States, 1980-1994. 2001. Vol. 50, no. 16, pp. 317-320. National Institute for Occupational Safety and Health. 2002. Mining facts for 2000. Pittsburgh, PA: Pittsburgh Research Laboratory. NIOSH Pub. No. 2002-145. National Institute for Occupational Safety and Health. 2000. Work-related lung disease. NIOSH Surveillance report, 1999. Morgantown, WV: Division of Respiratory Disease Studies. NIOSH Pub. No. 2000-105. U.S. Code ofFederal Regulations, Title 30-Mineral Resources, Parts 46 and 48. 1999. PRINCIPLES OF ADULT LEARNING: APPLICATION FOR MINE TRAINERS By Kathleen M. Kowalski1 and Charles Vaught2 3 ABSTRACT Safety and health professionals from all sectors of industry recognize that training is a critical element of an effective safety and health program. A major concern in the mining industry is how to train both an aging workforce and the expected influx of new miners and mine managers as older workers retire. Thus, a review of some of the basic principles of adult learning may be helpful to mine trainers. This paper discusses the principles of adult learning based on research in education and psychology. It stresses the importance of taking a systems approach to training, focusing on the relationship between the environment and technology, and understanding how workers interact with both. The authors argue that the principles of adult learning and a systems approach are fundamental to the delivery of effective training in the mining industry. Examples of training programs developed by the Pittsburgh Research Laboratory of the National Institute for Occupational Safety and Health are presented within two broad performance domains: routine and nonroutine skills. Basics of curriculum development are also briefly presented to provide the mine trainer with a template for program development. INTRODUCTION Recently, the mining population has undergone numerous changes, including increases in the age of employees, diversity of experience, and increased variety in age, ethnicity, and cultural background. These changes require a different way of thinking about the mining population with respect to training. Instead of following the traditional model of an instructor imparting knowl edge to passive learners, training must allow learners to draw on experience, link concepts to real-world situations, and transfer knowledge from one situation to another (Lankard,1995). Adults have their own unique ways in which they learn, and it is im portant for instructors to design training programs and materials around these ways. Caudron (2000) has noted that trainers fre quently do not teach the way adults learn. This thought is reflected in the mining industry, where trainers are usually experienced in specific content areas, but may not be knowledgeable in adult learning or various educational methods. Most mine managers seem to be sold on learning but not necessarily on training. Could this observation--that trainers are content-wise but not well versed in educational principles--be part of the disconnect? According to the 1999 American Society for Training and Development (ASTD) State of the Industry report (in Caudron, 2000), instructor-led classroom training is still the predominant way of teaching adults in the workplace. In fact, this study showed that 70% of all training still consists of an instructor talking about or sometimes demonstrating concepts. But research shows that adults, in general, don't respond well to "lecturers." These researchers note that the most unforgettable learning experiences occur through personal experience, group support (figure 1), or mentoring. With a renewed and expanded focus on training in the mining industry, as evidenced by formation of the Mine Safety and Health Administration's (MSHA) Educational Field Service, it is appropriate to review basic information on curriculum devel opment and adult learning. Learning by experience is important in that adults learn best by having experiences and reflecting on 1Research psychologist. Sociologist, Pittsburgh Research Laboratory, National Institute for Occupational Safety and Health, Pittsburgh, PA. Figure 1.--Group learning. 4 them. Group learning experiences are also important. In groups, learners can help each other understand the material and learn from each other. In addition, the context of the learning is important, as most adult learners want to know how what they learn will apply in the workplace. This paper presents basic knowledge of adult learning curriculum development and a model for a systems approach to training with the express goal of providing mine trainers with additional tools to enhance their effectiveness and meet the training challenges of the mining industry today. DEVELOPING A CURRICULUM First and foremost, in planning training classes, the following five points need to be considered and may provide a skeleton or checklist for the trainer (figure 2). 1. Clear goals What is the point of this training? What are the individual capabilities or expected outcomes of the training? It is im portant to clarify the instructional focus for yourself and for the trainees. 2. Content What content will support the stated goals? For example, if the goal is "to increase individual safety behavior around power sources at the mine site," what information should be presented to reach that goal? 3. Appropriate delivery mechanism Is it best to present the material through lectures, demon strations, videos, or software programs? Should trainees receive the instruction individually, with partners, in teams, or as a group? Teaching methods must address not only the content to be delivered, but different learning styles as well. No approach should be used just because it is the latest method of instruction. The delivery method needs to be carefully evaluated. For ex ample, teaching methods that draw on the knowledge of older workers in class and generate discussions with younger workers may be a very successful way to transfer knowledge, but that notion should be put to the test under given circumstances. 4. Assessment Assessment is key in planning an educational experience. Assessment should be built into the program. How will you know if your trainees have learned the content? How will you know if the learning goal has been reached? For example, a table-top simulation might have the teaching of critical escape skills as its goal. For these types of skills, a mastery of at least 90% of the exercise content is a reasonable standard (Cole et al.,1984). A lower performance is seen as undesirable because the real-world consequences can be severe. The measure used could be simply the exercise's total score expressed in a per centage of correct performance. Then, if only 20% of the in dividuals in the class attained performance scores at or above 90% mastery, the trainer would know he or she should offer some remediation. 5. Remediation Finally, all effective educational programs need a reme diation component. If the assessment indicates that the trainee "doesn't get it," a preplanned intervention is called for. This is particularly important when training in critical skills such as putting on a self-contained self-rescuer (SCSR). For instance, the "3+3" training protocol3 requires a trainee to demonstrate proficiency while being evaluated immediately after having received initial instruction in the task. If an error is committed, it is corrected by the instructor, and the entire process is repeat ed. This cycle of demonstration--remediation-- demonstration continues until the trainee exhibits immediate mastery of the donning process (Vaught and Cole, 1987). 1. Clear goals 2. Content 3. Appropriate delivery mechanism 4. Authentic assessment 5. Remediation Figure 2.--Checklist for developing a curriculum. ADULT LEARNING According to Caudron (2000), there are several important con cepts adult trainers should practice. She encourages the use of collaborative interaction, an atmosphere where learners and in structors support each other in the process both in and out of formal learning, and the use and encouragement of cooperative communication. She also suggests trainers remember that peo ples' feelings are critical in developing relationships in any learning situation. These concepts fit with adult learning theory. While there is no one theory or one best theory of adult learning, those that recognize that adult learners come to the learning situation from a particular environment and with a personal history seem most appropriate. Adult learning is about "the promotion of active learning grounded in the past experience of the learner and in the application of the knowledge at a personal level" (Puliyel, 1999, p. 513). That ap plication generally takes place in relation to places and things. It is important for a trainer to understand how adults learn, and it is important for him or her to understand the concept of a systems ap proach in order to plan appropriate and effective training. 3The "3+3" training protocol is explained in more detail in the section on "Application of Adult Learning Principles to Nonroutine Skills" in this paper. 5 KNOWLEDGE Until recently, theory in adult learning was approached from a psychological perspective, meaning that the focus was on indi viduals. Newer thinking, as suggested above, takes a broader per spective and includes the environments in which adult learners function as well as the dynamics of group learning (Merriam, 1993). According to Ference and Vockell (1994), adults respond best to learning that is active and experienced-based. Adults like inter active learning and learning they can relate to the basis oftheir own experiences. Thus, examples and illustrations need to be relevant to the trainees. Unlike children, adults are experts in their own right in specific areas. This expertise needs to be recognized and may be used to meet the learning goal. Adult learners are independent, and this independence should be considered when planning delivery methods and remediation. They are real-life centeredand desire problems, examples, and descriptions from real life (figure 3). In addition, adult learners are task-centered and problemcentered. Being problem-centered, adult trainees are quick to focus on a problem and so are solution-driven. Adults may also be seen as skill-seeking, as many times they are in training to acquire a new job skill and thus are positively motivated and self-directing. Adult learners are both internally and externally motivated. In other words, sometimes an adult will be motivated by the pleasure and satisfaction of learning something new or by the camaraderie of class interaction. Sometimes an adult will be motivated by the resultant increase in pay or certification at the end of the class. < Active < Experienced-based < Learner as expert < Independent < Real-life centered < Task centered < Problem centered < Solution driven < Skill seeking < Self-directing < Motivated Figure 3.-Some principles of adult learning. PRACTICE Practice is important to learning. In addition, how the practice is done makes a significant difference. In a recent study, Simon (2001) showed that in the short run, practicing several skills in separate but concentrated blocks led to better performance during practice than did interleaving (integrating one skill after another). However, in the long term, interleaved practice led to better learning than did block practice. This study also found that "People are often poor assessors of what they have learned." In some cases, this is not serious, but in others, such as in some surgical procedures, machinery operations, or putting on an emergency breathing apparatus, the consequences can be serious. EXAMPLES OF THE APPLICATION OF ADULT LEARNING PRINCIPLES WITHIN A SYSTEMS CONTEXT It is important for mine trainers to apply learning principles within a systems context (figure 4). In other words, the subject matter and approach need to be put within a broad, interrelated context for miners, so both technical issues and social and human behavior and their relationships can be considered. The term used for this type of approach is "sociotechnical systems" research. This approach was pioneered in British coal mines during the 1950's and recognizes that workers and technology interact within a physical and organizational environment. It is understood that change in one component of a system may have both intended and unintended consequences in other components. In mining, work conditions are dynamic, technology is being introduced at an ever-increasing rate, and the workforce, in a shrinking job market, is aging. Because interventions in this context must be multidimensional to be effective, researchers at the Pittsburgh Research Laboratory (PRL) have taken an interdisciplinary systems approach to worker safety and health training. The authors suggest that mine trainers under stand and incorporate this approach into their teaching of adult learners. Potentially hazardous situations that confront miners in their workplaces can be characterized as routine or nonroutine in nature. Elements that contribute to injury in either situation may involve the equipment employed, those procedures used to perform particular tasks, and the behaviors of individuals or groups. Most PRL research on broad problem topics, therefore, has tended to take all three of these elements into consideration. In teaching safety and health, trainers should include all dimensions. APPLICATION OF ADULT LEARNING PRINCIPLES TO ROUTINE SKILLS In mining, much human factors and training research has focused upon the reduction of death and injury from falls of roof during routine tasks associated with roof control. A contributor to roof fall fatalities is workers' relative inability to recognize hazards. Unfortunately, methods for teaching mine hazard recognition in the classroom have not changed much over the years. The format for such training usually consists of having workers view slides of hazards or participate in discussions of conditions in their particular workplace. These training approaches assume that informing workers of "problems" will have some impact at a later time when a miner happens to encounter similar hazards on the job. Questions relating as to whether learning will be transferred successfully from a classroom to the workplace are rarely addressed. The need for improved methods of teaching miners to recognize hazards has been addressed by PRL researchers using an innovative 6 r Physical and Organizational People and Technology Environment vJ Routine Nonroutine Equipment: Procedures: People: Bolter design ^Equipment: Roof scaling Procedures: Hazard training People: Figure 4.-A systems approach. SCSR design 3+3 method SCSR training form of instruction. This instruction combines the known ad vantages of table-top simulations with three-dimensional (3-D) slides. Adding 3-D slides to a simulation forms a truly unique training instrument. It figuratively "places" miners in a problem solving situation where surroundings can be visualized realistically using high-fidelity, 3-D pictures. This approach targets key prin ciples of adult learning in which the learning should be active, ex perience-based, real-life centered, task-and-problem centered, and solution-driven. Although these training materials were immediately appealing, little was known about whether they would lead to any improvement in hazard-recognition skills. Nor was it known whether such learning would transfer to the workplace. Thus, the researchers posed the following question: Can training that uses a latent image/3-D slide exercise improve a miner's ability to recognize roof and rib hazards? To answer this question, a realworld experiment was conducted with a small sample (Barrett and Kowalski, 1995). Six coal miners with similar job classifications and mining ex perience participated in the first experiment. The miners were as signed randomly to either an experimental group or a control group. The experimental group was trained with a latent image/3-D slide simulation in a classroom at the host mine's training center and took approximately 30 minutes to complete. Each miner worked individually through his problem booklet and responded to the questions. At certain points, exercise directions had the worker view a designated 3-D slide that accompanied a particular question. There were no discussions during the training session, and each miner worked at his own pace. To investigate the effectiveness ofthis training, a hazard recog nition task was set up in the mine. Twelve areas that contained roof and rib hazards similar to those found in the exercise were iden tified. These areas were part ofa mile-long route traversing two of the mine's major entries. Each area was marked by spray-painting a letter (A through L) on the ribs ofthe entry. No artificial hazards were prepared at any area; only ones that existed naturally were recorded. These then became keys for the recognition task. Hazard recognition performance was assessed as subjects from both groups walked through the mine and attempted to identify hazards in each marked area. Each miner was given a pencil and clipboard with 12 sheets of paper labeled A through L. The workers were instructed to walk as a group along the designated route and stop at each labeled station. They were given 1 minute at each stop to identify any roof or rib hazard they recognized. Subjects wrote their observations on the sheets provided. These written responses were done individually. At no time were group members permitted to talk to each other or discuss the task. Re searchers provided no feedback at any time during the entire experiment. The control group, of course, did not receive training prior to the walk-through. There were 20 points possible for the underground hazard recognition task. Table 1 shows the individual subject scores given as both the number of correct answers and a corresponding correct percentage. The table also shows means and standard deviations for both the control and experimental groups. Note that all experi mental subjects who had training prior to the walk-through scored higher than the control group. Given the small sample size, a Fisher Randomization t Test was applied to the data. This test confirmed with 95% confidence the hypothesis that the mean score of the experimental group was significantly greater than the mean score of the control group. Significantly, since human factors and training research and in terventions began, the number and rate of fatal roof fall acci dents (especially in small mines) has declined dramatically. In 1989, 17 deaths were attributable to rooffalls, while in 1990, there were 21. In 1994 and 1995, on the other hand, there were five and six roof fall fatalities, respectively. APPLICATION OF ADULT LEARNING PRINCIPLES TO NONROUTINE SKILLS Another broad problem topic that has occupied human factors researchers in the past few years concerns miners' emergency breathing equipment. Of particular interest has been the selfcontained, self-rescuer (SCSR), a 1-hour, oxygen-generating apparatus. Investigations of the Wilberg disaster and other major mine fires strongly suggested thatworkers had difficulty putting on their SCSR's in emergency situations, making their escape problematic. Subsequently, human factors researchers participated in performance studies that showed the need for hands-on training with the equipment. In the course ofthe research, personnel developed a simplified, standardized procedure, known as the "3+3" method, forputting on SCSR's. This method is based on the principles of adult learning (hands-on, task-centered, and skill-seeking). It lumps all the discrete tasks involved in putting on a SCSR into a logical sequence of three steps that must be completed to isolate a worker's lungs and three additional steps that prepare a worker for evacuating the workplace. The 3+3 method has been adopted almostuniversally by the coal industry and endorsed by CSE Corp. and Mine Safety Appliances as the approved procedure for donning the companies' models of person-wearable SCSR's. 7 Table 1.--Performance scores on underground hazard recognition task. Group, subject Controls: 1 ............................ 2 ............................ 3 ............................. Experimental: 1 ............................ 2 ...................... 3 ...................... Individual scores No%Mean no. 11 55 13 65 8 40 18 16 1575 90 80 _________________________ Group scores Stand. deviation Mean % Stand. deviation 10.7 2.5 53.3 16.3 1.5 81.7 12.6 7.6 A TRAINING PROGRAM USING THE PRINCIPLES OF ADULT LEARNING TO ENCOURAGE ADHERENCE A persistent problem in the coal industry has been nagging doubts about the reliability of SCSR's. One dimension of the reliability issue is the concern that workers do not adhere to manufacturers' recommended inspection and care procedures. To encourage these procedures, researchers developed a training package to teach miners how to conduct routine inspections of their SCSR's, to care for them properly between inspections, and to reinforce the relationship between inspection and care and performance of the apparatus when it must be used. The package was designed so that, after completing either the video session or the CBT module, the trainee would be able to--1 1. Conduct the daily required inspection according to the provided checklist. 2. Conduct the required 90-day inspection according to the manufacturer's recommended procedures. 3. Always properly care for a SCSR. 4. Determine when a SCSR should be removed from service. 5. Know the criteria that require a SCSR to be removed from service. The training modules contained in this package can be used together or separately as appropriate for any particular audience. The accompanying instructor's guide explains each module and lists related materials. In total, the package includes-- An Instructor's Guide that gives an overview of the training package and includes an inspection checklist. A 5-minute video that introduces care and maintenance. A computer-based training CD that covers inspection and care for individual trainees or groups. A screen saver to remind users of 3+3 donning procedures. Stickers designed to communicate the lifesaving function of SCSR's. The experience-based, task-centered, SCSR training is an ex ample of the effective use of adult learning principles. This work has been used to support the promulgation of a federal regulation requiring hands-on SCSR training for all people entering an underground coal mine. The overall success of training work on this topic is reflected, in part, by the documented accounts of 3+3-trained workers who have used SCSR's to escape underground mine fires. CONCLUSION A grasp of the relationships among the environment, tech nology, and workers and how these three interact is fundamental to the delivery of effective training. A better understanding of adult learning and how it can be applied within the two broad performance domains of routine and nonroutine is then more probable. NIOSH research in education and training seeks to offer a continuous array of data leading to economically justified training interventions based on adult learning principles and incorporating a systems approach. These data may be used to define realistic goals, methods, and procedures for successive improvements in safety, mining systems, work crew proficiency, and improved miner training. Such justification will serve to institutionalize increased investments in the workforce and support training in the mining industry. REFERENCES Barrett, E.A., and K.M Kowalski. 1995. Effective hazard recognition training using a latent-image, three-dimensional slide simulation exercise. U.S. Bureau of Mines Report of Investigations 9527. Caudron, S. 2000. Learners speak out. Journal of Training and Development, vol. 54, no. 4, pp. 52-57. Cole, H., J. Moss, F.X. Gohs, W.E. Lacefield, B.J. Barfield, and D.K. Blythe. 1984. Measuring Learning in Continuing EducationforEngineers and Scientists. Phoenix, AZ: Oryx Press. Ference, P.R., and E.L. Vockell. 1994. Adult learning characteristics and effective software instruction. Educational Technology, July-August, pp. 25-31. 8 Kowalski, K.M., C. Vaught, M.J. Bmich, L. Mallett, D. Reinke, L. Rethi, and W. Wiehagen. 2001. The evolving mining workforce: Training issues. Presentation at Thirty-Second Annual Institute on Mining Health, Safety, and Research Conference. Salt Lake City: University ofUtah, Aug. 6-10, 2001. Lankard, B.A. 1995. New ways oflearning inthe workplace. ERIC Digest 161. Columbus, OH: ERIC Clearinghouse on Adult, Career, and Vocational Education. 4 pp. Merriam, S.B., ed. 1993. Anupdate on adult learningtheory. New Directions for Adult and Continuing Education, No. 57. San Francisco, CA: Jossey-Bass Pub. National Institute for Occupational Safety and Health (NIOSH). 1996. Briefing document for the human factors research program. Pittsburgh Research Program. Pittsburgh, PA: Pittsburgh Research Laboratory. Puliyel, M.M., J.M. Puliyel, and U. Puliyel. 1999. Drawing on adult learning theory to teach personal and professional values. Medical Teacher, vol. 21, no. 5, pp. 513-515. Simon, D.A. 2001. Metacognition in motor learning. Journal of Experimental Psychology: Learning, Memory and Cognition, vol. 27, no. 4, pp. 907-912. Vaught, C., and H. Cole. 1987. Problems in donning the self-contained self rescuer. In Mining Applications ofLife Support Technology. U.S. Bureau of Mines Information Circular 9134, pp. 26-34. GETTING THROUGH TO GREENHORNS: DO OLD TRAINING STYLES WORK WITH NEW MINERS? By Launa Mallett,1 Dana Reinke,*2 and Michael J. Brnich, Jr.3 9 ABSTRACT Some segments of the mining industry, especially underground coal, have seen a large influx of inexperienced miners in recent years. It is anticipated that this trend will reach other mining segments over the next 10 years. This paper discusses the training needs of the younger generation of inexperienced workers who have just entered or are soon to enter the mines. Currently, many trainers are of the so-called Baby Boom generation. Can these different age groups learn to communicate across the generation gap? Even trainers who have been highly effective in the past should reassess their training styles and their classroom materials to determine if they are prepared to meet the needs of these new trainees. INTRODUCTION If you've passed your 40th birthday, then you are a gene ration or more older than the new miners who are showing up in training classes. These trainees are part of Generation X (Xers) and Generation Next (Nexters) and are taking their places in the workforce. Being from a different generation doesn't mean you can't teach new miners to work in safe and healthy ways. But it does mean that you can't expect them to think, look, believe, or behave as you do. You can't even expect them to be like you were when you were that age. Generation Xers and Nexters have grown up in a world quite different from the one that existed when you were younger. They have been affected by different life experiences. While they will mature, they are not likely to change much in their basic beliefs and attitudes. To train them successfully, you need to understand how these new workers differ from older generations in the workforce. GENERATIONS IN THE MINES There are currently people from four generations working in mines in the United States. Zemke et al. (2000) have categorized people into four distinct cohorts based on the years they were born. A cohort can be understood as a group of people sharing common experiences as they pass through life's milestones. They were born at approximately the same time, and started school, became teenagers, entered adulthood, and left the work force at roughly the same time. The four generational cohorts as defined by Zemke et al. are "Veterans" (birth years 1922-1943), "Baby Boomers" (birth years 1943-1960), "Generations Xers" (birth years 1960-1980), and "Nexters" (birth years 1980-2000). Other researchers may shift the years somewhat or use different titles, but most generally agree that individuals within each co hort share common life experiences and reactions to them. Studying these common threads and generalizations about cohorts can result in a greater understanding of all individuals. Just as cohorts' life experiences affect their attitudes and beliefs, these experiences shape learning styles and training 'Sociologist. Sociologist. 3Mining engineer, Pittsburgh Research Laboratory, National Institute for Occupational Safety and Health, Pittsburgh, PA. needs. Examining cohort learning styles, Martin and Tulgan (2002b) defined the groups with slightly different dates and sev eral different titles than Zemke et al., but agreed that there are four distinct groups and noted that each "has its quirks and preferences." They discussed the training preferences of each cohort as follows. Veterans. Years of experience have taught veterans to rely on tried, true, and tested ways of doing things. "When in com mand, take charge. When in doubt, do what's right." After years ofworking under command-and-control management, veterans must use their wisdom to face the radical changes in the new workplace. Baby Boomers. Boomers paid their dues and climbed the lad der under the old rules and now find themselves operating amidst constant downsizing, restructuring, and reengineering. Boomers still pride themselves on their ability to survive "sink or swim" management, but fewer today are willing to keep up the frenetic pace. Boomer women led the charge for workplace flexibility, and now many Boomers have caught on to the free-agent mindset. Generation X. Xers formed the vanguard of the free-agent workforce. Now Xers are growing up and moving into positions of supervisory responsibility and leadership, but they are not 10 settling down. Xers remain cautious and they know their security rests in staying on the cutting edge. They're still willing to sidestep rules to get things done smarter, faster, and better. Nexters. Coming ofage during the most expansive economy in the last 30 years, Nexters are the children of the Baby Boomers and the optimistic, upbeat younger siblings ofthe Gen Xers. The first cohort oftruly "global citizens," they are socially conscious and volunteer minded. Nexters have been told that they can do anything...and they believe it. They are poised to be the most demanding generation in history. Because cohorts are birth-year-based categories, people are lumped together by age. It is therefore tempting to attribute dif ferences to the amount of life experience in each group. But, "the generational clash playing out in the workplace today is not merely a matter of young versus old. This clash pits the oldfashioned expectations, values and practices of stability against the new reality of constant change and the consequent need for agility" (Martin and Tulgan, 2002a). This clash could also be playing out in training rooms among various trainees with different perspectives or between the trainer and some of his or her trainees. Managing those dif ferences will lead to more effective training. Additionally, un derstanding how differences in learning styles affect integration of knowledge from the classroom with workplace skills and practices is key to developing the new generation of workers. First, however, the perspectives and experiences of each gen eration must be determined before addressing how differences in learning styles might affect the way a trainer approaches teaching. GENERATION X Gen Xers have been in the mining workforce for years. They were hired one or two at a time and integrated into experienced work crews. Since there were few new employees hired at any one mine at any one time, the training needs of this new cohort didn't attract much attention. Their training started in traditional classrooms and continued as they worked side-by-side with and under the watchful eye of Veterans and Boomers. This informal apprentice model seemingly worked even though little special effort was given to developing the training skills of experienced miners. Gen Xers were comfortable with this model because they responded well to a learning environment where they got involved in a task, made mistakes, and received feedback. Whether or not this model will effectively prepare Gen X miners for the next phase of their careers remains to be seen. These employees, now in their 20's and 30's, are seeking and being put into increasingly responsible positions. Until recently, access to these roles had been impeded by the large number of Baby Boomers in positions of authority. Furthermore, during the lean years of the 1990's, few Gen Xers were being trained to fulfill leadership positions. As the Veterans and Boomers start looking toward retirement, there is a potential for crises in leadership that will call, in part, for a training solution. NEXTERS Just as Gen Xers are trying on new roles as leaders, they will soon become the experienced miners in the eyes of the Nexters. Members of the Nexter cohort are graduating from high school, technical schools, and colleges. They face very differentjob pros pects than Gen Xers did. Within the time of Nexters' careers, it is anticipated that a large number of Baby Boomers will reach retirement age and leave the workforce. Since there are fewer Xers available in the population to occupy the positions ofretiring Baby Boomers, there should be jobs left for the Nexter cohort. Some segments of the mining industry, especially under ground coal, are already opening their doors to large numbers of these young inexperienced workers. As retiring Veterans and Baby Boomers are replaced by young Nexters, it is likely that this cohort will enter the workplace more quickly than one or two at a time. Introductory miner training programs will have to be examined with this factor in mind. The informal apprenticeship model used for integrating Gen Xer's into the workforce will not work if there are many inex perienced miners working with only a few mentors. Research sug gests that Nexter workers need more structure and focused atten tion in the workplace than the preceding Xer cohort (Training Mag.com, 1999). While the mentoring framework can be success fully used with the Nexter cohort, it needs to be formalized within new miner training programs to be most effective. Even if the number of new workers is spread across shifts and crews, more experienced workers will be needed to act as mentors or trainers than in the past. This could lead to inconsistent training across a mine if training content and strategies are left to each individual experienced worker. To introduce this cohort to the mining work force effectively, how they are to be trained must be considered, as well as how to train the trainers. VETERANS AND BABY BOOMERS AS COACHES AND MENTORS As noted above, experienced miners play very important roles in the work lives of new miners. Workers from the Vet eran and Baby Boomer cohorts are the people who will have their work habits, both good and bad, taken as models by young miners still learning the ropes. Older workers are experts at their jobs, and some undoubtedly have developed into excellent teachers of their crafts. Capturing the knowledge and skill of these experienced workers is one of the mining industry's major challenges. Matching the right experienced workers with new employees can have consequences for many years because today's newly hired Nexters are tomorrow's mine managers. Many Veteran and Baby Boomer miners will find passing on their knowledge and skills to be highly rewarding. Some may even delay leaving the industry if they find fulfillment in a mentor/trainer type of role. Training can be provided to assist them as they develop into these roles. 11 TRAINING STRATEGIES FOR YOUNG MINERS A lot is being said about how to train Gen X and Nexter workers (Caudron, 1997; Salopek, 2000; Tulgan, 2000; Corley, 1999; Cannon, 1991; Wagshal, 1997; Wyld, 1994; Zemke et al., 2000). Frequently, however, the stories and articles focus on students who hope to become employees of accounting firms, banks, or computer firms. This targeting of future white collar employees by researchers is nothing new. Over a decade ago, a report on young workers stated, "Although studies of college students are abundant in social science research, research on young people who do not attend college and on young people beyond college age is scarce" (William I Grant Foundation Commission on Work, Family, and Citizenship, 1988, in Arnett, 2000). While the literature contains valuable information about the Gen X and Nexter cohorts, applying such information to young and future miners should be done with care. A key point that is repeated over and again in the literature is the importance of computer-based learning to young workers. However, little research seems to be available to confirm whether or not this is true of young blue-collar employees. There is no doubt that Nexters, as a cohort, are much more computer literate than any generation before them. They have been using computers in all sorts of settings, including education, and are likely to be comfortable with this medium. This does not necessarily mean, however, that all Nexters' training should be delivered in this manner. Additional research into generational training preferences has found that Nexters prefer training that allows them to work in groups to complete an activity. In the case of Gen X workers, they prefer to work on skill building on their own in a more nontraditional training setting. NIOSH researchers have started exploring the issue of training preferences for workers of all generations. In the summer of 2001, 88 miners at an underground coal mine and who were identified as part of the Gen X or Nexter cohorts were asked to choose three training strategies they most enjoyed. The workers chose training strategies from the list below. - Computer-based - Watching videos - Hands-on practice in classroom or lab - Class discussion - Practice at worksite - Lecture (teacher talking) - Group activities - Quiet reading - Games - Simulation or drill Miners were also asked to choose three training methods they thought were best to help them learn something new. Analysis of the resulting data argued against the claim that Generation X and Nexter workers preferred computer-based training and may surprise some experts on training young workers. The top three methods chosen as most enjoyed were "Hands-on practice in classroom or lab" (42.9%), "Practice at worksite" (41.7%), and "Watching videos" (32.1%). Computerbased instruction was not a top choice given by younger miners; only 15.5% chose computer-based instruction as the training method they liked best. Even fewer younger workers (3.6%) choose computer-based training as the method they preferred when learning a new skill or idea. Instead, they showed an overwhelming preference for train ing that would allow them to try the new skill. When learning a new skill or idea, they preferred "Hands-on practice in class room or lab" (61.9%), "Practice at worksite" (48.8%), and "Simulation or drill" (41.7%). This isn't to say that other strategies should never be used, but training that incorporates supervised practice in new skills is highly important to these young miners. Given their preference for hands-on practice, it stands to reason that Gen Xers and Nexters will respond to experienced miners who are willing to show them how to do the job. They will accept instruction from individuals whom they believe have the knowledge and skills they need. But for this type of re lationship to be built, experienced miners must be open to an swering a lot of questions and to finding ways to provide guid ance as the younger person learns. A potential obstacle to building these relationships is discussed in an article directed at the roofing industry. To effectively motivate those in Generation X, you must realize Gen Xers prefer to learn through mentoring and coaching. They want the information, skills, and competencies of people with more experience. A Gen Xer typically will think, "Why should I have to learn something the hard way when my experienced manager already knows it?" Mentoring and coaching Gen Xers is a difficult obstacle for the construction industry to surmount. Because construction is an industry in which "paying your dues" and "learning the hard way" have become the norm, the mindset will have to change for the roofing industry to attract and retain top-quality Gen Xers (Alafat et al., 2001). Introducing miners of the new cohorts to the workplace simply cannot be done "as we've always done it." Instead, the best training strategy for these individuals must be determined. One trainer described the experience of training younger workers as follows. 12 I knew that I was finally old when I muttered the words that I swore I would never say: "Kids these days...they don't know how good they've really got it!" Regardless ofwhether you feel this way, or whether you sympathize with the plight of the so- called slacker generation, you must understand ONE thing: What you feel does not matter! Your focus need NOT be on how to change these post-baby-boomers (aged 34 and younger), but rather, how to understand them (Dunne, 2000). SUMMARY During America's gold rush, many people "took to the hills" in search of their fortunes. The experienced miners called those obviously new to the trade greenhorns. There are many stories about their exploits and adventures. One can be found in various forms, but generally goes like this telling from 1939. Another time a greenhorn came into a small mining town looking for a mine. The boys after giving him the "once over" decided he was looking for shade. They told him that under a large tree near the camp would be a good place to start digging. The most pleasant part of the digging would be all the nice shade he would have from the tree. I'll be damned! The Greenhorn dug there, went down about seven or eight feet and he struck it rich. He took the odd-looking stuffthat he had found and asked a fellow in the camp ifthat wasn't gold. Poor guy, he didn't know gold from brass. To him rock was rock. Well, the boys told him it was gold. Hell, there wasn't anything else to do. He sold the mine for $70,000. Can you beat it? (Haight, 1939). Like the "greenhorn" in the different versions of this story, your inexperienced employees may become successful miners in spite of bad advice, but they are much more likely to become valuable additions to the mining industry with quality training targeted for their learning styles and needs. REFERENCES Alafat, T., N. Allen, and R.N. Magnus. 2001 Gauging Gen X: Gen X employees are different than others, and you need to know how to work with them. Professional Roofing, http://www.professionalroofing.net/past/apr2001/feature.asp Arnett, J.J. 2000. High hopes in a grim world. Youth andSociety, vol. 31, no. 1, pp. 267-286. Cannon, D. 1991. Generation X: The way they do the things they do. Journal ofCareer Planning and Employment, vol. 51, no. 2, pp. 34-38. Caudron, S. 1997. Can Generation Xers be trained?. Training andDevelop- ment, vol. 5, no. 3, pp. 20-24. Corley, T. 1999. Becoming an employer of choice for Generation X: The elements of the deal. Journal ofCareer Planning & Employment, vol. 59, no. 4, pp. 21-26. Dunne, J. 2000. Does your service training stick?: How to update your approach to appeal to Generation X and Y. Restaurant Report On-line, http://www.restaurantreport.com/features/ft servicetraining.html Haight, W.C. 1939. Gold mining lore: Interview of miner Carl Hentz from 1939. From the WPA Writers' Project, January 23, 1939, a report written by author William C. Haight of his oral history interview of a miner, Carl Hentz. http://htg-is.vianet.net/~artpike/etown14.htm Martin, C., and B. Tulgan. [ 2002a]. Managing the Generation Mix - Part I. Bruce Tulgan's winning the talent wars. http://www.rainmakerthinking.com/backwttw/2002/jan9.htm Martin, C., and B. Tulgan, [2002b]. Managing the Generation Mix - Part II. Bruce Tulgan's winning the talent wars. http://www.rainmakerthinking.com/backwttw/2002/feb19.htm Salopek, J. 2000. The young and the rest ofus. Training andDevelopment, vol. 54, no. 2, Feb. pp. 26-30. Training Mag.com. 1999. https://www.trainingsupersite.com/publications/archive/training/1999/911/91 1SB4.htm Tulgan, B. 2000. Managing Generation X: How to bring out the best in young talent. New York: W.W. Norton and Co. Wagshal, K. 1997. I became clueless teaching the Gen Xers: Redefining the profile ofthe adult learner. Adult Learning, vol. 8, no. 4, pp. 21-25. Wyld, D. 1994. The 13th generation and its revolutionary definition of "career." Journal ofCareerPlanning & Employment, vol. 55, no. 1, pp. 26-28. Zemke, R., C. Raines, and B. Filipczak. 2000. Generations at Work: Managing the Clash ofVeterans, Boomers, Xers, and Nexters in Your Work place. New York: American Management Association. AN OVERVIEW OF THE EVALUATION PROCESS FOR MINE TRAINERS By Launa Mallett1 and Dana Reinke2 13 ABSTRACT Interest in training evaluation in the mining industry was piqued with the promulgation of Title 30, Part 46, of the Code of Federal Regulations. Under Part 46.3, a training plan is considered to be approved if it contains certain information, including "The evaluation procedures used to determine the effectiveness of training." The present paper is a broad overview of training evaluation and is intended to give trainers and decision-makers a framework for planning or assessing training evaluation strategies. It discusses questions to consider when starting an evaluation plan, Kirkpatrick's model of evaluation categories, and various ways to collect data. It does not provide detailed instructions on how to develop evaluation methodologies, but reviews topics to consider when creating an evaluation plan. Training evaluation is a term that has many different meanings: assessing the quality of a course, effectiveness of materials used, teaching style of an instructor, or the comfort of a classroom. An evaluation can be done informally over lunch or with highly structured data-gathering tools. It can produce results that are useful to trainers, program administrators, corporate decision-makers, or no one. The key to a worthwhile evaluation is clearly defining why the evaluation is being conducted. Once the purpose is defined, planning the evaluation strategy can begin. For an evaluation to be effective, it should be incorporated into the development of the training activity itself. The training evaluation worksheet at the end of this paper can be used to guide evaluation planning. EVALUATION PLANNING Planning a Training Evaluation 1. Who is interested in the evaluation results? 2. What questions do they want answered? 3. What resources are available? The first step to developing an evaluation strategy is de termining who will be interested in the results. Assessment of a typical mine safety or health training course could be important to the trainer, company managers, a labor organization, state or federal government agencies, or others. An evaluation designed only to meet the Mine Safety and Health Administration's (MSHA) information needs may not contain results that com pany managers could use for future planning. One developed on the basis of a request from company managers may not help a trainer decide if the course is working. It usually isn't practical to gather enough information to meet everyone's information Sociologist. Sociologist, Pittsburgh Research Laboratory, National Institute for Occu pational Safety and Health, Pittsburgh, PA. needs, so the target audience for the evaluation must be clearly defined. After identifying who will use the evaluation results, the next step is to determine what the interested parties hope to learn from the evaluation. The kinds of decisions to be made based on the results of the evaluation should guide the evaluation design. Is information wanted about the quality of the instructor, the usefulness of training materials, the appropriateness of the topics covered, the achievement of the students, or something else? How the results will be used should also be considered. Will the course, the trainer, or the training location be changed on the basis of the evaluation findings? Will the findings be used to convince someone of the value of the training? An important determinant of the scope of training evaluation is the resources available. Some evaluation strategies are more resource intensive than others. The availability of personnel, time, dollars, equipment, and access to data will affect what can be accomplished (Dopyera and Pitone, 1987, p. 74). A balance between what would be interesting to know and what is practical to discover may have to be found. It is not resource efficient to gather and analyze more data than are required; neither is it productive to conduct an evaluation that fails to provide needed information. This is why it is important to define the reasons the evaluation is being conducted clearly before designing the strategy to be used. The questions in the box help focus the initial stages of evaluation planning. 14 KIRKPATRICK'S EVALUATION MODEL One way to think about what can be learned during an evaluation is to use the categories developed by Kirkpatrick. His four-level framework was first presented in a series of articles published around 1960 (Nichols, 2000) and go from level 1 (the easiest and least resource-intensive) to level 4 (the most difficult and expensive) (Kirkpatrick, 2001, pp. 122-132). Each level is described in table 1 and the text below. Over 40 years later, this framework is still being used to structure evaluations of training programs. "As we move from level 1 to level 4, the evaluation process becomes more difficult and time-consuming, although it pro vides information of increasingly significant results" (Clark, 1997). The questions the evaluation needs to answer and the resources available for the task should determine which levels will be included. Level 1: Trainee reactions are the easiest kind of assessment data to gather. This is not to say they are not important. If trainees do not see value in the training, they are not likely to translate the objectives of the course into useful knowledge and skills. When trainees find a course uninteresting, they will be less motivated to learn the material being covered. Furthermore, quality instruction will be wasted in a training environment that is not conducive to learning. A classroom that is too hot, cold, noisy, or small can defeat the purpose of the class before it be gins. While positive trainee reactions do not ensure that ob jectives are met, negative reactions guarantee a less-than-fullysuccessful transfer of knowledge and skills. Level 2: Measurements of learning are used to show whether trainees' knowledge and/or skills are changed by training. The best way to determine if changes are the result of specific train ing is to conduct an experiment in which the class is divided into two similar subgroups. Prior to training, both subgroups can be tested on the topics of interest either in writing or through observation. During this pretest, both groups should perform equally. Then only one of the subgroups is trained. After train ing, both subgroups are retested. If the trained group now per forms better than the untrained group, the training can be identified as the cause of the improvement. Often, however, it is not practical to leave a subgroup un trained, especially with regard to their safety and health. Less methodologically rigorous strategies can be used to assure that training is working. When the entire group is undergoing train ing, testing before and after the course can show any changes in knowledge or skills. While this cannot definitely prove that the change resulted from the training rather than some other external factor, if another reasonable explanation is not available to ac count for the improvement, the success of the training can be inferred. Level 3: Unlike levels 1 and 2, measuring a change in be havior must be done outside the classroom and with sufficient time elapsed for knowledge and skills to have been tried out in the workplace. The most elaborate plan for level 3 evaluation would include an untrained subgroup as described for level 2 and detailed testing of both subgroups in their workplaces before and after the training. This type of evaluation is resource intensive and isn't practical for all training sessions. But less intensive strategies can yield valuable results. [S]omething beats nothing, and I encourage trainers to at least do some evaluation of behavior, even if it isn't elaborate or scientific. Simply ask a few people: "Are you doing anything different on thejob because you attended the training program?" Ifthe answer is yes, ask, "Can you briefly describe what you are doing and how it is working out? If you are not doing anything different, can you tell me why? Did you learn anything that you can use on the job?" (Kirkpatrick, 2001, p. 128). Another strategy is to talk with the trainees' supervisors about any behavioral changes they have observed since the training was completed. Level 3 evaluation can be difficult be cause it must be conducted months after the training has been completed. This highlights the importance of planning an eval uation strategy when planning the course. Time must be scheduled for the follow-up level 3 data collection so it won't interfere with future training activities and projects. Level 4: Determining how training affects the organization is the most difficult evaluation to perform. Level 4 evaluations should be conducted when the value of the training or the training program to the overall organization needs to be assessed. A relatively simple example is measuring changes in sales numbers after training salespeople in a new skill. An increase in sales can be compared to the cost of the training and a bottom-line return on investment calculated. Unfortunately, many topics aren't that easy to quantify. Even the relatively simple sales example can be complicated by a number of other variables. Unless a control group is used, as discussed above, the economics of the region, the introduction of a competing product, or the end of a fad could distort the data to an extent that the impact of training is difficult to calculate. Level 1 - Reaction 2 - Learning 3 - Behavior 4 - Results Table 1.--Kirkpatrick's four-level evaluation scheme Measurement focus Trainees' perceptions Knowledge/skills gained Worksite implementation Impact on organization Questions addressed What did trainees think of this training? Was there an increase in knowledge or skill level? Is new knowledge/skill being used on the job? What effect did the training have on the organization? 15 To conduct a level 4 evaluation, it is important to define clearly the tangible results to be measured, such as a decrease in accident frequency, an increase in use of personal protective equipment, a reduction in maintenance costs, or an increase in production per shift. Once the desired result is identified, a means to measure changes is needed. Next, factors other than training that could influence the change should be identified so that they can be ruled out as the source of change, if possible. Finally, evidence that the training did cause the change being studied should be identified. "Most importantly, be satisfied with evidence, because proof is usually impossible to find" (Kirkpatrick, 2001, p. 129). GATHERING INFORMATION It is good to acquire data from a number of sources to obtain a more complete view of how the training is perceived and its impact. There are many kinds and sources of data that can be used to evaluate training. An obvious source is the individuals being trained. Useful information can also be obtained from the supervisors of these trainees. Content or training experts can be asked to review course materials, assist with test development, or critique a training session. Company documents can contain indicators of change, such as maintenance costs, accident frequencies, or number of grievances filed. The decisions made up to this point in the planning process guide the choice of data collection methods. Data can be ob tained in a number of different ways, and table 2 lists some commonly used techniques and data sources. Table 2 also describes when these methods would be used and what can be learned from each technique. To be used successfully, each of the data collection methods listed in table 2 requires knowledge about its development and im plementation. A more experienced program developer may be needed to assist a trainer with the integration of a particularmethod into a training course. Some methods, such as interpreting body language, require astute observational skills. Others, such as surveys and tests, do not require formal training to administer, but may require an experienced developer to construct questions that thoroughly address the training program and its effectiveness. The knowledge and skills ofevaluation plan developers, trainers, and/or other observers who will gather the data must be considered when determining the best method or methods for gathering evaluation data. Method Questionnaire....................................... Interviews............................................. Facial expressions/body language . . . Performance tests .............................. Written tests......................................... Workplace observations...................... Games .................................................. Group discussion ................................ Analysis of statistics............................ Table 2.--Training methods When used Before, during, or after training What can be learned Perceptions of trainees or supervisors Opinions of content or training experts Knowledge or skills Transfer of training to job Organizational impact Before, during, or after training Perceptions of trainees or supervisors Opinions of content or training experts Knowledge or skills Transfer of training to job Organizational impact During training Perceptions of trainees Before, during, or after training Trainee skills Before, during, or after training Trainee knowledge Before or after training Trainee knowledge or skills Transfer of training to job During training Trainee knowledge or skills Before, during, or after training Perceptions of trainees or supervisors Opinions of content or training experts Trainee knowledge or skills Transfer of training to job Before or after training Organizational impact 16 WHAT TRAINING EVALUATION IS NOT If the only result of interest is whether or not each trainee knows certain things or has acquired certain skills, then an evaluation of training is not needed. In this situation, it doesn't matter how each individual obtained the knowledge or skill, and, therefore, the effectiveness of a specific training activity is not important. A knowledge and/or skills test can be administered to each person after training is completed to determine who is at an acceptable level of performance. A pretest is not needed unless there is an option that permits trainees to skip the training class if they can pass the test. When the question becomes how to train those who are not performing adequately so they can pass a future test, then evaluation of the training becomes important. SUMMARY The only way to determine whether or not training is of value is to evaluate it. When objectives for the training are clearly defined, an evaluation plan can be designed to measure the training's effectiveness at achieving those goals. Sometimes company managers or outside organizations require evaluation data to assess a training program. Even when such outside influences are not present, it is in the best interests of a trainer to gather evaluation data routinely to assess course content, delivery methods, and teaching skills. If a course is going to be repeated, evaluation can guide changes to improve future sessions. If the course will not be repeated, evaluation could focus on the skills of the instructor with results being used for professional development of that trainer. The important thing is to decide what can and should be learned during training evaluations and then design a strategy to meet that goal. REFERENCES Clark, D. 1997. Instructional system development: Evaluation phase. Chapter 6. http://www.nwlink.com/~donclark/hrd/sat6.html. Dopyera, J., and L. Pitone. 1987. Decision points in planning the evaluation oftraining. More Evaluating Training Programs: A Collection ofArticlesfrom Training and Development Journal, D.L. Kirkpatrick, comp. Alexandria, VA: American Society for Training and Development, pp. 74-77. Kirkpatrick, D. 2001. The four-level evaluation process. Ch. 12 in What Smart Trainers Know: The Secrets of Success from the World's Foremost Experts, L.L. Ukens, ed. San Francisco: Jossey-Bass/Pfeiffer, pp. 122-132. Nichols, F. 2000. Evaluating training: There is no "cookbook" approach. http//home.att.net/~nickols/evaluate.htm. 17 Training Evaluation Worksheet Use this worksheet as a guide to help you plan the evaluation of your next training session. Plan the Evaluation 1. Who will be interested in the results? 2. What questions will be answered? 3. What resources (personnel, time, money, equipment) are available for evaluating the training program? Gather the Information What method(s) will be used to gather information? Data Collection Method Performed (circle choices) Before, During or After Training Before, During or After Training Before, During or After Training Before, During or After Training 19 INNOVATIVE ALTERNATIVES TO TRADITIONAL CLASSROOM HEALTH AND SAFETY TRAINING By Michael J. Brnich,1 1Jr., R. Lincoln Derick,2 Launa Mallett,3 and Charles Vaught3 ABSTRACT After thoroughly covering the health and safety training subjects required under Title 30, Part 48, of the Code of Federal Regulations (CFR), mine operators often find themselves with little or no time left in the class schedule for other important topics. This paper discusses a technique developed in partnership with RAG Twentymile Coal Co., Oak Creek, CO, for incorporating employee participation at all levels in fire prevention and safe equipment operation training outside of traditional classrooms. The process involves development of training modules consisting of short, 5- to 7-minute videotapes coupled with toolbox talks that ground the content of the videos within the context of a miner's workplace. INTRODUCTION Title 30, Part 48, of the Code of Federal Regulations (CFR) re quires underground mine operators to cover no less than 10 health and safety topics as part of new hire and annual refresher training. While the mandated topics are obviously important, many instruc tors would like to cover one subject in more depth and review additional subjects during class. Fire prevention and preparedness is one topic that some trainers think they do not have sufficient time to discuss in adequate detail in the context of an 8-hour refresher class. RAG Twentymile Coal Co. in western Colorado recognized the need to review fire prevention and preparedness, as well as other important health and safety topics, outside of its classroom training schedule. To meet this need, Twentymile Coal settled on using video training modules. In developing this training, the company had the following goals: 1. To find a method to conduct this additional training outside traditional annual refresher sessions, 2. To develop customized materials featuring the mine and its unique attributes, 3. To utilize input from mine employees and feature mine per sonnel as much as possible in the videos, and 4. To design the training so that it could be used for experienced as well as inexperienced miners. Under a cooperative research and development agreement (CRADA), researchers from the Pittsburgh Research Laboratory of the National Institute for Occupational Safety and Health (NIOSH) collaborated with Twentymile personnel to develop a series of customized training modules on mine fire prevention and preparedness. This paper discusses the process used by Twentymile Coal, provides an example of one of the training modules, and presents data supporting the utility of this type of training. PROCESS The purposes of the joint CRADA were to-- 1. Assist Twentymile in improving employees ' awareness ofmine fire prevention and preparedness by covering topics not normally discussed in annual refresher training, 2. Make employees part of the fire prevention and preparedness system, 3. Raise employees' awareness of the fire brigade's role, and 4. Develop and assess the effectiveness of content-specific video training modules tailored to particular needs. 1Mining engineer, Pittsburgh Research Laboratory, National Institute for Occupational Safety and Health, Pittsburgh, PA. ^Technical safety manager, RAG Twentymile Coal Company, Oak Creek, CO. 3Sociologist, Pittsburgh Research Laboratory, National Institute for Occupational Safety and Health, Pittsburgh, PA. Goal 1. One way to improve an emergency preparedness sys tem is to enhance prevention. This simple idea is often overlooked while elaborate response plans are being developed. Safety personnel at Twentymile Coal Co. recognized that they had given 20 a lot of attention to their capability to respond to major emer gencies, specifically underground mine fires, but had tended to neglect prevention and the basics of first response. The overall goal of the program, therefore, became to prevent fire-related emergency situations from happening or, at the very least, to control the events before a major response effort was needed. In addition, Twentymile Coal management saw the program as an opportunity to improve miners' level of awareness about management's commitment to fire preparedness. Goal 2. The program was built on the underlying message that everyone at a worksite is responsible for emergency prevention and response. Thus, it was important to involve mine personnel at all levels in the process of developing and presenting training materials on fire prevention and preparedness. Recent research has focused on the involvement of employees in development, structuring, and presentation oftraining materials. In a study ofthe use of participatory training techniques, Miles (1992) concluded that involvement of workers in the training process increased both job skills of the participants and their overall job knowledge. Goal 3. At the mine, which employs about 300 workers, there are two specially trained teams of volunteers. One is the fire brigade and the other is the mine rescue team. While team members are included in the training, employees who are not part of the special teams are the main focus of the program. Goal 4. The training modules detail basic fire prevention and first-response tasks and introduce some activities that would only be performed under the direction of someone with special training. The expectation is that, after training, each worker will know his or her responsibility during an emergency and will also understand the overall mine emergency system. For this portion of their pre paredness, mine personnel and researchers from the Pittsburgh Research Laboratory worked together to develop a program fo cusing on the basics. Implementation of the program at the mine began in July of 1998 and was completed in early 2000. TRAINING MATERIALS The program was started during a week-long session in January 1998 and included personnel having knowledge about content topic, effective training techniques, and video production techniques. Four initial video training modules were developed: "Introduction to Fire Prevention and Preparedness," "Conducting a Fire Risk Assessment," "Fire Prevention," and "Fighting Fire with Water." NIOSH researchers collaborated with Twentymile personnel in creating the outlines and scripts for each video. The fire brigade members involved in this work assisted with script writing and were filmed presenting unscripted segments about their areas of expertise. They also assisted with development of the safety talk guides that accompany each video. A contractor was hired to shoot and edit the video footage to create the final videos. Following development ofthe first four training video modules, four additional modules were created covering other topics con cerned with fire prevention and preparedness: "The Foam Gene rator," "Fire Suppression Systems," "Responding to a Fire: Fire Fighting and Evacuation," and "Using and Maintaining Fire Ex tinguishers." As with the previous video modules, mine personnel and NIOSH researchers provided content and training expertise. Mine fire brigade members provided assistance with script evalua tion and filming of various video clips. For the final videos, all footage was captured by an in-house videotographer from NIOSH. The safety talk guides are a key component of the fire pre vention and response basics program. While the videos briefly introduce topics and touch on concepts that everyone at the mine should know, the safety talks take the same topics and relate them to specific work locations. For example, one video discusses the equipment needed to fight fire with water. It includes information such as types of hoses and nozzles used to fight fires and where they are stored underground. The associated safety talk focuses on the equipment used to fight fire with water and where available in a given work area. After this talk, the employees should know what equipment is available near their work area and where that equipment is located. The videos introduce a subject and bring it to the attention of the employees. The safety talks relate the topic to specific work locations and provide a forum for questions and concerns. In all, five safety talk guides were authored for the first four video modules, including two guides for the module "Fire Prevention." The pairing of videos and safety talks allows material to be introduced to large groups and then targeted to small groups so that neither training segment takes much time. At the mine, the videos were presented as part of routine monthly production meetings. Every month or two a video would be shown during the preshift meeting. The safety talk guides were given to supervisors who already had the responsibility to provide such training on a regular basis. The length ofthe talks would vary by presenter and audience participation, but were designed to take 5 to 20 minutes. With this method, training was incorporated into the daily routine, ratherthan being put into special training classes thatrequired employees to be away from their jobs for hours or even days. AN EXAMPLE OF A TRAINING MODULE: EMERGENCY COMMUNICATIONS As mentioned earlier, two safety talks were created for the training module "Fire Prevention." One dealt with general issues of fire prevention and preparedness, including housekeeping and the mine monitoring system, while the other, "Mine Emergency Communication Using the Communication Triangle," focused on the content of warning messages. Research has shown that when 21 an emergency occurs, people often do not get the information they need to enable them to take appropriate action (Mallett et al., 1993, 1998). This safety training module presents a procedure in which mental cues can be used by senders and receivers of warning messages (Mallett et al., 1999). The safety talk was originally developed and field tested with Canterbury Coal Co. in western Pennsylvania. During the talk, miners learned about the six categories of critical information that should be provided during emergency communications. These are Who, Where, What, Miners, Event, and Response. Below is an explanation of these six communication categories. WHO. When reporting an emergency or receiving a warning, the first thing a miner must do is to identify him- or herself. This is important because people react differently depending on who gives them information. If a warning is received from an unknown person, the typical response is to try to gather more information before acting. Thus, significant time can be lost. WHERE. Telling or finding out where the problem is located is important. This may seem like common sense, but doesn't always happen. Forty-eight miners were interviewed and asked about their experiences following three mine fires that forced the evacuation ofmore than 60 miners through smoke. Only two ofthe 48 had known where the fire was as they were escaping, even though this information was known by either the dispatcher or the person who discovered the fire. As a result, miners had to make decisions about escape routes without knowing where the source of the fire lay. This lack ofknowledge also increased the stress on the miners because they didn't know how far they would have to walk to find fresh air. WHAT. Miners must tell or ask exactly what is happening. Again, this may seem like common sense, but such information is not always provided in an emergency. For example, during one serious mine fire, a warning was given for everyone on the section to evacuate. Miners who had been near the phone when the call came in went to gather others of their crew. One of these miners, a shuttle car operator, ducked under the check curtain and yelled to the miner operator, "Come on down to the mantrip! We're going out! " Since the belt was down and it was close to quitting time, the miner operator and his helper thought they were just leaving the section a little early. They went through their normal end-of-shift routine, including backing the continuous miner out of the cut, setting jacks, tightening check curtains, and disconnecting the power before reporting to the mantrip. Valuable time was lost. After providing or obtaining these three initial pieces of critical information, miners can then provide or obtain details about the response in progress. MINERS. Is anyone hurt? Has everyone been accounted for? When and where was a missing person last seen? EVENT. Will this problem require a first-aid kit or an ambulance? Should mine rescue teams be called or will just a couple of fire extinguishers do? RESPONSE. What's been done so far? How many people are on the scene? What equipment is on the scene? EVALUATION All training materials must be evaluated as to their effectiveness in teaching specific content. In conjunction with showing the first video, a questionnaire (see appendix) was given to all employees to assess their level of knowledge of and awareness about fire prevention and response, including the content of emergency warning messages. An identical follow-up questionnaire was administered in April 1999 after completion of the fourth training package. COMMUNICATION TRIANGLE Table 1 summarizes the percentage of miners identifying each ofthe six emergency communication elements on both the pre- and post-test questionnaires. In all categories, miners exhibited marked improvement. The data suggest this module was extremely useful as a teaching tool for improving emergency communication skills. On both the pre- and post-training questionnaires, trainees were asked to list three pieces of information that should be communicated in a fire warning message. Data were coded to place trainees' responses into the six categories discussed above. On the pre-test, 63% of the miners mentioned at least one of the six information categories. On the post-test, this number rose to 77.5%. As figure 1 illustrates, analysis of the data showed that more than three times as many miners (43%) were able to name three pieces of information following the training session than before the session (13%). Figure 1.-Percentage of miners identifying at least three com ponents of what information should be passed along in case of a fire or other emergency. 22 In addition, 14% of the miners on the pretest said they wanted information on what escape route to take in the event of a fire. This was seen as unrealistic by company safety staff. This number dropped to 8.6% on the post-test. Table 1.--Percentage of miners identifying each of the emergency communication elements ElementPre-testPost-test Who ................................... 1.3 25.8 Where................................ 61.0 75.8 What................................... 2.5 23.0 Miners................................ 7.6 10.7 Event................................... 29.7 32.0 Response ............................................. 14.823.0 DEMOGRAPHICS There was little change in the demographics ofminers between the administration of pre- and post-training questionnaires. Fewer miners completed the post-training questionnaire, but this decrease can be explained by noting that (1) no summer employees were working when the post-training questionnaire was given and (2) a number of miners were on vacation at the time because their children were home for spring vacation. Tables 2 and 3 present the basic demographic information. For purposes of data analysis, miners were grouped into two categories: age (less than 30 and 30 and older) and experience at Twentymile (2 or less years and over 2 years). Table 2.--Miner demographic data, years Before training . . After training . . Number Average age 236 37.1 Average years of mining experience 12.3 Average years of experience at Twentymile 5.3 178 36.7 12.2 5.6 Table 3.--Age and experience ranges for Twentymile miners, percent Age (1-29).......................... . Age (30 and over) ............. . Experience (0-2 yrs)........... . Experience (over 2 yrs) . . . . Before training 30.4 69.6 30.1 69.9 After training 31 69 20.4 79.6 OTHER FIRE PREVENTION AND PREPAREDNESS ISSUES On both questionnaires, miners were asked a series of questions to assess their awareness of fire prevention and preparedness at Twentymile. These included questions about the mine's general level of preparedness, the likelihood of a fire at the mine, knowledge of various fire prevention and preparedness activities, knowledge offire-fighting supplies and their location, and possible fire prevention strategies. Level ofFire Preparedness. Workers could select from "Legal requirements met," "More is done than required by law," and "Not all state/federal requirements met." Data presented in table 4 shows the percentage of change in responses between the pre- and post-tests, broken down by age and experience at Twentymile. Following the training, a greater percentage of younger, lessexperienced miners felt more is done than required by law compared with this same group before training. A slight increase in awareness was also seen for older, experienced miners. For all miners, 59.1% on the pre-test felt more was done compared to 63.4% on the post-test. Table 4.--Percentage of change between pre- and post-tests according to age and year of experience at Twentymile Level of preparedness _______Age_______ ______ Experience Under 30 and 0-2 Over 2 30ugyearsyears Law met................... -11.8 -2.2 -18.1 -1.4 More is done ............. 12.2 1.9 12.2 4.3 Law not met............... 1.0 0.35.71.7 LikelihoodofFire Occurring. Miners were asked how they felt about the likelihood of a fire on Twentymile property, underground at Twentymile, and in their work area. As table 5 illustrates, fewer miners in the post-test believed fire was likely compared with the pre-test. This shift might indicate that, since learning about Twentymile's efforts in fire prevention and preparedness, miners believe that a fire is less likely to occur. Interestingly, nearly onehalf of miners in both the pre- and post-test indicated that fire was not likely in their work area. This may reflect the sentiment that "It won't happen to me." Table 5.--Percentage of change between pre- and post-tests regarding the likelihood of a fire Location On Twentymile property............. Underground at Twentymile .... In your work area................... Overall -9.2 Age Years of experience at Twentymile Under 30 and 0-2 years Over 2 30 over________________years -10.4 -8.3 -7.3 -11.2 -5.8 -5.8 -7.5 -11.2 -7.1 -1.7 -10.5 -0.2 -2.3 -5.0 Similar trends were seen when the data were analyzed by workers' age and level of experience at Twentymile. Awareness ofVarious Twentymile Fire Prevention Activities and Programs. On both the pre- and post-test questionnaires, miners were asked about their awareness of the fire brigade, the mine rescue team, the mock drill at Empire Mine, smoke training, fire prevention training, and fire response training. The analysis suggested that awareness of fire prevention and preparedness at Twentymile increased, especially among less-experienced miners. Prior to training, nearly 77% ofthe workers said they knew about the fire brigade. After training, 85% reported knowledge of the fire brigade. Similarly, fewer miners on the post-test question naire gave "don't know about" responses in the remaining activity categories when compared to the pre-test responses. 23 FIREBOSS INTERVIEWS SUPERVISOR INTERVIEWS Besides administering the post-training questionnaire to miners, NIOSH researchers interviewed firebosses to obtain their views on the program and to determine if they had seen differences in fire preparedness and prevention since the start of the program. Overall, the firebosses felt that the training was good and that it generated discussion and increased aware ness among workers. Firebosses indicated overall housekeep ing at the mine was better since the start of the program. They felt sections and outby crosscuts were cleaner. The firebosses also said that rock dusting throughout the mine was much better. While the number of frozen or bad belt rollers had not changed, the firebosses felt response time for changing out rollers was better. They reported seeing no difference in belt alignment problems, but this item had not been specifically targeted in the training. Finally, firebosses said they had seen fewer bad fire extinguishers since the program started. In additionto interviewing Twentymile's firebosses, researchers also spoke with many of the supervisors who administered the safety talks to crews. Overall, supervisors felt the training modules and safety talks were good and covered topics not typically reviewed as part ofregular training. Many liked the fact the videos showed Twentymile Mine and included its own personnel. Most foremen felt the safety talk guides were easy to use and were about the right length for holding trainees' attention. One ex ception was the talk on the communication triangle, which most supervisors believed was too long. (This talk was originally devel oped for another purpose.) In general, foremen reported their crews talked about fire hazards after the safety talks. Some foremen said they had seen improvements in fire hazard awareness among crews. Several also stated that their crews were more aware of the correct placement and installation of water drops for fire fighting. Finally, most foremen said crews were doing a betterjob ofhousekeeping and were more aware ofbelt lines, hot rollers, and accumulations of material. LESSONS LEARNED ID CONSIDERATIONS From this collaborative effort in developing site-specific train ing modules, researchers and Twentymile Coal safety personnel obtained valuable information and identified several issues to be considered. Fire brigade members were willing and able to convey what they had learned in their special training. They knew what key points should be covered and were able to talk through the topics while being videotaped. Using them to help develop scripts and be "actors" greatly improved the return on investments in their fire-fighting training. For the program to be successful, Twentymile Coal believed that it was paramount for management to buy into the process and convey this commitment. Production managers, including the mine manager, general mine foreman, and the longwall coor dinator, were asked to give introductory remarks on the videos. Each manager was given a prepared script for a single video. They all agreed to present remarks, but were not comfortable with the task. It was felt they should have received the scripts sooner or should have been asked to introduce the videos in their own words. Producing training videos in a production environment requires maximum flexibility. Frequently, the person needed for a given segment is not available or a location can not be used as planned. Schedules change constantly, and alternate plans should always be prepared. Scripts should be written in advance of shooting footage whenever possible. The scripts will then guide what is to be shot. This is particularly important if a contract videographer is to be employed. Safety professionals, production managers, trained response personnel, and other employees all have ideas about emergency prevention and response. The issues important to each group can vary greatly. It is by gathering the concerns and solutions from all these groups that the most complete package can be created. CONCLUSION From the data, it is evident the video/safety talk modules have improved Twentymile employees' awareness of fire hazards, fire prevention, and fire preparedness. The prevention and response program was designed to target a given worksite and a specific hazard. The basic concepts and methods can be adapted to other companies and/or hazards. Twentymile Coal Co. has begun to ex pand the use of video training modules for other important safety topics, including proper pre-shift inspections on diesel scoops and roof bolters. The unique aspect of these types of training modules are that they are site specific. The content experts and video locations come from the chosen site. The local experts choose the targeted hazard and develop the content of the training. Outside con sultants may be required to obtain the video footage. The end result, however, is a unique training package that meets targeted needs, but that cannot be obtained off the shelf. 24 REFERENCES Mallett, L., M.J. Brnich, and C. and Vaught. 1998. Training to improve emergency communications skills. In: Proceedings of The International EmergencyManagement Society Conference (TIEMS `98) (May 19-22, 1998). Pp. 471-476. Mallett, L., C. Vaught, and M.J. Brnich. 1993. Sociotechnical communi cation in an underground mine fire: A study of warning messages during an emergency evacuation. Safety Science, 16, pp. 709-728. Mallett, L., C. Vaught, and M.J. Brnich. 1999. The emergency commun ication triangle. NIOSH Pub. No. 99-157. Miles, K.K. 1992. Use of participatory training techniques in right-to-know train-the-trainer course for New Jersey public employees. American Journal of Industrial Medicine, vol. 55, no. 3, pp. 36-40. 25 APPENDIX Fire Protection Questionnaire - Twentymile Coal Company 1. How likely is a fire to start in the following locations? (Circle the best answers.) On Twentymile property Very likely Likely Not likely Underground at Twentymile Very likely Likely Not likely In your work area Very likely Likely Not likely Not at all likely Not at all likely Not at all likely 2. Which of the following best describes Twentymile's level of fire protection activities? (Circle the best answer.) A. Not all state and federal requirements are followed. B. Legal requirements are met. C. More is done than is required by law. What do you know about the following Twentymile activities/programs? (Circle the best answers.) Fire brigade Participated in Know about Don't know about Mine rescue team Participated in Heard about Don't know about Mock emergency drill at Empire Participated in Heard about Don't know about Training in artificial smoke Participated in Heard about Don't know about Training related to fire prevention Participated in Heard about Don't know about Training related to fire response Participated in Heard about Don't know about 4. Please list three things that should be included in a fire warning message. 5 Please list three fire-fighting supplies found in your work area. 6. Please list three ways you can help prevent fires at Twentymile. Please answer the following questions for a study being done by Pittsburgh Research Laboratory. 7. Job title: ___________________________________________________________________________ 8. Age 9. Years mining experience 10. Years at Twentymile 11. Circle your direct employer: Twentymile Contractor CONSIDERATIONS IN TRAINING ON-THE-JOB TRAINERS By Bill Wiehagen,1 Don Conrad,*2 Tom Friend,3 and Lynn Rethi4 27 ABSTRACT On-the-job training (OJT) is a very common method of teaching job skills. Much of the training is done by experienced workers. This paper discusses small investments that can help improve the effectiveness of OJT. Effective OJT involves some structure and planning in the transfer of responsibility for task performance from the trainer to the novice. Assisting the OJT trainer could involve helping develop up-to-date job analyses and offering strategies for teaching and evaluating job skills. The scope ofthis paper addresses (1) organizational considerations supporting structured OJT, (2) general strategies for structuring OJT, (3) a typical approach for conducting OJT, (4) characteristics and duties of OJT trainers, and (5) limitations of OJT as a teaching method. Health, safety, and operational skills can be connected through training. Haul truck operator training is used to illustrate some of the concepts discussed in this paper. BACKGROUND AND SCOPE On-the-job training (OJT) is a very common method of teaching workers essential skills so they can perform a job safely and productively. OJT is often considered informal training, and across industries, organizations invest significantly more money in informal training than they do in formal training. Some researchers (Carnevale and Gainer, 1989) estimate the ratio to be from 1:3 to 1:6. That is, for every dollar invested in the classroom, 3 to 6 dollars are invested in informal training in the workplace. There is a range of activities defining what people refer to as on-the-job training. OJT might be a situation where workers essentially train themselves, that is, they watch someone do a job and rely on co-workers to show them the ropes. This is sometimes referred to as "following Joe around." We call this type of training on-the-job experience, or "unplanned OJT." This training has little structure (that is, no written plan or job analysis) and is almost always done by someone who has experience in the task. Sometimes it works well. Success in using unplanned OJT is usually dependent on the luck of the draw, that is, whether the informal trainer is competent at the task he or she is teaching, is motivated to teach, can organize the job into logical components, and knows something about good practices in teaching and evaluating. At the other end of the spectrum are more structured OJT strategies (Rothwell and Kazanas, 1990). Structured OJT is 'Industrial engineer, Pittsburgh Research Laboratory, National Institute for Occupational Safety and Health, Pittsburgh, PA. 2Training specialist, Mine Safety and Health Administration, Johnstown, PA. 'Consultant, Princeton, WV. 4General engineer, National Personal Protective Technology Laboratory, National Institute for Occupational Safety and Health, Pittsburgh, PA. useful if organizations want to increase the odds of workers learning to perform new jobs more effectively and quickly. This form of training involves a plan5 and is useful when the fol lowing considerations are present. Because mining technology is increasingly expensive and complex, decision makers may give more thought to how workers use and should use the technology. Training can work to enhance the fit between technology and how it is used. The riskier the job, the more training should be considered as a way to reduce risk or accelerate experience. Risk involves not only injury, but production downtime and unexpected maintenance. When hiring new workers or when workers rotate through several jobs, organizations may want to consider ways of ac celerating the learning curve and bringing new task performers up to speed quickly, as opposed to letting workers learn as they go (Rothwell and Kazanas, 1990). Organizations notice large, obvious levels of variability in task performance, and they are generally not happy with the consequences of that variability. Variability in procedures and 'Without a plan or structure, OJT is very informal and is often referred to as trial and error, learning by osmosis, or learning by experience. Informal OJT appears to save money early on in that (1) a plan does not have to be developed, (2) no time is invested in a job analysis, and (3) the trainer (or an experienced worker or supervisor) is doing other things. If an experienced worker or supervisor is instructing informally, they are teaching from memory and individual experience. For many mining jobs, the downstream costs ofinformal OJT can be quite high. Added costs can involve a greater risk of injury, additional downtime, and higher levels of property damage and machine maintenance. Money saved early can very likely result in greater risk and more money spent down the road. 28 how decisions are made on a job may affect an individual worker, a work crew, and/or the organization itself. It can affect the amount of time it takes to complete a task, the risk of injury, and product quality, and can likely affect costs (extra downtime, maintenance) connected with the job. Planned OJT can help reduce variability6 in how a task is initially performed. It gets workers on the same page for critical tasks. For tasks that are less critical, variability is less important (see Wiehagen et al., 1996; Lineberry and Wiehagen, 1996). ON-THE-JOB TRAINING IS PRACTICAL OJT is practical as productive work is accomplished while a worker is learning.7 With OJT, managers do not have to be concerned with training transfer. Transfer is the application of skills acquired in training to the worksite. 6Obviously, other factors outside of initial job training can affect long-term performance oftasks and quality output by individuals and work teams. 7Supervisors should understand that learning is the goal, not production. Why is this so? With OJT, the jobsite and the task being learned and performed are 100% real. Learning is doing. The re sults are evidenced by the work itself. Accomplishments are visible. PLANNED ON-THE-JOB TRAINING ACCELERATES LEARNING How organizations plan and conduct OJT is essential. A worker can take a long time to learn a task by trial and error. To reduce this time, decision-makers can examine ways of ac celerating the learning curve, especially for those who are new to a task. Accelerated learning means less risk and fewer costly mistakes that result in serious injury, production downtime, or increased levels of maintenance. Structured jobsite training accelerates the learning curve better and faster than traditional, more casual OJT approaches (see figure 1). Figure 1 shows time and proficiency as key variables--how to reduce the time it takes to learn a set of skills to an acceptable level. Also Figure 1.-Learning curves show proficiency versus time for structured on-the-job training versus on-the-job experience. 29 addressed is the value of examining the variability in the per formance ofcritical tasks that comprise the job.8 Structured OJT does not necessarily require a large investment of time and money, but it does take some careful thinking--that is, a plan. OJT does not require volumes of paper, but it does require (1) that a trainer be present, (2) that someone needs to learn a task, and (3) that something is written down, whether it is a guide or a checklist, to help organize what is taught and how skills will be assessed. Teaching with evaluation is a key component of structured OJT as feedback can be offered and questions addressed as the trainee is performing the task. If experienced workers are trained as OJT trainers, structured OJT can make good use of their knowledge and experience to help accelerate skills of those new to a task. Accelerated learning via structured OJT should reduce business risks and the unplanned costs associated with injuries, maintenance, and production downtime as more experienced miners retire and new employees are hired. The key issues are skill development, how to accelerate learning, and the amount of time it takes a worker to develop a skill to an acceptable level.9 This paper offers some considerations on making OJT more effective. Models are offered to provide considerations in the structure of OJT programs. The appendix highlights definitions of common terms (for example, on-the-job training, on-the-jobexperience), while the table provides critiques of OJT, military OJT, and formal apprenticeship programs. ORGANIZATIONAL CONSIDERATIONS OJT involves structure--a plan. Providing quality training is a way of developing and keeping good people. At the manage ment level, structured OJT only involves three steps. 1. Some thought should be paid as to who would be a good OJT trainer or coach for a particular job or task and how these people might be identified or selected to teach. 2. The selected person should be given some help. That help should come in two forms: A job analysis or written guide of what the task entails and Information about good practices in teaching and eval uating in an OJT environment. 3. Time should be allocated for the OJT trainer to teach and evaluate safe production skills for the specific task. In short, OJT provides the opportunity to accomplish objec tives important to organizations, whether the skill involves oper ating a haul truck, maintaining a conveyor belt, or supervising a work crew. In some organizations, OJT can become so ingrained that it is difficult, over time, to pick out the trainees from the train ers. Both teach and learn from each other. They solve problems that benefit the organization, their work crews, and themselves. Good teachers learn from their students and become better per formers and teachers. The transfer of knowledge is not one way. STRUCTURED ON-THE-JOB TRAINING: A GENERAL STRATEGY Implementing structured OJT involves three basic steps: assessment-training-evaluation. 1. Assessment: Finding out what employees already know about the job. T Decide how much of the job content is already known by the trainee.10 T Decide what needs to be taught. 2. Training: Providing the knowledge and procedural and motor skills for the job. T Teach the employee and fill in his or her gaps of knowledge and skills. Training should be based on a job analysis. The analysis should include aspects of safety, health, and production and address relevant aspects of maintenance and crew coordination. 3. Evaluation: Assuring that employees can perform the task. T Find out if the skills taught have been learned. Evaluation is follow-up111 to check if an individual's performance is inside or outside the performance envelope. "Envelopes" can differ in size; some are more open and flexible, others are more rigid. Often, this depends on conditions and decisions at the worksite. 8Reduced variability provides a tighter envelope ofperformance. Structured OJT coupled with coaching, or peer discussions, can allow workers to reduce variability in performing tasks, thus move well beyond minimum levels of acceptable job performance. High levels of variability in the performance of critical tasks can increase the risk of injury, production downtime, and maintenance. Variability is natural within a work system, that is, it will seldom equal zero. Unstructured OJT encourages large levels of variability in performing tasks. 9Trainee and employee learning should continue well after training has been completed. The organization may want to allocate time for follow-up or coaching after the initial OJT is completed. 10As new technology is introduced, it is important to reassess what experi enced workers know. 11Some skills are easily taught, others take more time and practice. Some consider training evaluation to be a progress or final test. However, in OJT, evaluation often involves continuous and casual observation. This is normal--evaluation is not necessarily a separate event. 30 CHARACTERISTICS AND PROCEDURES FOR STRUCTURED ON-THE-JOB TRAINING OJT is finite.12 It has a beginning and an end. It is task or job specific. It deals with minimum acceptable levels of perform ance that are often based on the expert judgment of an OJT trainer. Structured OJT requires the presence of a trainer to teach and assess skills. It also involves a written guide that breaks the job into tasks. This is typically called a job analysis, and it is an im portant prerequisite for structured training. A job analysis provides an orderly framework for teaching, learning, and evaluating. Some refer to a job analysis as an operating pro cedure, a job safety analysis, or a standard operating procedure. Many supervisors and production planners already do job analyses. Without a job analysis, OJT would be very limited be cause trainers13 would be teaching from memory and their own experience of how a job should be done. A job analysis offers a common framework for both the trainer and trainee. The job analysis should make sense to experienced task per formers and others that have a stake in the task. In a haul truck operation, experienced operators, production supervisors, and mechanics and repairmen who service the trucks could offer key input to a job analysis. They see the job from different per spectives and can offer insights. Job analyses are not all alike. Some are very meticulous and detailed (see Morris et al., 1982), and some are less analytic (see Hartley, 1999; Krupp and Applegate, 1983). The level of detail is normally related to the sophistication of training decisions.14 Regardless of the level of analysis, all job analyses should make the job of teaching accurate, logical, and easy. They provide a road map for teaching and evaluating. Because of the importance of job analyses in conducting quality job-specific training, the Mine Safety and Health Administration (MSHA) is leading an effort to develop and test a practical procedure for conducting on-site job analyses. The model, adapted from the Navy, addresses the duties and responsibilities of off-road haul truck operation. The assessment-training-evaluation model is intuitive and has been applied to OJT for some time, evolving into a generic training model with four steps: Preparation, Presentation, Ap plication, and Follow-up (Wilson et al., 1980). 1. Prepare: Put the learner at ease, find out what he or she al ready knows, and get the them interested in the job. Assessment is assumed to be part of preparation, that is, to determine if training is warranted and why. In other words, Is training nec essary? Is it important? (See Mager, 1999.) 2. Present: Tell, show, and illustrate one step at a time. Stress each key point, instruct clearly, completely, and patiently. 3. Apply: Have the learner do the job and make observations. Allow time for practice and look for opportunities to have the trainee explain key points. Have the person do the job until you know it has been learned. 4. Follow-up: Put the trainee on his or her own, check frequently, praise good work, re-instruct to correct poor work. This four-step OJT method is common and TRADITIONAL. It seems to fit pretty well into the three-step (assess-trainevaluate) general training model discussed above. CHARACTERISTICS OF GOOD TRAINERS OJT trainers must have competence in the task they are teaching. They also must like to teach or want to learn how. In the literature on peer training (also called tutoring), Fitch and Semb (1993) developed a simple model: ASK, which stands for attitude, skill, and knowledge. Attitude: Friendliness and desire to help others. Approach able trainers know how to encourage and invite questions and have good interpersonal skills. Some believe that, at the lowest level, teachers must prefer teaching to not teaching (see Fitch and Semb, 1993). Beyond that, they assert that a positive attitude can be trained. Skills: Good communication skills. An effective teacher coaches students to learn the material for themselves. Good 12However, on-the-job learning is continuous. 13The experienced person, now a trainer, may know the job so well that he or she will skip steps or miss presenting important information. 14If simulators are to be built, it is important to perform a thorough task analysis, that is, document duties, tasks, and elements and identify specific cues (for example, visual, audio, tactile, proprioceptive) used by workers to perform the task. See Morris et al., 1982. communication skills begin with listening. An OJT trainer does not do all the talking, but often listens and observes. This demonstrates patience. Early in the interaction, it is useful to assess what is known by asking questions and observing. Semb and his colleagues suggest that a common mistake made by tutors is that they are too quick to jump in; they lecture students before they listen. Peer trainers need to reinforce appropriate performance by providing knowledge of results and reinforcing key issues. Positive feedback is almost always better than negative. Knowledge: Peer trainers must know quite a bit about the task or job they are teaching. For many jobs, acceptable per formance includes an array of skills--cognitive, perceptual, pro cedural, and motor. DUTIES OF TRAINERS One plan for training OJT trainers would be based on an analysis of the training task--to assess, train, and evaluate. 31 Assess: Assessment is typically done in the form of questions to be asked of the trainee or observations to see what the trainee can do. It is a good idea to put the trainee at ease by having a relaxed atmosphere that helps the person feel comfortable. Training is not a big deal. If you want training to be accepted and valued, make it commonplace--just another day at the job. Asking questions is a way of engaging the trainee as an active participant in the training. It also gives the trainee the chance to verbalize responses. Not only does the trainer derive some idea of what that person knows, but the trainee learns that he or she is expected to participate. Asking questions also indicates to the trainee that the trainer is interested in teaching (Semb et al., 2000). Assessment concludes with some notion of a gap. Is there a gap in the safe, productive performance of the task or job for which the trainee is being trained? Will training help? Will further assessment help?15 Train: The notion of training in OJT is to shift task per formance from the trainer to the trainee. The key word is per formance. There are a couple of different approaches discussed in the literature. The first is trainer-centered; the other is learnercentered. A trainer-centered approach is what most of us experienced in school where the teacher takes on the responsibility for learning and does most of the talking. The student takes a more passive role. The trainee has little experience; the teacher is the specialist who must convey a body of knowledge via lectures, books, and videos. Students are motivated to learn externally because they have to pass a test. The teacher covers the content to be learned so the student gets the required information in some logical order. Motivation is controlled by the teacher via grades or other types of feedback (see Lawson, 1997). A learner-centered approach is one in which the trainee accepts responsibility for learning, that is, responsibility for learning is shifted from the teacher to the trainee. The trainee is an active participant in the training, asking questions and verbalizing responses to questions asked by the instructor (see Semb et al., 2000). The trainer, however, might listen more than he or she talks. The goal for OJT is the eventual transfer of responsibility for task performance from the trainer to the novice. It is a dynamic process where evaluation is continuous. This on-going evalua tion results in some final assessment of the trainee's per formance, either inside or outside the envelope. It is possible and likely for training to encompass both of these general processes. However, the learner-centered approach is thought to be more suitable for OJT (Lawson, 1997; Semb et al., 2000), since it requires active participation. With OJT, the work itself, not a grade, provides motivation.16 Many believe 15Ergonomics is the science of designing the work to fit the worker. Some jobs are so difficult and physically demanding that training will be of little help in reducing injury risk, improving product quality, or reducing downtime. Ergonomics and training can blend together to enhance the work process. 16Other motivators include achieving higher skill levels and higher pay. that the abilities and motivation of the teacher/trainer makes a considerable difference in learning. Evaluate: When we think of evaluation under traditional teacher-centered training, we often think of a test--The final exam. This is not the sole purpose of evaluation. However, tests that are well designed can help teach; they can provide moti vation for learning and can trigger questions and discussion. They help provide feedback to the employee. In OJT, the idea is to develop knowledge and skills, thus written or oral tests measure only part of the learning. As the trainee performs the task--operating a truck, for example--he or she is putting themselves to the test under the guidance of an instructor. In a very practical sense, evaluation becomes continuous.17 Consider a trainer giving the trainee the chance to talk his way through a task as the person performs it, such as a walkaround inspection of a haul truck. This technique can help the task performer learn by reinforcing procedures and consid erations about how to perform the task, much like a pilot's pre flight procedure. However, it also serves as an evaluation tool for the trainer because it offers an indication of how the trainee understands the task. Such a procedure gives the trainee the option to engage in the training process (Semb et al., 2000). The trainer asks the trainee questions at different steps in the process, which is a good way to embed evaluations with teaching.18 This implies continuous evaluation. The trainer updates his or her opinion of the trainee's competence. At one level, competence is either inside or outside the ac cepted envelope. Logically, some envelopes are larger or smaller than others. All items in the job analysis are not necessarily equal. Some may be conditional, such as "Perform this check if the temperature is below 10," "What are the factors that affect the uphill and downhill spacing of haul trucks?," and "Dumping over an edge is more risky under certain conditions and less risky under other conditions. What are those conditions? And why are they important?" Researchers suggest that evaluation should be incremental, continuous, and not beyond the capability of the trainee (Palinscar and Brown, 1984). This is one more reason why a job analysis is useful: It helps segment instruction so that compe tence can be assessed at the duty and task levels within the job. For example, a trainer would not be evaluating how a person operates a haul truck; he or she would be evaluating com ponents, such as how the individual performs the walk-around inspection, approaches the loader, or mounts and dismounts; how they decide where to dump; and how they would make use of back-up steering systems and brake systems. The trainer learns what the trainee can do by asking questions and making observations. 17Even though this three-step model makes evaluation appear as a separate entity, in reality, it is on-going. 18At the same time, the trainee should know that the "real" job is to learn, and good peer trainers learn from their trainees (see Semb et al., 2000). 32 All of these items would be identified in the practical job analysis. If the task is not practical nor desired, it should not be in the job analysis or operating procedure.19 Fortunately, job analyses do not live forever. One size does not fit all. As technology changes, job analyses need to be updated, so they have to be done reasonably quickly (see Hartley, 1999). Health, safety, maintenance, and other risks associated with performing the job, as well as what might be done to reduce risks, should be pointed out.20 Fitch and Semb (1993) suggest that effective teachers constantly compare the task goal with their diagnosis of the learner's ability and judgments about the type and amount of coaching needed. It makes sense that effective trainers also aim at a level of assistance slightly ahead of the trainee's level of achievement. Thus, effective trainers motivate and teach. ON-THE-JOB TRAINING AND COACHING OJT and coaching go hand-in-hand, but OJT is considered finite while coaching is continuous. OJT combined with informal approaches such as coaching can blend the two approaches. Employee interaction, sharing ideas and knowledge, coaching, ob serving fellow workers, supervisor guidance, and personal growth all have value and contribute to informal training (Brown, 1989). This is a key as organizational investments in informal training often exceed investments in formal or classroom training. According to Lawson (1996), skills and characteristics of good coaches include patience, enthusiasm, honesty, friendliness, concern for others, self-confidence, fairness, consistency, flexibility, resourcefulness, and empathy. The ability to motivate, teach, and offer feedback is the essence of coaching. Feedback is a form of evaluation. LIMITATIONS OF ON-THE-JOB TRAINING As a teaching method, structured OJT has limitations. These include-- Limitationsfor teaching nonroutine skills. For example, ifthe task is to teach truck operators how to operate on slippery haul roads, it would require drivers driving on slippery haul roads to teach and assess those skills. If the training is done on flat and dry surfaces, skills of operating the truck under those dry and flat conditions will be the only skills taught and assessed. Unusual operating conditions must appear at the worksite before these skills (operating contingencies) can be taught or learned. Thus, some unusual or nonroutine events are often difficult or too risky to replicate using OJT.21 Thus, if OJT is finite, then it is reasonable to assume that all skills cannot be taught during OJT. Limitationsfor teaching cognitive skills. For example, the work environment is not often conducive to teaching skills requiring re call because these skills require memorization and can often be learned only by drill and practice. Examples might involve impor tant specifications and components of a haul truck, stopping distances when loaded and unloaded, etc. Other forms of instruc tion, such as classroom, self-study, computer-based training, or the use of job aids are considered better for developing these cognitive skills. The OJT trainer should have an understanding of both the cognitive and procedural skills that comprise job per formance. However, many of these cognitive skills can be introduced in the classroom and reinforced on the job. It is im portant to consider the training environment. The workplace (for example, via OJT) is one environment, the classroom is another. SUMMARY OJT, which is often called "informal training," is a common and useful method for teaching and evaluating skills. Investments in OJT are quite significant, although difficult to estimate. Across all 19 It's not a bad idea to bench-test the job analysis to see if it is possible to perform the task the way it is laid out on paper. OJT trainers can do this as they are teaching just to make sure the job analysis makes sense. A practical job analysis saves time in teaching because it provides a road map. 20Risk will never equal zero, and down the road, workers may not perform the job as trained. 21That is why simulation (synthetic training) is considered by training professionals to be very useful--it addresses routine and nonroutine events, and skills that are difficult or too risky to teach at the workplace can be practiced. industries, estimates for training are quite large, from $60 to $210 billion (Carnevale and Gainer, 1989). The large gap is due to the difficulty of arriving at estimates ofthe amount ofinformal training for both large and small organizations. Organizations spend signif icantly more on informal training than they do on formal training. Skilled performance involves the integration of hazard awareness, recognition, and response with operational skills for a work task. Allen and Nawrocki (2000) suggest that there is a movement back to training via apprenticeship and OJT experience across industries. They suggest that targeted skills and knowledge will be tied to specific business objectives, that technologies (for example, multimedia) are available to assist 33 the learner, and that increasingly, larger share of the re sponsibility for learning will be placed on the learner. These same technologies may be very useful in assisting safety andskills trainers. Assisting the OJT trainer could involve helping provide up-to-datejob analyses and offering strategies for teaching and evaluating. It is also one way to capture the expertise of experienced workers, especially those who have a desire to teach. One approach supporting a focus on the skills ofthe OJT trainer is offered by Semb et al. (2000). While advances in technology may result in more sophisticated tools for conducting OJT, the knowledge andskills ofthe individual trainer will always be the most critical component ofOJT. These include both knowledge ofthe job and the ability to communicate that job effectively to the on-the-job trainees. This paper lays out a few considerations and references for planning OJT. We suggest that planned OJT could be a very practical way of accelerating the development of skills to benefit both safety and production. Opportunities exist for applied re search in examining concepts and practical strategies for OJT, peer training, coaching, and training OJT trainers. These pros pects will require worker involvement in the development and structure of both formal and informal training. ACKNOWLEDGMENTS We would like to acknowledge the following individuals for their reviews, comments, and insights on earlier drafts of this paper. These include James Baugher (MSHA); Bruce Blakemore and Henry Charpentier (Morton Salt); Harold Baugh and John Bodenhagen (Bridger Coal); and Ted Scharf, Priscilla Wopat, Fred Turin, and Bob Peters (NIOSH). REFERENCES Allen, J., and L. Nawrocki. 2000. Training in industry. In Training and Retraining, T. Sigmond and J. Fletcher, eds. New York: Macmillan. Pp. 237 266 Brown, J. 1983. Are those paid more really no more productive? Measuring the relative importance oftenure as OJT in explaining wage growth. Princeton Industrial Relations Papers. Carnevale, A., and L. Gainer. 1989. The learning enterprise. Alexandria, VA: American Society for Training and Development, and Employment and Training Administration, U.S. Dept. of Labor. Fitch, M.F., and G. Semb. 1992. Peer teacher training: A comparison ofrole playing and video evaluation for effects on peer teacher outcomes. Presentation at American Educational Research Association annual meeting, San Francisco, CA, April 1992, 26 pp. Fitch, M.F., and G. Semb. 1993. The ASKModel ofPeer Tutoring: Theory and Research. San Diego, CA: Navy Personnel Research and Development Center, 32 pp. Hartley, D.E. 1999. Job analysis at the speed ofreality. Amherst, ME: HRD Press, 107 pp. Krupp, K., and J. Applegate. 1983. Haulage truck training system. U.S. Bureau of Mines contract J038722 with Woodward Associates. Open-File Report 61-85, 224 pp. Lawson, K. 1996. Improving workplace performance through coaching. Urbandale, IL: American Media (Provant Media), 95 pp. Lawson, K. 1997. Improving on-the-job training and coaching. Alexandria, VA: American Society for Training and Development, 94 pp. Lineberry G. T., and W.J. Wiehagen. 1996. The cost relationship between performance engineering and human relations. Transactions, vol. 298, pp. 1932 1935. Mager, R. 1999 . What every manager should know about training: An insider's guide to getting your money's worth from training. Atlanta, GA: Center for Effective Performance (CEP Press), 139 pp. Morris, C.W., E.H. Conklin, and F.J. Bick. 1982. Development and fabrication of a continuous miner training system. Contract H0377025 with McDonnell Douglas Electronics Co. U.S. Bureau of Mines Open-File Report 140-83, 64 pp. Palinscar, A.S., and A.L. Brown. 1984. Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition and Instruction, vol. 2, pp. 117-175. Rothwell, W., and H.C. Kazanas. 1990. Structured on-the-job training (SOJT) as perceived by HRD professionals. Performance Improvement Quarterly, vol. 3, no. 3, pp. 12-26. Semb, G., J. Ellis, M. Fitch, and M. Kuti. 2000. On-the-job training (OJT): Theory, research, and practice. In Training and Retraining, T. Sigmond and J. Fletcher, eds. New York: Macmillan. Pp. 289-311. Semb, G., J. Ellis, M. Fitch, S. Parchman, and C. Irick. 1995. On-the-job training: Prescriptions and practice. Performance Improvement Quarterly, vol. 8, no. 3, pp. 19-37. Wiehagen, W.J., G.T. Lineberry, and L.L. Rethi. 1996. The work crew performance model: A method for defining and building upon the expertise within an experienced work force. Transactions, vol. 298, pp.1925-1931. Wilson, T.R., J. Olmstead, and R. Trexler. 1980. On-the-job training and social learning theory: A literature review. A special report prepared for the US Bureau ofMines by the Human Resources Research Organization under contract HO30828 (unpublished). 56 pp. 34 APPENDIX: DEFINITIONS AND CRITIQUES OF ON-THE-JOB TRAINING AND APPRENTICESHIP PROGRAMS IN THE UNITED STATES OJT is very common. It has a long history in which a trades man learned his job from a master craftsman via apprenticeship. We reviewed two studies (Wilson et al., 1980; Semb et al., 1995) that critiqued OJT training and apprentice-type training programs. These studies are valuable because the participating organizations were interested in learning more about how train ing methods or systems might be improved. OJT has several definitions. Some use it to describe any training at a worksite, either in a classroom or in the work loca tion. Others use OJT to differentiate between structured and unstructured approaches. Still others use the term to differentiate between site training and classroom (off-site) training. For this paper, the following definitions apply. On-the-job training (OJT): A method of training conducted at a worksite. It is finite. It may be scheduled (planned) or un scheduled (unplanned). It involves the interaction of a trainer and trainee, and often involves one-on-one instruction and dis cussion, so the trainer-to-trainee ratio is very small. Some refer to OJT as interaction between a journeyman (perhaps a super visor) and an apprentice. It could include classroom components that are very closely related to a specific task or job. Formal classroom training: The trainer-to-trainee ratio is rather large, and many students or trainees are taught by one teacher. Training is formal, scheduled, and time-limited. Skills obtained require application and transfer to the job. On-the-job experience: An informal method of learning that does not involve a trainer. Thus, there is no opportunity for in struction or evaluation (feedback) other than self-assessment. It is continuous, and it could include the use of job aids. On-thejob experience is useful for those who are knowledgeable about the work but need practice in performing a task. Job analysis: A method for breaking a job down into com ponents or steps. A fairly common hierarchy involves the following: A job is composed of several duties. The duties in volve the completion of tasks. Practical job analyses provide a tree of responsibilities that connect the job with duties with tasks. Coaching: Considered to be an informal (one-on-one) train ing method. It involves observations, questions, dialog, and feedback. Peer training: Considered to be a formal or informal (oneon-one) training method. Most research relates to the subject of tutoring for the development and transfer ofknowledge and cog nitive skills. Tutors are often viewed as coaches. Thus, peer training has direct relevance to OJT. Obviously, there are no perfect programs or training methods. Table A-1 summarizes the findings and results from these few studies. OJT and apprenticeship programs have a lot of strengths and are valued, and an examination of the difficulties can provide an opportunity for improvement. Table A-1.--Common difficulties with OJT and apprentice-type training programs (adapted from Wilson et al., 1980; Semb et al., 1995) OJT programs Apprentice programs Lack a trainer (closely resembles on-the- Trainee is sometimes treated as a helper or job experience). semi-skilled labor. Military OJT Written materials may be written above level trainee or trainer can understand. Lack a training plan (e.g., no job analysis). Trainees may fail to rotate through all job tasks. Inspection teams put too much emphasis on keeping training records and not enough on end results. Unscheduled (this may or may not be a problem). Classroom instruction may be poorly correlated Trainees may feel they are mis-assigned to tasks, with OJT. that is, not working in the area trained. This can affect their motivation to learn. Coordination of off-jobsite training and Production demands get most attention, and OJT can be difficult and/or poor. training is secondary. Training can be short-circuited. Proficiency tests can be passed without trainee demonstrating performance in some tasks. Structured OJT is most often found with very large employers. Trainers may lack knowledge and skill regarding instructional methods. Training material can be easily outdated or inappropriate. Completion of training is often based on hours, not competency. The training plan may not be based on actual job analysis. Always a problem keeping the materials up to date. Difficult and sometimes poor coordination between job knowledge and job proficiency training. Many front-line supervisors not trained in OJT methods. Poor coordination between training and follow-up. Trainee counseling sessions are either not held or may not be very meaningful when they are held. 35 RELEASING THE ENERGY OF WORKERS TO CREATE A SAFER WORKPLACE: THE VALUE OF USING MENTORS TO ENHANCE SAFETY TRAINING By Thomas W. Camm1 and Elaine T. Cullen2 ABSTRACT The mentor/leamer model is a time-honored approach to teaching, including within the mining industry. Miners know when they see excellence and have a deep respect for experts in their field. By using expert miners as mentors to other, less-experienced miners, training programs can be developed that will have a legitimacy and credibility that resonates with those being trained. Building upon concepts in current theories of adult education, this paper highlights the unique advantages of using mentoring as a teaching method that can make the educational experience both interesting and effective. Current NIOSH safety training materials use these concepts to deliver effective adult learning experiences for workers in the mining industry. INTRODUCTION "We are what we repeatedly do. Excellence then, is not an act, but a habit."--Aristotle When discussing worker safety training, a logical starting point is to ask the question: What is the goal of the training? Is it merely to satisfy regulatory requirements? Or is the goal to facilitate true learning about working safely? Asking what is the goal of the training is not a trivial question, for it determines the underlying philosophy of the entire safety training program. For a significant proportion of workers, most formal training has taken place in school in grades K-12. For many, memories of school and sitting in a classroom are memories of boredom and tedious exercises with little relevance to real life. The idea of sitting through a lecture with a test at the end does not stir pleasant emotions. Most of the models we have for teaching are based on teaching school children. When we consider the experiences most blue-collar workers had in school, it is no surprise that their reaction to these traditional learning settings tends toward ambivalence, reluctance, or even hostility. Yet, this is still the most common approach used for training adults in a work setting. There are, however, alternatives that can make training ses sions more than a tedious chore that must be endured to meet regulatory requirements. By using concepts from current adult learning theory, and as a particular example focusing on the dy namics of a mentoring approach to training, the goal of safety training can be moved from just trying to meet regulatory re quirements to a goal of facilitating true learning that has a last ing impact on helping workers to stay safe. Referred to in many ways (master/apprentice, teacher/protege, trainer/trainee, old hand/new recruit), the mentor/leamer concept provides an effec tive approach to safety training. ADULT LEARNING THEORY "Learn the fundamentals of the game and stick to them. Band-Aid remedies never last."-- Jack Nicklaus When discussing training in the workplace, we are talking about adult learners. Andragogy, the "art and science of helping adults learn" (Knowles, 1980, p. 43), has a different emphasis than pedagogy, the art and science of helping children learn (Knowles, 1980; Merriam, 1993). According to Knowles, 'Mining engineer. 'Activity chief, Spokane Research Laboratory, National Institute for Occupational Safety and Health, Spokane, WA. pedagogy-andragogy represent a continuum ranging from teacher-directed to student-directed, with both approaches appropriate for children and adults depending on the circumstances (Merriam, 1993, p. 8). The important distinction is the preference of adults in most circumstances to be more self-directed in their learning. Another component of adult learning that distinguishes it from pre-adult learning is addressed by Mezirow's theory of perspective transformation (Mezirow, 1990). Perspective transformation involves reformulating our assumptions to have a more inclusive, discriminating, permeable, and integrative perspective and to understand why we attach the meanings we 36 do to reality (Merriam, 1993). According to Mezirow, the hall mark of adult learning is becoming aware of how our pre suppositions constrain how we see things and then reformulating these assumptions for a better understanding ofthe world. When safe working practices and a philosophy of working safely be come a part ofworkers' basic assumptions about how to do their jobs, safety training becomes relevant to their lives and not just another required chore to endure. Adults are more likely to achieve this transformation if they can see it modeled in peers that they admire or trust. "Learning is not compulsory. . .neither is survival."--W. Edwards Deming, American statistician and quality-control expert The challenge for safety trainers is that an adult can be required to attend safety training, but they cannot be compelled to learn while they are there. Perhaps more importantly, they cannot be compelled to internalize and accept what is taught as part of their own belief system or way of working. "Bodily exercise, when compulsory, does no harm to the body; but knowledge which is acquired under compulsion obtains no hold on the mind."--Plato EVALUATING LEARNING According to Caldwell (1999), the following questions should be asked when evaluating training for adult learners. Is it-- Meaningful? Socially responsible? Multicultural? Reflective? That is, is some critical analysis used in development? Holistic? Global? Open-ended? Goal based? Traditionally, students were raised to do "seat work" when they were in the classroom, with most, if not all, of the class time spent with a teacher in front of the room lecturing to the students. The current generation of students has been taught using a variety of techniques, with a significant focus on ac complishing tasks and working cooperatively in teams. TRAINING METHODS Common contemporary training methods include a com bination of tools and delivery techniques. Lectures Videos Computer (CD, DVD, Internet) Simulation Hands-on Mentors Task training According to Knowles (1980), the essence of teaching adults lies not in the approach as much as in the relationship that exists between learner and teacher. To emphasize this point, Knowles often refers to the teacher as a facilitator, focusing on the con cept that adults prefer a learning environment in which they can participate. THE ADULT LEARNER "Researchers...have verified that a significant number of adults learn a great deal outside the control and confines of formal educational institutions."--Caffarella, 1993, p. 27 Workers learn most of what they need to know on the job (Wiehagen et al., 1994). Relationships with fellow workers will affect not only their attitude toward work, but also their attitude toward safety and training. Adults seek autonomy characterized by three major elements: Independence, Ability to make choices, and Capacity to articulate the norms and limits of their society (Chene, quoted in Caffarella, 1983, p. 29). Adults need information and involvement before learning and tend to ask three questions--How? What? Why? Current adult learning theory addresses the following charac teristics of adult learners (Knowles et al., 1998). A need to know why Self-directed Prior experience Readiness to learn Motivation Orientation to learning and problem solving Self-Directed Learning. Providing a certain degree of self direction in the training process is more likely to allow a trainee to follow his or her individual learning style. Most people learn best when a variety of learning methods is offered, but each person typically has a learning-style preference. Some learn best visually, others by hearing/audio, and still others with hands-on (tactile) training. A self-directed learning environment provides the opportunity to bring the previous experience of each worker to bear on the subject matter. Previous learning socialization and social orientation of the group can add to the efficiency of training and keep the locus of control with workers. "Adults have a deep psychological need to be generally self-directing" (Knowles, 1980, p. 43). Prior Experience of the Learner. Prior experience shapes our reality. Taking advantage of the wide range of individual differences among workers being trained adds a rich resource for learning. One advantage of using a worker's prior knowledge during training is the opportunity it provides for the workers to feel a sense of ownership in the training and enhance 37 their own sense of worth by making a positive contribution to the experience. A possible disadvantage of using prior experience is that unwanted biases may be presented that inhibit or shape new learning. Readiness To Learn. Adults are most ready to learn things that will help them cope with existing situations. Particularly relevant are tasks associated with moving from one de velopmental stage to another. An effective technique to induce readiness is through exposure to role models who excel in the skill or knowledge to be taught (Knowles et al., p. 67). Motivation To Learn. Wlodowski (1985) suggests that adult motivation to learn includes four desires: (1) success--to be a suc cessful learner; (2) volition--to feel a sense of choice in learning; (3) value--to learn something of value; and (4) enjoyment--to experience learning as pleasant. Adults are motivated to engage in learning experiences they see as practical and relevant to their lives, which either help them solve problems in their lives or that have internal payoffs. Orientation to Learning: Problem Solving. "Adults are motivated to learn to the extent that theyperceive that learning will help them perform tasks or deal with problems that they confront in their life situations. Furthermore, they learn new knowledge, understandings, skills, values, and attitudes most effectively when they are presented in the context of application to real-life situations. "--Knowles et al., 1998, p. 67 SOCIAL LEARNING THEORY Social learning theory has been around for over 60 years, incorporating the learning principles of reinforcement, punish ment, extinction, and imitation of models. There are currently several versions of social learning theory, but three basic ideas are common to all versions. (1) Response consequences (re wards, punishments) influence the likelihood that a particular behavior will be performed again in a given situation, (2) vi carious learning--learning by observing others--will take place in addition to learning by doing, and (3) learners are most likely to model behavior they observe in those with whom they identify (Stone, 1998). Two of these three ideas are relevant to the mentoring that takes place in mining. The first suggests that vicarious learning, or learning by watching others, is common, particularly when people participate directly in the observed act. The second says that people are more likely to pay attention to those with whom they identify or those to whom they are emotionally attached. In the mining industry, it is quite common to see new hires working with older, more successful miners. These older miners become role models and are effective teachers because their "students" identify with them and are willing to watch what they do and model it. This type of training is much more successful than trying to teach a new miner in a classroom. MENTORS OR COACHES Coaching: "[T]he process of equipping people with the tools, knowledge, and opportunities they need to develop them selves and become successful."--Hughes et al. The mining industry has historically depended heavily on the mentor/learner (master/apprentice) relationship to train new miners. Young or new hires are paired with older experienced hands (mentors) who teach them many things, including the art of staying alive. Billett (1994) suggests that this relationship is key to learning in skilled vocational jobs and that it is not only the activities that are important, but guidance and exposure to the work culture that makes up the learning experience. In his opinion, mentors (master miners) provide three essential attributes. Knowledge about what is important. Knowledge about how to do things right. Knowledge about the culture, including the values and attitudes that the learner must have to be successful in the current environment. The questions one must ask are "Why do learners pay at tention?" "What motivates them to listen to another miner?" Pegg (1999) would respond that mentors have credibility that has been gained in a variety of ways. It may have come through success as a miner, perceived "street-smarts," acknowledged ex pertise in a given area, personal presence or magnetism, or merely from age or experience. In any case, the learner sees a coach who has "been there-done that" and who could help them learn the ropes. This is a critical element in the relationship. For a successful learning experience, the learner must be willing to learn from the mentor. THE ART OF MENTORING So what happens in a mentoring relationship? Pegg (1999) argues that truly great mentors are those who help people find their own way to achieve success. Mentors teach the knowledge and skills the learner will need by helping them through the cycle of-- Challenges faced, Choices available, Consequences of available options, Creative solutions, and Results. The art of allowing the learner to fail in a safe environment is crucial to the learning process. The trainee must eventually be able to perform without the support of the mentor to become an effective or safe worker. A successful mentor will fade out after 38 making sure that the learner has the skills needed to perform. This process is included in the model suggested by Billett (1994) in his apprenticeship method of instruction. It involves four phases. Modeling, where the expert performs the task and discusses why it is done "this way." Coaching, where the mentor watches and monitors the learner, providing tips and feedback on how to improve. Coaching may also include performing the task again to reinforce modeling it. Scaffolding, in which the learner performs the task while the mentor is at a distance and not directly involved. The mentor may, however, have to do part of the task that the learner is not yet ready to perform. This phase primarily involves support. Fading, which is the gradual removal of support until the learner can operate without assistance or guidance. Mentoring in the mining environment is often an informal relationship. A young miner (the learner) looks around and iden tifies a more experienced miner who appears to be successful and approaches him to see if he is interested in teaching a new hand (become his mentor). This format is very common in Western noncoal mines. In many cases, the mentor initiates the relationship, recognizing the opportunity (or even the obliga tion) to give back to the younger generation, to "take someone under their wing" to ensure they learn to do things the right way. The mentor will coach the learner until he feels either that the trainee is wasting his (the mentor's) time or he has taught him enough to make him a valuable hand. If the learner does not believe the mentor is credible, however, very little learning will take place. In this training environment, the actual teaching is an on-going, constant interaction, rather than an isolated incident restricted to a training room. One ofthe most important concepts in a mentor/learner learn ing environment is identified by Billett (1994). "Developing learners' conceptual understanding of why things are done a certain way, and what will happen if they were not, is a key role for the expert." In other words, why should they care if things are done this way? Expert mentors must not only teach how to do things, they must clearly teach why and what will happen if things are not done in the proper manner. Becoming a master miner cannot be accomplished by classroom training alone. The skills to become a truly good miner are learned over many years and are the result of acquiring wisdom as much as skill and knowledge. Being a master becomes a part of who they are. Robert Pirsig, in his book Zen and the Art of Motorcycle Maintenance (1974, p. 148), describes it this way: Sometime look at a novice workman or a bad workman and compare his expression with that of a craftsman whose work you know is excellent and you'll see the difference. The craftsman isn't ever following a single line of instruction. He's making decisions as he goes along. For that reason he'll be absorbed and attentive to what he's doing even though he doesn't deliberately contrive this. His motions and the machine are in a kind of harmony. He isn't following any set of written instructions...it is art. An apprentice does not learn this easily. It is a lesson that is taught by the master in a mentoring relationship, in a learning environment honed by experience and reinforced by the culture. When the teaching relationship is successful, the learner is on the way to becoming a mentor for the next learner. REFERENCES Billett, S. 1994. Situating learning in the workplace: Having another look at apprenticeships. Industrial and Commercial Training, vol. 26, no. 11, pp. 9 16. Caffarella, R.S. 1993. Self-directed learning. In An Update on Adult Learning Theory, S.B. Merriam, ed. San Francisco: Jossey-Bass. Pp. 25-35. Caldwell, K.M. 1999. Curriculum development: Adult learners. Paper submitted for Doctoral Studies in Leadership class, Gonzaga University, Spokane, WA. Knowles, M.S. 1980. The Modern Practice ofAdult Education: From Pedagogy to Andragogy, 2nd ed. New York: Cambridge. Knowles, M.S., E.F. Holton, III, and R.A. Swanson. 1998. The Adult Learner, 5th ed. Woburn, MA: Butterworth-Heinemann. Merriam, S.B. 1993. Adult learning: Where have we come from? Where are we headed? InAn Update onAdultLearning Theory, S.B. Merriam, ed. San Francisco: Jossey-Bass. Pp. 5-14. Mezirow, J. 1990. Fostering Critical Reflection in Adulthood: A Guide to Transformative and Emancipatory Learning. San Francisco: Jossey-Bass. Pegg, M. 1999. The art ofmentoring. Industrial and Commercial Training, vol. 31, no. 4, pp. 136-141. Pirsig, R.M. 1974. Zen and the Art ofMotorcycleMaintenance. New York: Bantam Books. Pratt, D.D. 1993. Andragogy aftertwenty-five years. InAn Update onAdult Learning Theory, S.B. Merriam, ed. San Francisco: Jossey-Bass. Pp. 25-35. Stone, D. 1998. Social cognitive theory. http://www.med.usf.edu/%7Ekmbrown/Social Cognitive Theory Overview. htm . Univ. of South Florida, Dept. Community and Family Health. Retrieved May 28, 2002. Wiehagen, W.J., G.T. Lineberry, W.E. Lacefield, M.J. Brnich, and L.L. Rethi. 1994. The work crew performance model: A method for evaluating training and performance in the mining industry. U.S. Bureau of Mines Information Circular 9394. Wlodowski, R.J. 1985. Enhancing Adult Motivation To Learn. San Francisco: Jossey-Bass. 39 DEVELOPING TOOLBOX TRAINING MATERIALS FOR MINING By Floyd D. Varley1 and C.M.K. Boldt*2 WHAT IS TOOLBOX TRAINING AND WHY USE IT? Toolbox training is often described as short, informal training conducted at a worksite by technically competent persons for the benefit of a work team. The key feature of toolbox training is the focus on a work team and what is important to that group in its workplace. Toolbox training conducted by peers can connect miners and establish the feeling that the hazard is a real threat to them. This feeling can bridge the gap between delivery of a safety message and behavioral changes that can prevent an accident. Toolbox training is a popular form of "maintenance" training that should not be used to teach a new skill. The 10- or 15minute toolbox session is simply too short a time to teach or learn a new skill and test for skill development. However, the toolbox format is an excellent way to inform workers of changes in workplace rules, conditions, or hazards by bringing the dis cussion of change to the level of the work group and suggesting how a change will affect the way workers perform their jobs. Toolbox training should be structured to address a very specific hazard. That is, the hazard source should be identified and the effect of the hazard on the worker described. The temptation to discuss a safety subject in broad terms will result in an equally vague result. For instance, discussing "equipment guards" is inappropriate for a 15-minute training session. There simply is not enough time to cover all the hazards and their remedies. A more appropriate topic would be "placing guards on a conveyor return roller" or "hazards of cleaning a conveyor return roller." Each topic is concise enough to allow the trainer time to define a particular threat to the safety of miners and dis cuss ways to defend against that threat. The clearest way to a safe workplace is to identify hazards methodically and eliminate them. This cannot be done with sweeping, generalized training. Some best practices that will protect a worker from the hazard should be presented, and the hazard should be located at the spe cific worksite. Toolbox training is an opportunity for the work team to participate in learning and share experiences and knowledge, often through storytelling. Storytelling is an ancient form of passing along information in an entertaining way. It is enter taining because it draws the listener in with imagination and creativity. Stories told within work groups are a way for miners to exchange experiences in which they made mistakes that re sulted in an accident or a near-miss and learned a valuable les son. Thus, co-workers can learn from others without the pain or fear that accompanied the storyteller's learning path. To encourage these shared experiences, the people in the group must be comfortable with one another. Participation doesn't stop with listening to a near-miss story or informing the group of an unsafe work practice. Participation by sharing ex periences, knowledge, and skills should lead toward action in changing the unsafe tool, practice, or machine (Wallerstein, 1992). This power to change is called empowerment. Empowered workers use toolbox training sessions to discuss a problem; di rect their own knowledge, experience, and talents in the context of their work environment to solve the problem; and put into practice what they have learned by sharing (Baker, 1992). A worker is much more likely to implement changes they have contributed to creating than changes forced upon them. Toolbox training functions best when the group size is small (under 20) and composed of workers with similarjob functions. While combining all workers on a shift or multiple shifts may be convenient, the level of participation will drop dramatically as group size increases and participants are less connected. Similarly, when the work group participants are from different job functions, keeping the materials relevant to the individual becomes more difficult. The role of management in the toolbox training system is to communicate among various groups so that all groups at a mine site are hearing the same message. In very small operations, combining work groups can be productive if the trainer can promote open discussions and lead the group to solutions that address the needs of all and not let one group or individual dominate the discussion. WHY USE NARRATIVE METHODS An effective communications tool is to use stories as a means of describing a hazard and its consequences. Historically, using stories to transfer knowledge between individuals of the same 'Safety engineer. 2Civil engineer, Spokane Research Laboratory, National Institute for Occu pational Safety and Health, Spokane, WA. and different generations is well established. All good stories share the same elements required to communicate safety prin ciples. Stories require a setting (the part of the workplace), a plot (the circumstances that lead to a hazard), the solution (best practices), the lesson (what can happen), and the result (what will be done to prevent the hazard) (Dennehy, 1999). 40 Toolbox training, by definition, is not a place for long epic tales. However, telling stories, the narrative method, provides a way to open the door to empathy among workers regarding one individual's hazardous experience. The difference between toolbox training and traditional storytelling is the need to involve the workers by having them take part in the story. The narrative toolbox method begins with a short true story surrounding a hazardous condition that resulted in an accident. The setting and the plot have been established. The result, usually a bad one, for the individual in the story is also revealed. The next step is to lead the workers through the remaining parts of the story and guide them in rewriting the story as it applies to them, their workplace, and their reactions to the hazard. The first stop in story immersion is to bring out what best practices the victim did not use. Group discussions then lead to the question, "Could this happen here?" Now is the time to try to get the workers to open up about similar experiences they have had and what they learned. Following these revelations, the group should discuss the result for the team, that is, what will be done to protect against this hazard. This section of the training will, if successful, produce work for management in terms of things that must be acted upon, such as purchasing and installing signs or barriers, installing different guards, or scheduling follow-up task training. Effective use of storytelling and open discussions will result in time well spent in narrative toolbox training sessions. Writing down what the workers said and following up with feedback on the recommendations will help toolbox training be a part of an effective safety and health program. HOW TO BUILD A TOOLBOX Identify a Subject: The first step in creating toolbox training materials is to identify a subj ect relevant to the work group. Obvious examples would be accidents that have occurred in the work group, at the worksite, or at other sites within the company. Accidents outside of the work group can also be meaningful sources of training topics if the conditions or circumstances of that accident are present at the worksite. Other topics are the introduction of new work rules or policies related to safety, as well as new equipment or procedures that are about to be introduced in the workplace. Toolbox training can also be an effective means to reinforce topics related to citations issued by the Mine Safety and Health Administration (MSHA) or faults noted in your own safety audits. Special or nonroutine events, such as a con struction activity, a weather episode, an intense production per iod, or an extensive maintenance activity, could be used as tool box training topics to increase awareness of hazards associated with these events. In selecting a topic for toolbox training, remember to narrow the subject to specific actions, equipment, and/or hazards so that the short training period can both fully explore the subject and develop solutions. Describe the Hazard: After selecting the subject to develop into a toolbox talk, make a list of the hazardous characteristics of the subject. Describe what type of accidents and injuries can occur and the circumstances that create a hazard. This exercise will help narrow the subject to the specific topic to be covered in the training session. For instance, a subject such as housekeeping in the shop can cover several potential hazards. To specify a hazard such as slips and falls or fire will direct the housekeeping discussions to one of these very different aspects of house keeping. Identifying the potential result helps build empathy for the victim and interest in the topic by clearly stating the full range of consequences that could result from this hazard. Focusing only on fatalities can dilute the impact of a safety message because most people are not willing to recognize such a severe result as an outcome for themselves or their peers. Most miners will never experience a fatality in their workplace, and so news of such an event usually carries with it the impression that it happens to other people. Most miners do feel vulnerable to injuries and may better identify with the potential hazard and personal impact of these injuries because they or someone they know has probably had an on-the-job injury. A miner may take to heart a message about a broken arm that cost several week's wages more readily than a discussion of the same hazard in which someone died. By noting the circumstances that created this hazard, a trainer can create "what to watch for" lists to set the stage for discussions. This list is intended to provide a mental audit for miners to remind themselves of the conditions that could create a hazard. The list should relate to where potential hazards exist at a worksite. Listing circumstances not relevant to the work group, even if they may create more substantial risks, may diminish the message by creating a link to someone other than the miner. An example would be to note icy conditions as a po tential slip and fall hazard at a mine that does not experience freezing weather. Use a Story To Improve Empathy and Interest: Developing a sense of empathy toward the victims in a story of a real accident or incident is one of the best ways to convey the circumstances and consequences of a hazardous condition. The most common example of this tool in the mining industry is the use of MSHA's fatalgram. These short reports offer a basic description of a fatal accident. In addition to fatalgrams, MSHA has listed on its website all reportable accidents that have occurred in U.S. mines. These reports are available through the MSHA Data Retrieval System at http://www.msha.gov/drs/drshome.htm. This resource can be used to look at accidents reported by other mines so enough information can be acquired to frame a story for a training talk. 41 The system provides accident information by mine or mine operator. NIOSH is currently working on another web-based accident information source that will allow searches by type of accident or commodity being mined. A casual search of neighboring mines or mines in the region will usually produce a good supply of story material. Mines with different mining methods and mined commodities will have many of the same hazards facing employees. Story lines can also be developed from newspaper articles about accidents in the community. Many subjects, such as hand tool use and misuse, can be covered by stories found in local papers. Begin by clipping these stories and building a toolbox talk around the stories. In presenting a story, strive to duplicate the hazard description effort. Identify the hazard and the circumstances that created the hazard. Finish with the result, the injury to the victim. Embellish ments of the story may help sell the story as long as it can still be claimed as a true story. Offer Best Practices: After telling the story, the next step in the process is to make sure the injury result of the story is not repeated at the mine. Offer ideas for best practices that, had they been followed, would have prevented the accident. Reinforcing best practices followed in the story is also a good idea as it can help demonstrate that going part way in safety efforts is often not enough. Best practices are intended to be springboards to discussion. Visual aids such as pictures of someone doing the task the right way or the wrong way can help reinforce best-practice discussions and may help the competent trainer better understand the material. Elicit Participation: A miner's active participation in the training is probably the most important and beneficial aspect of narrative-style toolbox training. A way to elicit participation is by asking leading or open-ended questions. Examples of these common questions, when to use them, and what to expect are described below. The appendix to this paper is an example of a training module. "Has anyone here had a similar accident or a close call or know of one?" This question seeks to link the work group to the potential hazard. Ifminers feel free to express themselves, the trainer may find out about holes in the company's systems. For the future success of the training method and communications in general, information revealed in this forum should not lead to disci plinary actions. It is reasonable to council an employee in dividually if an incident should have been reported. When a employee volunteers an experience, it is important to follow up with the question-- "What do you do differently now to prevent the accident?" If there are no volunteers willing to admit a similar experience, add a question to connect the story to the miner's world, such as-- "What could the person in the story have done to prevent the accident?" Much like the follow-up to an employee's story, this question opens discussion to solving the potential problem. In all cases, connect the message to today with-- "Where in our mine could this same accident happen?" To reinforce the best practices, encourage discussion with a question such as-- "Are there other best practices we do or could use here?" This will open the floor to new ideas on solving the problem and may lead to the follow-up question-- "Is there anything that prevents us from using these best practices?" Other questions that connect the previous safety training to the present can be used, such as "Does anyone remember what we talked about last week?" or "Does anyone have a suggestion for a future safety topic?"It would also be useful to add follow-up questions such as "Last week Joe said he'd check the first aid kit. Was it done?" Document: The need for documentation will be based on how the toolbox training is to be used. At a minimum, the names of the miners who participated in the training should be recorded. If the intent is to use these training sessions toward Part 46 annual refresher requirements, each participating miner and the compe tent person leading the training must print their names on a log (see appendix) that identifies the talk, the date, the time spent, the location, and a note that the training is part of the 46.8c annual refresher requirement. In addition, Part 46 requires that training plans include a reference to the subjects covered in toolbox training if the toolbox meetings are to be used to satisfy the minimum training time requirements. The person responsible for training at the mine must sign either the log or another docu ment that summarizes the logs of several training sessions to certify that training has been completed and acknowledging that he or she knows the punishment for false certification. CONCLUSION Toolbox training can be a valuable part of a training program. It can be used to share safety information and provide a structured, but informal, forum for improving safety at a mine. Toolbox training requires preparation, active participation, and follow-up, but it can stimulate attention to everyone's health and safety on the job. 42 REFERENCES AND SELECTED READINGS American Conference of Governmental Industrial Hygienists (ACGIH). 1997. Tailgate meetings that work! A guide to effective construction safety training. National Education Pub. 9655, 324 pp. Baker, R., and B. Szudy. 1992. Hardware to hard hats: Training workers for action (from offices to construction sites). American Journal of Industrial Medicine, vol. 22, pp. 691-701. Cary, M., G. Van Belle, S.L. Morris, B. Cameron, and D. Bourcier. 1997. The role ofworker participation in effective training. New Solutions, vol. 7, pp. 23-30. Center to Protect Workers' Rights. Electronic library of construction occupational safety and health (eLCOSH). Training programs for safety and health. Available at http://www.cdc.gov/niosh/elcosh/docs/training/materials.html. Dennehy, R.F. 1999. The executive as storyteller.ManagementReview, vol. 88, no. 3, pp. 40-43. University ofCalifornia, Labor Occupational Health Program. 2001(1994). Tailgate training for California construction workers. Tailgate Meetings That Work! 2001 Guidebook. 200 pp. Wallerstein, N., and M. Weinger. 1992. Health and safety education for worker empowerment. American Journal ofIndustrial Medicine, vol. 22, pp. 619-635. 43 CDC TAILGATE TRAINING /Bosh PERSONAL PROTECTION EQUIPMENT AND PRACTICES - MODULE 2B HAZARD - Loose clothing, jewelry or hair catching in equipment RESULT - Cuts, burns, broken bones, death LOOK FOR - Loose clothes, jewelry, hair, pinch points REAL ACCIDENTS: On a cold February evening, a 20-year-old laborer with 1 year of experience was checking a head pulley. The sleeve of his sweater was caught by the pulley or edge of the belt, pulling in his arm and crushing it. 40 DAYS LOST. BEST PRACTICES: Don't wear ragged, loose, or hooded clothing. Keep sleeves and pant legs/cuffs tight. Keep long hair tied up and inside hard hat. Don't wear necklaces, scarves, or rings during work. Make sure all pinch points are well guarded.________________________ WHAT ABOUT OUR SITE?___________ ______________ COMMENTS Who remembers what we talked about last week? Was there something we needed to fix? What can get caught in pinch points? (Jewelry, shovels, pry bars, clothing, hair) Has anyone had or seen an accident or nearmiss with pulleys? What places/jobs in our mine have a high risk for catching clothing? (Show us.) Is there any other way to reduce the hazard? (Hair and clothing code, add/fix guards)______ 44 TRAINING RECORD DATE: MSHA MINE/CONTRACTOR NAME and ID No.: TRAINER: LENGTH: LOCATION Check If this Is Part of Annual Refresher Training (CFR 30, 46.8.c) PERSONS TRAINED (PRINT FULL NAME): I certify that the above training has been completed: False certification is punishable under 110(a) and (f) of the Federal Mine Safety and Health Act COMMUNICATING THE SAME MESSAGE WITH DIFFERENT MEDIA: AN EXAMPLE FROM HEARING LOSS PREVENTION By Robert F. Randolph,1 Jeffery L. Kohler,*2 and David C. Byrne3 45 ABSTRACT Multiple versions of an educational message can reach a diverse population more effectively than a single version. For instance, some workers are trained in formal classrooms while others are self-taught. The National Institute for Occupational Safety and Health has developed multiple versions of a hearing loss simulator (an interactive software package, a Web-based module, and an electronic slide show) to show how a single set of information can be readily adapted to different delivery methods. The three versions of the simulator were developed with a minimum of effort and expense compared to a single, less-flexible version. The interactive software is best for training sessions led by a hearing conservation professional, the Web pages are best suited for an individual worker, and the slide show is best for a small, more-generalized, training class. This paper will describe additional advantages and disadvantages of different delivery systems and will show what considerations are helpful in designing content that can be readily adapted to alternate presentations. WHY IS A HEARING LOSS SIMULATOR IMPORTANT? Although noise-induced hearing loss is the most common oc cupational disease (National Center of Health Statistics [NCHS], 1993), most people don't adequately protect themselves from harmful noise (Berger et al., 1996). Changing behaviors to increase hearing conservation has turned out to be especially challenging for a variety of reasons. In some cases, workers may not know how to protect their hearing. In other cases, obstacles may prevent them from taking action. Often, the obstacles are obvious--hearing protectors are not available, noise control solutions are expensive or otherwise impractical, or the worker has little control over reducing noise. A more subtle obstacle is lack of motivation to take pre ventive action. Clearly, nobody wants to have poor hearing. However, the threat of a potential hearing loss sometime in the distant future may not be enough motivation for action in the present, especially with all the other events vying for a busy worker's attention. Another problem is that because noise-in duced hearing loss is usually gradual and workers don't ex perience the same kind of physical pain associated with other types of workplace hazards, they don't realize that hearing nerves can be permanently damaged by excessive sound levels. To add to all of the preventative challenges, there are also a number of myths about hearing loss. Myth: I can build up a resistance to noise--my ears will even tually get "toughened up" so they won't get hurt. 'Research psychologist. 2Director. 3Research audiologist, Pittsburgh Research Laboratory, National Institute for Occupational Safety and Health, Pittsburgh, PA. In fact: There is no way to build a resistance to noise. Excess noise damages the cells and nerves of the ear and these cells and nerves cannot be repaired or replaced. Continued exposure results in continued damage, not "toughening." Myth: Noise can't hurt me unless it's painfully loud. Infact: Noise becomes potentially hazardous around 85 dBA4 and only begins to cause pain at much louder levels around 140 dBA. In between is a large range ofdangerous noise levels. Myth: I can duck in and out of a noisy place before it can affect my ears. In fact: Noise that is loud enough can damage the ears in stantly. Also, many short exposures can add up and cause damage similar to continuous exposure. Myth: My hearing will probably come back after I stay away from noise for awhile. In fact: Your hearing will never come back once the ear is permanently damaged. Myth: Even if I lose some hearing, I can get hearing aids--they will restore my hearingjust like my eyeglasses work for my eyes. In fact: Hearing aids don't work as well as glasses. At best, hearing aids will restore some ability to understand conver sation and experience the sounds around you, but they don't sound "normal." 4"A significant risk to miners of material impairment of health from work place exposure to noise over a working lifetime exists when miners' exposure exceeds an 8-hour time-weighted average (TWA8) of85 dBA." Mine Safety and Health Administration. Health Standards for Occupational Noise Exposure. Federal Register, vol. 64 no. 176, p. 49548, 9/13/1999. 46 All these myths reflect a misunderstanding of the mechanisms of hearing loss. In particular, they ignore the painless,cumulative damage that occurs to the sensory cells in the inner ear. They also ignore the irreversibility of hearing nerve damage. Before these myths can be debunked and workers can be receptive to taking action to protect their hearing, they must understand the nature of a noise-induced hearing loss. Un fortunately, it is difficult to describe such a subjective sensory experience, just as it is difficult to describe the concept of "pink" to a blind person. Rather than attempting to tell workers that their hearing will become "dull" and that they will have difficulty hearing high-pitched voices or understanding conver sation over background noise, safety trainers need to take a more direct approach by having workers experience hearing loss first-hand. Since it would be clearly unethical to have workers experience a true permanent hearing loss, or even a temporary threshold shift, a simulation is a realistic alternative. The hearing loss simulator developed by the National In stitute for Occupational Safety and Health (NIOSH) uses an es tablished standard method of estimating the effects of noise exposure. The specific formula is taken from an American National Standard Institute (ANSI) document entitled "Determi nation of Occupational Noise Exposure and Estimation of Noise-Induced Hearing Impairment" (ANSI S3.44-1996 [R2001]). This standard is based on a number of studies that report actual hearing levels in individuals who had a wide range of exposures to noise, including a population that was carefully screened to have had no noise exposure at all. By including nonexposed individuals, changes in hearing due to aging (known as "presbycusis") can also be predicted and separated out from noise-induced changes. Other researchers have used different populations and math ematical techniques to arrive at slightly different ways to cal culate the risk of noise-induced hearing loss (see Prince et al., 1997, for a discussion of the issue and of an alternative tech nique based on a NIOSH survey). There is also a great deal of variability in individual susceptibility to noise-induced hearing loss. However, there is no real dispute over the basic relation ship: Greater noise exposures over longer time periods result in more hearing loss. USING THE SIMULATOR FOR EFFECTIVE TRAINING Two primary goals drove the development of the simulator. The first goal was to make the results of excessive noise exposure as obvious as possible, and the second goal was to make the simulation as widely available as possible. The first goal was easy. Hearing loss simulation is an es tablished training technique that was already available in two basic forms, either "canned" recordings or specific demonstrations produced with specialized audio equipment. A typical recorded simulation would be an educational CD or audiotape that contains recordings of the sounds that both normal and hearing-impaired people would hear. The impaired tracks have been processed through filters to selectively reduce the frequencies most affected by noise exposure (typically in the range of 3000 to 6000 Hz). On some tracks, the loss is gradually "dialed in" so the trainee can hear the affected frequencies fade away little by little. On other tracks, the transition is abrupt, which serves to make changes immediately evident. Interaction with the simulation is limited to replaying the recordings and skipping back and forth between the normal and impaired sounds. Depending on the playback device in use, this may be cumbersome. A more interactive simulator is available as an audio in strument. These instruments are sophisticated electronic ma chines designed for use in audiological clinics, and they allow a clinician to control both the nature of the sounds (speech, background, etc.) and the type of impairments (high- versus low-frequency loss, etc.). The main drawbacks of these devices are their high cost and complexity so that a physician or audiologist is required to operate them. Because of these limitations, only a very small percentage of workers who may be at risk of noise induced hearing loss have the opportunity to experience a simulation. Therefore, wide dissemination became the most important remaining goal for an improved hearing loss simulator. Now that virtually all workers have access to a personal computer either at home, in a training facility, or at a local library, computer "interaction" became the primary focus for an inexpensive interactive simulator. NIOSH funded development work by Michael and Associates, Inc., State College, PA, to create a software version of the hearing loss simulator. Commercially available sound software libraries made this objective feasible at very low development costs, while the sophisticated sound capabilities of even the most modest computers made it possible to incorporate a large number of features. The full software package is an extremely flexible interactive training tool; however, its flexibility could become a liability in some training situations. Users must follow a series of steps just to set it up and get usable sounds from it, so they need to spend some time familiarizing themselves with the basic functions. Generating the correct sounds in the correct sequence requires following a training script or having significant expertise in the field of hearing loss prevention. Not all users will have the time or resources to make the best use of the full software package; instead, they will need something simpler and more straight forward. To meet this need, two other variants on the simulator were developed. One is a computerized interactive slide show, and the other is an Internet Web page. The advantages and disadvantages of each version in different training settings are described in table 1. 47 Table 1.--Advantages and disadvantage of three versions of hearing-loss simulator VersionAdvantagesDisadvantages Full simulator ............................ High flexibility Must be installed on PC Many scenarios Some learning time More functions Requires more trainer expertise Customizable sounds No background information Computerized slide show......... Can be used by individual trainee Only a few canned sounds available Includes background information Cannot be tailored to site or trainees Web page................................... Can be used by individual trainee Accessible by any Web-connected PC Only a few canned sounds available Simple and quick________________________ Cannot be tailored to site or trainees AUTHORIZING TOOLS Both the Web page and the slide show were constructed using basic, readily available software. The slide show was constructed using Microsoft5 PowerPoint 2000, and the Web page was built with Microsoft FrontPage 2000. Products from other software companies could also have been used-nothing about the Web page or slide show required any Microsoftspecific features. For instance, the Web page just uses standard HTML code and could have been generated in any HTML editor or even a generic text editor. The slide show makes use of simple animation and multimedia functions that are common to most other products on the market. SOUND PROCESSING The tools needed to generate the sounds in the simplified training packages were only slightly more specialized than the authoring tools. It was important to create filtered recordings that would simulate the individual frequency losses of a person with a noise-induced hearing loss. Fortunately, many such tools are available at modest cost ($100 or less). For the sounds in the slide show and Web simulators, Syntrillium's6 CoolEdit 2000 was used. First, the software ' s transformation function was used to create a filter with frequency characteristics similar to the hearing levels of a 45-year-old individual who had been exposed to 95 dBA of noise per 8-hour work day over a 25-year career. This represents a very noisy job, although there are some jobs that are even noisier. Then a recording of a male speaker reading a series of hearing loss messages (the same recording as used in the full simulator) and a combination of the male speaker and a mining background noise (continuous haulage machine) were processed. The resulting files were saved in both the common WAV format for the slide show and MP3 format for the Web page. By using the MP3 format, a significant file size reduction was achieved at the expense of a small loss in sound quality resulting from the format's "lossy" compression. This size reduction is important for Web pages because many users may have slow Internet connections, and the long download times required for uncompressed sounds would discourage users from accessing the simulator.* 6 5Microsoft Corp., Redmond, WA. 6Syntrillium Software Corp., Phoenix, AZ. WEB PAGE The Web page is the simplest version of the simulator (fig ure 1). It consists of a single page with instructions to listen to four sound samples by clicking on four icons in order. The icons play a normal male voice recording and the same voice as heard with a noise-induced hearing loss. Next, the user can hear the voice with machine background noise both with and without a hearing loss. The page is intended as a very brief introduction to the concept of noise-induced hearing loss and has no provision for adjusting exposure durations or modifying the types of sounds. These functions may be added to a later version of the site once the functions are evaluated in the full simulator. The target audience for this version is an individual worker who is accessing the Internet from home, a training room at work, or some other access site. In the future, the page will contain links to additional supporting publications and sites and allow downloading of other versions of the simulator. COMPUTERIZED SLIDE SHOW The computerized slide show has much more content and interactivity than the Web page. It uses the same preprocessed simulated sounds as contained in the Web page and embeds them into a series of slides. It also surrounds the simulation with a brief lesson about the nature and causes of noise-induced hearing loss and finishes with a review of actions that workers can take to protect their hearing. Selected slides for the background, simulation, and action portions of the presentation are shown in figure 2. This version of the simulator is mainly intended for safety and health instructors to use as part of their hearing loss training sessions. It is self-contained, requiring only a Windows PC to operate. Instructions on navigating from one screen to the next and activating the simulated sounds are displayed right on the screen. Although designed for use in small training rooms, its simplicity makes it also appropriate as a self-paced training exercise for an individual worker. FUNCTIONS AND CONTROLS OF THE FULL SIMULATOR The full simulator offers a number of controls to give the trainer flexibility to tailor the training to the audience and 48 training needs. This places considerable responsibility on the trainer, but the resulting interactive possibilities can be worth it. Below is a description of all of the essential functions in the prototype simulator that are currently being evaluated (see figure 3 for a view of the main control screen). These functions are likely to change somewhat in the final release and in subsequent versions as improvements are made on the basis of user feedback. CDC Home NIOSH Home | NIOSH Search | Site Map | Contact Us MINING Safety and Health Research Mining Home Research Products Data visitors About Us What does a hearing loss sound like? Loud noise can permanently damage your hearing. It especially affects the sounds that help you understand speech. To hear what this sounds like, try these computer-generated samples: Wofe. The sound samples are in the popular MP3 format. Normal Hearing First click on the picture to the right to hear a recording of a male voice reading several sentences (while this is playing, adjust your PC's volume so you can clearly understand what he's saying) Now, click on this link to hear the same recording altered to sound as it would for a person with a noise-induced hearing loss - in this case, a 45 year-old male worker exposed to 95 decibels (dBA) during his 8-hour work days over 25 years*. Hearing loss Background noise makes speech harder to understand, even with normal hearing Click on this link to hear the speech recording again with the sound of a coal continuous haulage machine added to the background. Normal hearing with background noise Finally, hear the same recording again (with background noise) with a simulated hpprinn In^q Hearing loss with background noise 30 Internet Figure 1.-Prototype Web page version of hearing loss simulator. 49 Foreground Sound Background Sound Human speech is used as the default foreground sound because it is both the most complex and most important signal workers need to perceive. The simulator allows the choice of either a male or female voice recording. However, the simu lator also allows the user to record a foreground sound of his or her choice through the computer's sound hardware. Some trainers could use this capability to record a special warning signal or other sound that is likely to be heard at a specific facility. Background sounds often severely tax a listener's ability to hear and/or comprehend the intended message. The simulator allows the choice of several types of background sounds, in cluding some recorded worksite sounds (continuous miner, haul age machine, drill) and some more generic background noises (male or female "speech babble," white noise, etc.). The user can also control the signal-to-noise ratio, that is, the relative loudness of the foreground and background sounds. In practice, a range of -10 to -20 signal-to-noise ratio seems to work best. 50 Figure 3.--Main control screen, full version of NIOSH hearing loss simulator. Noise Exposure Level Age The first ingredient of overall sound exposure is noise level. As expected, high-intensity sound levels cause much more hearing damage than lower levels. The effect of different levels of past noise exposure can be simulated by entering the desired A-weighted sound level in decibels. For simplicity, a single decibel A number is set in the simulator, although workers may correctly point out that the sound levels they are exposed to vary considerably over time. Because of this, the decibel A value should represent an estimate of the average exposure over the simulated time period, commonly referred to as the time-weighted average (abbreviated as TWA). Years of Exposure Time is the second major ingredient of exposure. This is set in years to represent a noisy period in the simulated indi vidual's life. For instance, it can be set to cover a single noisy portion of a career (for example, 10 years of working with a loud machine) or multiple noisy periods added together. The time entered assumes exposure during normal working days of around 8 hours, not continuous round-the-clock noise exposure. Some hearing loss occurs as people age, but age alone sel dom causes a severe hearing loss or deafness. One ofthe major lessons to be learnedfrom the simulator is that aging usually causes much less hearing loss than does noise exposure. The simulator shows this by demonstrating the hearing loss due to age alone. The age-related losses are usually mild and affect the highest frequencies the most. The additional and more sub stantial hearing loss due to noise can be added on top of agerelated loss to show the effects of noise, especially on speech frequencies. Gender Males tend to have higher levels of hearing loss than females who have had the same noise exposure, so the program allows the user to specify the simulated worker's gender. Population Distribution Noise does not affect everyone to the same extent. To ac count for variations within the population, the ANSI S3.44 standard specifies expected hearing loss for different 51 population fractiles. The program allows the user to specify the 0.1, 0.25, 0.5, 0.75, and 0.9 fractiles. For instance, a worker at the 0.1 fractile would have more hearing loss than 90% of the equally exposed population. Those at the 0.75 fractile would have more hearing loss than just 25% of the population. One use of this would be to show the range of impairment that could be expected. For instance, the 0.9 fractile could be used to reach the workers who believe (for whatever reason) that they may be less susceptible to noise-induced hearing loss. This 0.9 fractile level could be presented as the minimum loss that would be expected based on the set exposure, and that most workers exposed at this level will have even more hearing loss. Most workers will have no way of knowing their susceptibility to noise, so this control should usually be set on the expected population median (0.5 fractile) to start with. Invert Loss Once a loss is simulated, it can be "inverted" by clicking on the "Invert Loss" button. This has the effect of raising the sound level by an amount equivalent to the hearing loss at each frequency. To an imperfect extent, the boosted playback can compensate for a hearing loss to give a trainee an idea of what it would be like to regain his/her normal hearing. There will inevitably be imperfections in this illusion, however. The fidelity limitations of any mechanical playback system and the complexity of the auditory system make it impossible to "reverse" a hearing loss perfectly. Also, in cases of severe hearing loss, boosting the sound enough to compensate for a large deficiency could generate hazardous sound levels. Still, presenting hearing-impaired trainees with an approximation of normal hearing should be sufficient to show how much they have lost and reinforce how valuable their remaining hearing capacity is. Preset Generic Losses For a quick simulation of a hearing loss, there are three preset generic levels of noise-induced hearing loss that can be selected: mild, moderate, and moderate/severe. In each case, the greatest amount of loss is shown at 4000 Hz, with surrounding frequencies impaired to a lesser extent. Frequency Sliders A row of 10 slider controls for different frequency bands permits even more flexibility. Most of the time, frequency bands are automatically adjusted by the software to reflect a predicted hearing level as a result of noise exposure. They can also be directly manipulated by the user. This could be done, for instance, to enter a trainee's actual audiogram directly into the simulator program. Then, others who have no hearing loss could, in effect, hear with the same ears as the person whose test results were entered. Also, since each slider can be manipulated independently, the user can illustrate the effects of a hearing loss in each frequency band. For instance, a warning beeper may become much less audible as a result of a loss in a single frequency band. SCENARIOS The full power of the simulator can be shown by working through some instructive scenarios. Some ofthese were alluded to above, but a good training plan using the simulator should work through a series of scenarios using trainee input to tailor the lesson to the audience. Below is a list of some of the many possible scenarios that could be presented. OLDER WORKER, NOISE EXPOSED This scenario is one of the most important ones to include in a training session, especially with younger workers. A hypothetical older worker, perhaps nearing retirement, would be described. He (or she-the program can simulate either) should be characterized as in the range of 55-65 years old with 35-45 years of exposure to 90-100 dBA. Selection of numbers in these ranges can depend on what is typical in the user's workplace or industry. The trainer can demonstrate the significant hearing loss this worker will have going into retirement. exposure. This will serve to counter any assumption that the first worker's hearing loss was a natural consequence of aging. Instead, they will see that a relatively small amount of highfrequency loss is expected in older workers, but that noise exposure is responsible for much more of the damage. MID-CAREER WORKER Especially if the training class includes a large number of mid-career trainees, a worker with 10-20 years of exposure should be simulated. Based on this worker, several pro gressions can be followed. For instance, additional exposureyears can be added to show the accumulation of more hearing loss. It also allows comparison with an older nonnoise-exposed worker, which then allows the trainer to make the point that, with enough exposure, a 30-year-old worker may have, in effect, 50-year-old ears. INDIVIDUALIZED: INVERT LOSS OLDER WORKER, NO EXPOSURE Immediately following a demonstration about a hypothetical noise-exposed older worker, the trainer can set the exposure years to zero and simulate an equivalent worker with no The simulator can also be used as an individualized training and counseling tool. For instance, the trainer can show a worker how his/her hearing test results can be entered directly into the simulator using the frequency band sliders. By next selecting the "Invert Loss" function, the trainee can be given a 52 hint of what his/her hearing would be like if the hearing loss had been avoided. Switching back to the original loss profile, the trainer can then drag the sliders down to show the additional loss that would occur after further noise exposure. MAKING ADAPTABLE CONTENT Making a training product that can be adapted to several different formats can be either easy or difficult. Obviously, very different formats (say, a professional quality video and a small informational sticker) will make for a challenging conversion. Likewise, similar formats (say, an informational card and a briefbrochure) will be relatively easy. Regardless of how different the formats are, the conversions will be even easier if some simple steps are taken while preparing the content. Extra difficulties arise in adapting content that was not designed with conversion in mind. For instance, a training product is often developed for one medium and then shelved. When it becomes clear later that it would be beneficial to have other versions of the product, a considerable amount of new adaptation work usually needs to be done. For instance, if a video is produced and later someone decides to turn it into a booklet, they may then need to transcribe the narration for editing into printed text and re-photograph the visual elements. With a small amount of forethought and planning, a core set of content for a training product can be developed and used to "spin off' multiple versions. This will not eliminate all the work needed to tailor the content to different media, but it will reduce it considerably. TEXT Even the most visual training products usually have some text component. A video may have a script to be read by actors or a narrator. Signs and emblems are often accompanied by a user's or instructor's text. Many versions can be extracted from a single "master" text with appropriate modifications. This will be easier if the master is kept as a simply formatted, comprehensive electronic document. One good way to start this is to build a simple HTML Web page that's accessible to the development team. The team can then view and revise the master on a shared Intranet site. If HTML formatting is kept to a minimum (heading tags, simple tables, etc.), the resulting text can be easily imported into a word processing or desktop publishing package for more extensive formatting. PHOTOGRAPHS AND OTHER STILL IMAGES If producing video, take still pictures at the same time. For instance, commercial movies have almost always had profes sional photographers take "publicity stills' during filming. These still photos are almost always sharper and better posed than an individual frame from the movie and are essential for posters and other marketing materials. They also become useful later for books and other publications about the film. Even though training videos are produced on a much more modest scale than commercial movies, their example can still be followed. It is much easier to take still photographs of a scene set up for a video than to re-create it later. Also, although still frames can be extracted from a video stream, the results are of far lower quality than a decent still photograph. A high-quality digital master of each photograph should be kept in the development team's archive. If the originals were taken using conventional photographic film, a digital master can be made by scanning the negatives with a film scanner, or many photo labs will create high-quality digital images on a CD at the time of processing. From these master digital versions, smaller, faster-loading files can be converted using photo editing software. For printed materials, the images should have their resolution reduced very little, if at all. DRAWINGS, DIAGRAMS, AND ARTWORK As with photographs, it will be easiest to generate different versions of illustrations if there is a high-quality digital original. For these types of images, the best electronic format is referred to as "vector-based." For example, Windows metafiles, PostScript7 files, and most illustration software files are considered vector-based. These can be kept in vector format when used in document preparation or presentation software, but should usually be converted to a bitmap format (for example, GIF) for Web pages. Most illustration software will convert vector drawings into a bitmap of whatever size is needed for a Web design. AUDIO Sound recordings should be maintained in uncompressed digital format (for example, WAV). This can be later compressed, ifneeded, for limited-bandwidth presentation over the Web, but the compression cannot be reversed to obtain the original sound quality. For instance, the sounds used in the hearing loss simulator were recorded in CD-quality uncompressed digital format (44,100 16-bit samples per second). These sound files were used without further com pression in the full package and PowerPoint versions, but were compressed to 128 bit/sec MP3 format for the Web pages. This enabled significant reduction in sound file sizes; the "normal male" recording was reduced from 3.2 to around 0.5 MB with very little perceptible loss in sound quality. 7 Adobe Systems, Inc., San Jose, CA. 53 VIDEO New ways of showing video content are rapidly becoming practical. Videocassettes almost completely replaced film for training in the early eighties. Now, videocassettes are facing competition from digital versatile disks (DVD), streaming Web-based video, and other new technologies. The best way to keep video in a form that can readily be adapted is, again, to maintain a high-quality digital master. This has become relatively easy with the advent of inexpensive consumer-grade miniDV equipment. These camcorders and other devices connect to a computer though high-speed ports, and the down loaded video can be archived and edited with no further loss in quality. Basic video editing tools are now included with current computer operating systems for Windows and Macintosh8 systems, and more flexible software is available for less than $100. For use in a training room, the edited videos can be copied to tape a regular VCR or "burned" on a DVD or CD writer. If the video is also intended to be viewed on the Web, it can be converted to a compressed streaming format such as RealMedia9 or Windows Media10 (ASF) using tools that are inexpensive or even free. EVALUATING THE SIMULATOR'S EFFECTIVENESS The ultimate goal of all three versions of the hearing loss simulator is to reduce hearing loss by motivating workers to take self-protective actions. Behind this statement is a complex process that has several steps, each of which could be eval uated. First, has the message been communicated? That is, do the trainees understand that exposure to hazardous noise over a long enough period of time will result in an irreversible hearing loss? Next, how motivating is the message? How strong is the desire or intention to take action relative to all the trainees' other desires and needs? Third, what (if any) be havioral change resulted? Do the trainees maintain the noise controls on their equipment better? Do they wear earplugs or other hearing protection more often? Finally, the true outcomes must be evaluated, that is, do the trainees avoid noise-induced hearing loss as a result of their actions? INITIAL REACTIONS TO ALL VERSIONS The simulators are brand new, so evaluation is in just the first stages. Currently, NIOSH is working with organizations that want to use the simulators in their training to collect feed back from trainees. This feedback consists of questions about trainees' reactions to the simulator. Was it easy to understand? Could they hear the difference between the normal and sim ulated loss conditions? Did they learn something new? An swers to these questions will help refine the simulators and provide information about how best to deploy them. This eval uation is also appropriate to all versions of the simulator. KNOWLEDGE AND BELIEFS The next evaluation will look at how effective the simulators are at imparting knowledge and changing beliefs. Do trainees have a better understanding of the relationship between noise, exposure time, and hearing loss? Do they know that noiseinduced hearing loss is permanent? Are they still susceptible to the "noise myths"? Do they intend to take any specific actions to protect their hearing? These factors can be assessed through brief questionnaires or interviews. Ideally, they will be assessed at three points: before training, immediately after training, and several weeks following training to determine how much information was retained. For the full simulator and the slide show version, this information can be collected by the instructors. The Web version will offer the opportunity to collect this information online from users who agree to provide it. While asking online users to provide information can be convenient for both the users and the developers, there is much less control over who participates and other conditions that could affect the validity of the data. Consequently, the Web version will probably be evaluated with a known sample of participating users. BEHAVIORS AND HEARING LOSS OUTCOMES Changed knowledge and beliefs do not necessarily translate into effective hearing conservation actions, however. The be havioral and illness outcomes of the training, especially for the full version and the slide show version, will be investigated. The Web version, because ofthe otherwise beneficial openness of the Web, does not lend itself to this type of full evaluation. The NIOSH Hearing Loss Prevention Unit (HLPU) will be used in these efforts. The HLPU is a mobile testing trailer that can be taken to any training site for detailed hearing evaluations. This facility contains a system that can easily test one hearing conservation behavior: Correct use of earplugs. The multistation earplug fit-testing system can be used to de termine, through the use of specially designed headphones, how much noise reduction is achieved at each frequency. Better trained and motivated workers are able to obtain significantly more protection from their earplugs (Berger et al., 1996). If the simulator motivates workers to protect their hearing, the trained workers can be expected to take the time to fit their earplugs better. While it is important to evaluate hearing protection be haviors under controlled settings, behaviors at the workplace are a better predictor of long-term hearing conservation 8Apple Computer, Inc., Cupertino, CA 9RealNetworks, Inc., Seattle, WA. "Microsoft Corp., Redmond, WA. 54 efforts. In this evaluation, the hearing conservation actions taken by workers on the jobsite will be tabulated. For instance, do they maintain the noise control devices and treatments on their equipment? How many suggestions do they make about reducing noise? How well do they comply with administrative controls that are in the site's hearing conservation program? Finally, the ultimate outcome is the reduced incidence of noise-induced hearing loss. This can be assessed by long-term tracking of hearing levels as measured by a standard audiogram. Effective training should result in a lower rate of measurable noise-induced hearing loss. By tracking hearing levels, particularly between 3000 to 6000 Hz, changes in hearing thresholds that may reflect either reduced or continued noise exposure can be detected. While no one can determine whether the noise exposure occurred at work or off the job, it's not really necessary to distinguish between the two for these training efforts. These in-depth studies will be most feasible with the full simulator and the slide show version. Effective training will teach workers to protect their hearing regardless of where they are. The training message should emphasize that workers' responsibility for their own health does not begin and end at the front gate. Maintaining their hearing will have a positive impact on their work and the overall quality of their lives. REFERENCES American National Standards Institute (ANSI). 2001. American National Standard determination ofoccupational noise exposure and estimation ofnoiseinduced hearing impairment. ANSI S3.44-1996 (R2001). New York. Berger, E.H., J.R. Franks, and F. Lindgren. 1996. International review of field studies of hearing protector attenuation. In Scientific Basis of NoiseInduced Hearing Loss, A. Axelsson, H. Borchgrevink, R.P. Hamernik, P. Hellstrom, D. Henderson, and R.J. Salvi, eds. New York: Thieme Medical Pub., Inc. Pp. 361-377. National Center of Health Statistics (NCHS), Public Health Service. 1993. Vital and health statistics: Health conditions among the currently employed. United States, 1988. DHHS (PHS) Pub. No. 93-1514. Prince, M.M., L.T. Stayner, R.J. Smith, and S.J. Gilbert. 1997. A re examination ofrisk estimates from the NIOSH Occupational Noise and Hearing Survey (ONHS). Journal ofthe Acoustical Society ofAmerica, vol. 101, no. 2, pp. 950-963 lSiosh Delivering on the Nation's Promise: Safety and health at work for all people Through research and prevention For information about occupational safety and health topics contact NIOSH at: 1-800-35-NIOSH (1-800-356-4674) Fax: 513-533-8573 E-mail: pubstaft@cdc.gov www.cdc.gov/niosh DHHS (NIOSH) Publication No. 2002-156