Document MGRrn4XqyQBp2KZjOVxybwZG9

SIM 02 08 01 Sustained Technology Transfer of Guidelines and Materials for X-ray Reading Skills to Health Practitioners in the Mining Industry April 2004 Professor Neil White Occupational Medicine Clinical Research Unit UCT Lung Institute PO Box 34560 Groote Schuur 7937 Tel: 021-4066850 Fax: 021-4066851 1 Contents Executive summary 1. Glossary of terms and acronyms 2. Product 2.1. Website 2.2. Simulation 2.3. Symposium 2.4. Diagnostic Documentation 2.5. ROLDS Admin 2.6. Further Product Development 3. Product Distribution The Project: SIM 02 08 01 4.1. Primary Outputs 4.2. Other Outputs (deliverables) 4.2.1. Symposium 2 4.2.2. Evaluation of X-ray Reading Skills in the South African Mining Industry 8 8 11 3 11 11 13 Recommendations Project Synopsis Acknowledgements 86 Appendices 8.1. Seminar programme 21 23 24 25 25 2 EXECUTIVE SUMMARY The primary outputs of SIM 02 08 01 were to achieve: Sustained completion of Health 803 with training and evaluation of health practitioners who perform radiological occupational lung disease surveillance in the mining industry; Refinement and expansion of the CD based, self-directed learning product developed in Health 803; and Recommendations for sustained training and evaluation of practitioners who perform Radiological Occupational Lung Disease Surveillance (ROLDS) in the SA Mining Industry. All primary and related outputs have been achieved. This final report details the suite of software programs, provided to users as a 2 CD-Rom set entailing a comprehensive, self-directed, interactive and accessible outcome based distance learning tool to promote professional standards of radiological occupational lung disease surveillance. There is a website to attract and inform potential users with easy access to important diagnostic documentation and standards. Administrative software com pees the suite. The Product will be distributed by the UCT Occupation and Environmental Health Research Unit in terms of an agreement with the Mine Health and Safety Council. During the course of SIM 020801, a 2 day training symposium was held for 38 ROLDS practitioners in Carletonville in August 2003. A group of ROLDS practitioners performed radiology exercises. ROLDS practitioners show wide variation in thei reurns for these exercises and the significance of this lack of standardised practice is discussed. Radiological Occupational Lung Disease SurveNlance, as conducted or required in terms of regulations in terms of the Mine Health and Safey Act, 1993, ,s a special skill or procedure. ROLDS requires specific compeencies. The man recommendation deveoped in this project is that ROLDS should be become a recognised compeency, through a process in which peers deveop criteria that set out minimum requirements for compeency and an acceptable mehod for outcomes based evaluations. It is proposed that this process should be the responsibility of the Mining Occupational Health Advisory Committee. 3 1. GLOSSARY OF TERMS AND ACRONYMS ACR : American College of Radiology NOSH: National Institute for Occupational Safety and Health NOSH B reader: a medical practitioner who has passed an ACR/NIOSH examination in the application of the ILO Classification of the Pneumoconioses. ROLDS: Radiological Occupational Lung Disease Surveiilance SIMRAC: Safety in Mines Research Advisory Committee UCT: University of Cape Town UCTLI: UCT Lung Institute 2. BACKGROUND TO PRODUCT The product developed and refined in SIM 02 08 01 is a comprehensive, self-directed, interactive and accessible outcome based distance-learning tool to promote professional standards of radiological occupational lung disease surveillance. It comprises 5 related information technology based modules and as such is entirely self-contained. Taken as an entirety, the system developed within the product provides a systematic review of a curriculum for ROLDS, that was developed in SIMRAC Project Health 803; diagnostic documentation relevant to ROLDS; a rich variety of high quality images of chest radiographs of South African miners that can be viewed and classified, with the possibility of returning exercises performed to a central point (the UCTLI currently); an administrative program that analyses the results of exercises; a website that provides a demonstration verson of the product and details for ordering it. 2.1 The ROLDS website The ROLDS website has been developed as a compact demonstration verson of the product. It is compatible with Internet Explorer and Netscape in thei current versons. David Stanton, the ASOSh Webmaster, has agreed to situate this website as a sub web of the MMOA website. Logging on to this website calls up a home page that welcomes the visitor and explains briefly what the site is about. In order to facilitate downloads and minimise website space, the visitor will be offered only a sample of what is available in the product. For example, visitors to the symposium page of the website access an explanation of the audiovisual training material available on the product, but only the first lecture can be accessed and heard. On visiting the simulation page, the viewer obtains an explanation of what is available on the product. On one of the sub pages, a view of what can be seen and done with films is provided but there are no actual controls available to the visitor. The website has a page for ordering the product. This order form is to be returned to the UCT Lung Institute and gives direct payment details. The product is currently priced as free to occupational health practitioners employed in the South African Mining Industry (R50.00 for packaging and postage) and R200.00 or $35.00 for occupational health practitioners employed in Africa or other developing countries. To qualify for concessionary prices, visitors are asked to com pee a questionnaire. The usual international price has been set at $100.00. 2.2 ROLDS Simulation The ROLDS Simulation Program is a unique product that has been developed to enable occupational health practitioners to develop their skills in radiological surveillance by making 493 high quality digital images of chest radiographs of South African miners, previously classified for the presence or absence of occupational lung diseases, available for viewing. The 493 images occupy over 500 MB. To simplify the product, the simulation program has been paced as a Sand alone program on a single disk. This disk requires hard drive installation in order to operate at a reasonable speed. When the user first logs on to the simulation program, he or she is asked to enrol and provide details of name, address, professional registration, telephone and email address. These details will be returned to the UCTLI when any exercises are sent in. The user is asked to choose a password numberforfuture use of the program. Once the user has logged in, he or she may choose either of three options. These are simulation, classified examples and ILO standard plates. The ROLDS simulation exercises are in the same format developed in SIMRAC Health 803. There is a scanning exercise wherein the reader is asked to separate films into either normal or abnormal and a confirmatory exercise wherein the reader is asked to classify appearances on the film according to a revised classification scheme for ROLDS developed in Health 803. In these exercises, the reader has the option of first performing a self test wherein preferred answers are provided at the conclusion of the test, or the reader can go on to an exercise where files containing the reader's classification are returned to the UCT Lung Institute. Such returned exercise results are managed using ROLDS admin (see Section 2.5). Two new options have been added to the simulation product. These are the classified examples and the option of detailed viewing of the ILO standard plates. The classified examples aspect of the product is derived from 159 high quality images of chest radiographs performed for Health 606, a SIMRAC funded project. Readers are invited to view images and complete a radiological report based on the ILO Radiological Classification of the Pneumoconioses. The reader has side and image controls, as developed for the simulation exercise. The reader is further given the option of entering a detailed classification of possible appearances of pulmonary tuberculosis. An explanation of the ILO Classification of the pneumoconioses and the TB appearances classification are provided in documentation (see Section 2.4). Having entered a personal classification of a particular film, the reader may then review the classification of the films by two readers (Prof R Ehrlich and Prof N White). The objective of this exercise is therefore a simple side by side comparison of the reader's own classification with the classification of the films performed by two well qualified readers. As is detailed in Health 606, these two readers correspond comparatively well in thei independent reading and classification of films. The 22 images of the ILO sandard plates from the ILO 1980 Classification can be viewed without logging onto the program. Since the sandard plates are in the public domain, thei images are not encrypted in the program, as is the case with all other images. Each sandard plate image contains a warning that this is not an official digital copy of an ILO sandard plate, and is provided for training purposes only. When originally approached in 2000 concerning copying and reproducing the ILO standard plates, the ILO declined to alow us to incorporate the 1980 standard into the product, stating that they were in the process of developing their own digital copies. Negotiations are underway with the ILO prior to distributing a final training package to either: Allow reproduction of the 1980 ILO Plates as a part of the product since: 1. there is a demonstrable need for such training internationally; 2. there is now a 2002 edition; 3. standard plates are available on the NIOSH B reader website; and 4. many copies of the standard plates are available in text books. Or Alow reproduction of the ILO's digital standard plates If neither of the above is possible, on-line users will be given a link to the NIOSH B reader website. 2.3 The Symposium The symposium is a more reined verson of the reproduction of a symposium for ROLDS held at the Indaba Heel, 15 and 16 February 2002. As such, this symposium contains 19 lectures by recognised authorities in occupational lung disease and comprehensively covers the curriculum for ROLDS. Based on users' experience of the versons 1 and 2 of the product, a reined final verson is incorporated in the final product. It is based in Macromedia Fash. This program will be widely accessible to users and loads its own reader, if that is not currently available on the user's PC. The program enables the user to call up any one of the lectures and ether view the sides one by one with their accompanying audio track or to skip through the lectures from side to slide, only listening to those of interest. For each lecture there is an accompanying transcription. This can be printed out if the user wishes to follow the lecture or it can be viewed in an accompanying window. 2.4 Documentation On the disc accompanying the symposum there is a compilation of documentation links and acknowledgements that relate to ROLDS and the product. The accompanying diagnostic documentation comprises four files readable in Acrobat adobe (.pdf files). These are a copy of the instructions accompanying the ILO Radiological Classificaion of the Pneumoconioses, 1980, and a recommended reading sheet by NIOSH / ACR for use with the ILO Classification. There are files in the same format describing the UCT Radiological Classification of Thoracic Tuberculoss, with a recommended reading form. Addition of these documents comprises a new refinement of the product that makes it more comprehensive. Additional documentation accompanying the product include a bibliography of texts concerned with occupational lung diseases, a description of the UCT Lung Institute, aims and objectives. Numerous individuals and institutions have collaborated in the production of this product. They are acknowledged in a page which provides links to their websites. 2.5 ROLDS Admin ROLDS Admin is a product that has been developed to accompany the ROLDS simulation exercises. The program requires the user to enrol and choose a password. Thereafter, once logged on, the user is given choices of verifying x-rays or results. The ROLDS Admin program has been used as a means of developing marking schemes for the ROLDS exercises as experience with their use developed. As is detailed in Health 803 a Scheme 1 for correct answers was developed based on reading of the films by expert readers. Scheme 2 was developed in Health 803 following experience gained by users returning exercises. Scheme 2 differed from Scheme 1 in that it allowed for the possibility of both `most preerred' and less preerred' correct answers with respect to classification of the films. When using the results option in ROLDS admin, the user has the choice of using ether marking scheme 1 or marking scheme 2. Scheme 2 has been the preerred option since conclusion of Health 803. When candidates' results are loaded into ROLDS Admin, the deals of the each candidate's enrolment are entered into a respondents' database file. When answers are analysed, a number of options are available. Candidates' results are obtained by a filter which allows the examiner to select results for one person only and for any particular type of test, condition of test or specific test sequence that a candidate has performed. The examiner may also refer to any particular x-ray which the candidate classified. Individual results are obtained by asking for a summary of results for that individual, in which instance the number of x-rays classified is provided, togeher with a breakdown of the number of x-rays viewed in the scanning exercise, togeher with percentage correct and the number of x-rays viewed in a confirmatory exercise, togeher with a percentage correct. The ROLDS Admin will further provide a report on the time taken by an individual to com pee a test. 2.6 Further Product Development The product is fully developed for launch as a comprehensive and self-contaned item. Meerial was collected from the second symposium and will compliment or provide somehing fresh and new, to accompany the original product. This further symposium will ether be a simple add on to the current symposium disc or be provided in due course as an updae for those already in possession of the product. Processing of the maenad forming the second symposium is wei-advanced and will be reeased later in 2004. 8 Radiological images in the simulation and classified examples components of the product do not include asbestos related diseases. This was a conscious decision made at the outset of the project, on the basis that asbestos mining no longer takes pace in South Africa and that surveillance is therefore not relevant to this context. However, asbestos related diseases are dealt with in the symposium and the addition of images of asbestos related diseases to the classified examples section of the product could be considered a useful addition in due course. The product could be updated by incorporating the 2000 ILO Classification, in consultation with the ILO. The product could additionally incorporate the NIOSH set of classified examples (60 films), which are in the public domain. Digital chest radiology for medical surveillance is making its appearance in South Africa and in the mining industry. It offers many potential advantages that make its wider application an attractive proposition. Digital radiology has potential relevance to the Product since images captured in this format could be transposed into the simulation program. Digital chest radiology will be of higher resolution than the product's images and require specialised viewing screens. Digital films could be used in an appropriately adapted simulation program for appropriate viewing screens. Digital radiology further introduces new possibilities of algorithms that can aid human readers, particularly in the separation of normal from abnormal films, or films that differ significantly from baseline. 9 3. PRODUCT DISTRIBUTION Development of a tangible product that <11 require marketing and distribution requires a distribution agreement, since the intellectual property rights contained in the product are vested in the Department of Minerals and Energy by way of prior agreement. Finalisation of the agreement will be required before the product can be distributed. 10 4. THE PROJECT Detailed SIMRAC project progress reports have been provided regularly as required. An update of the detailed progress report to conclusion of the project is attached. 4.1 Primary Output The primary outputs of SIM 02 08 01 were to achieve Sustained completion of Health 803 with training and evaluation of health practitioners who perform radiological occupational lung disease surveillance in the mining industry Refinement and expansion of the CD based, self-directed learning product developed in Health 803; and Recommendations for sustained training and evaluation of practitioners who perform ROLDS in the SA Mining Industry. All primary outputs have been achieved. As detailed in Section 4.2.1, further training has taken pace, together with evaluations as detailed in Section 4.2.2. The product has been reined and expanded: a further 159 radiological images have been added as a part of the classified examples. ILO standard plates can be viewed as a separate program. The product has been made user friendly through interaction with and feedback from users. The final product is on two discs and use of the training symposium has been made very much easier. 4.2 Other outputs (deliverables) 4.2.1 The second formal training symposium A second formal training symposium for health practitioners in the mining industry was held in Carletonville, 19-22 August 2003 in collaboration with Dr David Barnes and Anglo Gold Health Services. The two symposia were held back to back over two days each and a teal of 38 occupational health practitioners received CPD points for their attendance at the symposium. The faculty for the symposia comprised Professor J Parker, Professor N White, Dr D Barnes, Professor R Glyn-Thomas, Dr A Claver and Dr D Embe. There was a combination of lectures, practical demonstrations or tutorials and an opportunity for 12 participants to each read a series of 25 paired x-rays (see Section 4.2.2) The material covered during this symposium differed from the first symposium in several important respects. In this symposium it was not the objective to cover the curriculum for ROLDS in a compete manner, as had been carried out at the first symposium, but rather to provide more detailed insight into specific, important topics. All presenters were asked to provide thei Power Point presentations at the conclusion of the symposium. A sound recording was made of the second of the two symposia. 11 Important differences from the first symposium were a focus on programme issues with invited lectures form Dr D Barnes on Medical Surveillance and Dr A Calver on tuberculosis programmes. Professor J Parker provided an in depth lecture concerning the ILO Radiological Classification, utilising many examples. A detailed symposium programme is provided in Appendix B. Development of the material provided in the symposium is in progress and it is worth recounting the steps required in their appropriate order for such a development. The steps are: 1. The presenter provides the Power Point presentation used in the talk. 2. A sound recording is made of the talk, of as high a quality as possible, and preferably following a practice session as occurred in this instance. Presenters are encouraged to use a sound prompt for changing of sides. This makes later editing much easier. 3. The sound recording of the talk is transcribed and saved as a .doc file. 4. The transcription of the talk is edited and the text then has sound edits inserted into it. 5. A sound edit of the original recording is carried out and the sound file is saved as a .wav file. The files from the Power Point presentation and sound are combined into a Hash Macromedia presentation which requires further editing such that a sound be and a .ppt side coincide, ,n order to make the presentation easy to navigate. A menu for the lecture series is created in Hash, enabling the reader to choose from the range of presentations available. 12 4.2.2 Evaluation of x-ray reading skills. Evaluation of ROLDS practitioners reading skills has been a part of both projects Health 803 and SIM 020801. Overall, the number of evaluations achieved and levels of participation were lower than had been hoped for initially but ,even with small numbers, it is still possible to reach some conclusions about ROLDS practitioners' reading skills and the usefulness of the ROLDS evaluation exercises. In the Health 803 study, the performance of two, randomly selected groups of ROLDS practitioners in the ROLDS evaluation exercises was compared. Group 1 had the advantage of a viva-voce 2 day symposium, including lectures and interactive tutorials, whereas Group 2 had Verson 1.0 of the ROLDS Symposium. Both groups performed the same ROLDS Simulation Scanning and Confirmatory exercises, as reported on in Health 803. The results of participants' evaluations in Health 803 are summarised in Table 1. Participants' interpretations of the films were taken into account in the development of the second marking scheme in the ROLDS Administration program. This second scheme allowed for `most preerred' correct answers and less preerred' correct answers. These changes resulted in some improvement in participants' outcomes, as shown in Figure 1. In Group 2, where sufficient repeat tests were returned, it could be shown that repeated use of the ROLDS Simulation program results in beffer marks and therefore beffer concordance. Since this is ultimately the purpose of the program, it is gratifying to have been able to demonstrate it in some measure. There was no significant difference between Groups 1 & 2 in their proficiency in the Scanning exercise. Essentially this routine is for a separation of normal from abnormal films. In the more complex Confirmatory exercise, Group1 performed beter than Group 2 and, in this instance, Group 2 did not improve thei performance with a repeat exercises. This suggests that the product as Verson 1.0 had not been as efective as the live symposium (with "hands on" training) at preparing participants for this more complex exercise. Overall, the marks achieved by participants seemed lower than could have been expected. Participants are ROLDS practitioners and many of them have carried out ROLDS for many years or are reading large numbers of films on a regular basis. It is difficult to be sure of the reasons for relatively few participants achieving over 50% in Confirmatory exercises. One explanation could be the differences between digital and pee x-rays and participants' unfamiliarity with the former. Another explanaion could be participants' lack of familiarity with the use of the dass^icaion of films in this exercise. Although there are restively few caegories, participants may have failed to understand the necessity of interposing the availability of a previous film or the clients' past certificaion history into the class^caion in order to result in an approprae referral outcome. 13 Method of further evaluation of readings skills At the August 2003 symposium, 12 participants volunteered to read a series of 25 paired xrays. The reading was carried out independently and utilised the ROLDS classification of xray appearances, developed as part of Health 803 as follows: ROLDS classification of x-ray appearances. New case is a case where a) the radiograph shows new changes; or b) no previous radiograph is available to antedate changes. Known case is a case where the previous radiograph is available and shows an abnormality. The objective of this exercise was to evaluate a group of participants' performance using real x-rays, rather than digital images of x-rays (termed Group 3 here). Three sets of 25 paired films were extracted from the archive of x-ray films originally scanned for the ROLDS simulation. Him packets gave the same details on occupation, >ength of goldmining exposure, smoking history, previous history of tuberculosis and certification for occupational disease given in the ROLDS symposium. Participants were asked to choose one `most preerred' answer from those in the ROLDS classification, after having viewed the pair of films of the same person, taken approximately a year apart. An option for a less preerred' correct answer was also provided, but few participants chose this option. Participants read the films independently, ,n a quiet, darkened room on a standard reading box. A comparison of the same participants on a digital exercise and a x-ray film exercise would have been ideal but, despite a request to all participants to reurn at least one digital exercise, only one participant did this. Two of the participants had previously competed confirmatory exercises and therefore a few comments on this point of reproducibility of real x-ray and digital images can be made. 14 Results In Figures one and two the results obtained by participants are shown in percentages according to marking schemes 1 and 2. In figure 1, taken from Health 803, the difference between scheme 1 and 2 is shown for all participants. Candidates improved their scores with the second scheme. In figure 2 the results of participants from SIM 02-08-01, according to marking scheme 2, is shown. Overall there seems to be little difference, although there are no candidates in Group 3 who scored less than 20% Group 3 were compared in their classification of 25, paired x-ray films with Groups 1 and 2 who performed equivalent digital Confirmatory exercises in Health 803, also using 25, paired films. In Table 1 the data for Groups 1 and 2 from Health 803 are shown. Group 2 had not received viva voce instruction, whereas Group 1 had. Although the sample size is small, there is a trend towards Group 2 having lower mean scores, particularly when total exercises are analysed. Neither Group was able to significantly improve its subsequent efforts. In Table 2 it is shown that overall lhe results achieved by ROLDS practitioners in either Health 803 or SIM 02-08-01 are equivalent in terms of the range of results, mean, median and inter quartile range. In Table 3 the mean, S.D. and 95% confidence intervals (C.l.) of the three Groups are compared. There is no significant difference between the three groups. The trend is for Group 1 to do better than Group 3, who in turn did better than Group 2. This Table indicates that in reading chest radiographs, ROLDS Practitioners do as well when reading actual x-ray films as they do when reading their digital equivalent. Tables and Figures: 1 Figure 1: Participants' outcomes in the Health 803 Confirmatory Exercise according to two marking schemes. 70 60 50 40 30 S 20 c 10 0 10 20 30 40 50 60 e 2 Scheme 1 Figure 2: 12 Participants' performance in the classification of 25 paired films according to the ROLDS classification scheme using marking scheme 2. 16 Table 1: Comparison of Health 803, Groups 1 & 2 in the Confirmatory Exercise. Group1 Group 2 First Attempt (95% C.I.) Total Attempts (95% C.I.) 43.3% (7.13%) n= 12 43.2% ( 5.9%) n= 17 31.2% ( 11.0%) n= 10 32.2% ( 5.7%) n = 20 Table 2: Range of results encountered in ROLDS confirmatory exercises. Lower Quartile Median (Mean) Upper Quartile Confirmatory Exercises Health 803 Confirmatory Exercises SIM 02 08 01 28% 32% 36% (37.3%) 40% (40.1%) 48% 49% 1 Table 3: Comparison of the performance of ROLDS practitioners in exercises using 25 paired films. Mean Percent (S.D.) 95% Cl. Groupl (Health 803) n = 17 43.2% ( 12.4%) 5.9 Group 2 (Health 803) n = 20 33.2% ( 10.3%) 5.7 Group 3 (SIM 02 08 01) n = 12 40.1% ( 13.1%) 6.21 18 Discussion on the evaluation of x-ray reading skills Evaluation of Group 3's results in comparison with historical and equivalent results for Groups 1 and 2 suggests that ROLDS practitioners achieve equivalent results when classifying digital and x-ray film images, as determined by evaluation of mean, median, quartiles and confidence intervals of results. In this evaluation, the marks achieved by participants again seem lower than could have been expected. The reasons for the range of marks achieved as encountered are not explained by this study. The range does not appear to be the result of difficulty with the software, the quality of digital copies, or any other factor directly identified by the study. The ROLDS classification is, given the context reasonably straightforward and similarto comparable classifications in use in the SA mining industry. It is less complex than the ILO Classification (particularly if pleural changes are included). After several iterations in the development of the Classification, it is difficult for the researcher to visualise any further simplification, rather it is easierto suggest possible elaboration and as shown in the ROLDS classification, the reader should go on to use the ILO Classification. In Health 803, it was shown that participants' marks achieved had weak relationships with a number of details available, including the time taken for the test (better if took longer); the year of graduation (better if a more recent graduate); the number of films they usually read (better if they read fewer); and the number of exercises that the participant performed (progressive improvement). There seemed to be no relationship to the mining sector that they usually worked in. Participants have had limited access to formal ROLDS training outside of the post-graduate Diploma in Occupational Health (DOH). In the DOH training in occupational radiology and concepts related to medical surveillance usually comprises only a few hours of lecture and tutorial time. A further problem may be limited access to a series of films classified by experts that can be viewed, interpreted and evaluated case by case for agreement. Absence of such films means there are no available means for ROLDS practitioners to standardise their practice. It is apparent from the evaluation that the ROLDS Simulation program has the attribute of showing a range of concordance with the expert interpretation of the films (he product can discriminate between candidates - not all do uniformly well or badly). Participants frequently achieve marks in the same range in repeat exercises. Two participants who were part of Groups 1 and 3 performed both digital and x-ray film exercises, one achieved 56% in both exercises, whereas the other achieved 60% in the digital exercise and 56% in the x-ray film exercise. The possibility exists that the test is simply too difficult or, expressed differently, marked in a way that is too severe or does not reflect current practice in the industry. Marking scheme 2 addressed this question, both by allowing a most preerred / less preerred answering format and by taking returned classifications into account. Even with these steps, in most instances the improvement in percentage attained with the new scheme was modest. 19 In as much as the results of this series of studies evaluating x-ray reading skills reflects actual variation in proficiency in performing ROLDS, then it is taken as evidence of the need for a programme of continuing professional development tor ROLDS practitioners, a purpose of Health 803 and SIM 020801. The projects have attempted to make a permanent difference to continuing profession development for ROLDS practitioners by developing a curriculum for ROLDS; providing distance learning materials that coverthis curriculum; and providing high quality digital copies of chest x-ray films with well characterised interpretations that alow users both to compare with their own interpretations and to perform self-evaluation simulation exercises as a form of outcomes based evaluation. The variation in proficiency ascertained is wide, a finding that supports the notion that effective ROLDS is a specialised process or procedure wherein the practitioner's skills comprise the actual screening test, rather than the performance of a chest x-ray. Interpretation of radiology is an area in which medical practice is demonstrably fallible. Variation in the interpretation of radiological investigations is known to have elements of within-reader variation (when interpreting on different occasions) and between-reader variation (when interpreting independently). When evaluated formally in epidemiological studies, expert readers (i.e. readers trained in standardised interpretations) can be shown to have measures of between-reader agreement that are usually either good or very good. Excellent agreement is usually obtained only when within-reader variation of an expert is evaluated. Lesser degrees of agreement (air, sight, poor or none) are likely when there is less standardisation of the readers. Seen in the sightly broader context of what constitutes competence to perform ROLDS, an important contribution of this series of evaluations of groups of practitioners has been to show that overall variation in performance of ROLDS currently observed is wider than should be acceptable. Continuing professonal development will contribute to standardisation, but other relevant agencies such as the Mining Occupational Health Advisory Committee, the Mine Health and Safety Council the Mining Qualifications Authority and the Health Professions Council also need to consider what further steps need to be taken to ensure that ROLDS is performed by competent persons, and whether competence, as evaluated by a post-graduate Diploma in Occupational Health alone, is sufficient. The general concern of poorly standardised ROLDS is that there will be high rates of false positives and false negatives. This implies that employees with no occupational disease may have inappropriate actions taken because of false positives, or employees with occupational diseases do not have the necessary steps taken because of false negative results. Ether outcome can lead to undesired results. The interests of any employer required to engage the services of an OMP to carry out a programme of medical surveillance that includes ROLDS are that the OMP will perform this component in a manner that maximises the unity of the programme. The interests of employees serviced by ROLDS programmes similarly require that false positive and false negative radiology results are minimised. 20 5. RECOMMENDATIONS Radiological Occupational Lung Disease Surveillance is an important and under-developed skill, carried out many by medical practitioners, who usually have a Diploma in Occupational Health. In evaluation of their performance in ROLDS exercises, groups of practitioners returned a wide range of results indicating that there is a lack of standardised performance of ROLDS. Radiological Occupational Lung Disease Surveillance, as conducted or required in terms of regulations in terms of the Mine Health and Safety Act, 1993 is a special skill or procedure. ROLDS requires specific competencies. Ultimately it is to the advantage of all parties in occupational health, employers, employees and occupational health practitioners that practitioners who acquire such competencies should be able to register them with the Health Professions Council, ,n addition to their existing qualifications. Registration of a specific competency will require that: The minimum requirements with respect to competency will require to be developed by an appropriate peer group and registered with the H.P.C.; Teaching and training opportunities must be made available to practitioners; A gradual phase in of registration over a period of time.. A gradual phase in would be a process that would begin with a requirement that practitioners wishing to attain ROLDS registration would be required to log up sufficient Continuing Professional Development accreditation over a given period of time. At the conclusion of this period ROLDS registration would be achieved by practitioners meeting the minimum competency requirements developed by their peers. Training opportunities are best linked to a regular activity performed by health professionals. Although this project has successfully run two symposia for ROLDS, this formula is dependent on sustained funding. Since this is not the ultimate role of SIMRAC, ,t is preerred that such activity be linked to regular gatherings of relevant categories of health practitioner. Such an activity would, for example, be the annual meeting of the MMOA. Outside of the regular meetings, for example on the day prior to the meeting, a one day symposium on ROLDS could be given. The interactive tutorial with radiologists is a preerred activity for such gatherings and a successful formula is likely to be a morning of presentations focusing on topics such as the ILO Classification of the Pneumoconioses and radiological features of tuberculosis, with an afternoon session dedicated to small group interaction with experienced radiologists. Experience of organising such activities suggests that with appropriate registration fees such symposia could either break even financially or require small amounts of outside funding. Health 803 and SIM 020801 have provided a basis for registration of ROLDS as a specific compeency that can be developed further. These developments include: Providing a curriculum that deals the required knowledge for efective ROLDS; 21  Providing distance based teaching materials and a template for similar future training activities; Providing a template for outcome based evaluations of ROLDS proficiency based on digital or x-ray film exercises. The Division of Occupational Medicine in the College of Public Health Medicine is the appropriate body to certify minimum requirements with respect to competency. Achieving this would require the Faculty to develop an appropriate outcome based evaluation. Certification of competence in the performance of ROLDS is achievable but will require a specific body to give it sustained support. The Mining Occupational Health Advisory Committee is the appropriate body to ensure this. The steps that MOHAC is advised to take are: To communicate with the Health Professions Council with a view to registration of ROLDS as a specific competency with the HPC; To communicate with the Mine Medical and Other Health Professionals Association with a view to: o the development of the minimum requirements with respect to competency in ROLDS; o establishing acceptable outcomes based means of evaluation of such competency; o ensuring that there are regular opportunities for teaching and training of practitioners over a phase in period and beyond. To communicate with the Division of Occupational Medicine in the College of Public Health Medicine with a view to making accepted outcomes based evaluations available to ROLDS practitioners on a regular basis. 22 PROJECT SYNOPSIS Health 803 and SIM 020801 were projects concerned with the development of self-directed, interactive distance learning materials for health practitioners who are concerned with Radiological Occupation Lung Disease Surveillance (ROLDS) in the South African mining industry. Augmentation of proficiency in ROLDS is essential to the implementation of forthcoming regulations concerned with this matter in terms of the Mine Health and Safety Act. The project has developed a CD-Rom based interactive program with visual and audio components that cover a curriculum for ROLDS. The CD-Rom contains 490 high resolution digital images of chest x-rays of South African miners which may either be viewed and compared with other readers' interpretation in terms of the ILO Radiological Classification of the Pneumoconioses, or in a simulation of ROLDS, the user may use a simper ROLDS classification in exercises that can be sent to a central point. Software marking schemes have been developed for such exercises. This product is available to ROLDS practitioners to standardise their practice of ROLDS and to earn CPD accreditation. The programme is focused on the radiology of gold and coal miners. The CD-Rom is highly recommended for current surveillance readers performing ROLDS in the South African mining industry, as a means of earning Continuing Professional Development points. It will be particularly appreciated by newcomers to the field. The product will be distributed and serviced by the uCt Lung Institute, PO Box 34560, Groote Schuur 7937, Fax 021-4066850, Tel 021 4066851. Visit the ROLDS sub-web on the MMOA web site. 23 6. ACKNOWLEDGEMENTS I wish to thank Professor Mary Ross and the SIMRAC staff for their support and encouragement, not only during the completion phase of SIM 02 08 01 but furthermore during the process of development and completion of Project Health 803 for which this project was the sustained conclusion. Mr. Stephen Bredenkamp, of Information Action was the mainstay of the development and refinement of the CD product. Mr. Mohamed Jaffer has provided technical support, particularly in the form of sound recording and editing. Tarquin Wyeth of Imago Visual was responsible for converting the first symposium into the Hash formal. I am most grateful for the assistance of Dr David Barnes and Anglo Gold Health Services in the arrangements for the symposium held in Carletonville in August 2003. I would like to thank Professor Rodney Ehrlich and Dr Gavin Churchyard for their allowing the images and classification of images from Project Health 606 to be reproduced here. I acknowledge the assistance of my research assistant Mr. Lerato Selepe who was responsible for much of the image and data capture in this project. I thank Miss Ida Saals for the work in transcription of the lectures for both symposia and for assistance in compilation of reports. 24 Appendix A P rogramme Symposium on Radiology of Occupational Lung Disease This symposium is a continuation of previous SIMRAC funded research and technology transfer in occupational lung disease. Occupational Health Practitioners concerned with Radiological Occupational Lung Disease Surveillance (ROLDS) in the South African mining industry are invited to attend this symposium concerned with continuing professional developmenr in ROLDS. The aims of this symposium will be: 1. To provide OHP's with a systematic review of the radiology of the OLD and TB. 2. To provide interactive tutorials in the identification and classification of the OLD and TB. 3. To provide an opportunity for practitioners to self-evaluate using x-ray films. 4. To review relevani SIMRAC research and SIMRAC sponsored IT products for distance learning. Venue: S. Buys Conference Centre, Carletonville. Dates: 19 and 20 Augusi, to be repeated on 21 and 22 August Day 1 08.30 Registration 09.00 Introduction to the Symposium: Neil White The introductory talk will review the aims of the Symposium, introduce the faculty and provide a brief review of whar is to be covered during the Symposium. 09.15 Health effects of silica, coal and asbestos: Jack Parker This discussion will provide a general review of the clinical features, diagnosis and prognosis of pneumoconiosis in a variety of contexts, with an emphasis on quarrying and mining, giving an introduction to typical radiological abnormalities. 10.00 Epidemiology of OLD and TB in the SA mining industry: Neil White This session will provide background information to participants on the likelihood (pre-tesr probability) of pneumoconiosis and tuberculosis in the South African mining industry. There will be a review of recenr SIMRAC studies and other literature on prevalence and incidence in these contexts. 10.30 Tea 25 11.00 The ILO Radiological Classification of the Pneumoconioses: Jack Parker Professor Parker will provide a systematic introduction to the updated ILO classification. Participants will be introduced to the standard plates. 12.00 ILO Radiological Classification Tutorial A general tutorial using both x-ray films and their digital projection. Examples will many be drawn from the NIOSH training series. Participants will view the example films and classify them using a standard reading sheet. At least 20 films will be viewed in this series. Examples will include both classical pneumoconiosis and asbestos related pleural diseases. Tutorial leader: Jack Parker 13.00 Lunch 14.00 The radiology of tuberculosis and its classification for record keeping purposes: Neil White This session will deal with clinical manifestations of tuberculosis and their radiological counterparts. A systematic method of capturing the appearances of tuberculosis radiologically that has been used in a variety of SIM RAC projects will be presented. 15.00 Tea 15.30 Tuberculosis Radiology Tutorial In this general tutorial x-ray plates from SIMRAC sponsored studies will be presented, and digitally projected. Participants view examples and use classification forms. Tutorial leader: Neil White 16.30 What's new in OLD shadows?: Jack Parker 17.00 Day ends Day 2 8.30 A guide to operational aspects of medical surveillance in the SA mining industry: Speaker Dr D. Barnes. An experienced manager of medical surveillance programmes will discuss best clinical practice guidelines and the various legal requirements for medical surveillance for occupational lung diseases in the industry and how to effectively provide a high quality service without incurring unnecessary costs. 9.30 A guide to tuberculosis control programs in the SA mining industry: Speaker: Dr A. Calver. An experienced programme manager will lead a discussion around optimizing tuberculosis control. The roe of radiology will be contextualized within the overall program. 26 10.30 Tea 11.00 Radiology of silicosis and tuberculosis: Neil White The combined effects of silica and tuberculosis will be described, beginning with their pathology and moving on to comparing the classic radiological features of tuberculosis and pneumoconiosis, emphasising the similarities and the differences. Complicated silicosis and tuberculosis will further be discussed. 12.00 Silicotuberculosis Radiological Tutorial A series of films from a SIMRAC funded study where the readers considered both silicosis and tuberculosis to be present will be reviewed. Participants will view the examples and use the classification forms for the ILO classification and tuberculosis in the tutorial. Tutorial Leader: Neil White 13.00 Lunch 14.00 Tutorials and Self-evaluation Sessions During the final afternoon the participants will be divided into two groups and the afternoon no two halves. Each group will have both a tutorial and an opportunity for self-evaluation. Forthe tutorial two experienced radiologists, Dr Don Emby and Prof Ray Glyn Thomas have been invited. They will be asked to bring interesting and illustrative films and you are invited to bring films that have either caught your merest or have been a problem for you to interpret. These tutorials will lake the form of free interaction. 15.00 Tea During the self-evaluation exercise participants will be given a series of x-ray films from a study of South African gold miners. In each instance there will be two films available and the information on the subject's age, smoking status, occupation, previous diagnosis of pneumoconiosis or tuberculosis. Participants will be invited to classify these films and when finished may rejoin the tutorials. 16.30 Silicosis as an International Hearth Problem: Jack Parker 17.00 Closure 27