Document wrOMDoabrdGzbGNZaggNw9pzd

Prepared by: CC: Date: To: Janet E. Collins, Ph D., R.D. Jay Vroom January 30, 2018 Dr. Nancy Beck, Mr. Rick Keigwin Comments on Office of Pesticide Programs' Framework for incorporating human epidemiologic & incident data in risk assessments for pesticides. Issue date: December 28, 2016. Found here. Questions from CLA members for open discussion/dialogue What is the appropriate way for EPA to integrate epidemiology results with toxicology results? What is the appropriate way to address the lack of published information on null results for epidemiology studies (i.e. selective reporting for results for specific pesticides)? If a publication does not provide sufficient disclosure of analysis, sensitivity, confounding, or dose-response, does OPP have a plan or requirement to either have the authors complete the required analyses or to acquire the raw data? General comments The US Environmental Protection Agency (EPA or Agency) Office of Pesticide Programs' (OPP) Framework for Incorporating Human Epidemiologic & Incident Data in Risk Assessments for Pesticides [EPA-HQ-OPP-2008-03 16) (2016 Framework)], states it is "not intended to be a reviewer's guide or manual or Standard Operating Procedure for assessing or using epidemiology data." Rather, OPP characterizes the 2016 Framework as a description of overall conceptual scientific consideration when evaluating epidemiology studies and highlights the importance of human data for risk assessment. The pesticide industry agrees that a systematic and transparent review process is necessary when evaluating epidemiology data. The 2016 Framework does incorporate some aspects of the 2010 SAP recommendations, such as developing an assessment of study quality. However, there remain important limitations in the 2016 Framework that require clarification and in turn, reduce its effectiveness. While this request is not exhaustive, as we continue to have concerns about the Framework and its implementation, three important areas are discussed more fully below and include: 1. Integration of epidemiology and EPA required guideline studies (the Weight of Evidence assessment) 2. Quality assessment (how will "appropriate and sufficient be defined?) 3. Transparency of epidemiology data (how will unpublished results be obtained?) * Representing the Riant Science industry 1156 15ih St, N.W. * Washington. D.C, 20005 202.296.15S5 * 202.483,0474 fax www.cropiiieamerica.org Sierra Club v. EPA 18cv3472 NDCA Tier 3/4 ED 002061 00038400-00001 Discussion Topics on the "2016 Framework" Page 2 of 5 It is our opinion that incorporating these suggestions into the 2016 Framework will enhance the potential to provide for a process that instills greater confidence in OPP and Agency integration of epidemiology into the human risk assessment process. 1. Integration of epidemiology and EPA-required guideline studies In its 2016 Framework, OPP has provided little guidance for weight of evidence (WOE). The OPP promises to use a WOE analysis, and that conclusions will be made on the preponderance of information rather than relying on any one study. The 2016 Framework presents existing guidance documents and continues to draw from the Bradford Hill criteria (guideline elements). Specific aspects from the Bradford Hill elements are listed. For many of the Hill elements, OPP first makes a strong point and then gives an excuse to dismiss it. In other words, the list is correct but the interpretation is weak and seemingly arbitrary. These elements were designed to be used collectively, to dispassionately evaluate a body of literature. When there is evidence of a causal effect, the reviewer does not have to offer excuses for each element. The 2016 Framework recommends using multiple data sources and information from different disciplines. One might interpret this as OPP will use toxicology when reviewing epidemiology. However, it appears that in practice, OPP is using epidemiology to the exclusion of toxicology. There is little discussion or even recognition in the 2016 Framework about the robust state of the science of registered pesticides. The 2016 Framework reads as if all epidemiology studies are new discoveries on new pesticides. The document makes little effort to incorporate the known with the new. OPP cannot both require Good Laboratory Practice-compliant studies and then turn a blind eye to their results when evaluating epidemiology studies, which often are conducted for a completely different purpose. The EFSA Panel on Plant Protection Products and their residues, in its assessment of the methodological limitations of pesticide epidemiology studies (EFSA, 2017), discussed incorporation of pesticide epidemiology studies into the regulatory risk assessment process [EFSA Journal 2017; 15(10):5007], While recognizing the potential important role that human health outcomes can play, the EFSA PPR (2017) cautioned against use of a single, not replicated, epidemiological study in the absence of other studies on the same substance, a practice the EPA did not follow in the assessment of the organophosphates. OPP discusses the NRC 2007 Tox21 report, that a strong WOE draws from the "best available information" from multiple data sources. OPP recognizes that epidemiology studies tend to report on widely used pesticides, which also have a significant body of data from toxicology, exposure, pharmacokinetics, mode of action (MOA) and adverse outcome pathway (AOP), etc. On page five of the Framework, OPP states that "it is noteworthy that the availability of a fully elucidated MOA/AOP is not [a] requirement for using epidemiology studies in human health risk assessment." When the epidemiology data and the guideline toxicology data are inconsistent and contradictory, either the WOE must be extremely robust with a detailed scientific rationale as to how and why the epidemiology, or toxicology data carry the greater weight. Neither the discounting of robust epidemiology data indicating an adverse association not observed in the guideline studies, nor the discounting of robust toxicology data for weak epidemiology data Sierra Club v. EPA 18cv3472 NDCA Tier 3/4 ED 002061 00038400-00002 Discussion Topics on the "2016 Framework" Page 3 of 5 suggesting an association not observed in the guideline studies or without a plausible MOA, is acceptable without a strong WOE j ustification to support the decision. It is not clear how data with a defined MOA will be integrated with information lacking a plausible MOA. 2. Quality assessment The integration of toxicology and epidemiology relies upon using the "best available information." Discussion of ideal study elements and quality criteria are part of the 2016 Framework. The Framework has a table on page 24 with five parameters for quality considerations ranked as high/moderate/low. However, there is little direction or discussion regarding how these will be used or how individual studies will be ranked. The rubric for study quality considerations (Table 2) uses vague terms such as "good", "moderately good" and "lowquality." Further, there is no discussion in the 2016 Framework about interpretation of study quality if one or more parameter scores low. In recent evaluations of glyphosate and the organophosphates, OPP classified epidemiology studies as being high, medium and low quality without providing details of its interpretation on individual elements. Notably, the quality evaluation was inconsistently applied in OPP's 2016 Updated Literature Review on Neurodevelopment Effects for the Organophosphates, in which several studies were excluded from the WOE based on a single quality element, while others were not (excluded using the same criteria). Transparency in these elements is an important requirement for future assessments. Following the table on quality considerations, the 2016 Framework goes into detail about direct and indirect approaches for exposure assessment in epidemiology studies. However, the reliability and validity of an exposure assessment is more important than defining if it is direct or indirect. The 2016 Framework also displays a portion of the biomarker evaluation instrument from Lakind et al., 2014, "Biomonitoring, Environmental Epidemiology, and Short-lived Chemicals (BEES-C)." OPP omitted consideration of Exposure Variability and Misclassification from the BEES-C instrument, which addresses the number of samples (spot vs. many) and reliability testing for this parameter. OPP concludes that exposure assessment methods should be able to provide exposure estimates that are reliable and valid. However, the 2016 Framework does not incorporate any quality recommendation to this effect. The importance of reliable and valid exposure data was emphasized in the scientific opinion of the EFSA PPR panel. The PPR panel specifically recommended that improvement in the accuracy of exposure measurement is increasingly important. While OPP recognizes the 2016 Framework is not designed to be a rigid guideline, the document lacks direction for peer reviewers and investigators. Alternatively, an expanded quality rubric that includes additional elements (from Table 2), such as sensitivity analyses to quantify the direction of bias and dose response analyses, would provide a minimum expectation for epidemiology data for risk assessment. Sierra Club v. EPA 18cv3472 NDCA Tier 3/4 ED 002061 00038400-00003 Discussion Topics on the "2016 Framework" Page 4 of 5 In summary, the 2016 Framework provides commentary on quality elements and considerations. However, the organization of the document and presentation of a quality rubric makes the actual approach unclear as to OPP's quality interpretation of epidemiology studies. 3. Transparency of epidemiology results Access to full analytical results (i.e. unpublished) from epidemiology studies is a critical difference from the EPA guideline data currently available for risk assessment. There are multiple places in the 2016 Framework that necessitate a conversation about access to data. In Section IIIA, OPP mentions missing data. Section IV discusses statistical analyses and null results. If indeed there are missing data or incomplete analyses, OPP should be recommending a discussion among OPP, registrant(s), and the epidemiology investigators to develop a plan to make the missing data available and/or conduct additional analyses. This is not present in the 2016 Framework. Apost-hoc review of published studies does not meet the same standard as regulatory data for HHRA. In the Problem Formulation section, OPP describes how it plans to define the scope of an analysis. OPP continues to point out that a review may be focused on exposure pathways and certain health outcomes. It notes on page 9, "If missing data are critical to the assessment, options are discussed as to how best to obtain that information." At first look, it seems like a good practice. However, it is unclear if these are standard questions posed for all pesticides and risk assessments or if they are raised after reviewing an epidemiology publication. Indeed, the 2016 Framework does not list how to obtain missing data (or if the study results will be used if the data are not available). The Data Collection section describes how OPP will search and report on published and unpublished sources. On page 10 the 2016 Framework states, "In the case of epidemiology, most studies are expected to be found in the open scientific literature. Although in some cases supplemental analyses or information may be available, dialogue with the researchers may provide additional, important information not published in the original paper in understanding and interpreting epidemiology studies." A comprehensive and systematic review demonstrates best practices. However, there is a notable lack of synchronization of the process of regulatory review of specific pesticides and the conduct and reporting of epidemiology results. In other words, the process to conduct and report epidemiology findings prior to the OPP search should be more systematic. The section on "Interpretation of null studies" fails to account for what is already known about the risk profile of a registered pesticide. OPP states that lack of associations will be evaluated carefully. The bulk of this section states the opposite. In fact, the 2016 Framework states, "the absence of evidence should not be interpreted as the evidence of absence" (page 35). Sierra Club v. EPA 18cv3472 NDCA Tier 3/4 ED 002061 00038400-00004 Discussion Topics on the "2016 Framework" Page 5 of 5 The 2016 Framework mentions the effects of publication bias that the published literature disproportionately excludes null findings. Contrary to OPP's view, the lack of human evidence, in the context of vast guideline studies, could be interpreted as evidence of absence. The null data could be interpreted as evidence below the No Observed Effect Level. Furthermore, when relying upon published epidemiology data, the OPP must account for the lack of disclosure of negative data. There is no discussion or guidance how to improve the status quo. In summary, it is well known that null (non-adverse) findings of epidemiology studies are not fully disclosed in the published literature. For both hazard identification and risk assessment, the ability to know and evaluate these results must be developed. Sierra Club v. EPA 18cv3472 NDCA Tier 3/4 ED 002061 00038400-00005