Document GEjZz551xYed5600kGgKZZ94

Message From: Sent: To: CC: Subject: Katya Tsaioun [ktsaioul@jhu.edu] 5/22/2017 8:37:29 PM Craig.Rowlands@ul.com [/o=ExchangeLabs/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=2faac2cbab564370ae25dc80951fff70-Craig.Rowla]; Paul Whaley [p.whaley@lancaster.ac.uk]; Manoj Lalu [________Ex:_6_______ j; Beck, Nancy [/o=ExchangeLabs/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=168ecb5184ac44de95a913297f353745-Beck, Nancy]; Thayer, Kris [/o=ExchangeLabs/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=3ce4ae3fl07749c6815f243260df98c3-Thayer, Kri]; Daniele Wikoff [dwikoff@toxstrategies.com] Martin Stephens [mstephl4@jhu.edu]; Sebastian Hoffmann [Sebastian.hoffmann@seh-cs.com];['"' J ''''''''`''^'6''` [/o=ExchangeLabs/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=a76a789blla04bl78ce2598165e66389[" Ex' 6 |Rob deVries [Rob.deVries@radboudumc.nl] SOT session #2 - proposal submitted Dear all, Please see below the EBTC SOT session proposal #2. Please forgive the last-minute edits Craig and I had to do to accommodate the comments of specialty sessions. Rob, thank you so much for taking the lead earlier and pulling it together, and my apologies for having to remove you from leadership due to SOT rules! :( Craig, your edits were really excellent. Thank you very much for your late contribution after you picked up from Rob! Katya TITLE: M oving beyond reliability and relevance in assessing toxicological study quality. Enhancing transparency and characterization of uncertainty in risk assessment. PRESENTATION TYPE: Workshop SECONDARY PRESENTATION TYPE: Symposium Chairs: Katya Tsaioun* & Craig Rowlands* (co-chairs) Speakers: Paul Whaley Manoj Lalu Nancy Beck* Kris Thayer Daniele Wikoff* *SOT Member SESSION DESCRIPTION: As the science of toxicology moves away from deterministic to probabilistic perspectives and characterizations, the topic of scientific quality of toxicological studies is going to gain progressively more prominent in the future of toxicology and risk assessment. Characterising the scientific quality of individual studies and taking this quality into account in the interpretation of evidence is at the heart of risk assessment. The concepts that are currently common to express scientific quality are `reliability' and `relevance'. The problem with these concepts is they are not always used consistently and there is at least ambiguity, if not disagreement, about Sierra Club v. EPA 18cv3472 NDCA Tier 12 ED 002061 00044760-00001 which aspects should be considered when assessing the reliability and relevance of a study and how to weigh these considerations in characterising uncertainty in the results of a chemical risk assessment. In this session, we will compare the current practices in chemical risk assessment with how study quality is assessed in systematic reviews, an evidence synthesis methodology widely used in medicine and of increasing interest in toxicology and environmental health research. The aim is to determine the extent to which the concepts and terminology used in systematic reviews can be applied to resolve the ambiguity surrounding the concepts of reliability and relevance, thereby advancing transparency and characterisation of uncertainty in chemical risk assessment. The workshop will be comprised of a series of highly focused presentations that evaluate the concepts of systematic reviews and their application in regulatory decision making in chemical risk assessment. The session will be of broad interest to investigators and regulators across environmental, industrial, consumer products, and pharmaceutical toxicology that perform product registration, and human and environmental risk assessments. Introduction (5 min) Katya Tsaioun ?  This will be a brief introduction that will provide the goals for the symposium, and introduce the presenters. Presentations (each speaker 30 min) Presentation #1 Title: Introduction to the concepts of study quality in systematic review and their application in assessing the reliability of toxicological studies Presenter: Paul Whaley, Lancaster Environment Centre, Lancaster University, UK Overview: The discipline of systematic review, with its origins in medicine, has developed its own concepts and vocabulary for characterizing and describing the relevance and reliability of clinical research for health care decision-making. This presentation will introduce these concepts and describe how they are used in evaluating the quality of healthcare research. Particular attention will be given to the value of distinguishing precision, bias and reporting quality in appraising the validity of individual studies; the importance of concepts including external validity, publication bias and consistency of the overall evidence base in characterizing the credibility of the results of a systematic review will also be discussed. Finally, using real examples from toxicology reviews, this presentation will explain how the vocabulary of research quality used in systematic review maps on to the concepts of relevance and reliability which are fundamental to toxicology, and how these concepts can be deployed by toxicologists to improve the characterization of uncertainty in the results of chemical risk assessments. Presentation #2 Title: Assessing threats to the "other" validities: external validity and construct validity: extrapolating from observed to target populations Presenter: Manoj M. Lalu, University of Ottawa/The Ottawa Hospital Research Institute, Ottawa, Canada Overview: Internal validity ("risk of bias") is the type of validity that is commonly assessed in the context of systematic reviews and for which standardized tools are available. This presentation will review two other concepts of validity, namely external validity and construct validity, and offer a practical approach to evaluate threats to each. External validity refers to the generalizability of findings across a range of experimental conditions. For instance, assessing external validity allows us to evaluate how generalizable toxicity findings in a cell culture model may be to a zebrafish model and/or a mammal model. Construct validity refers to the potential generalizability of findings from an experimental model to the human condition. For instance, a rigorous assessment of construct validity allows us to determine how closely a mammal model used to evaluate toxicity mimics humans that have been exposed to a similar toxin. A published assessment of the threats to these validities will be used to introduce the session participants to these concepts (i.e. a preclinical systematic review of stem Sierra Club v. EPA 18cv3472 NDCA Tier 12 ED 002061 00044760-00002 cell therapy for life threatening infections). Frameworks to evaluate external and construct validity will be introduced and the applicability of these frameworks to toxicologic examples will be demonstrated. Presentation #3 Title: Assessing Quality Beyond Risk of Bias and Internal Validity Presenter: Nancy Beck, US EPA Overview: While understanding the risk of bias/internal validity of a study is important, when doing systematic reviews to inform toxicity determinations for human health additional study characteristics can also play an important role. For instance, there are aspects of study design and methodology that are not captured when one only examines the internal validity of a study. In addition, the relevance of the study to the public health question being asked is also important. For example, a systematic review where the route of exposure and the concentrations tested are not relevant to the human exposure scenarios, will likely be less helpful to inform public health decision making. Asking questions about relevance will also help to inform whether a mode of action in an animal model is relevant to humans. A broader review of quality and relevance can easily be incorporated into a systematic review protocol. Moreover, traditionally the risk of bias evaluation has not been explicitly used or applied during the evidence integration step. While it may sometimes be used to tailor a meta-analysis, the risk of bias evaluation is not always incorporated into final systematic review determinations that may be made after evidence streams are integrated. This talk will discuss and provide specific examples where the risk of bias questions may not be sufficiently informative or specific to inform a full review of the quality and relevance of a study. Presentation #4 Title: Assessing the quality o f toxicological studies at an aggregated level: Applicability of the GRADE framework for environmental health and risk assessment Presenter: Kristina Thayer US EPA Overview: Within systematic reviews, the quality/reliability of the evidence is not only assessed at the level of individual studies, but also at the level of bodies of evidence, i.e. sets of studies of the same type of evidence grouped by outcome measure. One widely used approach for this type of assessment is the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework. GRADE represents a structured and transparent process to inform decision-making beginning with well-defined questions, followed by an assessment of the confidence in the evidence and leading to development of recommendations and decisions. GRADE is useful for systematic reviews, narrative reviews, or evidence assembled for rapid or urgent responses. Within GRADE, confidence in a body o f evidence can be rated down not only for risk of bias, but also for inconsistency, indirectness, imprecision, or publication bias. Moreover, confidence can be rated up for the magnitude of the effect, dose-response gradient, or direction and impact of residual plausible confounding. For over a decade, GRADE has been applied successfully to areas of clinical medicine, public health, and health policy, but experience with GRADE in environmental and occupational health is just beginning and will likely entail method refinement. However, initial applications of GRADE to environmental health topics, including animal toxicology, are promising and efforts are underway to apply the framework to in vitro and modelled evidence. This presentation will survey some examples of GRADE applied to toxicology evidence and summarize efforts among a variety of groups to assess the broad applicability of GRADE for environmental health. Presentation #5 Title: A practitioner's view on strengths and challenges of evaluating and integrating "study quality" in evidence-based assessments involving mechanistic, animal, and human evidence streams in food and nutrient risk assessment Presenter: Daniele Wikoff, ToxStrategies, Asheville, NC Overview: It is well established that the concepts of study quality should play a role in toxicological assessments. As we move towards establishing best practices in evidence based assessments, it has been recognized that many aspects of study quality are important, including both internal and external validity. However, in practice, toxicologists must integrate data from, often, large bodies of evidence, consisting of many study types in the context of hazard or risk. Through a series of case examples, this talk will demonstrate how Sierra Club v. EPA 18cv3472 NDCA Tier 12 ED 002061 00044760-00003 the rigorous standards o f evaluation of "study quality" can be met for multi-endpoint and multi-stream (i.e., human, animal, mechanistic) systematic reviews conducted in support of risk-based assessments. Particular focus will be on evaluation and integration of mechanistic data (including high-throughput data, other in vitro data, as well as in vivo data) to inform decision making. Discussion will be offered as to the utility and feasibility of evaluating study "quality" for a wide range of individual study methods and integrating such assessments into the body of evidence (e.g., considerations of quality/validity/relevance as they relate to confidence in the outcome), as well as integration of these methods relative to current practices (e.g., assessment of mode of action for carcinogens or derivation of health-based benchmarks). Presentations Total 160 min Sierra Club v. EPA 18cv3472 NDCA Tier 12 ED 002061 00044760-00004