Document Kzb8M2QnL6kzNm272bLk8rYQ

MEALEY'S Daubert Report Commentary Vol. 10, #1 January 2006 The Growing Role Of Litigation-Generated Science By William L. Anderson and Barry M. Parsons [Editor's Note: Mr. Anderson is a partner and Mr. Parsons is counsel in the Tort Practice at Crowell & Moring LLP in Washington, D.C. Both authors practice in tort and product liability cases on behalf of defendants. Copyright 2006 by the authors. Replies welcome.] If a study came out tomorrow in a peer-reviewed scientific journal documenting an association between, say, potato chips and diabetes, it might well form the basis for lawsuits against potato chip companies by diabetics. In fact, considering Daubert's heavy reliance on testing, peer review and publication, the new study could well insulate the plaintiffs' testifying experts from a Daubert exclusionary ruling. What if it subsequently became known that the study was funded by the plaintiffs' law firm and directed by their testifying causation expert, all for the express purpose of surviving a Daubert review? Should this make any difference in the way the court conducts its review of the expert's methodology and opinion? Knowing the lawyers were in the laboratory might make one a bit more cautious about the study's results. Lawyer involvement in scientific studies is a phenomenon that, we suggest, is on the rise and will require a modified approach under Daubert. Litigation is increasingly the source of new scientific studies because of the need for such research to support litigants' positions in tort and product liability litigation. This class of litigation is particularly dependent on scientific support -- the studies, publications, and pronouncements that are the bread and butter of the scientific community. With it, a plaintiffs' attorney can survive a Daubert motion and get a novel causation theory case to trial, where the plaintiff has significant advantages. Without it, the case may go the way of EMF and cell phone litigation, both of which, lacking any significant epidemiological support, have never taken off. On the defense side, in some jurisdictions significant testing and peer reviewed publications may be the only thing separating the company from a destroyed product and multimillion dollar settlements. Witness the cases of Bendectin, the subject of Daubert itself, taken off the market due to massive litigation even though later epidemiology demonstrated the product was not a teratogen, and the breast implant litigation, whose speculative causation assumptions were eventually corrected by epidemiology studies and a court-appointed scientific panel.1 The emergence of litigation-generated testing is likely a product of a number of influences. Certainly, the enormous money at stake in toxic tort, product liability and class action litigation, among others, has affected the demand for proof to support or eliminate new tort and product litigation. In addition, the amount of money collected by plaintiffs' attorneys to date in asbestos, tobacco, and other litigation has placed the cost of expensive scientific research easily within reach of the larger, national plaintiff firms. But perhaps the foremost driving force is the Supreme Court's 1993 Daubert standard and similar state court restrictions on novel expert testimony established since then. Daubert's increased scrutiny of expert methodologies, permitting judges to look closely at experts' theories and approaches, has led many courts to conclude that there are often no scientific clothes 1 Vol. 10, #1 January 2006 MEALEY'S Daubert Report on these expert emperors. What's a plaintiff attorney to do? Bolster the claim with a study or two, and get those studies published. Plaintiff firms are increasingly conducting their own litigation studies to support novel tort claims. The studies are usually conducted, or at least controlled by plaintiffs' testifying experts, and the attorneys may play a role in the concept, protocol, and conduct of the testing. The equation works the other way if there is sufficient scientific support for plaintiffs to get past an exclusionary ruling and take the case to a jury. Then the defendant company may need to consider commissioning testing to debunk the claim. The problem for defendants is particularly severe if the claim is novel and the defendant is before a court that permits plaintiffs' experts to speculate about cause in the absence of definitive studies.2 Either way, courts will have to deal with studies initiated not by scientists for science's sake, but by lawyers for litigation's sake. Litigation-based testing is not necessarily bad science, but it does pose some thorny issues for courts under Daubert, chiefly, concerns about bias and undue influence into the scientific process. The Daubert Court's approach -- relying heavily on the scientific community independently to review novel scientific ideas and discredit the bad ones -- gets turned on its head if the science is not independent at all but set up for the purpose of making the litigation go. In this article, we provide some background on litigation-generated studies, why scientific review by journals alone is probably not sufficient to address potential litigation bias in these studies, and how courts and parties should approach and address this kind of testing. Examples Of Litigation-Generated Studies Below are some concrete examples of litigation-generated testing and how it has played out in the courts. Note that all but two of these occurred after the Supreme Court's Daubert ruling in 1993. Bendectin litigation. Prior to the Supreme Court's review of the underlying Bendectin litigation in Daubert plaintiffs' epidemiologist performed a metaanalysis of the existing epidemiological studies showing no link between Bendectin and birth defects, and claimed to have found an increased risk of birth defects.3 The reanalysis was not published. The Court of Appeals noted that the opinions of plaintiffs' experts, including their epidemiology opinions, were "generated solely for use in litigation."4 After the Supreme Court remanded the case, the Ninth Circuit discussed the difference in reliability between independent research and litigation research but did not automatically exclude the litigation work. Instead, the court held that the party proffering the litigation-generated research should provide other evidence confirming that the expert's methodology "is based upon `scientifically valid principles.'"5 The expert must subject his methodology to "normal scientific scrutiny through peer review and publication."6 The court indicated that only those studies published in "reputable"7 journals and subjected to a "bona-fide process of peer review"8 would satisfy its test for scientific validity when the methodology at issue was conducted for litigation purposes.9 The metaanalysis was excluded by a New Jersey court,10 and the Daubert Ninth Circuit court rejected the overall Bendectin opinion under both prongs of Daubert.11 Autism and vaccines. One of the most highly publicized examples of litigation-generated science is that of Andrew Wakefield, a British doctor who published a dramatic study in the Lancet in 1998 purportedly identifying an association between children who received the MMR vaccine and autism. See Glenn Frankel, Washington Post, July 11, 2004, p. A01. The study caused an uproar, vaccinations dove, and lawsuits were filed. Only later was it revealed, through investigative work by the London Times, that Wakefield had previously received $90,000 from a plaintiffs' law firm to investigate this link, and he had used litigation plaintiff children and families as his study subjects. The researcher and the supposedly clinically-referred subjects both had significant interests in the outcome of the study -- to support litigation. Wakefield failed to tell any of this to his colleagues who published with him or to the journal. After the disclosures, the Lancet retracted the article, and Wakefield's colleagues retracted their participation. Multiple studies since then have not found any link, and the Centers for Disease Control, UK government, and World Health Organization today reject any link. See http://www.cdc.gov/nip/ vacsafe/concerns/autism/default.htm. In this instance, the newspaper gets the credit for uncovering information that the courts and parties to the cases might never have known. 2 MEALEY'S Daubert Report Vol. 10, #1 January 2006 Carpet emissions. In Ruffin v. Shaw Industries,12 plaintiffs' expert conducted air chamber toxicity testing on carpet emissions and claimed to have achieved significant toxic effects on rats. She did not publish her results. They were subsequently contradicted in studies performed by the Environmental Protection Agency, a university researcher, and two carpet manufacturers, none of which could replicate any injury to the rats. The Fourth Circuit Court of Appeals had little trouble rejecting the expert and her testimony due both to the litigation taint of the study and the inability of reputable labs to replicate her results. This case is unusual in that the litigation testing attracted significant independent review and analysis, which the Court could utilize in its review. Neighborhood toluene release. In Black v. Rhone-Poulenc,13 plaintiffs' expert performed an epidemiology study to connect a plant's toluene release to neighborhood health effects. The expert presented the results at two symposia but did not disclose the litigation link. The expert's firm was closely connected to the plaintiff attorney's firm, and the attorney played a significant role in selecting participants and interviewing study participants. The court excluded the testimony, finding that the studies were shot through with lawyer influence and insinuation: "The depth and breadth of litigation taint is so substantial the very validity of the study is compromised." Fungicide and birth defects. In a case defended by the authors, plaintiffs' counsel funded a series of studies designed to bolster their chief causation experts' opinion that a particular fungicide caused birth defects from dermal exposure during home use applications. One set of studies attempted to identify cellular level changes in in vitro (test tube) applications; another tested whole rat embryos; and in yet another the researchers applied the fungicide to human skin to attempt to show significant dermal penetration. Despite substantial flaws with all of the testing, the first study helped the plaintiff firm achieve a win in Florida, but two courts since have excluded the expert's testimony despite the litigation studies.14 Chlorine release. In Valentine v. Pioneer Chlor Alkali Co.,15 plaintiffs' expert conducted an epidemiological study to link a plant's chlorine release with neurological damage. All of the participants/exposed individu- als were current plaintiffs or involved in the litigation. The study was published in a minor journal. The court rejected the expert's testimony, noting that "[t]he assertions of plaintiffs' experts that exposure to chlorine damages the brain and nervous system are novel, and, as noted above, unsupported by scientific research extraneous to this litigation." Asbestos in schools. In Worthington City Schools v. Abco Insulator,16 a defense expert used data collected from defendant companies to publish an article showing that asbestos levels in school buildings were the same as ambient levels. Plaintiffs attempted to exclude the article and testimony because of its litigation taint. The expert was the lead author of the article but did not disclose his role in the pending litigation.17 Nonetheless, the court ruled that the article was trustworthy (and therefore admissible) since it was submitted for peer review and revised prior to publication. The court based its reasoning on the fact that the expert relied upon other articles -- including a government article and articles from the Health Effects Institute and the World Health Organization -- which published similar results.18 Fertility drug. In Lust v. Merrell Dow Pharmaceuticals,19 plaintiffs' expert relied on an article he published several years earlier on animal studies of a fertility drug, but failed to reveal initially that he had written the article while serving as a "professional plaintiffs' witness" in a separate case. That information became known during a deposition. The court rejected the expert's reliance on the article, despite its journal publication, because the litigation connection was not revealed during publication and because of the absence of any other publication supporting the expert's opinion. Insecticide and birth defects. In National Bank of Commerce v. Dow Chemical Co.,20 plaintiffs' expert published two articles in journals to support her opinion regarding a Dow insecticide and birth defects. The expert did not reveal her litigation role in the article. The court was troubled by the expert's long-standing role as consultant and testifying expert witness for twenty years in all of the cases at issue. The court also noted that the expert had "not published her protocols, reasoning or methodology" and thus foreclosed any real peer review and possible replication of her studies. The court concluded that the expert was en- 3 Vol. 10, #1 January 2006 MEALEY'S Daubert Report gaged in advocacy "based on suspicion and conjecture and litigation animus rather than science." Despite the flawed science involved in the examples above, legitimately conducted studies, even if for litigation purposes, may well consist of high-quality scientific research no less deserving of credit than non-litigation research. In the Bendectin cases, for instance, a series of animal and epidemiology studies in the 1970s and 1980s, at least one funded by Merrell, eventually generated a scientific consensus that Bendectin did not cause the alleged birth defects. These findings helped stem the tide of an unjustified and very destructive litigation.21 Similarly, early cases alleging cancer from EMF exposure died on the vine because the power industry had already conducted epidemiology studies showing no association.22 In the authors' own experience, defendant-sponsored stateof-the-art genetic testing in an agricultural product birth defect case, based on similar testing done independent of the litigation, recently formed the basis for a Daubert exclusion ruling by demonstrating that a genetic mutation caused the child's condition.23 The trick for the courts is to recognize the kind of ball they are playing with and handle it accordingly. Most importantly, litigation research carries risks of greater bias and must be examined more closely to ensure its bona fides and that the experts are properly interpreting its results. Daubert And The Role Of Litigation-Generated Science Litigation-generated science has much of its genesis in the test the United States Supreme Court applied in Daubert v. Merrell-Dow Pharmaceuticals in 1993. In Daubert the Supreme Court established the by-now familiar two-prong test of reliability and fit for expert testimony. The Court provided four non-exclusive criteria for reliability, at least two of which -- peer review and general acceptance -- depend upon independent scientific review of the expert's theories. Even if the methodology is properly scientific, the opinion derived from it must fit the circumstances of the case. Opinions that are not properly derived from the study or that do not support the scientific proposition before the court are not admissible under Daubert. The Daubert Court presumed that the scientific community would provide a buffer of independence that would knock down biased or speculative theories and thus give courts some comfort in excluding the experts. In some circumstances, as in the Bendectin litigation mentioned above, that might be the case. The courts were ultimately persuaded by a series of privately performed epidemiology studies demonstrating that drug carried no risk of birth defects.24 Daubert gets turned on its head, however, if the studies and their publication are not the result of independent research, but have been carefully orchestrated and performed, for instance, to support a preordained opinion for the very purpose of surviving a Daubert motion. If the lawyers are conceiving the study and potentially controlling it from start to finish, is this the kind of independent science that the Daubert Court envisioned? That we want in a courtroom? If we allow its use, do we treat litigation-science just like any other science or apply some different criteria? The courts in the cases discussed above reacted instinctually to litigation-based work, and they developed some ad hoc approaches to deal with it. Nevertheless, courts have not yet recognized the import of litigation science or formalized any process for analyzing it. Addressing The Problems Posed By Litigation-Generated Testing Just to play devil's advocate, we might ask, "Why treat litigation science differently than any other science?" All scientific studies, after all, are subject to bias and falsification. Scientists are under funding pressures, or the need to support their own theories, or the desire to achieve a dramatic result that will receive lots of attention. As only the latest example, witness the South Korean researcher recently caught forging evidence of human stem cell cloning, no doubt motivated by the enormous fame and publicity that the "discovery" initially brought him. The devil's advocates might argue, forget the litigation link -- just judge the studies on their merits and treat the funding or potential bias as, at most, a curiosity. This approach actually has some proponents in the scientific world. Some editors and researchers contend that disclosure of potential bias or conflicts of interest is not important, and requiring it only creates a false impression that funded research is biased research. They would take research from any quarter and simply judge it on its own merits.25 4 MEALEY'S Daubert Report Vol. 10, #1 January 2006 The trend in our society, however, is toward disclosure of all sorts of things, including possible side effects in pharmaceutical ads, the dangers of a hot cup of coffee, and the president's peccadilloes. Consider the recent upheaval at the National Institutes of Health over its scientists' links with outside interests and the restrictions the agency put in place as a result. It is unlikely this trend will reverse itself, even in the scientific community. Litigation science, we posit, creates an even greater need for close scrutiny, largely because of the intense adversarial nature of our litigation system and the pressures to get the "right" result. The instincts of the few courts to address litigation science are correct -- this research needs special attention. In an effort to help formalize that attention, we review some of the key problems with litigation science, and ways in which courts can address them: Potential bias: A court should be concerned whether the litigation pressure for a certain outcome has affected the validity and trustworthiness of the test. The hidden pressures in the academic and regulatory communities are nothing compared to the overt advocacy nature of our litigation system. If an academic might fear the "wrong" result could limit publication opportunities, the litigation expert and attorney could see their entire enterprise coming apart if the test supports the other side. Thus, the possibility of false or manipulated data is serious, and the degree of lawyer control exerted over the study is a critical factor in its reliability. Quality of the study: Good research work follows widely-accepted protocols for designing and performing a study. To the extent a study closely adheres to university or agency research guidelines; is well-documented; involves collaboration and review by disinterested scientists; and is subjected to quality control and blind analysis, the court can have more confidence that it is looking at a bona fide study. The converse is a smoking gun. Studies conducted in the back room, by one researcher, with poor documentation and little quality control should be highly suspect, even if they end up being published. The quality of publication and peer review: Because the value of publication in a peer-reviewed journal is considerable, the expert may submit the research and claim it has been peer-reviewed. There are two serious questions, however, if a litigation study claims to have been peer-reviewed and published. First, how serious is the peer review? Research led by the Journal of the American Medical Association has shown that publication peer review is all over the map -- sometimes "blinded" and sometimes not; sometimes by internal editors rather than outside reviewers; sometimes required and sometimes not.26 Many second- and third-tier journals struggle to fill up their pages. How much scrutiny will they undertake of an article that on its face appears legitimate? The quality of the journal and the nature of its peer review should be carefully scrutinized. In addition, certain journals seem to exist to advocate a particular side of an issue. If all the articles in a particular journal seem to have an agenda, what are the chances the peer review was legitimately independent? More seriously, the kind of peer review and general acceptance the Supreme Court had in mind is hardly accomplished by publication of one article in one journal. Scientists would never accept a novel causation claim based on one reported study. The scientific process, as set forth by Karl Popper and well-described in Peter Huber's book JUDGING SCIENCE, requires repeated research, revision of hypotheses, yet more testing, until ultimately there's been enough pounding on the post that it feels secure enough to build something on. This process takes place through replication in multiple studies, repeated conference presentations, and lots of back and forth in the scientific community. If the study is novel and litigation-generated, all the more should the court not simply pass on a single publication but examine whether the approach and finding have undergone true scientific community peer review. Transparency and disclosure: The summary of cases above illustrates that in today's disclosure-oriented society, litigation research runs into trouble if the researcher does not disclose the litigation link. Prior to the 1990s disclosure of conflicts in journals was not a particularly widespread requirement. Since 1989, the biomedical publishing community has engaged in a significant effort to create higher standards for peer review and conflict disclosure in journals, in part due to some high-profile embarrassments for major journals. The result is something called the Uniform Guidelines for Publication in Biomedical Journals, 5 Vol. 10, #1 January 2006 MEALEY'S Daubert Report which, among others things, now requires disclosure of conflicts of interest in journals that subscribe to the Guidelines.27 Yet even today many journals do not require disclosure of conflicts.28 Thus, both the opposing party and a court dealing with a published litigation study should enquire into whether the publication and its peer reviewers knew of the litigation link. The peer reviewers are almost entirely dependent on the honesty and integrity of the researcher, since they see only the article manuscript and have no way of knowing if the data has been falsified or "bent" to suit the litigation's purpose. The reviewers are entitled to know whether the article should, for instance, contain fuller descriptions and disclosures regarding the manner in which the protocol was established and the data was analyzed and interpreted, and the journal may require the author to produce or report more of the underlying work to ensure that the results are legitimate. The reviewers and journal may also wish to scrutinize more closely broad claims made in the article, intended for use in the courtroom, that may or may not be supported by the research itself. "Fit" problems: The research may be legitimate but can still be misused in litigation. This results in a "fit" problem under Daubert. Usually, the fit issue arises when the researcher attempts to extract meaning out of a litigation study that the study does not support. At first glance, it seems strange that the litigation expert would conduct a marginally relevant study to begin with, but this may well be the case. For example, if the epidemiology does not support the expert's position, the expert may turn to less relevant studies -- an in vivo (animal) or in vitro (laboratory cell test) study -- to attempt to find an arguable link between the substance and the disease. Juries, who are not familiar with the hierarchy scientists use that places epidemiology at the top of causation evidence, could well be persuaded to reject the epidemiology and believe the animal or in vitro studies. Flawed litigation studies often fail the fit test. Extrapolating human dose levels or causation evidence from high-dose animal is a classic example of a bad "fit" under Daubert, and one that many courts have rejected. Litigation animal studies were the focus of the courts' displeasure in two of the cases discussed above, Ruffin and Lust. Because of differences across species and the extreme dose levels and delivery methods (e.g., injection directly into the stomach by gavage) used in animal studies, results of these studies may well not fit the human causation theory the expert is supporting. Or the supporting research may involve a different cell line, organ, or process than that in the litigation. Another common approach is the attempt to extrapolate from studies of a different substance by contending that chemical similarities justify the extrapolation to the substance at issue. Any of these approaches, if used in the scientific community, would result in an extremely high rate of error because they are rarely accurate predictors of human toxicology. Daubert's gatekeeper role is a difficult one for judges but absolutely necessary to prevent courts from becoming the breeding ground for rampant and unproven speculative opinions. The task is difficult enough when it involves independent research. The gatekeepers are going to need considerable help recognizing and dealing with litigation-based studies to sort the legitimate from the unreliable. Endnotes 1. See Sanders, J., The Bendectin Litigation: A Case Study In The Life Cycle Of Mass Torts, 43 HASTINGS L.J. 301, 345-48 (1992); Marcia Angell, SCIENCE ON TRIAL (1996) (breast implant litigation analysis). 2. See, e.g., Donaldson v. Central Illinois Public Serv., 767 N.E.2d 314 (Ill. 2002) (permitting expert to testify about speculative link between coal tar exposure and neuroblastoma in the absence of any studies addressing that link). 3. 43 F. 3d 1311 (9th Cir. 1995). 4. Daubert, 509 U.S. at 584 (citing the Ninth Circuit opinion at 951 F.2d at 1131). 5. Daubert v. Merrell Dow Pharmaceuticals, 43 F.3d 1311, 1317-18. 6 MEALEY'S Daubert Report Vol. 10, #1 January 2006 6. Id. at 1318 (citing Peter W. Huber, GALILEO'S REVENGE: JUNK SCIENCE IN THE COURTROOM 209 (1991)). 7. Id. (emphasis added). 8. Id. n.6. (emphasis added). 9. The court recognized that "peer review and publication does not . . . guarantee" that the reported methodology is valid but stated that publication in a peer-reviewed journal "increase[s] the likelihood that substantive flaws in methodology will be detected." Id. (quoting Daubert, 509 U.S. at 593). 10. DeLuca v. Merrell Dow Pharmaceuticals, 791 F. Supp. 1042 (D.N.J. 1992). 11. 951 F.2d at 1316-22. 12. 149 F.3d 294, 297-98 (4th Cir. 1998). 13. 19 F. Supp. 2d 592 (S.D.W.V. 1998). 14. Castillo v. E.I. du Pont de Nemours and Co., 854 So. 2d 1264 (Fl. 2003); Bourne v. E. I. du Pont de Nemours and Company, 189 F. Supp. 2d 482 (S.D.W. Va. 2002), aff'd, 85 Fed. Appx. 964 (4th Cir. 2004), cert. denied, 125 S. Ct. 67 (2004); Bowen v. E.I. du Pont de Nemours and Co., 2005 WL 1952859 (Del. Super. 2005). 15. 921 F. Supp. 666 (D. Nev. 1996). 16. 616 N.E.2d 550 (Ohio App. 1992). 17. Id. 18. Id. (concluding that the admission of the article did not prejudice plaintiff's case). 19. 89 F.3d 594 (9th Cir. 1996). 20. 965 F. Supp. 1490 (E.D. Ark. 1996). 21. See Sanders, J., The Bendectin Litigation: A Case Study In The Life Cycle Of Mass Torts, 43 HASTINGS L.J. 301, 345-48 (1992). 22. See D. Bernstein, The Breast Implant Fiasco, 87 Cal. L. Rev. 457, 485-86 (1999). 23. See Bowen v. E.I. du Pont de Nemours and Co., 2005 WL 1952859 (Del. Super., June 23, 2005). 24. Hastings, Sanders, J., The Bendectin Litigation: A Case Study In The Life Cycle Of Mass Torts, 43 HASTINGS L.J. 301, 345-48 (1992). 25. See Eliot Marshall, Journals Joust over Conflict-ofInterest Rules, 276 SCIENCE at 524 (April 1997). 26. The authors reviewed the literature dealing with peer review practices in our article Daubert's Backwash: Litigation-Generated Science, 34 Mich. J. Law Reform, pp. 638-42 (2001). One of the authors also summarized this problem at the 2003 national annual meeting of the Council of Scientific Society Presidents. 27. The current version of the Uniform Requirements may be found at http://www.icmje.org. 28. One survey found that less than one-third of the journals identified in Uhlrich's International Periodicals Directory required any sort of conflict disclosure. See Richard M. Glass and Mindy Schneiderman, "A Survey of Journal Conflict of Interest Policies, INTERNATIONAL CONGRESS ON BIOMEDICAL PEER REVIEW AND GLOBAL COMMUNICATIONS, http://www.ama-assn.org/public/peer/ apo.htm. Another study found that only 39% of journals surveyed required disclosure of conflicts of interest. Cynthia D. Good, Stephen T Parente, Drummond Rennie, Suzanne W. Fletcher, A Worldwide Assessment of Medical Journal Editors' Practices and Needs -- Results of a Survey by the World Association of Medical Editors, 89 S. AFR. MED. J. 397 (1999). 7