Colloquium: Junk Science, the Courts, and the Regulatory State

Jeffrey Bossert Clark*

This article is part two of a two-part series covering the colloquium held on July 10, 1997 in Washington, D.C. The first part was published in the Fall 1997 issue of Environmental Law and Property Rights News.

Luncheon Address by Peter Huber

After the morning session had ended and lunch was nearly complete, Peter Huber rose to give his keynote address. Huber tried to flesh out what he viewed as the two most important words in the Daubert opinion: "falsifiability" and "reliability." At times, however, Huber could not help but comment on certain portions of the morning session that had aroused his interest.

Huber began by analyzing the word "falsifiability." He was struck by the fact that Chief Justice Rehnquist, in his dissent in Daubert, claimed not to know what the word meant. Therefore, Huber thought it might be profitable to explore the meaning of this word drawn from the philosophy of science espoused by Sir Karl Popper. See Daubert, 509 U.S. at 600 (Rehnquist, C.J., concurring in part and dissenting in part) ("I defer to no one in my confidence in federal judges; but I am at a loss to know what is meant when it is said that the scientific status of a theory depends on its "falsifiability," and I suspect some of them will be, too."); Karl Popper, Conjectures and Refutations: The Growth of Scientific Knowledge 37 (5th ed. 1989). Huber mused that Justice Blackmun, the author of Daubert, may also not have known what the word meant and it was likely that neither Justice Blackmun nor his clerks have ever read Popper’s principal works, at least not in their entirety. Huber, an MIT-trained engineer as well as a lawyer, set out to bridge the gap. In the simplest terms possible, Huber said that what Popper meant by "falsifiability" was that in order for a theory to qualify as science it must make predictions that are concrete enough to be proved wrong. Huber also traced Popper’s insights to Popper’s impatience with the unfalsifiable claims of contemporaries Marx and Freud.

Huber then quoted from the affidavit of one of the plaintiffs’ expert in Daubert, Dr. Shanna Swan. Huber mused that Dr. Swan had likely not written that affidavit herself because it was phrased in "lawyer-speak." After reading a 71-word passage containing a lot of double-negatives, Huber said it was time for the lawyer to "invite Popper in." Applying Popper’s falsifiability analysis, Huber demonstrated that Dr. Swan’s assertions were not science because they could not be proven wrong. Echoing a point made earlier by Dr. Ehrlich, Huber argued that it is impossible for science to prove ultimate negatives. Thus, it should come as little surprise that Jason Daubert and his parents eventually lost their case against Merrell Dow, making the positive spin put on Daubert by the plaintiffs’ bar right after the case was decided ring hollow. (On a lighter, but practical note, Huber explained that he had personally spoken to the Dauberts and that their name was pronounced /Daw-bert/ not /Dow-bert/ or /Do-bear/.)

Taking up the challenge laid down by Bryant, Huber asserted that he was perfectly content to have Daubert’s test to exclude junk science be applied in a totally neutral fashion, so that defendant experts were just as susceptible of being excluded. He had never maintained anything to the contrary, he retorted. Huber also responded to Bryant’s charge that Galileo’s Revenge was unscientific. "It’s true," said Huber. As if to say that Bryant’s point were irrelevant, Huber said that Galileo’s Revenge was merely "a polemic sold in bookstores."

Moving on to the second important word from Daubert, "reliability," Huber argued that this term was not equivalent to the term "validity." To understand the true meaning of "reliability," according to Huber, one must consult the eighteenth-century mathematician, Thomas Bayes. Huber explained "Bayes theorem" with a simple example. Suppose your grandma’s eyesight is 80 percent accurate (valid) and grandma tells you that she saw a yellow taxicab. Should grandma be allowed to testify to the taxicab’s color in court? Most judges (and most people) approach this question in the following way: 80 percent is pretty good accuracy -- I would allow grandma to testify; now maybe 60 percent or less would be too low. Such thinking misses half of what is important, as Bayes has demonstrated. Suppose your grandma told you she saw a yellow lion outside, would you still let her testify in court? Suppose your grandma told you she saw a yellow stegosaurus? Under Bayes theorem, what’s important to judging overall "reliability" is not just the characteristics of the observer ("validity") but the likelihood that what an observer claims he has seen is true in the world at large.

Huber then posed the question of how we obtain information about the extrinsic likelihood that observed (or predicted) events are true. In the case of grandma and the yellow taxicab, the Division of Motor Vehicles can give us information about what proportion of taxicabs are yellow. But in cases where new scientific issues are under consideration, there is no Division of Motor Vehicles to consult. What to do? According to Huber, at this point we have to make an estimate of extrinsic likelihood. How do we make such an estimate? The best estimate of extrinsic likelihood is derived from a range of observations, or in terms of the grandma analogy, by looking at what the whole gamut of grannies have to say about the color of the taxicab. Turns out that under Bayes theorem that comes down to doing something that looks a whole lot like assessing whether scientific theories have achieved general acceptance. Ironically, the Daubert decision, which held that the Federal Rules of Evidence had abrogated the general-acceptance test of Frye v. United States, 293 F. 1013 (D.C. Cir. 1923), has merely recreated Frye and to that requirement added the further requirement of falsifiability. For Huber, the Court has essentially come full circle and gone the older law one better.

Turning to questions, Huber at first first faced some skepticism about the Bayes theorem. Huber did his best to explain that the Bayes theorem really was true, although he acknowledged that it sometimes produces counter-intuitive results. As an example, he used the fact that although the current HIV test is 99.8% valid, seven out of ten people without the virus currently get false-positive test results because the disease is so rare in the population as a whole. This "cries out against my intuition," admitted Huber, but it is true nonetheless.

The same federalist difficulty put forth in the morning session was also served up to Huber. Huber’s response was: "You have to choose your poisons," implying that in this case, it is simply worse from a conservative perspective to allow juries to pass on whether theories qualify as real science than to give judges greater powers as gatekeepers to do the same. Sounding a variation on Judge Hand, Huber asked the rhetoric question of why we don’t put legal questions to juries -- "Ladies and Gentlemen of the jury, here is the text of Rule 10b-5. Please tell us what it means."

Lastly, I asked Huber the purely legal question of whether his reading of the meaning of "reliability" in Daubert was justified since Daubert specifically makes general acceptance a single factor in the determination of what is truly scientific rather than a determinative one and because the opinion appears to use the words "reliability" and "validity" interchangeably. Huber acknowledged the latter difficulty, but seemed to say that his reading of Daubert was plausible and that it made for better policy. "Who knows what the Court really meant by the term?," asked Huber.

Many of the ideas Huber expressed at lunch are contained in his new book, Judging Science: Scientific Knowledge and the Federal Courts (MIT Press 1997), and Huber recommended that Federalist Society members pick up a copy.

Afternoon Session on Science and Toxic Torts: Who Decides and How

Panelists included Professor David Bernstein, George Mason Law School, co-editor of Phantom Risk: Scientific Inference and the Law (MIT Press 1993); Jackson Sharman, III (Moderator), Lightfoot, Franklin & White, Vice Chairman of Programs for the E.L. Wiegand Practice Group in Environmental Law & Property Rights.

Professor David Bernstein dominated the afternoon session because of the unexpected absence of plaintiffs’ bar representative Anthony Z. Roisman of the law firm of Cohen, Milstein, Hausfield & Toll. Moderator Jackson Sharman, however, brought the welcome perspective of a grizzled lawyer fighting to defend corporations against "junk science" in the unreceptive state-court systems of Alabama and Mississippi. Sharman summarized the sophistication of many in the Mississippi judiciary by telling the story of a judge whose name I have altered slightly to "Billy Bob." At one point in a proceeding, Judge Billy Bob looked down at Sharman and said, "Cases, I don’t need no cases!" Sharman challenged Professor Bernstein to give him some practical advice that would be useful in such situations (a tall order).

Not one to be taken off his game plan merely because his adversary was a no-show, Professor Bernstein referred to an article quoting Roisman in the June 22, 1997 edition of the Houston Chronicle: "This isn’t about who’s right -- this is about who has the right to give an opinion. That's a mistake courts make. In the field of toxic exposure, there is room for scientists to have an opinion before there is a scientific consensus. Some cases are ahead of the curve. In those cases, the jury is at least as well-equipped as the judge to decide -- not who’s right, but who should win." Mike Tolson, Matter of Proof -- Courting Billion-Dollar Consequences -- Changing Rules on Scientific Testimony Could Have a Big Impact on Torts, Especially Breast-Implant Lawsuits, Hous. Chron. 6/22/97, available in 1997 WL 6564872. For Professor Bernstein, this approach is seriously in error. Lawsuits where scientific claims are at issue must be judged by a "rule of fact" as much as a "rule of law." Scientific truth, or "who’s right" in the words of Roisman, should matter according to Professor Bernstein. Justice means more than simply giving both plaintiff and defendant their day in court and urging the jury to follow its conscience.

Professor Bernstein reviewed a number of alternative legal explanations for why "junk science" should be excluded from the courtroom. First, suggesting that Judge Hand’s insights are now obsolete, Professor Bernstein argued that the notion that experts should be treated as exceptional cases in the law of evidence because they can offer opinions is no longer true because the Federal Rules of Evidence now allow lay witnesses in some cases to offer opinions. Professor Bernstein also rejected an explanation based on jury incompetence because Daubert rejected this argument. See Daubert, 509 U.S. at 596 ("respondent seems to us to be overly pessimistic about the capabilities of the jury and of the adversary system generally. Vigorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof are the traditional and appropriate means of attacking shaky but admissible evidence.").

What’s left, according to Professor Bernstein? The new governing principle is that any expert scientific testimony must be capable of being cross-examined. In other words, the heart of Daubert is its emphasis on falsifiability. As Professor Bernstein explained, modern Popperian scholarship equates falsifiability with criticizability. In the courtroom this means -- will litigants be able to attempt to undermine effectively the expert testimony presented by the other side? A useful companion question to ask in this regard is -- can the expert make quantifiable predictions based on his theory?

The reason for emphasizing falsifiability is that experts should not be allowed to speculate in the courtroom. Speculation is particularly an evil to be avoided in the judicial context because lawyers go shopping for experts. The other side need never be told how many experts were approached before the hiring side found what it had been looking for. Peer review and the general-acceptance factors of the Daubert test were similarly deployed by the Court according to Professor Bernstein in order to ensure that expert speculation is eliminated or minimized. To these tools Professor Bernstein added Judge Kozinski’s focus on whether the expert’s work had been generated solely for the purposes of litigation. Unless unscientific evidence is excluded from the jury’s view, according to Professor Bernstein, juries are inclined to "throw up their hands" and decide cases based on sympathy or the relative congeniality of opposing counsel. Professor Bernstein directed anyone who doubts this conclusion to consult the transcript of the comments made by jurors in the Laas breast-implant trial. SeeFRONTLINE:Breast Implants on Trial, Feb. 27, 1996, available in LEXIS, Nexis Library, SCRIPTS File.

In addition to the Eleventh Circuit’s wayward decision in Joiner, Professor Bernstein suggested that Federalist Society members should read the D.C. Circuit’s opinion in Ambrosini v. Labarraque, 101 F.3d 129 (D.C. Cir. 1996) (Rogers, J.), cert. dismissed, 117 S. Ct. 1572 (1997). The plaintiff in Ambrosini claimed that her child’s birth defects had been caused by her ingestion of the drugs Bendectin and Depo-Provera while pregnant. Based on Daubert, the D.C. Circuit had earlier held that the plaintiff’s Bendectin-related expert testimony should be excluded. In Ambrosini the court concluded that the expert testimony relating to the plaintiff’s claims regarding Depo-Provera should be treated differently, primarily because in the case of Depo-Provera the defendants had not produced the same body of epidemiologic evidence that they had mustered against the claim that Bendectin causes birth defects. In Professor Bernstein’s view, this approach is wrong for two reasons. First, it contradicts Daubert because that decision requires that admissible expert testimony qualify as science. Since the same flimsy sorts of animal studies had been presented to support the plaintiff’s Bendectin claims, testimony regarding the Depo-Provera claims should also have been excluded. In the words of Professor Bernstein, this approach was erroneous because "something’s either science or it’s not." A plaintiff’s evidence cannot be transmogrified into science based on a defendant’s inability to produce evidence on the other side. Second, and more obviously, the plaintiff has the burden of proof. It was thus fundamental error in Ambrosini to give dispositive weight to the lack of contrary evidence presented by the defendant when deciding whether to grant a Daubert motion. (The Joiner opinion is similarly guilty of improper burden-shifting. See Joiner, 78 F.3d at 537 (Smith, J., dissenting) (citing Daubert).)

Professor Bernstein also added to his reading list in this area the Ninth Circuit’s decision in Hopkins v. Dow Corning Corp., 33 F.3d 1116 (9th Cir. 1994), cert. denied, 513 U.S. 1082 (1995). The district judge in Hopkins should have excluded the plaintiff’s experts according to Professor Bernstein. (It will come as little surprise to Federalists that the district judge in Hopkins was Thelton Henderson, the same judge who struck down the California Civil Rights Initiative on logic that was tantamount to arguing that affirmative action is not only constitutionally permissible but constitutionally compelled. See Coalition for Economic Equity v. Wilson, 946 F.Supp. 1480, rev’d 110 F.3d 1431 (9th Cir. 1997).)

Continuing with his attack on the absent Roisman, Professor Bernstein turned to Roisman’s assertion that cases are sometimes ahead of the curve. Professor Bernstein was willing to entertain the possibility that a few true Galileos might somehow find their way to testifying for plaintiffs. The problem, Professor Bernstein argued, is a practical one -- most of the new "scientific" theories advanced in the courtroom turn out to be wrong. Perhaps a different set of legal rules should obtain if the opposite were true, but it is not. Thus, keeping a few Galileos out of the courtroom is a small price to pay to obtain the benefits of excluding a host of quacks.

Professor Bernstein associated Roisman’s approach with that of fellow Professor E. Donald Elliott. Particularly irksome to Professor Bernstein is Professor Elliott’s assertion that "Toxic tort cases are about good and evil, about corporate greed and indifference, and about risk of the unknown. But above all, toxic tort cases are about redefining our public morality for a new era in which we must confront the troubling truth that we do not fully comprehend the relationships between the things that we have made and our health and well-being." Planning and Managing Mass Toxic Tort Cases, C534 ALI-ABA 605, 611 (1990). "Redefining our public morality" "is a bit much to ask of our tort system," Professor Bernstein maintained. Professor Bernstein also found Professor Elliott’s pioneering sense of justice questionable since Professor Bernstein believes that plaintiffs should have to establish that there has truly been a victim before being allowed to secure a recovery from a potentially blameless party. Mere status as a corporation and the environmental track record of corporations generally should never be enough to change the normal rules of evidence.

Sharman then put his own question to Professor Bernstein before opening up the floor more generally. In Sharman’s view, since most juries approach cases in good faith and do their best to muddle through even complicated scientific issues, the problem in this area of the law is judges who allow "junk science" to go to juries and thereby either confuse them or provide them with a handy justification for indulging their prejudices. Implying that the presentations of the day had operated on perhaps too theoretical a plane, Sharman asked Professor Bernstein for practical advice on to deal with judges like Judge Billy Bob, who often say that Daubert-like arguments are really arguments about the sufficiency of the evidence. Given that perspective, such judges are unwilling to "cut the legs out from under" plaintiffs at an early stage of the litigation. Professor Bernstein could only reiterate his point that the falsifiability prong of Daubert is easily translatable into a plea to a judge to force the side propounding "junk science" to "give us something we can cross examine." Professor Bernstein conceded, however, that the admissibility and sufficiency inquiries in this area of the law were intertwined to such a degree, however, that it is hard to give simple advice about how to sway judges inclined to frame admissibility issues as matters of sufficiency.

The next question to Professor Bernstein came from an audience member who was troubled by the reality that much of the science bearing on commonly litigated issues is performed by the corporate defendants themselves or by other industry-affiliated scientists rather than pure academics. Don’t plaintiffs in toxic tort cases superficially appear to have a point when they advance claims of bias? To this Professor Bernstein suggested that expanded use of neutral scientific panels should be investigated, such as Warren had advocated in the morning session. (It appears to the author that a further useful response to overly-simplistic arguments for even-handed application of Daubert is that there are solid reasons for judges (and juries) to give more credence to corporate science over plaintiff-generated science. While completely neutral science is the ideal, corporate science is at least monitored in many areas by federal or state regulators. Much corporate science is in fact performed to satisfy regulatory requirements. The extensive testing required by the Federal Insecticide, Fungicide, and Rodenticide Act comes to mind readily. Plaintiff-driven science is far more questionable because there really are no external checks on its validity other than judges who faithfully apply Daubert (or in the state courts that have not followed Daubert, Frye). Many hired-gun experts can make a good enough living as frequent witnesses that they cease even to guard their professional repuations.) Professor Bernstein also referenced a work in the Federalist Society’s anchor journal by audience member, Paul Taylor, who explored the common-law self-critical analysis privilege, which prevents voluntarily performed corporate investigations from being used by plaintiffs against the corporations that generated the information. See Note, Encouraging Product Safety Testing by Applying the Privilege of Self-Critical Analysis When Punitive Damages Are Sought, 16 Harv. J. L. & Pub. Pol’y 769 (1993).

Another of the positions advanced by Professor Elliott that Professor Bernstein criticized was the suggestion that the burden on Daubert-like questions of admissibility should be shifted to defendants when plaintiffs can show that there was some effort by a defendant to conceal material information (usually through the mechanism of the "discovery tort" discussed in the morning) from past or present plaintiffs. One audience member intrigued by this concept asked Professor Bernstein whether he took his distaste for this burden-shifting idea so far as to reject the doctrine of spoliation, which applies a judicial inference that destroyed evidence was damaging to the reponsible party’s case. Professor Bernstein responded in the negative -- there is an important difference between withheld evidence that is eventually turned over and evidence that is destroyed. Sharman agreed with Professor Bernstein and echoed Warren’s concerns in the morning session about the rise of the "discovery tort."

The next question put to Professor Bernstein was whether the Seventh Amendment requires plaintiffs to be given the opportunity to present the views of minority scientists to juries. In Professor Bernstein’s view, Seventh Amendment objections to the exclusion of evidence are red herrings and the Supreme Court rightly gave them short shrift in Daubert. Codes of evidence have never been thought unconstitutional. By contrast, the trio of summary judgment cases in 1986 seems to have impinged on the right to a jury trial far more than Daubert in Professor Bernstein’s view and those cases are good law. Provocative ideas like Judge Hand’s solution to "junk science" do raise novel Seventh Amendment questions for Professor Bernstein, however. Professor Bernstein also endorsed a recommendation by Professor George Priest that juries should be required to at least write down the reasons for their verdicts in complex cases. Professor Bernstein suggested the possibility, however, that once the curtain hiding Oz was torn away such a reform might have the effect of toppling the civil-jury-trial system we use in products-liability cases.

Lastly, Professor Bernstein was asked about how case reports in the medical literature contribute to new litigation crazes. He pointed out that scientists, like members of the media, have an incentive to make news and thus there is a bias in the scientific literature, at least when considering an issue for the first time, to search for a causal link between some product or service and the illnesses of users. In closing, Professor Bernstein recognized that while peer review is an important factor in analyzing whether a theory can truly claim scientific status, it should not be thought dispositive. He pointed to the mild scandal over an article published in the Journal of the American Medical Association (JAMA), purporting to establish that breast-implants caused nursing problems. There were many flaws in this study, however, and Professor Bernstein opined that it never should have been published. See Jay P. Mayesh & June A. O'Hea, Second-Generation Breast Implant Claims: A Tough Road to Hoe, 5 Med./Leg. Aspects of Breast Implants No. 3 (1997), available in WESTLAW, TP-ALL database.

Speaking for those who attended this Colloquium, I can say that I thoroughly enjoyed it and I encourage attendance at the next such event.

*Jeffrey Bossert Clark is an associate at the law firm of Kirkland & Ellis and a member of the Federalist Society’s E.L. Wiegand Practice Groups in Administrative Law & Regulation, Environmental Law & Property Rights, and Federalism & Separation of Powers.


2001 The Federalist Society