Summary
striking part of rebuttal report because "Plaintiffs will not be allowed to 'sandbag' Defendants with new analysis that should have been included at the very least in [expert's] opening merits report."
Summary of this case from Rovid v. Graco Children's Prods. Inc.Opinion
Case No.: 11-CV-02509-LHK
04-04-2014
IN RE: HIGH-TECH EMPLOYEE ANTITRUST LITIGATION THIS DOCUMENT RELATES TO: ALL ACTIONS
ORDER RE: DEFENDANTS' MOTIONS REGARDING DR. LEAMER AND DEFENDANTS' JOINT MOTION FOR SUMMARY JUDGMENT BASED ON MOTION TO EXCLUDE TESTIMONY OF DR. LEAMER
On January 9, 2014, Defendants jointly moved to strike portions of Dr. Edward Leamer's reply report. ECF No. 557 ("Strike Mot."). Plaintiffs filed an opposition. ECF No. 600 ("Strike Opp."). Defendants filed a reply. ECF No. 714 ("Strike Reply"). On January 10, 2014, Defendants jointly moved to exclude the testimony of Dr. Leamer. ECF No. 570 ("Leamer Mot."). Plaintiffs filed an opposition. ECF No. 604 ("Leamer Opp."). Defendants filed a reply. ECF No. 715 ("Leamer Reply."). On January 9, 2014, Defendants filed a joint motion for summary judgment based on their motion to exclude Dr. Leamer's testimony. ECF No. 556. Plaintiffs filed an opposition. ECF No. 603. Defendants filed a reply. ECF No. 712.
The Court held a hearing on these motions on March 27, 2014. Having considered the briefing, relevant law, and oral argument, the Court GRANTS in part and DENIES in part Defendants' motion to strike Dr. Leamer's report, DENIES Defendants' motion to exclude Dr. Leamer's testimony under Daubert, and DENIES Defendants' joint motion for summary judgment.
I. BACKGROUND
Plaintiffs Michael Devine, Mark Fichtner, Siddharth Hariharan, and Daniel Stover, on behalf of themselves and a class of those similarly situated, filed the instant litigation against Defendants Adobe Systems Inc. ("Adobe"), Apple Inc. ("Apple"), Google Inc. ("Google"), Intel Corp. ("Intel"), Intuit Inc. ("Intuit"), Lucasfilm Ltd. ("Lucasfilm"), and Pixar. ECF No. 65. Plaintiffs allege that the Defendants entered into several bilateral agreements with each other pursuant to which the parties to the agreement would not cold call each other's employees. Id. ¶ 55. The crux of Plaintiffs' complaint is that these bilateral agreements together form an overarching conspiracy that suppressed wages for all of Defendants' employees. Id. Plaintiffs contend that Defendants' agreements violated Section 1 of the Sherman Antitrust Act, 15 U.S.C. § 1, and Section 4 of the Clayton Antitrust Act, 15 U.S.C. § 15.
Plaintiffs filed a Consolidated Amended Complaint, the operative complaint, on September 13, 2011. See id. Defendants filed a Joint Motion to Dismiss the consolidated amended complaint on October 13, 2011, see ECF No. 79, and, with leave of the Court, Lucasfilm filed its separate Motion to Dismiss on October 17, 2011, see ECF No. 83. Following full briefing on both motions and a hearing on January 26, 2012, see ECF No. 108, the Court granted in part and denied in part Defendants' Joint Motion to Dismiss and denied Lucasfilm's Motion to Dismiss on April 18, 2012, see ECF No. 119.
On October 1, 2012, Plaintiffs filed their motion for class certification, in which Plaintiffs sought to certify a class made up of all of Defendants' employees during the conspiracy period. After full briefing and a hearing, the Court granted in part and denied in part the motion on April 5, 2013. See ECF No. 382 ("April Order"). In that order, the Court denied the motion to certify the class, but appointed interim Co-Lead Counsel and Class Counsel. Id. The Court's analysis focused on the predominance requirement of Rule 23(b)(3) of the Federal Rules of Civil Procedure. The Court found that Plaintiffs had not demonstrated that common questions would predominate with respect to the antitrust impact element of Plaintiffs' claim. Id. at 29. The Court, however, gave Plaintiffs leave to amend to address the Court's concerns in light of the fact that Defendants had not produced the discovery needed for class certification. Id. at 47, 52.
On May 10, 2013, Plaintiffs filed a supplemental motion for class certification, seeking certification of a narrower class of technical employees. While the motion was pending, Plaintiffs reached a settlement with Pixar, Lucasfilm, and Intuit, which the Court has preliminarily approved. After full briefing and a hearing, the Court granted Plaintiffs' motion for class certification on October 24, 2013. ECF No. 531 ("October Order"). The Court certified the class of technical employees because Plaintiffs had met their burden under Rule 23. Defendants sought interlocutory review of the Court's class certification order. On January 14, 2014, however, the Ninth Circuit exercised its discretion to deny Defendants' petition for immediate review. ECF No. 594.
On March 28, 2014, after full briefing, the Court denied the Defendants' individual motions for summary judgment filed by Adobe, Apple, Google, and Intel. See ECF No. 771. This Order addresses Defendants' joint motion to strike Dr. Leamer's reply report, Defendants' joint motion to exclude Dr. Leamer's testimony under Daubert, and Defendants' joint motion for summary judgment based on their motion to exclude the testimony of Dr. Leamer.
II. LEGAL STANDARDS
A. Motion to Exclude Testimony under Daubert
Federal Rule of Evidence 702 allows admission of expert opinions based on "scientific, technical, or other specialized knowledge" if such an opinion would "help the trier of fact to understand the evidence or to determine a fact in issue." Fed. R. Evid. 702. Expert testimony is admissible if it is both relevant and reliable. Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 589 (1993). When considering expert testimony, the trial court acts as a "gatekeeper" by assessing the soundness of the expert's methodology to exclude "junk science." Estate of Barabin v. AstenJohnson, Inc., 740 F.3d 457, 463 (9th Cir. 2014); see Kumho Tire Co. v. Carmichael, 526 U.S. 137, 147-48 (1999); Daubert, 509 U.S. at 589-90. An expert witness may provide opinion testimony if: (1) the testimony is based upon sufficient facts or data; (2) the testimony is the product of reliable principles and methods; and (3) the expert has reliably applied the principles and methods to the facts of the case. Fed. R. Evid. 702. Under Daubert, in determining reliability, courts can consider (1) whether a theory or technique "can be (and has been) tested;" (2) "whether the theory or technique has been subjected to peer review and publication;" (3) "the known or potential rate of error;" and (4) whether there is "general acceptance" of the methodology in the "relevant scientific community." Daubert, 509 U.S. at 593-94. "[F]ar from requiring trial judges to mechanically apply the Daubert factors . . . judges are entitled to broad discretion when discharging their gatekeeping function." United States v. Hankey, 203 F.3d 1160, 1168 (9th Cir. 2000) (citation omitted). The proponent of the expert has the burden of proving admissibility by a preponderance of the evidence. Lust v. Merrell Dow Pharmaceuticals, Inc., 89 F.3d 594, 598 (9th Cir. 1996); Daubert, 509 U.S. at 592 n.10.
The Supreme Court has cautioned that "Daubert's list of specific factors neither necessarily nor exclusively applies to all experts or in every case." Kumho Tire, 526 U.S. at 141.
Rule 702 "mandates a liberal standard for the admissibility of expert testimony." Cook v. Rockwell Int'l Corp., 580 F. Supp. 2d 1071, 1082 (D. Colo. Dec. 7, 2006); Daubert, 509 U.S. at 588 (Rule 702 is part of the "liberal thrust" of the Federal Rules of Evidence); Dorn v. Burlington N. Sante Fe R.R. Co., 397 F.3d 1183, 1196 (9th Cir. 2005) ("The Supreme Court in Daubert [] was not overly concerned about the prospect that some dubious scientific theories may pass the gatekeeper and reach the jury under the liberal standard of admissibility set forth in that opinion[.]"). Thus, the inquiry into admissibility of expert opinion is a "flexible one," where "[s]haky but admissible evidence is to be attacked by cross examination, contrary evidence, and attention to the burden of proof, not exclusion." Primiano v. Cook, 598 F.3d 558, 564 (9th Cir. 2010) (citing Daubert, 509 U.S. at 596). The "district judge is 'a gatekeeper, not a fact finder.' When an expert meets the level established by Rule 702 as explained in Daubert, the expert may testify and the jury decides how much weight to give that testimony." Id. (citation omitted).
B. Motion to Strike Testimony
Rule 26(a)(2)(B) provides that an expert witness's opening report must contain "a complete statement of all opinions the witness will express and the basis and reasons for them" together with "the facts or data considered by the witness in forming them" and "any exhibits that will be used to summarize or support them." Fed. R. Civ. P. 26(a)(2)(B)(i)-(iii). Rebuttal disclosures of expert testimony are "intended solely to contradict or rebut evidence on the same subject matter identified by another party" in its expert disclosures. Fed. R. Civ. P. 26(a)(2)(D)(ii). "Rule 37(c)(1) gives teeth to these requirements by forbidding the use at trial of any information required to be disclosed by Rule 26(a) that is not properly disclosed." Yeti by Molly, Ltd. v. Deckers Outdoor Corp., 259 F.3d 1101, 1106 (9th Cir. 2001). This rule requires the exclusion of untimely expert witness testimony, unless the "part[y's] failure to disclose the required information is substantially justified or harmless." Id. (citation omitted). The moving party bears the burden of showing a discovery violation has occurred. See Hernandez ex rel. Telles-Hernandez ex-rel. Telles-Hernandez v. Sutter Medical Center of Santa Rosa, 2008 WL 2156987, at *13 (N.D. Cal. May 20, 2008). Once that burden is satisfied, the burden shifts and the nonmoving party must prove that its failure to comply with Rule 26 was either justified or harmless. Yeti by Molly, 259 F.3d at 1107.
III. ANALYSIS
A. Defendants' Motion to Strike and Motion to Exclude Dr. Edward Leamer
Defendants move to strike portions of Dr. Leamer's December 2013 reply report and to exclude Dr. Leamer's testimony under Daubert. Specifically, Defendants move to strike Dr. Leamer's use of a 50% statistical significance theory to defend his "conduct regression," Dr. Leamer's arguments relating to the "total new hires" variable in his conduct regression, and Dr. Leamer's arguments relating to his use of real compensation in his conduct regression. Defendants move to exclude under Daubert Dr. Leamer's testimony, raising four specific challenges to his conduct regression model.
The Court first sets forth the relevant history of expert reports submitted in this case, a summary of Dr. Leamer's conclusions, and a summary of significance testing as necessary context and background for Defendants' motions to strike and exclude Dr. Leamer's testimony. For reference, the Court notes that all the challenges in Defendants' motion to strike and motion to exclude pertain solely to Dr. Leamer's conduct regression model detailed below.
1. Summary of Expert Reports and Dr. Leamer's Analysis
At the class certification stage, Dr. Leamer submitted four expert reports on Plaintiffs' behalf: (1) October 1, 2012 ("Class Cert. Opening Rep."), ECF No. 190; (2) December 10, 2012 ("Class Cert. Reply Rep."), ECF No. 558-4; (3) May 10, 2013 ("Suppl. Class Cert. Rep."), ECF No. 558-5; and (4) July 12, 2013 ("Suppl. Class Cert. Reply Rep."), ECF No. 454-4. In addition, at the class certification stage, defense expert Dr. Kevin Murphy submitted a report on November 12, 2012 ("Murphy Class Cert. Rep."), ECF No. 230, and a supplemental report on June 21, 2013 ("Murphy Suppl. Class Cert. Rep."), ECF No. 440.
Dr. Murphy challenged Dr. Leamer's analysis, concluding that individualized inquiries predominate over common ones in this case for the purpose of determining impact.
On October 28, 2013, Dr. Leamer filed his opening merits report ("Leamer Opening"), ECF No. 558-6. On November 25, 2013, defense expert Dr. Lauren Stiroh submitted her rebuttal merits report challenging Dr. Leamer's analysis ("Stiroh Rebuttal"), ECF No. 558-7. On December 11, 2013, Dr. Leamer submitted his reply report ("Leamer Reply Rep."), ECF No. 558-8.
Edward E. Leamer, Ph.D, is the Chauncey J. Medberry Professor of Management, Professor of Economics, and Professor of Statistics at the University of California, Los Angeles. Dr. Leamer earned a B.A. in Mathematics from Princeton University in 1966, and a Masters in Mathematics and a Ph.D. in Economics at the University of Michigan in 1970. Class Cert. Opening Rep. ¶ 1. He has published on the topics of econometric methodology and statistical analysis, international economics, and macro-economic forecasting, including on the subject of inferences that may appropriately be drawn from non-experimental data. Id. ¶ 2.
Dr. Murphy also submitted a merits expert report on November 25, 2013, but his report did not contain an assessment of Dr. Leamer's October 2013 merits report.
Plaintiffs submitted four reports from Dr. Leamer in support of their argument at the class certification stage that common issues predominate for the purpose of assessing classwide impact and damages. In Dr. Leamer's first report in October 2012, Plaintiffs asked him to evaluate whether classwide evidence was capable of showing that the anti-solicitation agreements artificially reduced the compensation of: (1) members of the class generally, and (2) all or most members of the class. Class Cert. Opening Rep. ¶ 10(a). In addition, Plaintiffs asked Dr. Leamer a second question—to assess whether there was a reliable classwide method capable of quantifying the amount of suppressed compensation suffered by the class. Id. ¶ 10(b). Dr. Leamer answered both questions in the affirmative.
The three elements of Plaintiffs' antitrust claim are: (1) violation of antitrust law; (2) injury, or "impact"; and (3) damages. In re New Motor Vehicles Canadian Export Antitrust Litigation, 522 F.3d 6, 19 n.18 (1st Cir. 2008) (citation omitted).
Because this Court certified only a class comprised of technical, creative, and research and development employees, the Court omits all discussion in prior expert reports relating to Plaintiffs' putative class of all employees. October Order at 10-11.
As explained below, Dr. Leamer's analysis with respect to the first question proceeded in two steps. First, Dr. Leamer explained that economic theory, documentary evidence, and multiple regression analyses were capable of showing that the anti-solicitation agreements tended to suppress employee compensation generally by preventing class members from discovering the true value of their work. Id. ¶¶ 11(a)-(b), 63. Second, Dr. Leamer illustrated how economic theory, documentary evidence, and statistical analyses are capable of showing that this suppression of compensation affected all or nearly all class members. Id. ¶¶ 11(c), 64.
Dr. Leamer first concluded that classwide evidence was capable of showing that the anti-solicitation agreements suppressed compensation of class members generally. This first step was supported by principles of information economics, such as "market price discovery." Dr. Leamer noted that, when evaluating labor markets, economists often use a market equilibrium model, which "presume[s] that market forces . . . work rapidly enough that virtually all transactions occur at approximately the same price—the 'market price' which equilibrates supply and demand." Id. ¶ 71. In reality, when labor market conditions change, high transaction costs and limited information flow can slow the process by which transaction prices reach market equilibrium. Id. ¶¶ 72-73. "Market price discovery" is the process by which participants in a market search for this equilibrium. Id. ¶ 71.
Dr. Leamer opined that the high transaction costs—including time, money, and personal dislocation—involved in searching for high tech jobs limit the number of existing workers seeking new employment. Id. ¶ 74. Defendants and other high tech companies value potential employees who are not actively looking for new employment opportunities ("passive candidates") more than those who are looking for new jobs ("active candidates") because currently satisfied employees: (1) tend to be perceived as more qualified, diligent, and reliable; (2) often have training, on-the-job experience, and track records that save the hiring company search and training costs; and (3) are valuable assets that, if hired away from rivals, can harm competitors. Id. ¶ 62. Thus, recruiting these passive candidates by cold calling is both an important tool for employers and a key channel of information for employees about outside opportunities. Id. ¶¶ 57-62, 75.
Dr. Leamer hypothesized that, by restricting cold calling and other competition over employees, Defendants' anti-solicitation agreements impaired information flow about compensation and job offers. Defendants' inhibition of employees' ability to discover and obtain the competitive value of their services meant employees were afforded fewer opportunities to increase their salaries by moving between firms and deprived of information that could have been used to negotiate higher wages and benefits within a firm. Id. ¶¶ 71-76. In addition, by limiting the information available to employees, Defendants could avoid taking affirmative steps, such as offering their employees financial rewards and other forms of profit sharing, to retain employees with valuable firm-specific skills. Id. ¶¶ 77-80.
Dr. Leamer relied on documentary evidence as further support for the link between the anti-solicitation agreements and compensation reduction. Id. ¶¶ 81-88. He also performed regression analyses which utilized Defendants' internal compensation data to illustrate class members' undercompensation by comparing compensation during the conspiracy with compensation in a conspiracy-free, but-for world. Id. ¶¶ 135-46, Figs. 20-24. Dr. Leamer concluded that the regression analyses showed that the anti-solicitation agreements artificially suppressed compensation at each Defendant. Id.
"A regression is a statistical tool designed to express the relationship between one variable, such as price, and [independent] variables that may affect the first variable. Regression analysis can be used to isolate the effect of an alleged conspiracy on price, taking into consideration other factors that might also influence price, like costs and demand." In re Aftermarket Auto. Lighting Prods. Antitrust Litig., 276 F.R.D. 364, 371 (C.D. Cal. July 25, 2011) (citation omitted). The coefficient for any given independent variable measures how the dependent variable responds, on average, to a change in that independent variable. Federal Judicial Center, Reference Manual on Scientific Evidence 336 (3d ed. 2011) ("Ref. Manual"). In other words, regression coefficients represent the mean change in the dependent variable for one unit of change in the independent variable, holding other independent variables in the model constant.
Dr. Leamer's second step was to opine that economic theory, documentary evidence, and statistical analyses were capable of showing that this compensation suppression had widespread effects—i.e., that suppression of compensation affected all or nearly all class members. Id. ¶ 101. Dr. Leamer first relied on economic theories of loyalty, fairness, and internal equity to explain how the adverse effects on compensation due to Defendants' anti-solicitation agreements would have been felt by employees who would have received a cold call or had a significant chance of receiving a cold call and employees who are linked to these groups due to internal equity considerations. Suppl. Class Cert. Reply Rep. ¶¶ 27-28. In other words, Dr. Leamer contended that labor markets rely on committed long-term relationships built on trust, understanding, and mutual interests. Class Cert. Opening Rep. ¶ 102. Thus, both employers and employees seek ways to turn the market transaction into secure long-term relationships, which "can come either from commitment (emotional or financial) to the mission of the organization, or from jointly owned firm-specific assets." Id.Companies thus attempt to create loyalty "by getting buy-in from the firm's mission and by making the place of work as appealing as possible." Id. ¶ 103.
"One foundation of employee loyalty is a feeling of fairness that can translate into a sharing of . . . [a firm's] rewards with more equality than a market might otherwise produce." Id. ¶ 104. Firms seek to promote a feeling of fairness among employees to maintain or to increase employees' commitment and contentment, which also leads to higher levels of productivity. Suppl. Class Cert. Rep. ¶ 16. Dr. Leamer explained that, "[t]o maintain loyalty, it is usually better for a firm to anticipate rather than to react to outside opportunities, since if a worker were to move to another firm at a much higher level of compensation, coworkers left behind might feel they have not been fairly compensated. That can have an adverse effect on worker loyalty, reducing productivity and increasing interest in employment elsewhere." Class Cert. Opening Rep. ¶ 105.
Dr. Leamer opined that the information conveyed by an outside offer or a cold call could stimulate a response by management that could extend beyond the specific individual who received the cold call. Suppl. Class Cert. Rep. ¶ 15. Even though the market may not mandate a rise in compensation for these similar individuals until they actually receive an outside offer, "preemptive improvements" can minimize the disruption to employee loyalty that might occur when an employee discovers the she was undercompensated. Class Cert. Opening Rep. ¶ 105. Thus, "[c]old-[c]alling—as well as just the threat of [c]old-[c]alling—puts upward pressure on compensation." Id. ¶ 106. Dr. Leamer opined that "a broad preemptive response is completely analogous to salary increases that are tied to information provided by employment services regarding the compensation offered by the 'market.'" Suppl. Class Cert. Rep. ¶ 15. Essentially, Dr. Leamer opined that the "response to bursts of cold calls and, even more, the response to the threat of cold calls" would raise internal equity concerns that would spread the impact throughout the class. Suppl. Class Cert. Reply Rep. ¶ 27. Dr. Leamer also noted that the documentary evidence showed that Defendants each employed company-wide compensation structures that included grades and titles, and that high-level management established ranges of salaries for grades and titles, which left little scope for individual variation. Class Cert. Opening Rep. ¶¶ 121-22.
Dr. Leamer also utilized statistical analyses as evidence that the anti-solicitation agreements broadly affected members of the class. Id. ¶¶ 120-34. These regressions were based on Defendants' salary structures and compensation data. Id. ¶¶ 127-30, Figs. 11-14. These "Common Factors Analyses" assessed Defendants' "firmwide compensation structures, and the formulaic way in which total compensation was varied over time." Id. ¶ 128. According to Dr. Leamer, approximately 90 percent of the variation in any individual employee's compensation could be explained by common factors "such as age, number of months in the company, gender, location, title, and employer." Id.; see also id., Figs. 11-14. Dr. Leamer concluded that "[t]he fact that nearly all variability in class member compensation at any point in time can be explained by common variables means there was a systematic structure to employee compensation at each of the Defendant firms." Id. ¶ 130. Dr. Leamer opined that these rigid wage structures, and the fact that the coefficients in his regressions did not vary substantially over time, suggested that "compensation of class members tended to move together over time and in response to common factors," such that the effects of the anti-solicitation agreements would be expected to be experienced broadly. Id.
Dr. Leamer further opined that the evidence showed "a persistent salary structure across employees consistent with important elements of equity in the Defendants' compensation practices." Id. ¶ 134. Dr. Leamer relied on five compensation movement charts that depicted changes in the base salaries and total compensation for ten major job titles at Apple between 2006 and 2009, and the ten major job titles at Google between 2005 and 2009. Id., Figs. 15-17. Dr. Leamer contended that these charts offered further evidence that compensation for different positions tended to move together over time (i.e., if software engineers received a raise, so did account executives). Id. ¶¶ 133-34. Based on this evidence, Dr. Leamer opined that the anti-solicitation agreements that focused on subsets of workers would nonetheless have broader effects because of a desire on Defendants' part to maintain the overall salary structure. Id.
The second question Plaintiffs asked Dr. Leamer to assess was whether there was a classwide method of quantifying the total amount of suppressed compensation suffered by the class generally. Id. ¶ 10(b). Dr. Leamer concluded that a regression could quantify the estimated cost to the class resulting from Defendants' challenged conduct—in terms of wage suppression during the periods when anti-solicitation agreements were in effect for each Defendant. Id. ¶¶ 141-48. This is the regression model Defendants challenge in Defendants' instant motion to strike and motion to exclude Dr. Leamer's testimony. Dr. Leamer's model, to which the Court previously referred as the "conduct regression," uses the real annual compensation of each employee in each year as the dependent variable, and includes various independent variables designed to account for factors including: (1) age, sex, and years at the company; (2) the effects on compensation caused by the anti-solicitation agreements; (3) the effects caused by factors specific to each Defendant (e.g., firm revenue, total number of new hires, etc.); and (4) the effects caused by the industry. Id.; id. Fig. 23.
The model is intended to predict the average effect of the anti-solicitation agreements on compensation, holding other compensation-related variables constant. The critical independent variable is the general "conduct variable," which represents "the fraction of months in each year during which the employer was involved in one or more of the agreements." Id. ¶ 145. This variable "estimate[s] the immediate impact of the illegal conduct." Id. ¶ 146. The model also includes three interaction variables representing the interaction between the general conduct variable and employee age, employee age squared, and the hiring rate at an employee's firm to allow for the possibility that the agreements had effects that varied over time, across firms, and across individuals. Id. ¶ 145. Dr. Leamer also identifies these interaction variables as "conduct" variables separate from his general conduct variable. Class Cert. Reply Rep. ¶ 107.
In a regression model, an "interaction" variable is the product of two other variables that are included in the regression model. Ref. Manual at 316. The "interaction variable essentially allows the expert to take into account the possibility that the effect of a change in one variable on the dependent variable may change as the level of another explanatory variable changes." Id. Here, Dr. Leamer "interacted" these variables to allow for the possibility that the firms' illegal behavior had different effects on employees of different ages, or had different effects on employees at firms that had been doing different amounts of hiring relative to their total number of employees.
More specifically, the conduct regression estimates the effect of the anti-solicitation agreements by contrasting compensation during the periods when the anti-solicitation agreements were in effect with compensation before and after the anti-solicitation agreements. Class Cert. Opening Rep. ¶ 136; Class Cert. Reply Rep. ¶ 72. The model generates percentages—or regression estimates—by which Defendants undercompensated the class employees in each of the conspiracy years. See Class Cert. Opening Rep., Fig. 24 ("Estimated Impact on Technical Employee Class Total Compensation"). Dr. Leamer used this model to show that the anti-solicitation agreements suppressed compensation of the class generally, and to estimate the average or net under-compensation at each Defendant firm during the period in which the anti-solicitation agreements were in effect. See id. Dr. Leamer contended that this model could be used in a formulaic fashion to calculate aggregate damages to the class. See id. ¶ 148.
Dr. Leamer's general conduct variable is an indicator for when the challenged agreements were in effect. Leamer Opening ¶¶ 20-21, 44-45. It is a "zero-one" variable that is turned "on" for a particular defendant during the period when that defendant allegedly participated in any of the challenged agreements. Id. It takes on a value of one in the years when a defendant had an agreement and zero otherwise. Id. First, the model is run with the conduct variable with a value of one. Id. Second, compensation is calculated (the regression is run) with the conduct variable turned off to reflect what compensation would have been had there been no non-compete agreements. Id. The difference in compensation between these two runs is the estimated reduction in total compensation due to the agreements. Id. The impact of the agreements on wage per year is the coefficient on the general conduct variable—i.e., if the coefficient is 0.0559, total compensation was reduced by 5.59% in one year. Leamer Reply Rep. ¶ 85.
In his October 2013 opening merits report, Dr. Leamer explained that his original conduct regression that utilized individual employee compensation data and was outlined in his October 2012 report (hereinafter "original conduct regression") continued to be the best approach for estimating the total impact on the class as well as the damages the class suffered. Leamer Opening ¶¶ 24, 29-31. Dr. Leamer concluded the class was undercompensated by $3.06 billion as a result of the agreements. Id. ¶ 46; Fig. 7. In order to address Dr. Murphy's prior criticisms of his conclusions at the class certification stage, Dr. Leamer also ran his original conduct regression with clustered standard errors as Dr. Murphy recommended. Id. ¶ 28; Ex. 2 ("Compensation Model [Without Clustered Standard Errors]"); Ex. 3 ("Compensation Model with Clustered Standard Errors"). Dr. Leamer concluded that while clustering the errors changed the standard errors for each variable (in comparison to a regression without clustered errors), the change had "no impact on the estimates of damages." Id. ¶ 26.
Dr. Leamer made only "minor changes" to the original model to reflect updated data and changes to the composition of the class. Leamer Opening ¶¶ 2, 16, 19, 32, 39.
Standard errors measure the likely difference between the estimated value for a variable's coefficient and its true value. Ref. Manual at 281. "An estimate based on a sample is likely to be off the mark, at least by a small amount, because of random error. The standard error gives the likely magnitude of this random error, with smaller standard errors indicating better estimates." Id. at 243; see also Daniel L. Rubinfeld, Reference Guide on Multiple Regression 467 (Federal Judicial Center, 3d ed. 2011) ("Rubinfeld").
2. Statistical Significance and Null Hypothesis Testing
The Court now provides an overview of null hypothesis testing, which is discussed throughout Defendants' motions. Statisticians often measure the accuracy of a regression model's estimates using what is called "significance testing" or "null hypothesis testing." Ref. Manual at 241. Statisticians determine whether the results are statistically significant enough such that they can reject the "null hypothesis" of zero effect, which means that the independent variable being tested has no actual impact on the dependent variable and that whatever relationship is shown in the model occurred due to random chance. Id. at 342, 354. In this case, the null hypothesis of zero effect would be that the anti-solicitation agreements had no actual impact on compensation.
"Significance level" is a term of art used in significance testing. Id. at 287. "The significance level measures the probability that the null hypothesis will be rejected incorrectly." Id. at 320. If there is less than an X% probability the independent variable's coefficient could have occurred simply due to random chance, then the null hypothesis can be rejected at the X% significance level. If there is more than X% probability that the result occurred by chance, the null hypothesis cannot be rejected at the X% significance level. In other words, if the coefficient is statistically significant at the 5% significance level, there is no more than a 5% likelihood that one would observe that relationship between the independent variable and dependent variable merely by chance. A 5% significance level "indicates that the demonstrated relationship between the variables would occur in a random sample five times out of one hundred[.]" White v. City of San Diego, 605 F.2d 455, 460 (9th Cir. 1979). The smaller the significance level at which one rejects the null hypothesis, the greater the confidence one has that the null hypothesis has been correctly rejected and that the regression's estimate is correct.
Statistical significance is determined by reference to a "p-value." Ref. Manual at 241. A "p-value" for a variable tests the null hypothesis that the coefficient for that variable is equal to zero. Id. at 320. It represents the "probability that a coefficient of this magnitude or larger could have occurred by chance if the null hypothesis were true." Id. If the p-value is less than or equal to the selected significance level, the null can be rejected because the result is said to be "statistically significant" at that level, which means the probability that the observed association is the result of chance rather than a true association is less than the stated significance level. DeLuca v. Merrell Dow Pharms., Inc., 911 F.2d 941, 945-47 (3rd Cir. 1990). If the p-value is greater than the significance level, then the result is said to be statistically insignificant at that level, which means there is insufficient evidence at the selected significance level to reject the "null hypothesis" of the observed association being a product of chance rather than a true association. Sander Greenland, The Need for Critical Appraisal of Expert Witnesses in Epidemiology and Statistics, 39 Wake Forest L. Rev. 291, 298 (2004). For example, a variable with a p-value greater than 0.05 means that the variable's coefficient is not statistically significant at the 5% significance level and that one cannot reject the null hypothesis that the variable has no effect on the dependent variable.
"Often, the null hypothesis is stated in terms of a particular regression coefficient being equal to 0." Ref. Manual at 320.
Statisticians can also test the null hypothesis by looking at the variable's "t-statistic." Ref. Manual at 342. If the t-statistic is less than 1.96 in magnitude, then at the 5% level, the statistician cannot reject the hypothesis that the estimate equals zero, so the estimate is said to not be statistically significant at the 5% level. Id. at 343. Conversely, if the t-statistic is greater than 1.96 in absolute value, the statistician concludes the true value of the coefficient is unlikely to be zero, the null can be rejected, and the estimate is deemed statistically significant at the 5% level. Id.
A t-statistic of 2.57 in magnitude or greater is associated with a 1% significance level. Ref. Manual at 343 n.83.
3. Defendant's Motion to Strike Portions of Dr. Leamer's Reply Report
Defendants move to strike three sections of Dr. Leamer's December 2013 reply report, claiming these new opinions should have been included in his October 2013 opening merits report. Strike Mot. at 1-2. For the reasons set forth below, the Court GRANTS IN PART AND DENIES IN PART Defendants' motion. The Court addresses each of Defendants' three contentions in turn.
a. Motion to Strike Paragraphs 75-90 & Figs. 15-16
First, Defendants contend Paragraphs 75-90 & Figs. 15-16 should be stricken as improper rebuttal because Dr. Leamer argues for the first time that his original conduct regression with clustered standard errors, see Leamer Opening, Exhibit 3, should be evaluated using a 50% statistical significance level if null hypothesis testing is to be used to assess the reliability of his model. Strike Mot. at 3-4. The Court agrees, and thus precludes Dr. Leamer from testifying about that opinion at trial.
In his December 2013 reply report, Dr. Leamer argues for the first time that if null hypothesis testing is to be used, a 50% level should be used to determine the statistical significance of the variables' coefficients in his original conduct regression with clustered errors, and opines that the coefficient on the general conduct variable is statistically significant at that level. Leamer Reply Rep. ¶¶ 75-90 & Figs. 15-16. Dr. Leamer presents the theory that this 50% level is the necessary result of balancing the risks and costs of "Type I" and "Type II" statistical errors, and offers an analysis of the relationship between these two types of errors. Id. He claims that in Dr. Stiroh's November 2013 rebuttal report, Dr. Stiroh failed to conduct this balancing test before choosing the 5% level to evaluate Dr. Leamer's model and concluding that the general conduct variable's coefficient was not statistically significant at that level. Id. ¶¶ 84-85 (referring to Stiroh Rebuttal ¶¶ 166-72; id. Ex. V.14). The Court agrees with Defendants that Dr. Leamer's new theory is untimely disclosed and should have been included in his October 2013 opening report. By presenting this analysis for the first time in Dr. Leamer's reply, Plaintiffs have deprived Defendants of the opportunity to respond. The following summary of the various reports in this case illustrates how Dr. Leamer could have and should have included this specific analysis in his reports prior to his December 2013 reply report.
A Type I error in this case would be a finding of classwide impact and damages when there were none. A Type II error would be a finding of no classwide impact and damages when in fact there was classwide impact and damage. Leamer Reply Rep. ¶ 83.
Plaintiffs argue that Dr. Leamer's analysis in his reply is proper rebuttal because it responds to Dr. Stiroh's criticism that Dr. Leamer's original conduct regression with clustered errors is unreliable because it fails to meet the 1%, 5%, or 10% levels, and also because Dr. Murphy never made this criticism, so Dr. Leamer "could not possibly have anticipated this [argument in his opening merits report]." Strike Opp. at 4. Plaintiffs are incorrect. Below, the Court first sets forth where Dr. Murphy made this precise criticism, and sets forth Dr. Leamer's responses to that criticism in Dr. Leamer's various reports, which notably do not mention any theory that a 50% significance level should be used to evaluate his original conduct regression model with clustered errors. Dr. Leamer had four expert reports in which he could have responded to Dr. Murphy's criticism in the manner he does in his December 2013 reply report, but he did not.
In his November 2012 class certification report, Dr. Murphy explicitly made the criticism that Dr. Leamer's original conduct regression with clustered errors is unreliable because it fails to meet the 1%, 5%, or 10% levels. More specifically, Dr. Murphy explained that Dr. Leamer's conduct regression failed to account for the fact that compensation for employees within the same firm is correlated. Murphy Class Cert. Rep. ¶ 126. Dr. Murphy contended that, given this correlation, Dr. Leamer should have clustered the standard errors in his model. Id. Critically, Dr. Murphy opined that when the errors are clustered, the general conduct variable's coefficient is not statistically significant at the 1%, 5%, and 10% levels when null hypothesis testing is conducted, and also that Dr. Leamer's final "undercompensation" percentages were not statistically significant at the 5% level. Id. ¶ 128; Ex. 21B ("Dr. Leamer's [] Regression Using Corrected Standard Errors"); Ex. 22B ("Dr. Leamer's Undercompensation Estimates Are Not Statistically Significant [at the 5% level]."). Dr. Murphy further noted, "The p-values imply that Dr. Leamer's estimates are completely consistent with there being no true effect of the desired conduct and his estimates resulting entirely from random factors unrelated to that conduct. Thus, once properly analyzed, Dr. Leamer's conduct regression provides no meaningful evidence that the challenged agreements reduced compensation[.]" Id. ¶ 128. He also emphasized Dr. Leamer did "not even acknowledge in his report that his reported standard errors and resulting t-statistics . . . were not meaningful." Id. ¶ 126.
Dr. Leamer responded to Dr. Murphy's critique in his December 2012 reply report but did not do so by setting forth his theory that a 50% significance level should be used to evaluate his original conduct regression with clustered errors. Rather, he argued that clustering standard errors is only one way of controlling for correlations between employees. Class Cert. Reply Rep. ¶¶ 76, 78, 82-83. Another approach would be to include variables to explain the commonalities across firms and capture the common sources of variation between employees. Id. ¶ 76, 83. He also emphasized that yet another approach would be to use an alternative regression model that utilized firm-wide compensation averages for each defendant as opposed to individual employee compensation data, which his original conduct regression utilized. Id. ¶ 103, 106; Figs. 12 & 14; see also Leamer Opening ¶ 29. Dr. Leamer conceded that in this alternative conduct regression model, not all the variables were statistically significant at the "conventional [5] percent and [10] percent levels. However, the T-values on the conduct coefficients are relatively high and provide evidence that the negative coefficients did not occur by mere chance." Class Cert. Reply Rep. ¶ 107.
Dr. Leamer noted that his regression model had already included one such variable, revenue. Class Cert. Reply Rep. ¶¶ 82-83.
Dr. Leamer's alternative model showed that of the five conduct variables' coefficients, two were statistically significant at the 1% level and three were not significant at the 1%, 5% or 10% levels. Class Cert. Reply Rep. Fig. 14.
In his May 2013 supplemental class certification report, Dr. Leamer did not respond to Dr. Murphy's criticism. Dr. Murphy's rebuttal supplemental class certification report in June 2013 again raised the same criticism, see Murphy Suppl. Class Cert. Rep. at 27, but Dr. Leamer's supplemental reply report in July 2013 did not address that criticism except to say that "[t]he work I have done so far establishes the robustness of my damages model[.]" Suppl. Class Cert. Reply Rep. at 31.
In Dr. Leamer's May 2013 supplemental expert report, Plaintiffs asked him to respond to questions raised by the Court related to whether Dr. Leamer's initial methodology could show classwide impact. Dr. Leamer found that his additional analyses confirmed his "original finding of a somewhat rigid pay structure at each Defendant firm that would have transmitted the effects of the agreements broadly, including throughout the Technical Class." Supp. Class. Cert. Rep. ¶ 13.
In his October 2013 opening merits report, Dr. Leamer addressed Dr. Murphy's criticism, but did so in a different way than his December 2012 reply report by actually running his original conduct regression with clustered errors as Dr. Murphy had recommended. Leamer Opening ¶ 28; Ex. 2 ("Compensation Model [Without Clustered Standard Errors]"); Ex. 3 ("Compensation Model with Clustered Standard Errors"). Dr. Leamer opined that although clustering the errors changed the standard errors for each variable (in comparison to his regression which did not cluster the errors), the change had "no impact on the estimates of damages" because the variables had the same exact coefficients in both models. Id. ¶¶ 26-28; Exs. 2 & 3. Again here, Dr. Leamer did not set forth his theory that a 50% significance level should be used to evaluate his original conduct regression with clustered errors.
As the above timeline reflects, the fact that Dr. Murphy made the exact same criticism as Dr. Stiroh in Dr. Murphy's November 2012 report demonstrates that Dr. Leamer knew about this criticism long before Dr. Stiroh's report and thus had four reports before Dr. Leamer's December 2013 reply in which he could have set forth his theory that a 50% significance level should be used to evaluate his original conduct regression with clustered errors. Yet he did not. For example, in his December 2012 reply, Dr. Leamer did not defend the reliability of his original conduct regression with clustered errors against Dr. Murphy's attack by rebutting that the statistical significance of that regression should be evaluated at the 50% level. Rather, he ran an alternative regression he claimed obviated the need for clustering. Class Cert. Reply Rep. ¶ 103, 106. In his October 2013 merits report, Dr. Leamer did run his original conduct regression with clustered errors but still did not defend the reliability of that model by stating that statistical significance should be evaluated at the 50% level. To the contrary, Dr. Leamer's own exhibit displaying the regression with clustered errors reports that the general conduct variable's coefficient is not statistically significant at the 1%, 5%, and 10% levels, and does not report statistical significance at any other level. Leamer Opening, Ex. 3. Dr. Leamer never explains in the body of his report that this result is not problematic because statistical significance should be evaluated at the 50% level, nor that his results were in fact significant at that 50% level. Simply put, Dr. Leamer's theory is untimely disclosed because he could have and should have included this theory in his opening merits report to allow Defendants the opportunity to respond. Further, Plaintiffs cannot reasonably characterize Dr. Leamer's new theory as simply proper rebuttal to Dr. Stiroh's decision to measure his original conduct regression with clustered errors at the 5% level because Dr. Leamer himself reported and utilized the same 5% level against his model in his October 2013 report.
Dr. Stiroh makes this precise point when stating that in Dr. Leamer's Exhibit 3, the general conduct variable's coefficient is not statistically significant. Stiroh Rebuttal ¶ 168. Dr. Leamer highlights this point in the body of his report for the first time in his December 2013 reply. Leamer Reply Rep. ¶ 75 (noting that his original conduct regression with clustered standard errors "leave[s] the estimated conduct coefficient 'statistically insignificant' at the conventional 5% level[.]").
Dr. Leamer also used the 1%, 5%, or 10% levels in four of his other charts in his opening merits report. Leamer Opening, Exs. 2, 4-6 (reporting t-values and noting whether each variable coefficient was statistically significant at the 1%, 5%, or 10% levels).
In Dr. Leamer's reports prior to his October 2013 report, Dr. Leamer also analyzed the statistical significance of many of the coefficients in varying models using the 1%, 5%, and 10% levels. See, e.g., Class Cert. Opening Rep. Figs. 20 & 23; Class Cert. Reply Rep. Figs. 12, 14, 16-19; Suppl. Class Cert. Rep. ¶¶ 41 (noting "a t-statistic in excess of 2 in absolute value is said to produce 'statistically significant' estimate[s] by conventional [5%] standards."), Fig. 1 (reporting statistical significance at the 1%, 5% and 10% levels).
This Court's April Order further illuminates why Dr. Leamer's new theory is untimely disclosed. The Court held that "the fact that, when the errors are clustered, the Conduct Regression's results are not statistically significant at the 95 percent level does not persuade the Court that the regression is inadmissible (although this failure might affect the model's probative value)." April Order at 42. The Court noted, "To the extent there are other variables that may improve the accuracy of the Conduct Regression . . . , Dr. Leamer is encouraged to include them in his next report." Id. at 43 n.15. Thus, the Court explicitly asked Dr. Leamer to explain in his upcoming reports any further response he had to Dr. Murphy's argument that his results were inaccurate because the conduct variable's coefficient in his original conduct regression with clustered errors was not statistically significant at the 5% level. Dr. Leamer's 50% theory would have been precisely such a response, but Dr. Leamer did not include it in any report until his final December 2013 reply report.
"Confidence intervals . . . are statistical estimates of the range within which there can be reasonable confidence that a correlation or prediction is not the result of chance variability in the sample on which the correlation or prediction was based[.]" ATA Airlines, Inc. v. Fed. Exp. Corp, 665 F.3d 882, 895 (7th Cir. 2011). Every confidence interval is the complement of a respective significance level. A 95% confidence interval reflects a statistical significance level of 5%. Cook, 580 F. Supp. 2d at 1101 ("[A] confidence interval can also be used to infer the p-value and thus can be used as a surrogate test for significance. A 95% confidence interval, for example, that does not include the null hypothesis [] indicates that there is a less than 5% chance that the observed association is the result of random error or chance. . . .This is equivalent to a p-value of less than .05, meaning the study result is 'statistically significant.' [] Conversely, if the null point falls within the 95% confidence interval, then the study result is not deemed 'statistically significant' under a significance level of .05."). Because a 5% significance level is associated with a 95% confidence interval, statisticians sometimes colloquially refer to the 5% level as the 95% level.
Nor did Dr. Leamer explain this theory in his November 2013 deposition. When asked by Plaintiffs' counsel to admit that the general conduct variable's coefficient in Exhibit 3 of his October 2013 report was "not statistically significant," he simply responded, "That's correct." Brown Decl, ECF No. 573, Ex. 3, at 1044 ("Nov. 2013 Leamer Dep.").
In sum, because Plaintiffs waited until after Defendants had filed their last expert report for Dr. Leamer to offer a new theory, Plaintiffs have violated Rule 26(a)(2)(B)'s requirement that an expert witness's opening report contain "a complete statement of all opinions the witness will express and the basis and reasons for them" together with "the facts or data considered by the witness in forming them." Fed. R. Civ. P. 26(a)(2)(B)(i)-(iii). Plaintiffs will not be allowed to "sandbag" Defendants with new analysis that should have been included at the very least in Dr. Leamer's opening merits report. Oracle Am., Inc. v. Google Inc., No. C 10-03561 WHA, 2011 WL 5572835, at *3 (N.D. Cal. Nov. 15, 2011) (granting motion to strike and noting expert disclosure schedule "was designed to forestall 'sandbagging' by a party with the burden of proof who wishes to save its best points for reply, when it will have the last word, a common litigation tactic."). Dr. Stiroh had no chance to rebut Dr. Leamer's theory because expert discovery has closed. "This immunity, combined with the element of surprise, would be unfair." Id. While Defendants have proven a discovery violation, Plaintiffs have not proven that their failure to comply with Rule 26 was either justified or harmless. See Yeti by Molly, 259 F.3d at 1107; Negrete v. Allianz Life Ins. Co. of N. Am., 2013 WL 6535164, at *24 (C.D. Cal. Dec. 9, 2013) ("[P]laintiffs do not explain how their tardy disclosure was either substantially justified or harmless under Fed. R. Civ. P. 37(c)(1)."). Accordingly, the Court strikes ¶¶ 75-90 & Figs. 15-16 of Dr. Leamer's reply as untimely disclosed and improper rebuttal. Defendants' motion to strike ¶¶ 75-90 & Figs. 15-16 is thus GRANTED.
Plaintiffs' argument that it would be prejudicial not to allow Dr. Leamer to point out the flaws in Dr. Murphy's 5% level, see Strike Opp. at 5-6, ignores the point that Dr. Leamer should have included his 50% theory in his opening merits report so that Defendants would have a chance to respond. Further, while Plaintiffs cite Scientific Components Corp. v. Sirenza Microdevices, Inc., No. 03 CV 1851(NGG)(RML), 2008 WL 4911440, at *7 (E.D.N.Y. Nov. 13, 2008), for the proposition that when "the alleged confusion in the report in chief turns on a subtle scientific distinction that neither side's experts have previously discussed, it is not only permissible but also obligatory for the rebuttal expert report to provide technical background information adequate to illustrate the point," that case is inapposite because Dr. Murphy did previously discuss the same criticism that Dr. Stiroh raises in Dr. Murphy's November 2012 report.
Nothing in this Order prevents Dr. Leamer from testifying to one of the opinions set forth in his December 2012 reply, which Dr. Leamer expressly incorporated by reference into his December 2013 merits report and attached as an exhibit to his merits report, Leamer Opening ¶ 1. In his December 2012 reply, when describing an alternative model that utilized firm-wide compensation averages, Dr. Leamer included a one line reference to a 50% significance level (equivalent to a 0.5 p-value), and suggested that the conduct variables' coefficients in this model may be reliable because they meet the 50% level. See Class Cert. Reply Rep. ¶ 107 ("The p-value on all conduct coefficients is less than 0.5 which suggests that it is more likely than not that the compensation of employees was decreased during the period of the agreements."). While Dr. Leamer will not be allowed to testify, as laid out in his December 2013 reply, that his original conduct regression model with clustered errors is reliable because its general conduct coefficient meets the 50% significance level, Dr. Leamer may testify to the exact opinion disclosed in his December 2012 reply, notably that the fact that his alternative conduct regression model's conduct coefficients pass the 50% level "suggests that it is more likely than not that the compensation of employees were decreased during the period of the agreements." Id.
Plaintiffs make much of the fact that Dr. Leamer "does not advocate point null hypothesis testing." Strike Opp. at 4; Leamer Opp. at 8-9 (Dr. Leamer "has never used point null hypothesis testing in this case."). Dr. Leamer's testimony on this point has been inconsistent. In his December 2013 report and October 2012 deposition, he claimed he had not conducted null hypothesis testing. Leamer Reply Rep. ¶¶ 77-78, 82; Omnibus Brown Decl., ECF No. 716, Ex. B, "Oct. 2012 Leamer Dep." at 220, 1236-37, 1243-44 (claiming the t-statistics in his exhibits were simply "standard things that come rolling out of computer packages" and not an indication he did hypothesis testing). Yet in that same deposition, he conceded he did. Oct. 2012 Leamer Dep. at 1239 ("I would admit that [I did hypothesis testing in this case] . . . I'm doing [] the hypothesis testing exercise[.]"); id. at 1237 ("I pursued . . . the hypothesis testing task."). In his November 2013 deposition, he stated that the computation of statistical significance at the 1%, 5% and 10% levels in his models was "done setting the null hypothesis to zero" because "that's the way that it's usually done in econometric literature[.]" Nov. 2013 Leamer Dep. at 1038-40. Yet he simultaneously claimed that it was not his "choice. That's just a standard operating procedure that economists use. When it comes to estimating damages, I'm trying to argue that that is a poor idea." Id. at 1039. Regardless of whether Dr. Leamer advocates the use of null hypothesis testing in this case, the Court finds that Dr. Leamer did in fact conduct such testing in this case because he conceded he did so, and further finds that he conducted such testing by using the 1%, 5% and 10% levels.
b. Motion to Strike Paragraphs 115-20 & Figs. 17-18
The Court now addresses Defendants' request to strike Paragraphs 115-20 & Figs. 17-18 of Dr. Leamer's reply on the ground that these sections contain new arguments in support of Dr. Leamer's "total new hires" variable. Strike Mot. at 8. Defendants cite Dr. Leamer's assertion that this variable is the "most statistically significant variable" and that omitting it would "wreak havoc" on the other coefficients. Id. (citing Leamer Reply Rep. ¶¶ 115, 117). Defendants also claim Dr. Leamer impermissibly provides "new analyses aimed at justifying the variable's inclusion in his model." Id. (citing Leamer Reply Rep. ¶¶ 118-20). The Court disagrees that Dr. Leamer's arguments constitute improper rebuttal. In her report, Dr. Stiroh attacks Dr. Leamer's use of this variable by claiming that the variable's coefficient has the "wrong" sign (i.e., negative) because the coefficient implies that as the firms are doing more hiring, the firms pay their employees less, which runs contrary to basic economic principles, Stiroh Rebuttal ¶¶ 161-65, and by advocating that the variable should be omitted because it wrongly "combines the impact of the hiring by firms with whom each Defendant has a[n anti-solicitation] agreement with the impact of hiring by other Defendants." Id. ¶¶ 183, 186-88. Dr. Leamer was entitled, in direct response to Dr. Stiroh's opinion, to explain in detail greater than his opening report the statistical significance of this variable compared to that of the other variables and why omitting it would be incorrect. Leamer Reply Rep. ¶¶ 115-17. Defendants also mischaracterize the substance of Dr. Leamer's testimony at ¶¶ 118-20 & Figs. 17-18, wherein Dr. Leamer does not provide any new analysis to "justify" the inclusion of the variable. Rather, Dr. Leamer simply responds to Dr. Stiroh's criticism that Dr. Leamer "has not provided an explanation for why the unusual results are reasonable," Stiroh Rebuttal ¶ 165, by explaining that negative coefficient may have resulted either because the variable was "identifying periods of weak labor markets," or because "high levels of hiring may leave the impression that replacements are easy to find, [thus leading to lower] wages of incumbents." Leamer Reply Rep. ¶ 119-20. Such a response is within the realm of proper rebuttal testimony, as it is clearly "intended solely to contradict or rebut evidence on the same subject matter identified by another party." Fed. R. Civ. P. 26(a)(2)(D)(ii); see also Kirola v. City & Cnty. of S.F., No. C-07-3685 SBA (EMC), 2010 WL 373817, at *2 (N.D. Cal. Jan. 29, 2010) ("Rebuttal disclosure is not automatically excluded solely because it includes evidence that was absent in the original expert disclosure."). Accordingly, the Court DENIES Defendants' request to strike Paragraphs 115-20 & Figs. 17-18 of Dr. Leamer's reply.
This variable is one of the independent variables in Dr. Leamer's model which represents the sum of all new hires by all Defendants in a given year. Leamer Opening Fig. 5.
Dr. Murphy did not make this criticism in any of his reports in opposition to class certification.
c. Motion to Strike Paragraphs 108-110
Third, Defendants seek to strike Paragraphs 108-110 on the grounds that Dr. Leamer "introduces for the first time a justification for using real compensation as a metric in Dr. Leamer's model instead of nominal compensation." Strike Mot. at 9. The Court denies Defendants' request. Dr. Leamer has utilized real compensation in all his regressions since his October 2012 report. See, e.g., Class Cert. Opening Rep. Figs. 20 & 23 (dependent variable is total annual compensation of each employee divided by the CPI to adjust for inflation); Leamer Opening ¶¶ 20, 41, Exs. 2-6. Dr. Murphy did not challenge Dr. Leamer's use of real compensation in any of Dr. Murphy's reports in opposition to class certification. In her rebuttal report, Dr. Stiroh opines that Dr. Leamer's conduct regression is unreliable because he utilizes real compensation by adjusting the model for inflation; Dr. Stiroh claims that "running the model on nominal figures would be expected to produce a more accurate result." Stiroh Rebuttal ¶ 174. Dr. Stiroh opines that when nominal compensation is utilized, the resulting damages estimate is $1.8 billion as opposed to the $3.06 billion Dr. Leamer's model estimates. Id. ¶¶ 175-76; Ex. V1.1. Dr. Stiroh thus claims that Dr. Leamer's damages appear to be caused by "changes in inflation[.]" Id. ¶ 176. In response, Dr. Leamer claims Dr. Stiroh's critique is invalid because using nominal compensation assumes that the labor market determines nominal, not real wages, which is contrary to mainstream economic thinking. Leamer Reply Rep. ¶¶ 108-110. Such an opinion is appropriate rebuttal because Dr. Leamer is entitled to respond to Dr. Stiroh's criticism. While Dr. Leamer did not explicitly state in his opening report the rationale for using real as opposed to nominal compensation, that does not mean his rebuttal report violates Rule 26. This is because to exclude Dr. Leamer's response to Dr. Stiroh's challenge here would create a rule whereby experts would feel the need to include "vast amounts of arguably irrelevant material" in their opening reports "on the off chance that failing to include any information in anticipation of a particular criticism would forever bar the expert from later introducing the relevant material." Crowley v. Chait, 322 F. Supp. 2d 530, 551 (D.N.J. March 16, 2004). Accordingly, the Court DENIES Defendants' request to strike Paragraphs 108-110 of Dr. Leamer's reply.
Defendants argue that if the Court does not strike the testimony Defendants cite, Defendants should be granted leave for Dr. Stiroh to submit a reply report. Strike Mot. at 10. The Court denies this request because Dr. Leamer's opinions regarding the total new hires variable and nominal compensation do not comprise improper testimony, and thus there is no need for a surreply.
4. Defendants' Motion to Exclude Dr. Leamer's Testimony Under Daubert
Defendants move to exclude Dr. Leamer's conduct regression under Daubert on four grounds: (1) the general conduct variable lacks statistical significance and the Court should reject Dr. Leamer's attempt to justify his model based on a 50% significance level; (2) the regression fails to distinguish between any alleged impact from the anti-solicitation agreements and conduct not at issue; (3) the "total new hires" variable is inconsistent with Plaintiffs' theory of harm; and (4) the regression is incapable of showing each class member was injured. Leamer Mot. at 1-2. The Court disagrees with Defendants with respect to all four challenges.
While Defendants claim in their conclusion section that "Dr. Leamer's proposed testimony regarding alleged impact and damages is unreliable and should be excluded in its entirety," Leamer Mot. at 15 (emphasis added), the substance of Defendants' motion challenges only Dr. Leamer's opinions relating to his conduct regression model.
As a preliminary matter, the Court notes that it held in its October Order that Dr. Leamer's conduct regression was "statistically robust," supported by the economic literature, and "capable of calculating classwide damages." October Order at 82; see also April Order at 35. Further, numerous courts have held that regression analysis is generally a reliable method for determining damages in antitrust cases and is "a mainstream tool in economic study." In re Industrial Silicon Antitrust Litig., No. 95-2104, 1998 WL 1031507, at *2 (W.D. Pa. Oct. 13, 1998); Petruzzi's IGA Supermarkets. Inc. v. Darling-Delaware Co., Inc., 998 F.2d 1224, 1237-41 (3d Cir. 1993) (admitting regression analysis for use in calculating antitrust damages); In re Flat Glass Antitrust Litigation, 191 F.R.D. 472, 486 (W.D. Penn. Nov. 5, 1999) ("[R]egression analysis is one of the mainstream tools in economic study and it is an accepted method of determining damages in antitrust litigation.") (citation omitted). With this context in mind, the Court addresses each of Defendants' arguments in turn below.
a. Defendants' First Daubert Challenge
Defendants' first challenge is that Dr. Leamer's conduct regression is unreliable because two of its variables lack statistical significance at the 1%, 5%, and 10% levels when null hypothesis testing is used. Leamer Mot. at 6-8. Defendants cite how Dr. Leamer concedes that his general conduct variable in his original conduct regression with clustered standard errors has a "large standard error" and thus its coefficient is not statistically significant at the 1%, 5%, and 10% levels. Id. at 6 (citing Nov. 2013 Leamer Dep. at 1036, 1044 & Brown Decl, ECF No. 573, Ex. 4, Dec. 2013 Leamer Deposition at 1258); see also Leamer Opening, Ex. 3 (demonstrating that the general conduct variable is not statistically significant at the 1%, 5%, and 10% levels). Defendants point out the same flaw in one of Dr. Leamer's other variables, which represents the interaction between the general conduct variable and the hiring rate per Defendant firm. Leamer Mot. at 6. The Court rejects Defendants' argument.
Defendants' other request, see Leamer Mot. at 8-10, to exclude Dr. Leamer's attempt to justify his original conduct regression with clustered errors based on his 50% significance theory, as set forth in his October 2013 opening report, is moot because the Court has precluded Dr. Leamer from testifying about that opinion on other grounds. See supra Part III.A.3.a.
In null hypothesis testing, "standard errors" determine the statistical significance of a variable's coefficient—i.e., determine whether the model provides statistically reliable evidence that the true value of the estimate (the independent variable's coefficient) is different from zero. ECF No. 574, "Stiroh Decl." ¶ 3. In addition, the fact that a coefficient is not statistically significant at a certain significance level means the null hypothesis (that the independent variable has no actual effect on the dependent variable) cannot be rejected at that significance level. Here, Dr. Leamer's own exhibit, which reports the results of his original conduct regression model with clustered standard errors, reports that two of his variables, including the general conduct variable, are not statistically significant at the 1%, 5%, and 10% levels. Leamer Opening, Ex. 3. Defendants argue this means Dr. Leamer's regression has been unable to estimate those variables' coefficients "with sufficiently reasonable precision to conclude their true value — or the impact of the challenged agreements — is different from zero." Leamer Mot. at 7.
The Court finds that the fact that these two variables are not statistically significant at the 1%, 5%, and 10% levels goes to the weight, not the admissibility of Dr. Leamer's model. As an initial matter, the Court acknowledges that there is certainly ample evidence that these three levels are the "conventional" levels statisticians typically use. ATA Airlines, 665 F.3d at 895 (noting that a 95% confidence interval - which reflects a statistical significance level of 5% - is "the standard criterion of reasonable confidence used by statisticians"); Contreras v. City of L.A., 656 F.2d 1267, 1273 n.3 (9th Cir. 1981) ("[A] .05 level of statistical significance . . . is generally recognized as the point at which statisticians draw conclusions[.]") (citation omitted); Madani v. Equilon Enterprises LLC, CV 04-10370 JVS JTLX, 2009 WL 2148664 (C.D. Cal. July 13, 2009) ("The 'generally accepted' rates in the economic community [are] 5-10 %[.]") (citation omitted); Ref. Manual at 251-52 (statistical analysts typically use the 5% and 1% levels); Omnibus Brown Decl., ECF No. 716, Ex. H, Jeremy Foster et al., Understanding and Using Advanced Statistics 1, 6 (2006) (noting that these are the three conventional levels used); id. Ex. K, MARNO VERBEEK, A GUIDE TO MODERN ECONOMETRICS 31 (2d ed. 2004) (same); id., Ex. N, R. Carter Hill, William E. Griffiths & Guay C. Lim, PRINCIPLES OF ECONOMETRICS 710 (4th ed. 2011) (same); Rubinfeld at 431 (same).
This notwithstanding, the fact that Dr. Leamer's model fails to meet these three levels does not convince the Court that his model is so methodologically flawed as to warrant exclusion. For one thing, Plaintiffs cite evidence that null hypothesis testing is not a requirement of statistical analysis, because it is not the only test of reliability statisticians use. Plaintiffs also present evidence that some scholars believe that the conventional levels should not be blindly applied in every case but that a level should be selected after a careful consideration of the particular study at hand. See Harvey Decl., ECF No. 607, Ex. 20, William H. Kruskal, Tests of Significance, in 2 INT'L ENCYCLOPEDIA OF STATISTICS 955 (William H. Kruskal & Judith M. Tanur ed., 1978) ("Significance testing is an important part of statistical theory and practice, but it is only one part, and there are other important ones."); id., Ex. 19, PETER KENNEDY, A GUIDE TO ECONOMETRICS 61 (2003) (noting that the opinion that "hypothesis testing is overstated, overused, and practically useless as a means of illuminating what the data in some experiment are trying to tell us" is "shared by many") (citation omitted); id., Ex. 17, R.A. FISHER, STATISTICAL METHODS AND SCIENTIFIC INFERENCE 45 (3d. ed. 1973) ("[I]t would clearly be illegitimate for one to choose the actual level of significance . . . as though it were his lifelong habit to use just this level."). There is also case law in support of these scholarly positions. See, e.g., Cook, 580 F. Supp.2d at 1091 ("[S]cientific endeavor takes many forms, many of which do not involve hypothesis testing."); Kadas v. MCI Systemhouse Corp., 255 F.3d 359, 362 (7th Cir. 2001) ("The 5 percent test is arbitrary; it is influenced by the fact that scholarly publishers have limited space and don't want to clog up their journals and books with statistical findings that have a substantial probability of being a product of chance rather than of some interesting underlying relation between the variables of concern.").
Second, Defendants have not cited, nor has this Court found, any case holding that a regression model must reject a null hypothesis of zero effect at least at the 10% significance level in order to be admissible. In fact, there is authority holding otherwise. See, e.g., Cook, 580 F. Supp. 2d at 1102, 1105 (rejecting argument that "statistical significance is a threshold requirement for establishing the admissibility of expert testimony involving the use of statistics" and holding that neither "the Tenth Circuit ([nor] any other court) has adopted a rule barring admission of any epidemiological study that was not statistically significant at the 95-percent confidence level."); Kadas, 255 F.3d at 362 (rejecting the idea that a study is inadmissible as a matter of law just because it is less statistically significant than the 5 % level). Even Defendants' own expert, Dr. Murphy, conceded that a model's results need not necessarily be statistically significant to be reliable. ECF No. 297-14, Murphy Dec. 2012 Deposition at 366 ("Question: Is it your opinion that in order for a statistical analysis to be reliable it must produce a statistically significant result? Answer: Not necessarily. That's doesn't have to be true . . . But statistical significance is one thing you do look at."). Dr. Stiroh also conceded that she could not identify any econometrics textbook which states that a coefficient has to be statistically significant at the 5% level to be reliable evidence. Cisneros Decl., ECF No. 605, Ex. JJJ, Stiroh Dec. 2013 Deposition at 183.
In re Silicone Gel Breast Implants Prods. Liab. Litig., 318 F. Supp. 2d 879 (C.D. Cal. Apr. 22, 2004), cited by Defendants, is inapposite as it does not stand for the proposition that a study is inadmissible unless it produces statistically significant results at the conventional levels. Henricksen v. ConocoPhillips Co., 605 F. Supp. 2d 1142 (E.D. Wash. Feb. 11, 2009), is similarly inapposite because it did not involve any regression model but concerned a qualitative study measuring benzene exposure of a very small sample size of twenty-one study participants.
Reliance on statistical significance to determine the admissibility of expert evidence has been rejected by some courts in non-antitrust contexts. See, e.g., Kadas, 255 F.3d at 362 (age discrimination suit). As the court observed in criticizing such reliance, "[l]itigation generally is not fussy about evidence; much eyewitness and other nonquantitative evidence is subject to significant possibility of error, yet no effort is made to exclude it if doesn't satisfy some counterpart to the 5 percent significance test." Id.; see also Rendon v. AT & T Technologies, 883 F.2d 388, 397-98 (5th Cir. 1989) (rejecting argument that there is a strict legal benchmark requiring a particular number of standard deviations to demonstrate that data has statistical significance); Heller v. Shaw Industries, Inc., 167 F.3d 146, 158 (3d Cir. 1999); Waisome v. Port Authority of New York & New Jersey, 948 F.2d 1370, 1376 (2d Cir. 1991); MacDissi v. Valmont Industries, Inc., 856 F.2d 1054, 1058 n.3 (8th Cir. 1988).
Finally, the Ninth Circuit has held that lower courts are "not to confuse the role of judge and jury by forgetting that 'vigorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof,' rather than exclusion, 'are the traditional and appropriate means of attacking shaky but admissible evidence.'" United States v. Chischilly, 30 F.3d 1144, 1154 (9th Cir. 1994) (quoting Daubert, 509 U.S. at 596). Heeding this admonition, this Court previously denied Defendants' Daubert challenge to Dr. Leamer's conduct regression at the class certification stage by holding that "the fact that, when the errors are clustered, the Conduct Regression's results are not statistically significant at the 95 percent level does not persuade the Court that the regression is inadmissible (although this failure might affect the model's probative value)." April Order at 42 (rejecting Defendants' Motion to Strike Dr. Leamer's Testimony, ECF No. 210, at 16). The Court reasoned it was sufficient that Dr. Leamer's model could be "attacked by cross examination, contrary evidence, and attention to the burden of proof." Id. at 50 (citing Primiano, 598 F.3d at 564). Because Defendants have provided no compelling reason why the Court should deviate from that conclusion, Defendants' first Daubert challenge is DENIED.
b. Defendants' Second Daubert Challenge
Defendants' second Daubert challenge is that Dr. Leamer's regression is incapable of segregating the impact on compensation attributable to the challenged agreements from the effects on compensation attributable to Defendants' other agreements and unilateral conduct. Leamer Mot. at 10-12. Defendants cite to agreements Intel had with Pixar and Apple, and unilateral policies Google adopted with respect to two non-defendant companies. Leamer Mot. at 11. Defendants claim Dr. Leamer conceded that his general conduct variable, which is a dummy variable that is turned on when the challenged agreements were in effect, "will pick up [the compensation suppression effects stemming from] anything that is applicable to [the class period from 2005 to 2009] when the [variable] is turned on." Brown Decl., ECF No. 573, Ex. 1, Oct. 2012 Leamer Dep. at 329; see also id. at 340 ("To the extent that these [other cold-calling restrictions] are coincident in time with . . . these [challenged] bilateral agreements they had, and to the extent that they suppress wages during that period of time, it's going to be picked up by the conduct variable[.]"). The Court rejects Defendants' argument.
Defendants cite an internal Google document which notes the existence of do-not-cold-call policies effective January 20, 2006 with respect to OpenTV Corporation and Invidi Technologies Corporation. ECF No. 573, Brown Decl., Ex. 13. Plaintiffs do not appear to dispute the existence of these policies.
This dummy variable technique is not uncommon in regression analysis. Ref. Manual at 313 ("In an antitrust case, it may be a variable that takes on the value 1 to reflect the presence of the alleged anticompetitive behavior and the value 0 otherwise").
The Ninth Circuit has held that an antitrust plaintiff is required to distinguish between losses attributable to lawful competition and those attributable to unlawful anticompetitive conduct. City of Vernon v. Southern California Edison Co., 955 F.2d 1361, 1371-72 (9th Cir. 1992). This is because the antitrust laws are intended to compensate plaintiffs only for losses caused by a defendant's unlawful behavior. Litton Sys., Inc. v. Honeywell, Inc., CV 90-4823 MRP (EX), 1996 WL 634213, at *2 (C.D. Cal. July 24, 1996). The Supreme Court recently affirmed the rationale underlying this principle in a case concerning whether a class could be certified under Rule 23, noting that "a model purporting to serve as evidence of damages . . . must measure only those damages attributable to that theory." Comcast Corp. v. Behrend, 133 S. Ct. 1426, 1433 (2013).
As a preliminary matter, the Court notes that although it need not resolve this question, Defendants' challenge at least appears to present a purely hypothetical problem. Dr. Leamer explained his model in fact controls for any compensation suppression effects stemming from unchallenged conduct, unless all the unchallenged agreements or policies were the same exact duration as the unlawful agreements—i.e., started on the first day of the class period in 2005 and ended on the last day of the class period in 2009. ECF No. 573, Ex. 1, Oct. 2012 Leamer Dep. at 340; see also Oct. 2012 Leamer Dep. at 1025-27, 1029 ("[I]f there were comparable [unchallenged] agreements struck in place prior to the conspiracy period and after the conspiracy, then [the unchallenged conduct's effects are controlled for] in the statistical analysis.") (emphasis added)). Because Defendants have not presented evidence that the unchallenged conduct satisfies this criteria, the problem to which Defendants allude appears to be hypothetical and Dr. Leamer's model should not be excluded due to any alleged failure to segregate out any suppression of compensation attributable to Defendants' unchallenged conduct.
In fact, the evidence suggests that the agreements to which Defendants cite do not fit this criteria. Defendants have submitted evidence suggesting the Intel/Apple agreement began in 2007, which is one year into the class period. Brown Decl., ECF No. 573, Ex. 11 at 82-83, 110. The Intel/Pixar agreement apparently started in 2008. Id., Ex. 12 at 158-62. Defendants have presented no evidence concerning when these agreements ended. Defendants concede that Google's unilateral policies were effective January 20, 2006, which is also well into the class period. Leamer Mot. at 11. Defendants cite evidence they claim shows that Google removed its Do Not Call List from Google's internal staffing website and staffing library in 2009. Id. Ex. 14 (Google internal email on September 29, 2009 suggesting Google suspended any do-not-cold-call policies by removing them from internal staffing website and staffing library).
Yet even assuming Dr. Leamer's damages estimate includes some effects from unchallenged conduct—i.e., that Dr. Leamer should have included a special control to account for effects of unchallenged conduct without simply assuming, as he did, that it was not the case that all the unchallenged agreements or policies were the same exact duration as the unlawful agreements, see Oct. 2012 Leamer Dep. at 1028-29—the Court finds that Dr. Leamer's model need not be excluded under Daubert for failure to satisfy the disaggregation requirement, as explained below.
First and foremost, it is not clear that the Supreme Court's holding in Comcast, a case arising in the Rule 23 class certification context, is applicable in the present Daubert context, when the Court is tasked with evaluating whether expert testimony is reliable and relevant to the jury's consideration at trial of the facts as applied to substantive antitrust law. In Comcast, the plaintiffs, more than two million Comcast subscribers, had alleged four different types of antitrust injury that they claimed collectively resulted in subscribers overpaying for cable TV service, Comcast, 133 S. Ct. at 1430-31, but the district court only found one theory amenable to common proof at the class certification stage. Id. at 1431. Despite this determination, the district court accepted the plaintiffs' damages model even though it holistically calculated damages stemming from all four impact theories. Id. Because the model "failed to measure damages resulting from the particular antitrust injury on which petitioners' liability in this action is premised," the Supreme Court held that the plaintiffs had failed to prove a method of quantifying damages on a classwide basis and class certification was thus improper. Id. at 1433-35. In the midst of so holding, the Court noted that the regression model "did not isolate damages resulting from any one theory of antitrust impact" and thus failed the requirement that "a model purporting to serve as evidence of damages in this class action must measure only those damages attributable to that theory. If the model does not even attempt to do that, it cannot possibly establish that damages are susceptible of measurement across the entire class for purposes of Rule 23(b)(3)." Id. at 1431, 1433 ("There is no question that the model failed to measure damages resulting from the particular antitrust injury on which petitioners' liability in this action is premised."). Comcast is thus a case which discussed the requirements for showing Rule 23 predominance. The precise question the Court addressed was whether "certification was improper because [plaintiffs] had failed to establish that damages could be measured on a classwide basis." Id. at 1431 n.4. Notably, the Court did not address standards of admissibility of expert testimony under Daubert. Accordingly, it is not at all clear that Comcast's holding concerning Rule 23 predominance - that "a model purporting to serve as evidence of damages in [a] class action must measure only those damages attributable to [plaintiffs'] theory," id. at 1433, in order to serve as a basis for showing that damages can be proven on a classwide basis - is applicable to the Daubert stage when courts are evaluating whether an expert's model is admissible under Rule 702. In fact, the Court noted that it was not addressing the question of whether the damages model at issue was admissible evidence under Rule 702. Id. at 1431 n.4. Further, the Court expressly noted that its ruling "turn[ed] on the straightforward application of class-certification principles," and noted, while addressing the dissent, that this case provided "no occasion for" discussion of "substantive antitrust law." Id. at 1433. In contrast, issues raised at the Daubert stage no doubt implicate substantive antitrust law, as the entire issue is whether an expert's testimony will be "relevant" to the jury's consideration at trial of the facts as applied to substantive antitrust law.
Plaintiffs' expert admitted that the model calculated damages resulting from the alleged conduct "as a whole" and did not attribute damages to any one particular theory of impact. Comcast, 133 S. Ct. at 1434. The model assumed the validity of all four theories of antitrust impact initially advanced by Plaintiffs: decreased penetration by satellite providers, overbuilder deterrence, lack of benchmark competition, and increased bargaining power. Id.
Yet even assuming Comcast and the Ninth Circuit cases citing the disaggregation principle do apply in the Daubert context, the Court finds Dr. Leamer's model is not inadmissible because Defendants read Comcast's disaggregation holding too broadly. This is because the rationale underlying Defendants' argument—that Comcast holds that a damages model must precisely segregate out effects of every possible factor, including legal conduct, that could impact the dependent variable, in order to be admissible under Daubert—directly contravenes well-established Supreme Court and Ninth Circuit authority holding that damages in antitrust cases often cannot, and therefore need not, be proven with exact certainty. Zenith Radio Corp. v. Hazeltine Research, Inc., 395 U.S. 100, 123 (1969) ("[D]amages issues in [antitrust] cases are rarely susceptible of the kind of concrete, detailed proof of injury which is available in other contexts."); J. Truett Payne Co. v. Chrysler Motors Corp., 451 U.S. 557, 566 (1981) (expressing "willingness to accept a degree of uncertainty" in antitrust damage proof given that "[t]he vagaries of the marketplace usually deny us sure knowledge of what plaintiff's situation would have been in the absence of the defendant's antitrust violation"); Knutson v. Daily Review, Inc., 548 F.2d 795, 811 (9th Cir. 1976) (proof of damages is sufficient "if the evidence show[s] the extent of the damages as a matter of just and reasonable inference, although the result be only approximate") (citation omitted); Moore v. James H. Matthews & Co., 682 F.2d 830, 836 (9th Cir. 1982) ("[A]n antitrust plaintiff is only obligated to provide the trier-of-fact with some basis from which to estimate reasonably, and without undue speculation, the damages flowing from the antitrust violations.") (citation omitted); see also In re Scrap Metal Antitrust Litig., 527 F.3d 517, 533 (6th Cir. 2008) ("[T]he antitrust cases are legion which reiterate the proposition that, if the fact of damages is proven, the actual computation of damages may suffer from minor imperfections.") (citation omitted). Defendants' argument is even belied by the Comcast Court's own acknowledgment that "[damages] [c]alculations need not be exact" in antitrust cases and that Comcast did not change substantive antitrust law. Comcast, 133 S. Ct. at 1433. Accordingly, the most plausible reading of Comcast is that by mandating that a damages model in a class action case "measure only those damages attributable to [plaintiffs'] theory," the Supreme Court did not alter this fundamental principle of antitrust law by requiring that an expert's model precisely tailor, in a fool-proof way, the connection between the damages claimed and the anticompetitive conduct alleged in order to be admissible under Daubert. Rather, the Court was concerned with damages models that attempt to calculate damages stemming from various theories of antitrust impact which were not at issue—i.e., damages models that "do[] not even attempt to [measure damages attributable to plaintiffs' theory]." Id.
The Ninth Circuit cases that recite the disaggregation principle do not address the admissibility of the expert's analysis under Daubert or otherwise, but rather consider the sufficiency of such evidence to prove other matters. For example, in City of Vernon, the plaintiff alleged the defendant engaged in anticompetitive practices by disallowing plaintiff's use of defendant's power transmission lines, maintaining rate schedules that were discriminatory, and preventing plaintiff from acquiring power from alternate suppliers. 955 F.2d at 1363. The plaintiff's damage study assumed that "all of [these] acts contributed to the damage figure." Id. at 1373. The district court found that some of the acts were lawful and that the damage estimate thus "failed to segregate the losses, if any, caused by acts which were not antitrust violations from those that were." Id. at 1372. Accordingly, the court granted defendant summary judgment because plaintiffs had presented no evidence of damages. Id. at 1372. On appeal, the Ninth Circuit affirmed because the plaintiff's aggregated damage proof (which encompassed claims which were dismissed) was unduly speculative and could not support a damage recovery. Id. at 1373. The Court notes that at the hearing on Defendants' motion to exclude Dr. Leamer's testimony, Defendants could not cite to any case in the Ninth Circuit which applies or addresses the disaggregation principle in the Daubert context.
Here, Dr. Leamer's model does not suffer from the critical flaw in Comcast. It is undisputed that Dr. Leamer's model evaluates damages resulting from only one theory of antitrust injury—a decrease in compensation due to the challenged anti-solicitation agreements. Nor is Dr. Leamer's model one that "does not even attempt to" measure damages stemming only from the challenged agreements. Id. His model expressly controls for many other variables that impact compensation in an effort to ensure that the estimated damages result only from the challenged conduct. See Leamer Opening ¶ 19; Leamer Reply Rep. ¶¶ 91-93. This includes factors like employees' age and gender, worker tenure, and location differences, industry effects including San Jose information sector hiring, and employer effects including firm revenue and firm hiring. Id. Further, Dr. Leamer has stated that his model controls for, and thus segregates out, the effect of unchallenged conduct, so long as certain assumptions hold. While Plaintiffs have presented no evidence that these assumptions did in fact hold, the fact that Dr. Leamer's model presents some uncertainty as to whether some compensation-related effects of unchallenged conduct are included in the damages estimate does not provide a basis for exclusion. It is sufficient that Dr. Leamer's model is substantially more narrowly tailored — with respect to the connection between damages and challenged conduct — than the damages model at issue in Comcast. Because this Court concludes that Dr. Leamer's model can provide the "trier-of-fact with some basis from which to estimate reasonably, and without undue speculation, the damages flowing from the antitrust violations," Moore, 682 F.2d at 836, Dr. Leamer's model will not be excluded for failure to segregate out any effects of unchallenged conduct. Accordingly, Defendants' second Daubert challenge is DENIED.
This is precisely what distinguishes Defendants' other cited cases, where damages models failed to take into account critical variables that could have impacted the independent variable at issue. See Concord Boat Corp. v. Brunswick Corp., 207 F.3d 1039, 1056 (8th Cir. 2000) (antitrust damages expert conceded his model, which "ignored inconvenient evidence," completely failed to account for various critical market events that could impact the independent variable); Blue Cross and Blue Shield United of Wisconsin v. Marshfield Clinic, 152 F.3d 588, 593 (7th Cir. 1998) (excluding statistical study which failed to correct for any other factor that could have affected the independent variable, price in clinical services, except for just one, thus effectively attributing the "entire difference [in price] . . . to the [anticompetitive conduct.]").
Defendants also assert in a footnote that "Dr. Leamer's model [] cannot isolate the impact of the [anti-solicitation] agreements on compensation from other significant [macroeconomic and microeconomic events] during the class period," such as "the 2008-2009 recession, which would have negatively impacted compensation" or "the effect of Defendants' different responses to the recession in setting compensation." Leamer Mot. at 11, 12 n.5 (recapping Stiroh Rebuttal ¶¶ 198-203). Defendants are incorrect. Dr. Leamer directly responded to Dr. Stiroh's criticism by noting he did control for these two particular factors by including "highly pertinent market and firm-specific recession-sensitive variables" like firm revenue, firm hiring, total number of new hires, firm profit, and San Jose Information sector hiring. Leamer Reply Rep. ¶¶ 91-93. Indeed, the fact that Dr. Leamer's model includes various macro-economic variables to control for these factors distinguishes this case from In re REMEC Inc. Securities Litig., 702 F. Supp. 2d 1202, 1273-75 (S.D. Cal. Apr. 21, 2010) (excluding regression model under Daubert), which Defendants cite, Leamer Mot. at 11 n.4. In that case, the expert had made "no attempt to account for other possible causes" and failed to "incorporate major [macroeconomic] independent variables." In re Remec, 702 F.Supp.2d at 1273 (citation omitted).
c. Defendants' Third Daubert Challenge
The Court now addresses Defendants' third Daubert challenge that Dr. Leamer's "total new hires" variable is inconsistent with Plaintiffs' theory of harm. Leamer Mot. at 12-14. Defendants rely on Comcast's requirement that "any model supporting a plaintiff's damages case must be consistent with its liability case[.]" Comcast, 133 S. Ct. at 1433 (citation omitted). Leamer Mot. at 2. The Court disagrees with Defendants. Defendants make three arguments in connection with their third Daubert challenge, and the Court addresses each in turn.
In Comcast, the Supreme Court held that the plaintiffs' inability to match their damages model with any one theory of liability meant the plaintiffs' damages case was not "consistent with its liability case[.]" Comcast, 133 S. Ct. at 1433.
i. Defendants' Claim that Dr. Leamer's Total New Hires Variable is Inconsistent with Plaintiffs' Theory of Harm
Defendants argue Dr. Leamer's conduct regression is inconsistent with Plaintiffs' theory of harm because it fails to account for the fact that under Plaintiffs' theory, the impact of a Defendant's "increased recruiting and hiring [] on another Defendant would depend on whether there was a[n anti-solicitation] agreement between those two firms." Leamer Mot. at 13. Defendants claim this is because Dr. Leamer uses a "total new hires" variable that is the sum of all new hires by all Defendants in a given year. Defendants claim that because Dr. Leamer applies this same variable to every employee in the class regardless of employer, Dr. Leamer "assumes the impact of increasing hiring by all Defendants is the same on an employee at Google (which had three [anti-solicitation] agreements) as it was on an employee at Adobe (which had only one [anti-solicitation agreement])," which Defendants claim is an assumption "fundamentally at odds" with Plaintiffs' theory of the impact of the anti-solicitation agreements. Id. They claim that to correct for this error, Dr. Stiroh "split" the total new hires variable into component parts so that the model would reflect new hiring by firms that had anti-solicitation agreements with each other "separately from new hiring by firms that did not have such agreements with each other." Id. (citing Stiroh Decl. ¶¶ 8-10); Leamer Reply at 8. In essence, Defendants' claim boils down to an argument that Dr. Leamer's total new hires variable is "improperly aggregated" because it combines the impact of the hiring by firms with whom each Defendant has an anti-solicitation agreement with the impact of hiring by other Defendants. Id.
Defendants do not argue this case poses the same problem as in Comcast where the plaintiffs' damages model calculated damages resulting from various theories of impact, thus creating a situation in which the plaintiffs' damages case was inconsistent with its liability case because the model could not attribute damages to only the one theory of impact left in the case. Nor could they, as there is no dispute that Dr. Leamer's model evaluates only one theory of antitrust injury - a decrease in compensation due to the anti-solicitation agreements.
This variable was included by Dr. Leamer as a "macro-factor to control for the overall demand for labor by all defendants." Leamer Reply Rep. ¶ 131.
Specifically, Dr. Stiroh removed the "total new hires" variable from the model, and inserted three new variables: (1) total number of hires of [do-not-cold-call] firms, i.e., the number of hires of the firms with which a particular Defendant had agreements; (2) total number of hires of non-[do-not-cold-call] firms, i.e., the number of hires of the firms with which a particular defendant had no agreements (for the ADOBE variable, it is computed as the total number of hires by all non-APPLE defendants); and (3) the conduct variable interacted with the total number of new hires of [do-not-cold-call] firms. Stiroh Rebuttal ¶ 187-88.
The Court is not convinced that Dr. Leamer's decision to use an aggregated total new hires variable means his model "is at odds with Plaintiffs' theory of harm," Leamer Reply at 8. Defendants' argument is based on the assumption that under Plaintiffs' theory, "the impact of a Defendant's increased recruiting and hiring [] on another Defendant would depend on whether there was a[n anti-solicitation] agreement between those two firms," and accordingly, the "impact of an increase in recruiting and hiring activity at Intel, [for example,] would be different with respect to an employee at Google (which had a[n anti-solicitation agreement] with Intel) than it would be for an employee at Adobe (which did not have a[n anti-solicitation agreement] with Intel)." Leamer Mot. at 13. Yet Defendants fail to persuasively explain why or how Plaintiffs' theory must lead to this conclusion. The contours of Defendants' argument are not entirely clear. However, Defendants' argument appears to be that, under Plaintiffs' theory, the impact on compensation—i.e. on employees' wages—of one Defendant's increase in hiring should be smaller on a second Defendant who had an anti-solicitation agreement with the first Defendant as compared to the impact on a third Defendant who did not have an agreement with the first Defendant. This is so, Defendants appear to contend, because Defendant pairs who did not have agreements faced no limitations on information flow between them, while Defendant pairs who were parties to an agreement did face such limits. Leamer Mot. at 12. Presumably, Defendants' claim is that because there is more information flow about new job options between two firms without an anti-solicitation agreement, an increase in hiring at one Defendant firm would lead to a greater increase in wages at the second Defendant firm because (a) the employees at the second firm would be more aware of the potential salaries and benefits that come with the job openings at the first firm, and (b) this awareness will force the second firm to raise its employees' wages in order to ensure that its employees stay.
Defendants do not explicitly state whether under Plaintiffs' theory, the impact of a Defendant's increase in hiring on another Defendant should be greater or smaller when the Defendant pair has an anti-solicitation agreement, compared to a Defendant pair that does not.
The Court need not resolve the question whether Plaintiffs' theory necessarily leads to this conclusion, as Defendants claim it does, because the critical point is that Defendants have failed to explain, both in their briefing and at the hearing, why and how Dr. Leamer's inclusion of an aggregated total new hires variable in his model means his model is "inconsistent" with this allegedly logical implication of Plaintiffs' theory. Further, Defendants' argument is particularly unpersuasive given that Dr. Leamer's model not only includes a total new hires variable to control for the overall demand for labor by all defendants, but also "include[s] a different variable of hiring by each firm," Leamer Reply Rep. ¶ 131; Leamer Opening, Ex. 3, variable #27. The Court notes that Defendants did not respond to the Court's question at the hearing regarding why the existence of individual hiring variables for each firm in Dr. Leamer's model, see Leamer Reply Rep. ¶ 131, does not address Defendants' concern that Dr. Leamer's use of an aggregated total new hires variable means his model is somehow inconsistent with Plaintiffs' theory of harm.
Ultimately, while framed as an argument that Dr. Leamer's model violates Comcast, Defendants' argument is, in essence, that Dr. Leamer's model fails to include variables that take into account the "distinction between hiring among Defendants with a[n anti-solicitation] agreement and other hiring." Leamer Reply at 8. That is a prototypical concern that goes to weight, not admissibility. Bazemore v. Friday, 478 U.S. 385, 400 (1986) ("Normally, failure to include variables will affect the analysis' probativeness, not its admissibility."). Accordingly, the Court rejects Defendants' argument that without Dr. Stiroh's suggested changes to Dr. Leamer's model, Dr. Leamer's model is inconsistent with Plaintiffs' theory of harm and must be excluded.
Further, while the Court need not resolve whether Dr. Stiroh's solution to this alleged problem is statistically sound, Dr. Leamer explains that her approach is not necessarily sound but is one way to effectively change around the signs of the regression coefficients. By removing the "total new hires" variable which has a large t-statistic compared to many of the other coefficients, see Leamer Opening, Ex. 3 (absolute value of 4.84), Dr. Stiroh's approach "wreak[s] havoc on the [other] coefficients," thus disrupting the final damages estimate. Leamer Reply Rep. ¶ 115. Dr. Leamer's conclusion is not without support, see Edward Leamer, "A Result on the Sign of Restricted Least Squares Estimates," Journal of Econometrics, 3 (1975) at 387-90. The fact that Dr. Leamer's opinion finds support in his own scholarship weighs at least slightly in favor of finding his model does not fail the reliability prong. Daubert v. Merrell Dow. Pharm., Inc., 43 F.3d 1311, 1317 (9th Cir. 1995) ("One very significant fact to be considered is whether the experts are proposing to testify about matters growing naturally and directly out of research they have conducted independent of the litigation, or whether they have developed their opinions expressly for purposes of testifying.").
The Court notes that Defendants mischaracterize one of Plaintiffs' arguments, claiming Plaintiffs "try to justify Dr. Leamer's total new hires variable as a 'macro-factor' that controls for overall labor demand." Leamer Reply at 8. That is incorrect. Plaintiffs note that the variable controls "for the overall demand for labor by all Defendants." Leamer Opp. at 14 (emphasis in original) (citation omitted). Dr. Leamer has consistently testified the same. Leamer Reply Rep. ¶ 131.
ii. Defendants' Claim that Dr. Leamer's Total New Hires Variable has the "Wrong" Coefficient Sign
In the section concerning their third Daubert challenge, Defendants also take issue with how the total new hires variable has a negative coefficient, which they claim is "contrary to basic economic principles" because it indicates a negative relationship between Defendants' total hiring and employee compensation—i.e., that as Defendants hire more employees, they pay their employees less. Leamer Mot. at 13-14 (summarizing Stiroh Rebuttal ¶ 163). The Court is not persuaded that the variable's negative coefficient deems Dr. Leamer's model so unreliable such that it fails Daubert's reliability prong, for Dr. Leamer provides various plausible explanations as to why the negative coefficient is not necessarily an unexpected outcome. First, he explains that "dynamic" regressions like this one sometimes lead to results that may appear counterintuitive at first. Leamer Reply Rep. ¶ 60; see also Cisneros Decl., ECF No. 605, Ex. NNN, Leamer Nov. 2013 Dep. at 1008. Dr. Stiroh provides no rebuttal to this point in her declaration submitted in support of Defendants' motion to exclude. More importantly, Dr. Leamer provides at least two plausible, even if not persuasive, market-based explanations as to why it is not "wrong" for the total new hires variable to have a negative coefficient—i.e., why it makes sense for increases in new hires to be negatively correlated with increased compensation. Leamer Reply Rep. ¶ 119 (explaining that periods of economic recovery after a recession are typified by periods of ramped-up hiring but persistent low wages as employers bring back laid-off employees; it is only later that the labor market "tighten[s] enough to put upward pressure on wages"); id. ¶ 120 (setting forth another explanation that high levels of hiring may leave the impression that replacements are easy to find, thus holding down wages of incumbents due to their poor bargaining position); see also In re Plastics Additives, No. 03-CV-2038, 2010 WL 3431837, at *18 (E.D. Pa. Aug. 31, 2010) (dismissing plaintiffs' argument that coefficients had the "wrong" sign, which indicated a negative relationship between price and demand, because while "Plaintiffs ha[d] shown that the coefficients . . . were inconsistent with theoretical explanations," they had not "given any basis for their expectation that the conditions in the markets . . . would be consistent with economic theory," and crediting defense expert's opinion that market conditions could lead to an inverse relationship between demand and price). This Court need not decide whether Dr. Leamer's market-based explanations are in fact correct, as "[t]he evidentiary requirement of reliability [under Daubert] is lower than the merits standard of correctness." In re Paoli R.R. Yard PCB Litig., 35 F.3d 717, 744 (3d Cir. 1994); Primiano, 598 F.3d at 564 ("[T]he test under Daubert is not the correctness of the expert's conclusions but the soundness of his methodology.") (citation omitted). In light of Dr. Leamer's at least plausible explanations, the Court declines to conclude that the existence of a negative coefficient on the total new hires variable means Dr. Leamer's applied methodology was so flawed as to warrant exclusion of his regression at trial. It is noteworthy that the Ninth Circuit has held that a court may admit even "somewhat questionable testimony if it falls within 'the range where experts might reasonably differ, and where the jury must decide among the conflicting views . . .'" S.M. v. J.K., 262 F.3d 914, 921 (9th Cir. 2001), as amended by 315 F.3d 1058 (9th Cir. 2003) (citation omitted).
A "dynamic" regression model, or a "distributed lag model," is one in which the statistician regresses dependent variable "y" at time t on the present and past values of independent variable "x." Larry D. Haugh et al., Identification of Dynamic Regression Models (Distributed Lag Models Connecting Two Time Series), 72 J. Am. Stat. Assoc. 121 (1977).
Dr. Leamer provided the same two explanations in his December 2013 deposition. Cisneros Decl., ECF No. 605, Ex. OOO at 1189.
Maintaining this distinction between the evidentiary requirement of reliability and the higher standard of whether the expert's conclusions are correct "is indeed significant as it preserves the fact finding role of the jury." In re TMI Litig., 193 F.3d 613, 665 n.90 (3d Cir. 1999).
Dr. Leamer also explains that the negative sign may be the result of "collinearity among the variables," Leamer Reply Rep. ¶ 61, which means that multiple correlated independent variables are competing to explain the same dependent variable. Id. at 61, 72; see also ABA Section of Antitrust Law, Proving Antitrust Damages: Legal and Economic Issues, Ch. 6, "Economics and Regression Analysis" at 150 (2010) ("Proving Antitrust Damages"). While the Court acknowledges that collinearity may cause regression estimates to becomes less precise, see Realcomp II, Ltd. v. F.T.C., 635 F.3d 815, 834 (6th Cir. 2011); Ref. Manual at 324; Rubinfeld at 465, Defendants have not raised this issue nor provided any argument as to why this would deem Dr. Leamer's model unreliable under the Daubert standard. Other courts have admitted regressions even in the face of expert disagreement regarding whether collinearity posed a problem. In re High Fructose Corn Syrup Antitrust Lit., 295 F.3d 651, 660-61 (7th Cir. 2002) (refusing to second-guess district court's admission of defense regression analysis where parties' experts disagreed on whether collinearity problem had been resolved or if regression was fundamentally unreliable). This is not surprising given that the concept of collinearity is not a methodology, but a common phenomenon that results when using the methodology of regression analysis. Daubert, 509 U.S. at 595 ("The focus [of the admissibility inquiry], of course, must be solely on principles and methodology, not on the conclusions that they generate.").
iii. Defendants' Claim that Dr. Leamer's Conduct Regression is Unduly Sensitive to Intel
In connection with their third Daubert challenge, Defendants also claim Dr. Leamer's regression is unduly "sensitive" to changes in Intel's hiring and that the "the damages allegedly caused by [anti-solicitation] agreements between other Defendants turns on Intel's behavior." Leamer Mot. at 14; Leamer Reply at 10. In support, Defendants pose a hypothetical they claim demonstrates that "changing the start date of Intel's alleged participation [from 2005 to 2006] has an enormous and irrational influence on the estimates of Dr. Leamer's model." Leamer Mot. at 14. Defendants claim that when the 2006 date is utilized, Dr. Leamer's damage estimate is reduced by over one billion dollars, and that "the enormous effect this relatively minor change has on Dr. Leamer's model underscores its inherent unreliability." Id. at 14 n.6 (summarizing Stiroh Rebuttal ¶¶ 179-80). The Court concludes that any alleged sensitivity in Dr. Leamer's model to Intel's data does not deem his model so inherently unreliable such that it must be excluded from the jury's consideration under Daubert.
The Court recognizes that sensitivity tests can be utilized as one way to test the reliability of regression estimates, as Dr. Leamer himself has acknowledged. Brown Decl., ECF No. 215, Ex. 1 at 351 (noting a "sensitivity analysis ... [is an] exploration of how sensitive [a model's] conclusions are to a choice of variables."); Leamer Reply Rep. ¶ 92; see also Rubinfeld at 436-37 ("Estimated regression coefficients can be highly sensitive to particular data points. Suppose, for example, that one data point deviates greatly from its expected value, as indicated by the regression equation . . . It would not be unusual in this situation for the coefficients in a multiple regression analysis to change substantially if the data point were removed from the sample."). Yet the Court is not convinced that Dr. Leamer's model is in fact unduly sensitive to Intel's data or that any alleged sensitivity means his regression model fails Daubert's reliability prong. Defendants do not explain or provide any evidence as to why a reduction of $1 billion, namely around thirty percent of Dr. Leamer's original $3.06 billion damages estimate, inherently means or suggests that Dr. Leamer's damages estimate improperly "turns on Intel's behavior." Leamer Mot. at 14. It might very well be the case that when the start date of a particular agreement is changed for any other defendant, the damages estimate is similarly reduced by such a large sum, or larger. Defendants do not provide any further information so that the Court may make an appropriate comparison. Further, the Court observes that it does not seem at all unexpected or "irrational," as Defendants characterize it, for the damages estimate to be reduced by a large percentage when a 2006 start date for Intel is utilized as opposed to a 2005 start date. This is because a model that utilizes a 2006 start date neglects to take into account the compensation suppression that would result for an entire year from a cease of cold-calling at the Defendant—i.e., Intel - whose employees comprise the majority of the class, or 40, 357 members out of the 64, 613 person class. Leamer Reply Rep. Table 1.
See also Edward Leamer, Global Sensitivity Results for Generalized Least Squares Estimates, 79 J. Am. Stat. Assoc. 867-70 (1984) (considering sensitivity of regression estimates).
"Sensitivity analysis is the study of how the variation in the output of a model (numerical or otherwise) can be apportioned, qualitatively and quantitatively, to different sources of variation, and how the given model depends on the information fed into it. . . . It allows the analyst to assess the effects on inferences of departures from the assumptions made and the data values, [and] detect outliers or wrong data values . . ." Enrique Castillo, et al., A general method for local sensitivity analysis with application to regression models and other optimization problems, 46.4 Technometrics 430 (2004); see also D.W. Bacon et al., A profile-based approach to parametric sensitivity analysis of nonlinear regression models, 43.4 Technometrics 425 (2001) ("Predictions from a nonlinear regression model are subject to uncertainties propagated from the estimated parameters in the model. Parameters exerting the strongest influence on model predictions can be identified by a sensitivity analysis.").
At the same time, Defendants also claim that the change they implement (changing Intel's participation date from 2005 to 2006) is just a "minor modification," Leamer Reply at 10, but fail to explain why such a change would be "minor" compared to other changes in assumptions that one could make to the model.
Dr. Leamer alludes to this point when explaining that the fact that the damages estimate changes substantially when the date of Intel's agreement is changed by one year does not mean his model is unreliable but is an expected outcome. Leamer Reply Rep. ¶ 112 ("[C]hanging the date of the conspiracy would be expected to have substantial changes in the measured effect of the conduct. It's not just that a portion of Intel's employment is being removed from the class, but that some suppressed compensation is then being treated as 'normal.'"). While Dr. Leamer does not explain in further detail why the resulting change in his estimate does not render his results unreliable, "gaps" in an expert's reasoning may go to the weight of the expert evidence, not its admissibility. Campbell ex rel. Campbell v. Metro. Prop. & Cas. Ins. Co., 239 F.3d 179, 186 (2nd Cir. 2001).
Even putting aside the question whether Dr. Leamer's damages estimate is improperly driven by Intel's data, Defendants do not cite, nor has this Court found, any case holding that the sensitivity of a dependent variable to one or more independent variables categorically means the model must be deemed "junk science" under Daubert. AstenJohnson, 740 F.3d at 463; c.f. Hartley v. Dillard's, Inc., 310 F.3d 1054, 1061 (8th Cir. 2002) ("Only if the expert's opinion is so fundamentally unsupported that it can offer no assistance to the jury must such testimony be excluded.") (citation omitted). In light of these considerations, and this Court's "broad discretion" in deciding whether evidence is reliable and helpful to the trier of fact, see Hankey, 203 F.3d at 1168, the Court concludes that Defendants' argument goes to the weight, not admissibility of Dr. Leamer's model, and that Defendants may appropriately raise their concerns on cross-examination or through Dr. Stiroh's testimony. It will then be up to the jury to assess the credibility of the experts—i.e., whether and to what degree any alleged sensitivity of the model to Intel's data means that the predictive value of Dr. Leamer's regression is low or its regression estimates are imprecise. Wyler Summit P'ship v. Turner Broad. Sys., Inc., 235 F.3d 1184, 1192 (9th Cir. 2000) ("Weighing the credibility of conflicting expert witness testimony is the province of the jury."). Accordingly, the Court rejects Defendants' argument that Dr. Leamer's conduct regression is unduly sensitive to Intel. In sum, the Court DENIES Defendants' third Daubert challenge.
At the hearing on Defendants' motion to exclude Dr. Leamer's testimony, Defendants could not cite to any case which holds that the sensitivity of a dependent variable to one or more independent variables in a regression model means the model must be deemed unreliable under Daubert.
d. Defendants' Fourth Daubert Challenge
The Court now addresses Defendants' fourth and final Daubert challenge. Defendants claim "Dr. Leamer cannot rely on his conduct regression to establish the existence of classwide impact when he admits the model is incapable of showing that each class member was injured." Leamer Mot. at 15. Defendants accordingly assert that "Dr. Leamer's opinion that there was a classwide impact must be excluded." Id. The Court denies Defendants' request.
In antitrust cases, "[p]roof of injury (whether or not an injury occurred at all) must be distinguished from calculation of damages (which determines the actual value of the injury)." Newton v. Merrill Lynch, Pierce, Fenner & Smith, Inc., 259 F.3d 154, 188 (3d Cir. 2001); Catlin v. Washington Energy Co., 791 F.2d 1343, 1350 (9th Cir. 1986) ("[T]he requirement that plaintiff prove 'both the fact of damage and the amount of damage . . . are two separate proofs.' ") (emphasis in original) (citation omitted). Defendants' challenge implicates the element of "[a]ntitrust 'impact'—also referred to as antitrust injury—[which] is the 'fact of damage' that results from a violation of the antitrust laws." In re Dynamic Random Access Memory (DRAM) Antitrust Litig., No. 02-1486, 2006 WL 1530166, at *7 (N.D. Cal. June 5, 2006). Courts have indeed held that plaintiffs must prove every class member was injured by the alleged violation in order to prove the element of impact. See In re Hydrogen Peroxide Antitrust Litig., 552 F.3d 305, 311 (3d. Cir. 2008) ("[E]very class member must prove at least some antitrust impact resulting from the alleged violation."); Blades v. Monsanto Co., 400 F.3d 562, 571-72 (8th Cir. 2005) (plaintiffs must be able to prove injury to each class member); DRAM, 2006 WL 1530166, at *7 (same).
Here, Defendants correctly note that Dr. Leamer concedes his regression does not determine whether any individual class member was impacted. Brown Decl., ECF No. 573, Ex. 1, Oct. 2012 Leamer Dep. at 44, 56-57. However, Defendants' argument fails because their main basis for exclusion hinges on a misleading characterization of Dr. Leamer's opinion regarding impact. While Defendants claim Dr. Leamer relies on his regression model to establish the existence of "classwide impact" as defined by Defendants—i.e., that every class member was in fact impacted—Dr. Leamer has never opined that his regression proves that every class member was in fact impacted. Rather, he has consistently stated that his regression provides reliable proof that the anti-solicitation agreements had a general impact on the class. See Class. Cert. Opening Report at 62 (noting his regression is "capable of showing that the non-compete agreements artificially suppressed compensation to the members of [the technical] class generally") (emphasis added); Leamer Opening ¶ 2 ("I describe[] a methodology (regression analysis) for showing impact and calculating damages to the Defendants' workforces as a whole . . .") (emphasis added); id. ¶ 17 (stating the model "estimate[s] the impact of the illegal conspiracy on the total compensation of Class members."). Thus, Defendants' asserted basis for exclusion—that "Dr. Leamer relies on his model to do what he has admitted it cannot do: prove injury to all class members despite admitting it cannot measure injury to individuals," Leamer Mot. at 15—is incorrect.
Dr. Leamer explained that his regression estimated total undercompensation per defendant per year. Brown Decl., ECF No. 573, Ex. 1, Oct. 2012 Leamer Dep. at 56.
Putting this mischaracterization aside, the Court observes that Defendants also frame their argument in a different way by claiming the regression must be excluded because "Plaintiffs cannot use such a model to satisfy their burden of proving classwide impact." Leamer Mot. at 2 (emphasis added). This argument also fails because it rests on either one or both of two faulty assumptions—first, that Dr. Leamer's regression must singlehandedly suffice to prove that each class member was impacted in order to be admissible evidence, and second, that Dr. Leamer's regression is not relevant to the question of whether each class member was impacted. The former assumption is incorrect because, while the Court made no finding at the class certification stage that the regression itself was capable of demonstrating impact to every class member, neither Rule 702 nor Daubert requires that an expert's testimony, in part or in whole, singlehandedly prove an element of the offering party's case for it to be admissible. Obrey v. Johnson, 400 F.3d 691, 695 (9th Cir. 2005) (noting expert evidence need not establish any element of a claim or defense to be admissible under Daubert); Adams v. Ameritech Servs., Inc., 231 F.3d 414, 425 (7th Cir. 2000) ("[T]he question before us is not whether the reports proffered by the plaintiffs prove the entire case; it is whether they were prepared in a reliable and statistically sound way, such that they contained relevant evidence that a trier of fact would have been entitled to consider."); City of Tuscaloosa v. Harcros Chems., Inc., 158 F.3d 548, 565 (11th Cir. 1998) (expert's study and testimony "need not prove plaintiffs' case by themselves; they must merely constitute one piece of the puzzle that the plaintiffs endeavor to assemble before the jury.").
Such a finding was not required for Plaintiffs to attain class certification. In re Cardizem CD Antitrust Litigation, 200 F.R.D. 326, 340 (E.D. Mich. Apr. 3, 2001) ("To show impact is susceptible to class-wide proof, Plaintiffs are not required to show that the fact of injury actually exists for each class member.").
As for the latter assumption, to the extent Defendants' argument is that Plaintiffs should not be able to rely on Dr. Leamer's model as evidence that each class member was injured—i.e., that Dr. Leamer's regression is irrelevant to the issue of classwide impact—their argument fails. This Court already concluded, when ruling on Plaintiffs' first class certification motion, that Dr. Leamer's conduct regression was a reasonable methodology capable of showing that the anti-solicitation agreements caused "generalized harm to the class." April Order at 38, 43. The Court reaffirmed that conclusion in its October Order. October Order at 60 ("[T]he Conduct Regression analysis is also capable of demonstrating a general classwide impact."). The Court now concludes that even though Dr. Leamer's model is not capable of demonstrating specific injury to each class member on its own accord, it is highly probative to that issue. See In re TFT-LCD (Flat Panel) Antitrust Litig., M 07-1827 SI, 2012 WL 555090, at *5 (N.D. Cal. Feb. 21, 2012) ("Even if regression models are not enough, standing alone, to establish classwide impact, they may nevertheless be relevant to the issue."). This is because a reasonable jury could find that Dr. Leamer's model—which this Court has held is capable of proving generalized impact to the class—in combination with the other evidence presented by Dr. Leamer and documentary evidence separate from Dr. Leamer's analysis, strongly suggests that each class member was impacted. Notably, Dr. Leamer provides substantial evidence that economic theory, documentary evidence, and statistical analyses separate from his conduct regression are capable of showing that the anti-solicitation agreements suppressed the compensation of "all or virtually all" class members. See supra, Part III.A.1. The Court also held in its October Order that "Plaintiffs marshal substantial evidence, including documentary evidence and expert reports . . . [which] suggests that all technical employees—not just those who would have received cold calls but for the anti-solicitation agreements—may have been impacted by the agreements." October Order at 31 (emphasis added); id. at 51 ("The extensive documentary evidence Plaintiffs present [] supports their theory that they will be able to prove the impact of the antitrust violations on a classwide basis."); id. at 33 (concluding "Plaintiffs submitted thousands of pages of documents . . . which support Plaintiffs' theories of classwide harm."). Because the Court finds that Dr. Leamer's regression model will be helpful to the jury's assessment of classwide impact, the model is relevant and thus admissible. See United States v. Rahm, 993 F.2d 1405, 1413 (9th Cir. 1993) (holding that encompassed in the determination of whether expert testimony is relevant is whether it is helpful to the jury, which is the "central concern" of Rule 702). Accordingly, Defendants' fourth Daubert challenge is DENIED.
These "thousands of pages" included "documentary evidence on the importance of cold calling as a recruitment tool and the effect of the preclusion of cold calling on the Technical Class as a whole," "evidence of Defendants' rigid compensation structure and importance of internal equity," and "documentary evidence that Defendants viewed each other as labor competitors, which may have resulted in individual Defendants' wage suppression depressing other Defendants' employees' wages." October Order at 33.
Defendants' citation to In re Plastics Additives, 2010 WL 3431837 (E.D. Pa. Aug. 31, 2010), is unavailing. There, a district court in the Eastern District of Pennsylvania found that the plaintiffs had not demonstrated that antitrust impact was "capable of proof by evidence common to the class" and thus denied class certification. Id. at *19. In doing so, the court held plaintiffs' regression model could not "serve as proof of impact common to the class" because the model said "nothing about individual class member experience" and plaintiffs' expert had conceded that his "industry-wide regression results are in no way indicative of individual impact" and "do not help determine whether each class member suffered any impact[.]" Id. at *15-*16 (emphasis added). Here, in contrast, this case is not at the class certification stage, and Dr. Leamer does not concede that his model is not at all probative to whether each class member suffered an impact. The Court also notes that the In re Plastics court utilized a higher standard at the class certification stage than this Court, which held at the class certification stage that Dr. Leamer's model "support[ed] Plaintiffs' theories of common impact of harm," October Order at 52, 72, despite the fact that his model did not purport to show individualized impact to each class member. --------
Ultimately, the Court concludes the jury is the proper body to decide whether or not and, if so, to what extent, Dr. Leamer's model should be discredited based on the various objections Defendants have raised. Bouman v. Block, 940 F.2d 1211, 1225 (9th Cir. 1991) ("Whether the statistics are undermined or rebutted in a specific case would normally be a question for the trier of fact."). The Ninth Circuit has held that "as a general matter, so long as the evidence is relevant and the methods employed are sound, neither the usefulness nor the strength of statistical proof determines admissibility under Rule 702." See Obrey, 400 F.3d at 696. This Court held at the class certification stage that Dr. Leamer's conduct regression was "statistically robust," supported by the economic literature, and "capable of calculating classwide damages." October Order at 82. None of Defendants' arguments persuades the Court to change that conclusion, and thus Defendants' challenge to Dr. Leamer's conduct regression is denied.
B. Defendants' Joint Motion for Summary Judgment Based on Defendants' Motion to Exclude the Testimony of Dr. Leamer
Defendants jointly move for summary judgment based on their motion to exclude Dr. Leamer's testimony. ECF No. 556. Defendants' sole argument in support of their joint motion for summary judgment is that "[w]ithout Dr. Leamer's expert report and testimony, Plaintiffs have no evidence of classwide impact or damages and cannot prove the essential elements of their antitrust claim." Id. at 1. Because this Court denies Defendants' motion to exclude Dr. Leamer's testimony in full, Defendants' joint motion for summary judgment based on their motion to exclude Dr. Leamer's testimony is also DENIED.
IV. CONCLUSION
For the foregoing reasons, the Court GRANTS IN PART and DENIES IN PART Defendants' motions:
• Defendants' motion to strike Dr. Leamer's testimony is GRANTED in part as to Dr. Leamer's new 50% statistical significance theory and DENIED in all other respects.
• Defendants' motion to exclude Dr. Leamer's testimony under Daubert is DENIED.
• Defendants' joint motion for summary judgment based on their motion to exclude Dr. Leamer's testimony is DENIED.
IT IS SO ORDERED.
Dated: April 4, 2014
/s/_________
LUCY H. KOH
United States District Judge