Opinion
Civil No. 19-2875 (RBK/JS)
12-02-2020
IN RE: VALSARTAN, LOSARTAN, AND IRBESARTAN PRODUCTS LIABILITY LITIGATION
OPINION
This Opinion addresses a regrettable discovery dispute involving Teva's use of technology assisted review ("TAR") in connection with its ESI production. The dispute illustrates the unfortunate avoidable consequences that occur when a party does not meaningfully and timely meet, confer and collaborate regarding complex and costly ESI discovery. Given the stakes at issue and the lessons to be learned, a lengthy discussion is warranted.
The moving defendants are Teva Pharmaceuticals USA, Inc., Teva Pharmaceutical Industries Ltd., Actavis LLC, Actavis Pharma, Inc., and Arrow Pharm (Malta) Ltd. (collectively "Teva" or the "Teva defendants").
Presently before the Court is Teva's request for an Order foreclosing additional review of documents that, based on its unilaterally developed and administered TAR, are predicted to be "non-responsive." In the alternative Teva seeks to shift to plaintiffs the cost of its further manual review of alleged non- responsive documents that its Continuous Multi-Modal Learning ("CMML") platform predicts are non-responsive. The parties' dispute has been the subject of extensive briefs and lengthy oral argument. At bottom the Court must decide if the terms of the Court Ordered ESI Protocol that requires timely good faith cooperation and collaboration amongst the parties takes precedence over Teva's unilaterally adopted CMML platform.
Although Teva suggests the terms "non-responsive" and "irrelevant" are interchangeable, this is not technically correct. Teva acknowledges that some of the "non-responsive" documents it does not want to review are arguably relevant although it questions their importance.
As discussed herein, the Court denies Teva's request to impose on plaintiffs its unilaterally adopted CMML platform since the adoption violates the Court Ordered Protocol. Teva's request for cost shifting is also denied. Plaintiffs' request that Teva continue to manually review its non-responsive documents is denied, as well as plaintiffs' request for sanctions. Plaintiffs' back up request asking that Teva produce all its non-responsive documents with a privilege filter but no manual review is also denied. Rather than granting the relief requested by the parties, the Court Orders Teva to review its non-responsive documents using the TAR protocol the parties negotiated but failed to consummate in August 2020.
Background
The present dispute has a long history. This multidistrict litigation ("MDL") concerns adulterated prescription Valsartan blood pressure medication made, sold, dispensed, etc. by defendants. Plaintiffs allege the Valsartan they ingested was contaminated with cancer causing chemicals that has or will cause past, present and future personal injuries and economic losses. Three master complaints have been filed. Two putative nationwide class actions assert medical monitoring and economic loss claims. The third master complaint alleges Valsartan users contracted various forms of cancer from ingestion of the contaminated drug. To date approximately 670 separate personal injury complaints have been filed with an expectation of more to come.
On December 18, 2019, this MDL was expanded to include Losartan and Irbesartan. Doc. No. 325.
Given the size and complexity of this matter there is no question that from the outset of the litigation the parties knew the stakes were high. Although at present no one knows for certain the precise amount in dispute, given the popularity of Valsartan the potential exposure in this MDL may reach into the hundreds of millions and perhaps a billion dollars. The parties, therefore, knew or should have known early on that ESI discovery was going to be costly and extensive. This being the case the use of TAR was or should have been reasonably contemplated at the outset of the litigation.
Soon after the litigation was filed attorneys with substantial experience in MDL cases entered their appearances. To their credit counsel quickly formed into liaison groups and prepared the organization and background documents typically executed in MDL cases. The lead defendant group consists of the manufacturers of the active pharmaceutical ingredient used in Valsartan and the finished dose manufacturers (collectively "manufacturers"). The other groups are the wholesalers and retailers/pharmacies. Separate liaison counsel was appointed for each of these liaison groups. Teva belongs to the target manufacturer group.
Turning to the ESI dispute at issue, on June 18, 2019, the Court entered CMO No. 8, the parties' stipulated "Electronic Discovery Protocol" ("Protocol") [Doc. No. 125]. As it pertains to the present dispute the key language is contained in Section II. which reads:
II. Search Terms for Electronic Documents
The parties agree that they will cooperate in good faith regarding the disclosure and formulation of appropriate search methodology, search terms and protocols, and any TAR/predictive coding prior to using any such technology to narrow the pool of collected documents to a set to undergo review for possible production. The parties agree to meet and confer as early as possible to discuss, inter alia:
The Protocol also includes a provision requiring the parties to meet and confer regarding a dispute as to whether they should produce documents in conformity with the Protocol, and if the parties cannot reach agreement "the issue shall be brought to the Court's attention for resolution[.]". See Protocol at 15 (Section IV.C.).• Search methodology(ies)to be utilized (including but not limited to Boolean searches and technology assisted review/predictive coding).
The implementation of the Protocol has not been smooth. The manufacturer defendants negotiated with plaintiffs for months about the search terms and custodians to use which finally resulted in an Order entered by consent on December 23, 2019 that approved the parties' lists. [Doc. No. 328]. All manufacturer defendants agreed to use the same search terms but separate custodian lists were negotiated and approved for each manufacturer.
Although Teva was on notice before December 2019 of the core group of custodians and search terms that were going to be used in connection with its document search, including the priority custodians Teva identified, Teva (and the other manufacturers) refused plaintiffs' requests to conduct sample runs or hit reports to determine if the proposed search terms were overbroad. As a result, when the manufacturers finally ran the reports after the entry of the December 23 Order, they complained that the search terms they agreed to were unduly burdensome and asked for relief. This caused plaintiffs to complain bitterly that defendants were "moving the goalposts." Eventually plaintiffs relented and accommodated Teva and the other manufacturers by agreeing to narrow the previously agreed to search terms. See Doc. Nos. 485 (Mylan), 487 (ZHP) 489 (Hereto), 490 (Torrent), 493 (Teva) and 497 (Aurobindo). Teva's amended search term list was entered on June 24, 2020. [Doc. No. 493].
The December 23 Order included a caveat that the search term list could be modified. However, plaintiffs and the Court did not envision that wholesale revisions would be requested.
The original date for the manufacturer defendants to complete their ESI production was May 29, 2020. See Dec. 12, 2019 Order ¶7, Doc. No. 303. Subsequently, at defendants' request and over plaintiffs' objection, the completion date was extended to November 29, 2020, with the proviso that rolling productions would occur on July 15, September 1, October 1, and November 1, 2020. See Apr. 20, 2020 Order, Doc. No. 416.
Because of the virus situation in China, ZHP's deadline was extended to June 30, 2020. Id.
On July 1, 2020, a year after the Protocol was entered, and more or less a year after the parties started negotiating search terms, seven months after the entry of the first Order setting the search terms, a week after the June 24 Order narrowing the original search term list, and only two weeks before Teva's first rolling production was due, Teva informed plaintiffs for the first time "that the Teva defendants will be utilizing a continuous multi-modal learning ("CMML") platform to assist defendants with review and production of the Teva defendants' electronically stored information ("ESI")." See July 14, 2020 Letter, Ex. A, Doc. No. 516-3. Teva described CMML as a "machine-learning technology that enables a computer to prioritize relevant documents based on limited human input." Id. According to Teva, as contrasted with TAR 1.0 which uses a seed set to train the application:
The CMML workflow is an iterative process, where the training of the application is continuous and does rely on a seed set. Rather, CMML has the flexibility to begin prioritizing documents based on any subset of documents that have been evaluated and coded with a responsive or non-responsive coding value. As more documents are reviewed and submitted as subsequent training examples, the CMML application improves in its ability to distinguish between responsive and non-responsive content. CMML's reliability has been confirmed from both a statistical/academic perspective[.]July 14, 2020 Letter Brief ("LB") at 3-4, Doc. No. 516.
If Teva had planned to only use CMML to "prioritize" documents for production, perhaps plaintiffs would not have objected. However, plaintiffs were apoplectic about Teva's statement that "[i]f at some point, the CMML system is indicating that there is a population of documents unlikely to be responsive (such that it will be unduly burdensome for the Teva Defendants to review them), [Teva] will inform Plaintiffs of the same." Id. Teva added it would "inform" plaintiffs if there is ultimately a population of documents [it does] not intend to review. Id. And Teva wrote, "the Teva Defendants are currently reviewing all documents if or until the CMML platform indicates such review would no longer be efficient and/or reasonable." Id. at 1.
Plaintiffs' main objection to Teva's July 1 revelation was that if they had known Teva contemplated the use of TAR they would not have agreed to limit the review of the custodians' documents to only those that contained the designated search terms. Plaintiffs insist that, had Teva made its intentions known, they would not have wasted countless hours negotiating over search terms, nor would they have agreed to Teva's plan to layer TAR review with search terms. Plaintiffs' position is that, "search terms and technology assisted review are alternatives in this setting" and that CMML is an "alternative - not an adjunct - to use of search terms." Id. at 3. Plaintiffs have made their position clear that they steadfastly object to "layering" TAR with a set of documents that has been limited to only those that contain the designated search terms.
Having heard plaintiffs' objection to its use of CMML, Teva asked the Court to "confirm Teva is authorized to use CMML in its review and production process and that Plaintiffs must cooperate in good faith with Teva as appropriate utilization of CMML technology continues moving forward." Id. at 6-7. Thereafter, at the parties' request, the Court, Teva's counsel, plaintiffs' counsel, and the parties' ESI consultants, attempted to reach an agreement. After extensive discussions which resulted in much progress, all substantive TAR related disputes related to the search methodology to be used were resolved and an agreed upon CMML protocol was ready to be signed but for two issues about which Teva would not relent. One, Teva would not agree to the entry of a Court Order to memorialize the parties' agreement but instead insisted that the agreement not be put "on the record." Two, Teva would not agree to permit plaintiffs to review 5000 alleged non-responsive documents to evaluate and validate Teva's CMML platform.
As a result of its steadfast refusal to agree to the disputed terms, Teva informed the Court on August 5, 2020 [Doc. No. 544] that it was withdrawing its request that the Court address the TAR issue. Teva wrote, "the fundamental disagreement is that the Teva Defendants cannot agree to a non-confidential validation protocol which permits Plaintiffs to review non-responsive documents[.]". Id. at 2. Teva also informed the Court that it would not use TAR to eliminate non-responsive documents to be reviewed and that it would review all of its documents manually even though it recognized its manual review would be "extremely burdensome." Id. at 3. There the matter stood until Teva filed the present application before the Court.
Ordinarily the Court would not disclose details of the parties' confidential discussions. However, Teva "opened the door" by referring to these discussions in its letter briefs.
Unbeknownst to plaintiffs, since the parties efforts to resolve their TAR dispute broke down in August 2020, Teva undertook an effort to self-validate its electronic review. Teva first revealed to plaintiffs this was done shortly before the present October 13, 2020 application was filed. Teva acknowledges plaintiffs had no input into its CMML platform although Teva posits it used a "gold-plated" standard.
Teva alleges it undertook to review a uniform random sample of 15,000 documents from the designated high-priority custodians that the CMML platform indicated are non-responsive. Oct. 13, 2020 LB at 2, Doc. No. 594. Teva alleges that out of the 15,000 documents, and after a quality control process that it controlled and directed, the reviewing attorneys deemed only 109 documents to be responsive, an "elusion of only 0.73%" Id. Teva also alleges that almost all of the 109 documents are duplicate of those already produced or are "marginally relevant." Id. Teva contends its review of the 15,000 documents involved 330.6 hours of attorney review time, almost all of which was wasted since 99% of the documents it reviewed will not be produced. Id. Teva also contends that unless the Court grants its request it will be "forced" to review 260,375 documents for the high-priority custodians that are likely to be irrelevant and a waste of time. Id. at 8. This total does not include the "non-responsive" documents from the other custodians to be searched. Teva buttresses its argument with supporting Declarations from its consultant, Dr. Maura A. Grossman. [Doc. Nos. 594-1, 616-2]. Dr. Grossman co-invented CAL and attests to the fact she is "widely considered one of the foremost experts in technology-assisted review ('TAR')." Decl. ¶6; Supp. Decl. ¶2. Teva also submitted a Declaration from its ESI vendor, Consilio, which attested to the fact that to date 164,028 documents from six "high-priority" custodians have been reviewed with 260,375 documents remaining from these custodians. See D. Ketchmark Decl. ¶¶2-4, Doc. No. 594-2.
Based on its analysis Teva contends it would be grossly disproportionate to require it to review the documents designated non-responsive by its TAR. Teva argues it "[s]hould not be forced to spend months and millions of dollars reviewing documents that it already knows are likely to be non-responsive based on well-known technology that has become the norm in eDiscovery." Oct. 13, 2020 LB at 2-3 (emphasis in original). In the alternative, Teva argues that if it is "forced" to review non-responsive documents its review costs should be shifted to the plaintiffs, "who have unreasonably attempted to thwart Teva's efforts to leverage this now industry standard technology." Id. at 3. Teva summarizes its request as follows:
Teva respectfully requests an Order foreclosing additional review of documents that based on state-of-the-art TAR and e-Discovery best practices are non-responsive. Alternatively, Teva respectfully requests that the Court shift the cost of Teva's further non-responsive document review by ordering plaintiffs to reimburse Teva's costs and fees associated with reviewing documents that CMML predicts are non-responsive.Id. at 6.
Plaintiffs oppose Teva's application. At bottom, plaintiffs argue that Teva violated the Court Ordered Protocol and, therefore, it cannot now use TAR to narrow the pool of documents to review. Plaintiffs argue the parties worked tirelessly to agree on search terms to use and it is too late to change course to layer TAR with search terms. According to plaintiffs, "[t]he time for Teva to request a layered document review process, with TAR layered upon an already narrowed search terms document set, was prior to entry of the ESI protocol and search terms Order - which was agreed to by Teva, not seven months after the search terms were finalized and Ordered." Oct. 30, 2020 LB at 8 (emphasis in original), Doc. No. 612. Moreover, plaintiffs argue Teva's TAR review should not be approved because it was done without their input. Plaintiffs argue, "[i]t would be grossly inequitable to rely on the data quoted by Teva, or to give any benefit of the doubt to Teva, since the entire process was tainted. Plaintiffs were locked out of the process so it is not possible for plaintiffs to respond from a level playing field." Id. at 4. Plaintiffs conclude by arguing:
The only equitable outcome is for the Court to deny [Teva's] application, give Teva the chance of either complying with its obligation to review the documents identified with the search terms with no cost shifting, or producing all documents it chooses not to review to plaintiffs, and to assess sanctions against Teva for its pattern of conduct on this issue, which has severely prejudiced plaintiffs.Id. at 2-3.
Discussion
1. The Requirements Imposed by the Court Ordered Protocol Rather than Teva's Proportionality Analysis Controls the Outcome of the Parties' Dispute
The Court's first order of business is to identify the issue to be decided since the parties have materially different views. Before tackling this issue the Court will identify what it does not have to decide. The Court does not have to decide if TAR is an appropriate discovery tool. We are past the time when parties and courts view TAR as an outlier. See generally Hyles v. New York City, 10 Civ. 3119 (AT)(AJP), 2016 WL 4077114, at *2 (S.D.N.Y. Aug. 1, 2016). Indeed, plaintiffs do not object to the general notion that a party can use TAR so long as its use is transparent and timely disclosed, and the parties collaborate in good faith about its use. See July 14, 2020 LB at 4, Doc. No. 513. ("Plaintiffs agree that machine-learning software can be useful and can be a reasonable alternative to the application of search terms to collect documents for production in some situations.").
The Court also does not have to decide if Teva's CMML application works or if Teva's consultant is qualified to attest to its general effectiveness. In other words, this is not a referendum on CMML or Teva's consultant. For present purposes the Court can assume CMML generally works and Teva's consultant is qualified. Further, the Court does not have to decide if there are instances when a party may layer a document production with search terms and TAR. Ample case law exists to support Teva's position that in appropriate instances layering may be done. See Teva's Nov. 18, 2020 LB at 6-7 (citing cases). (Nonetheless, the decision in Bridgestone Americas, Inc. v. Int'l Bus. Machines Corp.,No. 3:13-1196, 2014 WL 4923014, at *1 (M.D. Tenn. July 22, 2014), is noteworthy. Although the court permitted plaintiff to use predictive coding after an initial screening was done with search terms, the court noted the process required "openness and transparency in what Plaintiff is doing [since this is] of critical importance").
Last, the Court does not have to decide if plaintiffs can dictate to Teva the manner in which Teva must review and produce its ESI. The Court agrees with the line of cases that holds that a producing party has the right in the first instance to decide how it will produce its documents. Hyles, supra, at *2. As will be discussed, however, this general principle is trumped by the requirements in an agreed upon ESI Protocol memorialized in a Court Order.
As noted, the parties dispute whether the Protocol controls the outcome of their dispute. Teva says no, plaintiffs say yes. Teva argues, "[a]t this point, the issue before the Court is not what the ESI Protocol permits or does not permit as it relates to CMML. Rather, the Court must rule on a straightforward proportionality e-Discovery dispute." Oct. 13, 2020 LB at 2. Teva also argues, "[t]his remains a straightforward proportionality motion." Nov. 18, 2020 LB at 10. Plaintiffs disagree. Plaintiffs focus on the requirements in the Protocol and argue that Teva violated the Protocol. See Oct. 30, 2020 LB at 7. ("Teva clearly violated both the letter and the spirit of the ESI protocol by waiting until July 2020 to bring up the [TAR] issue, and by using their TAR methodology prior to meeting and conferring with Plaintiffs.").
The Court agrees with plaintiffs and rules it must first decide what the Protocol requires and whether Teva violated these requirements. In the Court's view there is no legitimate question that the Court's Order trumps Teva's proportionality argument. If the protocol has been violated the Court's task is to decide the relief to be granted, not to do a proportionality analysis under Fed.R.Civ.P. 26(b)(1). If the Court agreed with Teva and did a proportionality analysis in the first instance, it would have to ignore and discount the troublesome history of the parties' search term discussions. This the Court will not do. Another reason to reject Teva's argument is that if the Court addressed Teva's proportionality argument and ignored the Protocol, it may incentivize parties to skirt the requirements in a Court Order.
2. Teva Violated the Protocol
Neither side has cited any controlling case law interpreting the relevant language in the Protocol. The Court, therefore, must tackle the issue. As is evident from the language in the Protocol, Teva is not barred from using TAR. Indeed, the Protocol explicitly recognizes that TAR may be used. However, before TAR is used to "narrow the pool of collected documents to a set to undergo review for possible production," the parties must cooperate in good faith regarding TAR's disclosure and formulation and timely collaborate on its use. Importantly, the Protocol requires the parties to "meet and confer as early as possible" regarding TAR/predictive coding.
Relying upon the express language in the Protocol, and the goal of fostering and encouraging transparency and collaboration, the Court holds the Protocol required Teva to timely disclose its use or possible use of TAR when Teva objectively knew or reasonably should have known that it might use TAR to reduce the universe of documents to review. The Protocol also required Teva to disclose its possible use of TAR when its use was reasonably foreseeable. In addition, the Court holds the Protocol required Teva to timely collaborate with plaintiffs on its CMML platform to be used before, not after, it implemented the Protocol. Applying these standards, the Court finds that Teva violated the Protocol by not timely disclosing its use or possible use of its CMML platform to reduce the universe of documents to review and by attempting to foist on plaintiffs a protocol about which plaintiffs had no input.
To be clear, the Court does not find that Teva acted in bad faith. No evidence has been presented to show that Teva delayed disclosing its TAR for strategic reasons or for a nefarious purpose. The Court accepts Teva's representation that it did not make a final decision to use TAR to reduce the universe of documents to review until shortly before its July 1, 2020 letter to plaintiffs. However, a finding of bad faith is not necessary to conclude that a party did not act in good faith within the meaning of the Protocol. Further, a finding of bad faith is not necessary to rule that a Court Order has been violated and consequent relief may be awarded. Tracinda Corp. v. DaimlerChrysler AG, 502 F.3d 212, 242 (3d Cir. 2007); Montana v. City of Cape May Board of Freeholders, C.A. No. 09-755 (NLH/JS), 2013 WL 11233748, at *10 (D.N.J. Sept. 20, 2013).
Teva did not comply with the requirements in the Protocol since it is seeking the Court's blessing to use a CMML platform it unilaterally adopted. By no means is the Court holding that Teva must give in to plaintiffs' demands. Instead, the Protocol requires the parties to meet and confer in good faith to attempt to reach agreement. This does not occur if one side or the other unilaterally adopts a TAR protocol "late in the game" and argues it should be approved by the Court. This also does not occur when a party's TAR protocol is presented to its adversary as a fait accompli. The time to meet and confer in good faith is before a TAR protocol or CMML platform is adopted and used, not after.
The Court has no hesitation in finding that Teva did not timely disclose its intent to use TAR. The clearest case is Teva's recent communication advising plaintiffs it had already been using TAR, combined with Teva's present application asking the Court to approve its unilaterally adopted protocol. After all, Teva's November 6, 2020 communication to plaintiffs concerning the CMML platform it developed and used was made one year after the Court Ordered Protocol was entered, after months of search term negotiations resulting in the entry of two Court Orders (December 23, 2020 and June 24, 2020) defining and then narrowing the search terms to be used, after the parties negotiated and almost finalized an agreed upon protocol in August, and only two weeks before Teva's first scheduled rolling ESI production was due.
The Court agrees with plaintiffs that, "Teva's intention to use CMML should have been disclosed at the earliest possible time." July 14, 2020 LB at 4. Contrary to Teva's position, this occurred before Teva subjectively decided to use CMML. Since the backbone of TAR's use is transparency and collaboration, Teva should have disclosed to plaintiffs in the fall that it might use TAR since at that time is was objectively reasonable and foreseeable that Teva might use TAR in the future. This would have enabled the parties to fairly collaborate on the protocol to use. "Electronic discovery requires cooperation between opposing counsel and transparency in all aspects of preservation and production of ESI." William A. Gross Const. Associates, Inc. v. American Mfrs. Mut. Ins. Co., 256 F.R.D. 134, 136 (S.D.N.Y. 2009). Early disclosure would have likely prevented the parties from going down the rabbit hole of laborious search term negotiations. As is evident by what happened here, "the failure to engage in a collaborative search and sampling strategy can often yield discovery dysfunction." Lawson v. Love's Travel Stops & Country Stores, Inc., C.A. No. 1:17-CV-1266, 2019 WL 7102450, at *5 (M.D.Pa. Dec. 23, 2019).
Teva acknowledges when it met with plaintiffs in November 2019 it was already consulting with TAR specialists. Decl. of Teva's Counsel at ¶¶5-6, Doc. No. 616-1. It is implausible, therefore, that at that time it was not objectively foreseeable that TAR might reasonably be used to eliminate documents to manually review. Rather than keeping this a secret until July 1, 2020, and waiting until November 2020 to ask the Court to approve a CMML application it unilaterally adopted, Teva's possible use of TAR should have been disclosed at the earliest possible time. Given the stakes in the case, the volume of ESI likely to be requested, and the fact Teva was consulting with TAR specialists in the fall, it was objectively reasonable for Teva to foresee in the fall, and certainly before the December 23, 2019 Order was entered, that it reasonably might use TAR in the future. Teva is a sophisticated party and was certainly familiar with TAR's benefits when it was negotiating search terms with plaintiffs. The Protocol required the parties to disclose their potential use of TAR before it was used and implemented, not after.
Besides contradicting the timely notice requirement in the Protocol, Teva also did not comply with its meet and confer obligation. The Protocol required the parties to "cooperate in good faith regarding the disclosure and formulation of appropriate search methodology..., and any TAR/predictive coding prior to using any such technology to narrow the pool of collected documents to a set to undergo review for possible production." Teva did not cooperate regarding the disclosure and formulation of its CMML platform since it was adopted without any input or knowledge of plaintiffs. Nor did Teva meet and confer as early as possible with plaintiffs since Teva proposes to use a TAR that it did not notify plaintiffs about until its mind was already made up. "The best solution in the entire area of electronic discovery is cooperation among counsel." William A. Gross, 256 F.R.D. at 136 (citation omitted). Unfortunately this did not occur. To be sure, the Court is not weighing in on the effectiveness of CMML. It may very well be that Teva's methodology performs as advertised. However, what the Court holds is that it will not approve a TAR methodology that was unilaterally adopted and implemented since this contradicts the requirements in the parties' Protocol.
The Court is not asked to address, and is not deciding, whether a unilaterally adopted TAR protocol is appropriate in the absence of a Court Order dictating what must be done.
The fact that Teva may have used a similar Protocol to what was used in the Broiler antitrust litigation is of no moment. Nor is the fact that in Broiler the protocol called for layering search terms and TAR. See In Re Broiler Chicken Antitrust Litig., No. 1:16-CV-08637, 2018 WL 1146371 (N.D.Ill. Jan. 3, 2018). One reason is because the Court is not presently deciding whether Teva's protocol is effective but instead is addressing in the first instance what is required under the Protocol. Further, Broiler is inapposite since what happened here is not remotely similar to what happened in that case. As plaintiffs point out (Oct. 30, 2020 LB at 14-17), the timing and process regarding the entry of the protocol in Broiler is markedly different than this case. The Broiler protocol was negotiated by the parties early in the case unlike here where Teva is proposing to disregard the requirements in a Court Ordered Protocol by using a late disclosed CMML platform it unilaterally adopted.
In order to fairly evaluate Teva's CMML rather than simply accepting Teva's representations at face value, plaintiffs would have to take discovery about discovery. This effort would be wasteful and distracting and is not likely to help advance a decision on the merits. Too much time has already been spent on the parties' TAR dispute.
Also, unlike here, in Broiler the parties and court did not undertake months of negotiations and argument over search terms with no inkling that TAR would be proposed for use. In addition, in this case the parties agreed to search terms without the benefit of hit counts, sampling, and false positives, all of which Teva refused to provide.
Nothing prevented Teva from running these reports. See Pltfs.' Nov. 18, 2020 LB at 2, Doc. No. 635. ("Teva's claim that it could not have done so before all of the custodians were agreed to is obviously inaccurate. Testing is routinely run on the custodial sets of those custodians who are clearly going to be included.").
Plaintiffs argue that when the parties' search term negotiations took place there was no expectation TAR would be used. Having participated in a good deal of the parties' discussions and knowing that TAR was not mentioned, the Court credits plaintiffs' argument that "negotiations would have looked very different if Teva had timely disclosed its intentions, and the parties had a procedure like that in Broiler Chicken in place at the beginning of the process." Oct. 30, 2020 LB at 16. Further, the Broiler protocol required the parties "to be reasonably transparent regarding the universe of documents subject to targeted collections or culling via search terms and/or TAR/CAL." 2018 WL 1146371, at *1. This did not occur here as to Teva's CMML platform. The Broiler protocol also required a party using TAR to disclose before its use, inter alia, a general description of the process, including how it will train the algorithm and what quality control measures will be taken. Id. at *2. As to the CMML platform Teva asks the Court to approve, nothing of the sort occurred here. The fact that the Broiler protocol was acceptable for use in that case does not necessarily mean its use is appropriate in this litigation.
The Court does not question Teva's representation that it did not subjectively decide to use TAR until June 2020. However, the Court rejects the notion that pursuant to the Protocol Teva's duty to meet and confer in good faith about its TAR methodology was not triggered until Teva subjectively and definitively committed to using TAR or, as Teva suggested during oral argument, when there was a "substantial likelihood" TAR would be used to narrow the universe of documents to review. Otherwise, the opposing party, in this case plaintiffs, is at the mercy of Teva and cannot meaningfully contribute to the protocol to be used. The Court will not countenance a situation where for months plaintiffs were led to believe and proceeded under the reasonable assumption that Teva would do a manual search term review of its ESI and then, after a substantial effort was expended in this direction, Teva changed course and unilaterally adopted and used its unilaterally adopted CMML platform. The Court credits plaintiffs' argument:
[W]hen the parties here were mediating the initial search terms..., Plaintiffs had no expectation that there was going to be any possibility of further restriction of the corpus of documents to be reviewed using TAR. This holds doubly true for the narrowed search terms agreed to by the parties in June 2020. Both negotiations would have looked very different if Teva had timely disclosed its intentions[.]October 30, 2020 LB at 16.
3. Teva's Arguments are not Persuasive
Teva argues plaintiffs should accept their protocol and validation at face value. It argues, "after additional review, no validation protocol is necessary, as Teva has detailed data to demonstrate to plaintiffs, and also the Court, that its CMML platform is working consistent with Teva's representations[.]" Oct. 13, 2020 LB at 2. If we lived in a perfect world devoid of the skepticism and doubt that pervades litigation, perhaps this could occur. However, we know this is not the case. It is common and accepted practice that parties collaborate with regard to TAR validation measures. The Court will not weigh in on the effectiveness of Teva's methodology except to note that plaintiffs have made a primie facie argument why Teva's protocol is inadequate. See generally Decl. of Jonathan Jaffe at 2, Doc. No. 612-1. ("[T]here are fundamental flaws in TEVA's approach.").
Plaintiffs point to the differences between what Teva previously agreed to and what it seeks to unilaterally impose. For example, plaintiffs argue Teva did not start with the entire set of all custodial files rather than the narrowed search term set. In addition, plaintiffs argue Teva did not use all the "core discovery" documents to educate the system, it may have excluded certain documents from TAR review, it denied plaintiffs the ability to submit training documents, it gave plaintiffs no reports on its review as the review proceeded, it failed to employ an agreed upon stopping criteria, and it failed to preclude email threading. See Oct. 30, 2020 LB at 4.
Teva underestimates the prejudice to plaintiffs by its late disclosure. The Court also rejects Teva's argument that "[t]he type of transparency exhibited by Teva is exactly what the ESI protocol in this case require[s]" Nov. 18, 2020 LB at 2. Plaintiffs' ESI consultant, Jonathan Jaffe, speaks to this:
In the search term negotiations in which I was heavily involved, plaintiffs negotiated with the understanding that TAR and CAL would not be used in the review and production process, as no defendant indicated that it would be using TAR or CAL for review and production....Plaintiff agreed to search term modifications that were only reasonable if TAR and CAL were not going to be used. If this intent had been disclosed, my strong advice would have been not to agree to the modifications and that the present objections would have been presented at that time.Decl. of Jonathan Jaffe at 3. Jaffe also states, "[w]ithout the full engagement of the Plaintiffs from the outset, ..., the production resulting from application of CAL to the truncated set of documents utilized provides no reasonable assurance of fairness or adequacy." Id. at 9.
The Court disagrees with Teva's argument that it made an appropriate TAR disclosure within the meaning of the Protocol. Although the Protocol was entered in June 2019, and intensive search term negotiations took place over the year, prior to July 1, 2020 Teva can point to only one fleeting reference to a` TAR disclosure at a November 15, 2019 meeting with plaintiffs. See generally Decl. of Teva's Counsel, Doc. No. 616-1. Teva's counsel attests, "[t]he parties held a short discussion of TAR during the meeting," they agreed the ESI protocol addressed the use of TAR, and "plaintiffs requested to be informed if Teva intended to use TAR to make review determinations without attorney's eyes on the documents." Id. at ¶7. Teva's response was that it was "evaluating whether or not to use TAR but had not made any determination as to how or whether [it] would do so in the future." Id. at ¶8. This disclosure was inadequate. The simple reservation that Teva might use TAR in the future did not satisfy Teva's duty to "cooperate in good faith regarding the disclosure and formulation" of TAR, nor did it satisfy the duty to meet and confer "as early as possible" regarding TAR.
Plaintiffs have a different recollection of the parties' November 15, 2019 TAR discussion. Plaintiffs' position is that, "Teva told Plaintiffs counsel that their vendor had TAR capabilities, that they were not going to use it for purposes of searching their documents, and that they were not sure they were or were not going to use it at all. Oct. 30, 2020 LB at 5. Plaintiffs also state that Teva did not reserve its right to modify its document review obligations at a later date and that if this reservation was made they would have objected. Id. at 5-6.
Even if Teva did not finally decide until June 2020 to use TAR, it should have foreseen in the fall of 2019 that this would or was objectively reasonably likely to occur. The Court holds that pursuant to the duties imposed by the Protocol this knowledge triggered Teva's duty to meet and confer with plaintiffs about a TAR methodology to be used. It was inappropriate for Teva to involve plaintiffs and the Court in intensive search term negotiations and disputes without also disclosing there was a reasonable prospect TAR would be used to winnow its documents to be reviewed. Had a timely disclosure been made the parties' TAR dispute would not have dragged on and instead of now spending their resources on this discovery dispute, the parties could have instead focused on the start of fact depositions which is imminent
Teva argues its duty to disclose TAR was not triggered in June 2020 since at that time it was only using CMML for "prioritization" purposes and not to exclude documents for manual review. However, the Court has already ruled that Teva's duty under the Protocol was triggered before the December 23, 2019 Order was entered. Further, before Teva's July 1, 2020 notification letter to plaintiffs Teva knew or reasonably should have known that its "priorization" effort was a prelude to an effort to exclude documents for manual review.
The Court does not question Teva's representation that it "understood it could leverage technology if the search terms yielded too many hits for a manual review." Nov. 6, 2020 LB at 2. However, Teva neglected to consider the requirement in the Protocol that it meet and confer as early as possible regarding TAR. The time to meet and confer with plaintiffs was before the parties and the Court proceeded under the reasonable assumption that a manual search term review would be done, and not on the eve of the due date of the first rolling production.
Teva's attempt to foist responsibility for a misunderstanding on plaintiffs is off base. Teva argues because it "never made any affirmative statements to Plaintiffs that it was no longer reserving its right to [use TAR]...Plaintiffs were well-aware that Teva was still considering whether to use TAR[.]" Id. at 2. Teva also argues plaintiffs did not make it clear they would not agree to layer TAR and search terms and that plaintiffs would not negotiate search terms until Teva made a final decision on TAR. Nov. 18, 2020 LB at 6. To repeat, the Court holds that pursuant to the Court Ordered Protocol Teva had an affirmative duty that was triggered in the fall of 2019 to timely reveal it was reasonably possible or probable that it would use TAR to limit the universe of documents to review. It was not up to plaintiffs to infer, guess, or speculate if this would be done. Teva is a sophisticated party with extensive litigation experience. It knew or should have known from the outset of the litigation that the stakes were high and plaintiffs were likely to ask for the production of millions of documents. This being the case Teva should have made appropriate disclosures and contingencies if and when it finally decided to use TAR. Teva should have disclosed it was reasonably possible it might use TAR before the extensive search term negotiations took place. The simple reservation that no final decision was made yet is insufficient. If an appropriate disclosure was made it would have enabled plaintiffs to take this into account in the parties' negotiations. Teva acknowledges plaintiffs specifically made their concerns known at their November 2019 meeting. Waiting seven months to finally make its disclosure was a violation of the parties' Protocol.
If in the future plaintiffs anticipate an extensive ESI search and production will be done, one takeaway from the parties' dispute is that plaintiffs should make it clear from the outset of their ESI discussions that their search term negotiations are contingent on whether their adversary will use TAR to eliminate documents from a manual review.
A case remarkably similar to the present dispute is Progressive Casualty Ins. Co. v. Delaney, No. 2:11-CV-00678-LRH-PAL, 2014 WL 3563467 (D. Nev. July 18, 2014). In Progressive Casualty the parties agreed to an ESI protocol that provided that plaintiff would use agreed upon search terms to review its documents. Like this case, in the midst of the review a party, the plaintiff, decided the cost to review the documents was prohibitive and it asked the Court to permit it to apply its unilaterally implemented predictive coding or TAR to the set of documents that contained the agreed upon search terms. Defendant objected to plaintiff's request to modify the agreed upon ESI protocol midstream. The Court agreed with defendant and noted that plaintiff was asking for a "do-over" of its own invention that lacks transparency and cooperation regarding the search methodologies applied. Id. at *10. Rather than granting plaintiff's request to use predictive coding, the Court ordered plaintiff to produce all "hit" documents without a manual review, with the proviso that plaintiff could apply "privilege filters" to the "hit" documents and the production would be subject to clawback.
As in Progressive Casualty the Court denies Teva's request to unilaterally impose on plaintiffs its CMML platform. A long line of cases holds that TAR requires, "an unprecedented degree of transparency and cooperation among counsel in the review and production of ESI responsive to discovery requests." Id. at *10; accord Youngevity Int'l, Corp. v. Smith, No. 16-CV-00704-BTM(JLB), 2019 WL 1542300, at *12 (S.D. Cal. April 9, 2019). In the cases that have approved TAR, "the courts have required the producing party to provide the requesting party with full disclosure about the technology used, the process, and the methodology[.]". Progressive Casualty, at *10. This was not done here. The time for Teva to make its TAR disclosure was before the "die was cast," not afterwards. The Court will not endorse a TAR protocol that was unilaterally adopted by a producing party without any input from the requesting party. While the Court does not endorse plaintiffs' hyperbole, it is sympathetic to its argument:
It would be expressly inequitable to rely on the data quoted by Teva, or to give any benefit of the doubt to Teva, since the entire process was tainted. Plaintiffs
were locked out of the process so it is not possible for plaintiffs to respond from a level playing field. That was the purpose of the protocol plaintiffs negotiated, including the validation, to yield data that the parties could jointly rely on.Oct. 30, 2020 LB at 4.
Teva's attempt to distinguish Progressive Casualty is fruitless. It is true that unlike here the defendant in that case delayed producing documents. However, this was not the crux of the decision. The lynchpin of the opinion was that like here the parties negotiated a search term protocol and review and then a party "changed horses midstream" and proposed to use a TAR protocol that it unilaterally adopted. When it declined to bless the plaintiff's new TAR proposal the court in Progressive Casualty wrote:
In this case, the parties negotiated an ESI protocol which was adopted by the court as an order[.]...Had the parties worked with the e-discovery consultants and agreed at the onset of this case to a predictive coding-based ESI protocol, the court would not hesitate to approve a transparent, mutually agreed upon ESI protocol. However, this is not what happened. [Plaintiff] agreed to search the universe of documents identified in the stipulated ESI protocol using search terms. [Plaintiff] had the option to produce all non-privileged documents reviewed without manually reviewing them [.] Alternatively, [plaintiff] had the option of manually reviewing the "hit" documents[.] [Plaintiff] chose the latter and began manually reviewing the ESI[.]Id. at *9. This is precisely what happened here. The court in Progressive Casualty reasoned as follows: "[plaintiff] proposes a 'do-over' of its own invention that lacks transparency and cooperation regarding the search methodologies applied." Id. at *10. For the same reason, the Court will not bless Teva's "do-over."
Teva's other arguments do not carry the day. Teva insists that plaintiffs are attempting to "control Teva's method of electronic document review." Oct. 13, 2020 at 3. Not true. Plaintiffs are instead attempting to compel compliance with the Court Ordered Protocol. Teva cites Kaye v. N.Y.C. Health & Hospitals Corp., 18-CV-121W7 (JPO)(JLC), 2020 WL 283702 (S.D.N.Y. Jan. 21, 2020), for the proposition it can adopt the ESI of its choosing so long as it reveals the collection criteria used, the name of the CAL software, and how it intends to validate its review results. Nov. 18, 2020 LB at 2. Not so since the Protocol requires timely transparency and collaboration. Significantly, Kaye is not controlling since in that case the court did not address what was required under a Court Ordered Protocol. Nor was the court in Kaye dealing with a situation where a party operated for a year under the reasonable assumption that a search term review would be done but the producing party later changed course. Teva argues plaintiffs are not prejudiced by Teva's CMML process. Nov. 6, 2020 LB at 1. Again, for the reasons already discussed this is not true. See also Pltfs.' Nov. 18, 2020 LB at 4-5. Plaintiffs are prejudiced by being shutout of Teva's CMML platform and the validation process Teva proposes to use.
Teva argues plaintiffs steadfastly refuse to allow for the use of CMML at all. Nov. 6, 2020 LB at 2, Doc. No. 616. Not true as evidenced by the fact the parties were on the verge of consummating an agreed upon protocol in August 2020. Teva argues it was not until the final search terms were agreed to and approved that it realized TAR was necessary to timely and cost-effectively manage its document review." Id. at 4. The Court discounts this argument since Teva should have known there was an objectively reasonable prospect it might use TAR months earlier. Teva argues, "plaintiffs have offered no reason to believe, or even suspect, that Teva's CMML process was deficient." Nov. 18, 2020 LB at 3. Not true for the reasons already discussed. See n. 11, supra.
Teva argues plaintiffs did not negotiate in good faith after it asked plaintiffs to bless the CMML platform it unilaterally adopted. It even goes so far to state, "Plaintiffs were never willing to meaningfully work with Teva." Id. at 2. The Court disagrees for several reasons. One, Teva's argument is belied by the fact the parties were able to reach agreement on a protocol but for Teva's objection to two provisions. Two, since Teva presented its CMML application as a fait accompli plaintiffs' objection is understandable. Three, Teva neglected to follow the provision in the Protocol that disputes be presented to the Court for resolution. See Protocol at Section IV.C. (p. 15). Throughout the course of the case the parties have not been shy about presenting their discovery disputes to the Court. Teva did not ask the Court to decide whether the two disputed provisions should be implemented. Instead, Teva foreclosed a reasonable opportunity to put this dispute to bed by not agreeing to use the non-disputed portions of the protocol the parties agreed upon over the summer or presenting the disputed issues to the Court for final resolution. Plaintiffs' argument is apropos: "Teva inexplicably failed to implement the validation protocol agreed to by the Plaintiffs when Teva went ahead and unilaterally applied its own protocol to narrow the review set anyway, without notice to Plaintiffs, in violation of the ESI protocol." Nov. 18, 2020 LB at 4.
4. Unless Otherwise Agreed to by Plaintiffs, Teva Must Use the TAR Protocol the Parties Negotiated Over the Summer
Having ruled that Teva's application is denied, the Court must decide how to proceed going forward since Teva still has to address its set of documents that have not been reviewed. Plaintiffs argue that if Teva's application is granted and Teva is relieved of its obligation to review non-responsive documents, "the only fair and workable solution would be for the entire set of allegedly non-responsive documents to be produced to Plaintiffs." Oct. 30, 2020 LB at 17. Contrary to Teva's argument, this position is not completely out of the mainstream and has been done in other cases. Plaintiffs cite to precedent to support the notion a court may order alleged non-responsive documents to be produced without a manual review first being done by the producing party. Plaintiffs also ask for sanctions against Teva because Teva has "drained valuable resources from Plaintiffs' litigation effort," and Teva "wasted an enormous amount of the Court's time, for weeks working with the parties to try and negotiate an agreed to TAR protocol." Id. at 17. Plaintiffs sum up by arguing, "the fair outcome is to sanction Teva for the time and financial costs expended by plaintiffs in continually responding to Teva's effort to artificially cut off review and deprive plaintiffs of documents." Id. at 17-18.
See n.17, infra. In any event, as one court noted, "in the world of ESI, new perspectives and approaches are needed to complete discovery in an efficient and reasonable manner." Adair v. EQT Prod. Co., No. 1:10CV00037, 2012 WL 2526982, at *4 (W.D.Va. June 29, 2012).
Given the Court's ruling that there has been a violation of the Court Ordered Protocol, the Court has wide discretion to fashion an equitable remedy. Farmers & Merchs. Nat'l Bank v. San Clemente Fin. Group Sec., Inc., 174 F.R.D. 572, 585 (D.N.J. 1997). In deciding what to do the relief to be granted must be tailored to the harm identified. Montana v. City of Cape May Bd. Of Freeholders, C.A. No. 09-755 (NLH/JS), 2013 WL 11233748, at *10 (D.N.J. Sept. 20, 2013). And, the Court must be mindful that all Court Ordered relief originates from equity. Bull v. United Parcel Serv., Inc., 665 F. 3d 68, 83 (3d Cir. 2012).
To put it bluntly, the thought that Teva or plaintiffs might have to spend millions of dollars to manually review irrelevant or marginally relevant documents is more than mildly disturbing. This is contrary to the mandate in Fed.R.Civ.P. 1 which counsels that courts should construe, administer and employ the federal rules to "secure the just, speedy, and inexpensive determination of every action and proceeding." The Court is cognizant, therefore, that if it bars Teva from using TAR Teva will continue its manual review. The Court is also aware that if it grants plaintiffs' request, plaintiffs may manually review Teva's documents.
In order to save costs Teva can choose to produce its non-responsive documents without a manual review but it has declined to do so. If Teva is correct that its non-responsive documents are irrelevant, cumulative or immaterial, their prompt production will not prejudice Teva, especially since Teva is protected by a clawback provision and a Fed.R.Evid. 502(d) Order.
Instead of directing Teva to manually review its documents or directing Teva to produce to plaintiffs its non-responsive documents before Teva reviews them as was done in Progressive Casualty, the Court will Order what it deems to be an eminently fair and equitable resolution of the parties' dispute. The Court will permit Teva to do a TAR review of its non-responsive documents but Teva must use the protocol negotiated by the parties over the summer. Teva can hardly complain since its consultant was part and parcel of all of the parties' discussions and compromises and approved their meeting of the minds. Plaintiffs cannot complain because Teva will conduct a TAR review pursuant to a protocol they approved.
The Protocol is attached as Exhibit B to plaintiffs' Oct. 30, 2020 LB.
The Court will direct that the TAR protocol to be implemented include the two provisions originally objected to by Teva, neither of which the Court finds controversial or bothersome. The final TAR protocol shall be memorialized in a Court Order. This will hopefully reduce or eliminate disputes down the road. In addition, amongst the validation protocols that must be included in the protocol, plaintiffs shall have the right to review at the end of Teva's production 5000 alleged non-responsive documents plaintiffs designate for review. The Court does not understand why this provision is so bothersome to Teva. Plaintiffs are not, as Teva insists, asking Teva to produce its irrelevant documents in discovery. Instead, plaintiffs are asking to review a relatively small number of documents as part of their validation review. Teva's insistence that it is unheard of for alleged non-responsive or irrelevant documents to be produced either by court order or by agreement is not correct., In addition to denying Teva's request asking the Court to approve its CMML platform, the Court denies Teva's application to shift costs to plaintiffs since the Court will not direct Teva to manually review its non-responsive documents.
Progressive Casualty, supra; In re: Generic Pharm. Pricing Antitrust Litig., MDL 2724, 2019 WL 8106511 (E.D.Pa. Oct. 24, 2019), mandamus den. by In re Activis Holdco U.S., Inc., 2019 WL 8437021 (3d Cir. Dec. 6, 2019); In re: 3M Combat Arms Earplug Products Liab. Litig., No. 3:19-md-2885 (N.D. Fla. 2019)(Pretrial Order No. 12)("TAR Protocol"); Williams v. Taser Intern., Inc., 1:06-CV-0051-RWS, 2007 WL 1630875, at *6 (N.D. Ga. June 4, 2007).
Given the Court's view that the two provisions Teva objects to are not unreasonable, the Court disagrees with Teva's repeated statements that it had "no choice" but to manually review its non-responsive documents and that it was "forced" to do so. Oct. 13, 2020 LB at 2. Instead, Teva made a voluntary choice how to review its documents.
The Court is not ignorant of the fact that Teva may have to incur additional costs to implement a different TAR protocol. However, the additional cost is likely to pale in comparison to the millions Teva claims it will incur if its non-responsive documents are manually reviewed. Also, Teva proceeded at its own risk when it decided to use a new protocol of its own choosing instead of the non-objectionable portions of the protocol the parties agreed to in principal. Further, the Court is aware of the substantial resources plaintiffs had to unnecessarily spend the past seventeen months dealing with search term issues when they were willing to address the use of TAR from the outset of the MDL.
Conclusion
In sum, the Court repeats its regret that an enormous amount of time and energy has been spent on a dispute that was avoidable if the parties were fully and timely transparent and collaborated as envisioned by their Protocol. For all the reasons discussed herein the Court concludes that Teva violated the Protocol. The Protocol required the parties to meet and confer in good faith as soon as possible about the methodology and formulation of TAR to reduce the number of documents to manually review. This duty arose when there was an objectively reasonable or foreseeable likelihood or possibility that TAR may be used to cull the set of documents to review. This duty was triggered in the fall of 2019, before the December 23, 2019 search term Order was entered. Teva did not comply with the Protocol when it waited until July 1, 2020, to disclose for the first that it already adopted a TAR methodology that undoubtedly was a prelude to asking to remove documents from a manual review. Teva's present request asking the Court to approve its CMML platform also violates the Protocol because Teva unilaterally adopted its CMML platform without any input from plaintiffs.
For the reasons discussed herein, the Court is reluctant to impose a harsh penalty. Instead, an equitable solution is to require Teva to conduct its review of non-responsive documents using the TAR protocol the parties almost finalized but for two provisions. Given the Court's view that the two provisions Teva objected to are reasonable, the Court will direct that the parties' protocol be memorialized in a Court Order and that plaintiffs be permitted to review 5000 alleged non-responsive documents of their choosing.
Teva may apply a privilege filter to the 5000 documents plaintiffs designate. Also, for good cause shown Teva may apply to the Court to review certain of the 5000 documents in camera.
For all the foregoing reasons, Teva's application is denied as is plaintiffs' counter application. An appropriate Order memorializing the rulings in this Opinion will be separately entered.
The Court has no objection if the parties agree to modify the protocol negotiated over the summer. The Court also has no objection if plaintiffs agree to use Teva's CMML application. However, even if the parties agree on changes to the TAR protocol to be used, the protocol shall be memorialized in a Court Order and shall include the disputed provision regarding plaintiffs review of 5000 alleged non-responsive documents. The court is hopeful that this Opinion is the last word on the parties' TAR dispute and they will work cooperatively to move forward in an efficient and productive manner.
s/ Joel Schneider
JOEL SCHNEIDER
United States Magistrate Judge DATED: December 2, 2020