Bringing it Full Circle in Rio Tinto, Technology-Assisted Review Series Part 10

10 / 14

Seven years after it first rose to prominence in eDiscovery, technology-assisted review remains an important, and at times controversial, tool in the eDiscovery practitioner’s toolkit

In “Still Crazy after All These Years,” we discussed the slow but steady growth in the importance of TAR.  In “In the Beginning Was da Silva Moore,” we discussed the first case to address TAR.  In “Questions of Choice in Kleen Products,” we discussed an attempt to force the use of TAR, and in “Reported Results in Global Aerospace,” we discussed the first instance of reported TAR results.  In “A Negotiated Protocol in In Re: Actos,” we discussed a successfully negotiated TAR protocol.  In “At a Judge’s Direction in EORHB,” we discussed a Judge ordering TAR use unprompted.  In “Debating Process and Transparency in Biomet,” we discussed questions of process and transparency.  In “Changing Horses Midstream in Progressive and Bridgestone,” we discussed trying to switch to a TAR approach after agreeing to a traditional one.  In “Seeking Approval and Perfection in Dynamo,” we reviewed the questions of whether to seek prior approval and how good is good enough.  In this Part, we review Judge Peck’s second prominent TAR case.

The next prominent TAR case for us to review is Rio Tinto PLC v. Vale, S.A., et al. (S.D.N.Y. Mar. 2, 2015), which brings us back full circle to our beginning with another decision from Magistrate Judge Peck.  This is another case in which the parties reached an agreement about the use of TAR, but the Magistrate Judge’s order accepting their agreement still provides an excellent summary of the state of TAR acceptance as of spring 2015, after the first three years of cases.

Black Letter Law

In his review of those early cases, Magistrate Judge Peck identifies several key points about the then-current state of TAR acceptance by parties and courts.  First and most importantly, he identifies the same trend of judges accepting its use that we have, and provides a statement on it that has been widely quoted since: “In the three years since Da Silva Moore, the case law has developed to the point that it is now black letter law that where the producing party wants to utilize TAR for document review, courts will permit it.”  [emphasis added]

Three Key Points from Magistrate Judge Peck

Beyond that declaration, Magistrate Judge Peck also makes three important points about the primary issues still unresolved by case law:

  • First, he writes that “[o]ne TAR issue that remains open is how transparent and cooperative the parties need to be with respect to the seed or training set(s)” and explains that opinions are divided on this issue – both among practitioners, and in the case law. As he writes, “where the parties do not agree to transparency, the decisions are split and the debate in the discovery literature is robust.”  [emphasis added]
  • Second, he points out the fact that seed set and training set disclosure is not the only validation option available to requesting parties and suggests alternatives that might be less contentious than full disclosure:

In any event, while I generally believe in cooperation, requesting parties can insure that training and review was done appropriately by other means, such as statistical estimation of recall at the conclusion of the review as well as by whether there are gaps in the production, and quality control review of samples from the documents categorized as non-responsive.  [internal footnote omitted; emphasis added]

  • Third, he discusses the standards by which we judge review and production methodologies, and like the Judge in Dynamo, concludes that TAR should not be held to a higher standard than traditional human review is:

One point must be stressed – it is inappropriate to hold TAR to a higher standard than keywords or manual review.  Doing so discourages parties from using TAR for fear of spending more in motion practice than the savings from using TAR for review.  [emphasis added]

The Search Term Dispute

After this order, but before the case was eventually dismissed, the attempt to utilize TAR in this case resulted in some additional disputes and orders.  First, in a letter to the Magistrate Judge dated April 6, 2015, the parties raised a number of unresolved discovery disputes, including the question of applying search terms to reduce the document collection before beginning the TAR process.

The Defendant reported that it had collected a “Document Universe of nearly 2 million documents” pursuant to the agreed discovery plan, and after evaluating this collection, it determined that:

. . . the low prevalence of responsive documents in the Document Universe . . . risked undermining and burdening the process of assessing the accuracy of the predictive coding process and leading to review of a disproportionately large number of documents in relation to the number of responsive documents in the Document Universe.  [emphasis added]

The agreed upon discovery plan allowed for the parties to “use search terms and other criteria . . . to reduce the volume of the Document Universe” if “the Responding Party determines it to be reasonable and appropriate,” and based on its assessment of the collection’s low prevalence, the Defendant concluded it would be “reasonable and appropriate” to use search terms first.

The Defendant applied “a set of over 150 search terms,” reducing the collection size by about 75%.  It also evaluated a “Statistically Valid Sample of 2,398 documents” from the pool of materials eliminated by the search terms and “found zero responsive documents excluded by the search terms.”

The Plaintiff, however, objected to the Defendant’s use of search terms before TAR, arguing that search terms shouldn’t be used before TAR due to their inferiority.  The Plaintiff cited several of the same sources we’ve previously discussed, including Judge Peck’s own prior comments.  The Plaintiff also objected to the terms chosen and to the Defendant’s interpretation of “reasonable and appropriate.”

Magistrate Judge Peck concluded he was “bound by the parties’ protocol” and approved the Defendant’s use of the search terms before TAR, despite his personal view that “such pre-culling should not occur in a perfect world.”  He also allowed the Plaintiff an opportunity to suggest additional search terms to be used, and the Plaintiff requested more than 120 additional search terms, which “more than doubled” the document population to “over 800,000 documents.”  Despite this huge increase, the Defendant sampled the new search results and concluded that “those search terms did not add new responsive documents.”

The Special Master Dispute

The ongoing disputes over the best way to proceed with and validate the TAR effort also led the Defendant to request the appointment of TAR expert Maura Grossman as a Special Master.  The Plaintiff objected to this as well, but Magistrate Judge Peck granted the request.  Even with her participation, process disputes and complications continued to arise regarding training, validation, and other specifics.

Upcoming in this Series

In the next Part of this series, we will review the Hyles case, the third and final prominent TAR case from Magistrate Judge Peck in our survey.

About the Author

Matthew Verga

Director of Education

Matthew Verga is an electronic discovery expert proficient at leveraging his legal experience as an attorney, his technical knowledge as a practitioner, and his skills as a communicator to make complex eDiscovery topics accessible to diverse audiences. A fourteen-year industry veteran, Matthew has worked across every phase of the EDRM and at every level from the project trenches to enterprise program design. He leverages this background to produce engaging educational content to empower practitioners at all levels with knowledge they can use to improve their projects, their careers, and their organizations.

Whether you prefer email, text or carrier pigeons, we’re always available.

Discovery starts with listening.

(877) 545-XACT / or / Email Us