Explore

Keywords with TAR and Key Takeaways, Assisted Review Series Part 9

9 / 9

Over the nine years since it first rose to prominence in eDiscovery, technology-assisted review has expanded to include numerous new tools, more potential workflows, and a variety of legal issues

In “Alphabet Soup: TAR, CAL, and Assisted Review,” we discussed TAR’s rise to prominence and the challenges that it has created for practitioners.  In “Key Terms and Concepts,” we discussed the key terms and concepts practitioners need to know.  In “Applications, Aptitudes, and Effectiveness,” we discussed use cases, relative merits, and overall efficacy.  In “Are You Allowed to Use TAR?,” we discussed case law on authorization to use TAR.  In “Can You Be Compelled to Use TAR?,” we discussed case law on being compelling or directed to use TAR.  In “When Is the Right Time for TAR Process Objections?,” we discussed case law on objection timing.  In “What TAR Process Results Are Objectionable?,” we discussed case law on how good is good enough.  In “Is Process Transparency Required?,” we discussed case law on process transparency requirements.  In this final Part, we review case law on using keywords with TAR, as well as key takeaways.


The final question for us to address, before we review our key takeaways from across the series, is whether you can use keywords and TAR together.  Due to the enormous volume of materials collected in some cases, and due to the challenges associated with discovery from low-prevalence collections (even when using a TAR approach), some parties wish to use keyword searches first followed by a TAR approach applied to just the results of those searches.  This is not an approach typically recommended by the developers of TAR tools, and some parties have objected to their opponents taking it.

Using Search Terms and TAR Together

Several cases have allowed parties to apply search term filtering to a document population before applying a TAR approach to the results, over the objections of the requesting parties (and, in one case, over the stated preference of the judge):

  • In In Re: Biomet, the initial collection work done by the defendants yielded about 19.5 million documents.  They then used keyword searching to cut that number down to 3.9 million documents, and then deduplication cut that to about 2.5 million documents.  The defendants then opted to leverage a TAR approach on those 2.5 million documents, and the plaintiffs objected to applying TAR after filtering with keyword searches, rather than applying it to the full collection of 19.5 million documents.  To make this argument, the plaintiffs pointed to the same studies used in prior cases to support the use of TAR, which show the superiority of TAR to keyword searching.  The Judge framed the issue in a different way, however:

The issue before me today isn’t whether predictive coding is a better way of doing things than keyword searching prior to predictive coding.  I must decide whether [the Defendants’] procedure satisfies [their] discovery obligations and, if so, whether [they] must also do what the [Plaintiffs] seek[].  [emphasis added]

After reviewing the work performed, the Judge concluded that the defendants’ process did, in fact, satisfy their discovery obligations (reasonableness).  The Judge also considered the costs that would have been associated with the plaintiffs’ requested approach and concluded that a cost of millions of additional dollars would outweigh the potential benefit gained from finding some more of the remaining relevant documents (proportionality).

  • In Bridgestone, the plaintiff asked if it could switch to using a TAR approach – after having used keyword searches – due to the enormous volume encountered using the originally agreed discovery plan. The Defendant objected to what it called an “unwarranted change” from the originally-agreed protocol and argued that “it is unfair to use predictive coding after an initial screening has been done with search terms” [emphasis added].  The Magistrate Judge ultimately allowed the plaintiff to “switch horses in midstream” and use TAR, concluding that whether and when to use TAR comes down to a “judgment call” focused on efficiency and the needs of the specific case:

In this case, we are talking about millions of documents to be reviewed with costs likewise in the millions.  There is no single, simple, correct solution possible under these circumstances.  [emphasis added]

  • In Rio Tinto, the parties reached an agreement about the use of TAR that was memorialized by court order, and their protocol included the option to utilize keyword searches prior to using a TAR approach:

If the Responding Party determines it to be reasonable and appropriate, the Responding Party may use search terms and other criteria (the “Culling Criteria”) to reduce the volume of the Document Universe.

Later, in a letter to the Magistrate Judge dated April 6, 2015, the parties documented a number of discovery disputes, including the plaintiff’s complaint that the defendant “unilaterally decided to apply search terms to its Document Universe without first consulting with” the plaintiff, which led the Magistrate Judge to allow the plaintiff an opportunity “to suggest ‘reasonable’ additional search terms.”  In a later order, the Magistrate Judge referenced the issue and commented that “The Court itself felt bound by the parties’ protocol, such as to allow keyword culling before running TAR, even though such pre-culling should not occur in a perfect world” [emphasis added].

In at least one case, the court declined to allow the use of keyword searches until after the TAR process was complete:

While the parties agree on many issues relating to discovery and particularly electronic discovery, they unfortunately cannot reach agreement over whether the universe of electronic material subject to TAR review should first be culled by the use of search terms. . . .  They have left the matter for the Court to decide.  The Court does so rather reluctantly. . . .  Be that as it may, having reviewed the letters and proposed orders together with some technical in-house assistance including a read of The Sedona Conference TAR Case Law Primer, 18 Sedona Con. J. ___ (forthcoming 2017), the Court is satisfied that FCA has the better position.  Applying TAR to the universe of electronic material before any keyword search reduces the universe of electronic material is the preferred method.  The TAR results can then be culled by the use of search terms or other methods.  [emphasis added]

Although courts’ preference for allowing producing parties to choose their methods likely means it will continue to be permitted, using keyword searches before applying a TAR approach may draw greater scrutiny to your process and its results.

Key Takeaways

Over the course of this series, we have taken a look at seventeen cases, plus numerous articles and other sources, and together, these materials have provided us with an effective overview of the technology-assisted review landscape.  Here are the eight key takeaways from that overview: 

  • There Are Two Main Types of TAR Approaches

TAR approaches encompass a family of similar-but-distinct workflows that rely upon similar-but-distinct mathematical approaches and software tools, which can be broadly grouped into two categories we have called TAR 1.0 and TAR 2.0.  TAR 1.0 approaches are first-wave, predictive coding-style approaches.  They take longer to set up but provide more statistical information and are often better for larger or more complex projects.  TAR 2.0 approaches are second-wave, continuous active learning-style approaches.  They are faster and easier to set up and, typically, cheaper to use, making them suitable for smaller, simpler projects than TAR 1.0 approaches.

  • TAR Approaches Can Be As Effective As – or More Effective Than – Older Approaches

Numerous studies have shown that, when used correctly, TAR approaches are at least as effective as older approaches.  This is, in part, because TAR approaches are good and, in part, because keyword searches and human review are not as good as many practitioners assume.  In reality, untested keyword searches are often fairly ineffective and even the best reviewers make numerous mistakes due to simple human fallibility or due to differing understandings of relevance and privilege.

  • You Can Use Tar, and You Don’t Need Permission

As clearly shown by da Silva Moore, Global Aerospace, In Re: Actos, Bridgestone, Dynamo, Rio Tinto, and others, courts in the United States generally accept the use of TAR approaches as viable alternatives to older, manual methods.  Additionally, we have seen on-the-record acceptance of its use in Ireland, England, and Australia.

Moreover, as emphasized in Dynamo and others, parties are not required to seek prior judicial approval for the use of TAR approach any more than they would be required to do so for an older approach. 

  • You Can’t Compel Tar Use, but the Judge Might

Although you are free to use TAR yourself, you cannot compel another party to use TAR if they prefer another methodology.  As emphasized in Kleen Products, Hyles, In Re: Viagra Prods., and In Re: Mercedes-Benz, courts adhere to Sedona Principle 6 and the idea that producing parties are in the best position to choose appropriate methods.  The requesting party’s interest is in the results rather than the process employed.

While you cannot compel a party to use TAR, judges have directed parties to use TAR to expedite matters in cases like EORHB, Independent Living Center, and Winfield.

  • Process Objections Should Be Held Until after Production

As emphasized in da Silva Moore, Dynamo, Hyles, and others, parties are not meant to raise adequacy objections to another party’s discovery processes until after at least some production has actually occurred.  There is no mechanism for objecting to a hypothetical efficacy issue with a process that has yet to be carried out; objections must be rooted in identified issues with actual results (as also noted in Entrata).

Whether a party has satisfied their obligations can only be determined as a fact-based, post hoc analysis like the one conducted in Winfield.

  • Processes and Results Must Be Reasonable and Proportional, Not Perfect

There is an oft-repeated refrain in discovery that the expectation is reasonableness not perfection.  This is drawn from the requirement in Federal Rule of Civil Procedure 26(g) that a “reasonable inquiry” be made to ensure the completeness of discovery responses, as well as from the case law addressing this issue (e.g., Dynamo).  Thus, if a TAR approach in question was a reasonable inquiry, its results will generally be deemed acceptable even if imperfect.  The reasonableness of what has been done is evaluated using a post-hoc, fact-based proportionality analysis (e.g., da Silva Moore, Hyles, and Winfield)

  • Process Transparency Might Not Be Compelled, but It Is Preferred

For a time, the pattern in early cases (e.g., da Silva Moore, Global Aerospace, In Re: Actos, and Bridgestone) made it seem like a high level of transparency might be a requirement for TAR use, despite the facts that: (a) transparency in those cases was often offered voluntarily to help overcome objections to what was then a novel approach and (b) traditional document review approaches were never that transparent (or expected to be).

In cases where parties have tried to compel such transparency (e.g., In Re: Biomet, Winfield, and Entrata), however, judges have declined to compel most of what’s been requested, citing relevance, privilege, and work product concerns.  Although In Re: Biomet didn’t compel the transparency, it did include a warning that a lack of voluntary transparency might negatively affect the application of judicial discretion.

  • You Can Probably Use Search Terms before Tar, but It Is Not Preferred

Several cases (e.g., In Re: Biomet, Bridgestone, and Rio Tinto) have allowed parties to apply search term filtering to a document population before applying a TAR approach to the results, over the objections of the requesting parties (and, in one case, over the stated preference of the judge).  In at least one case (FCA), the court declined to allow the use of keyword searches until after the TAR process was complete.  Although courts’ preference for allowing producing parties to choose their methods likely means it will continue to be permitted, using keyword searches before applying a TAR approach may draw greater scrutiny to your process and its results.


For Assistance or More Information

Xact Data Discovery (XDD) is a leading international provider of eDiscovery, data management and managed review services for law firms and corporations.  XDD helps clients optimize their eDiscovery matters by orchestrating precision communication between people, processes, technology and data.  XDD services include forensicseDiscovery processingRelativity hosting and managed review.


About the Author

Matthew Verga

Director of Education

Matthew Verga is an electronic discovery expert proficient at leveraging his legal experience as an attorney, his technical knowledge as a practitioner, and his skills as a communicator to make complex eDiscovery topics accessible to diverse audiences. A thirteen-year industry veteran, Matthew has worked across every phase of the EDRM and at every level from the project trenches to enterprise program design. He leverages this background to produce engaging educational content to empower practitioners at all levels with knowledge they can use to improve their projects, their careers, and their organizations.

Whether you prefer email, text or carrier pigeons, we’re always available.

Discovery starts with listening.

(877) 545-XACT / or / Subscribe for Updates