Grading Papers: Measuring Human Review, Testing Classifiers Series Part 3
Just as a search or a TAR tool is making a series of binary classification decisions, so too are your human reviewers, and the quality of those reviewers’ decisions can be assessed in a similar manner to how you assessed the quality of a search classifier. Depending on the scale of your review project, employing these assessment methods can be more efficient and informative.
Read More
Show Your Work: Contingency Tables and Error Margins, Testing Classifiers Series Part 2
Sampling can be used to test your search classifiers – whether keyword searches, TAR software, or other tools – by calculating their recall (efficacy) and precision (efficiency). Doing so requires a previously-reviewed control set, contingency tables, and some simple math.
Read More
Pop Quiz: How Do You Test a Search?, Testing Classifiers Series Part 1
Beyond estimating prevalence, there are other opportunities to replace informal sampling of unknown reliability with formal sampling of precise reliability. Imagine iteratively refining searches for your own use, or negotiating with another party about which searches should be used, armed with precise, reliable information about their relative efficacy. Using sampling to test classifiers can facilitate this.
Read More
Red Hots, Hot Docs, and the Ones that Got Away, Estimating Prevalence Series Part 3
Now that we understand the necessary sampling concepts, let’s apply those concepts to our candy contest and figure out how many red hots we think are in the jellybean jar. In order to do so, we will need to identify our sampling frame, select our desired confidence level, and select our desired confidence interval.
Read More
Key Sampling Concepts for Winning the Candy Contest, Estimating Prevalence Series Part 2
In order to use sampling to estimate how many red hots are mixed into the jellybean jar, we need to understand some basic sampling concepts, including: sampling frame, prevalence, confidence level, and confidence interval, as well as how each affects required sample size.
Read More
Finding out How Many Red Hots are in the Jellybean Jar, Estimating Prevalence Series Part 1
Despite years of discussion in the eDiscovery industry about the power and importance of sampling techniques, many practitioners remain unfamiliar with what they can accomplish with them and when, outside of TAR, they might do so. There are opportunities across the phases of an eDiscovery project to replace guesses based on anecdotal evidence with actual estimates based on formal sampling.
Read More
Modern Challenges and Key Takeaways – eDiscovery Investigations Series, Part 5
In part five of our eDiscovery investigations series, we discuss modern challenges in eDiscovery investigations and key takeaways from eDiscovery investigations. Our discussion covers mobile devices and BYOD practices, as well as alternate communication channels like secret message apps as modern challenges in investigations. Check out our five key takeaways after a review of eDiscovery in investigations.
Read More
The Need to Be Prepared for Later Litigation – eDiscovery Investigations Series, Part 4
“Come, Watson, come!” he cried. “The game is afoot. Not a word! Into your clothes and come!” – Arthur Conan Doyle, The Adventure of the Abbey Grange.
The majority of eDiscovery work takes place in the context of litigation, but a significant amount of it takes place instead in the context of investigations. Although the available ESI and the available eDiscovery technologies are the same, the realities of handling investigations are different in some ways worth discussing.
Read More
The Need for Nuanced Analysis and Review – eDiscovery Investigations Series, Part 3
Analysis and review is the process of figuring out what happened by investigating your collected evidence, and that process is made more challenging when relevant individuals have actively tried to conceal what’s happened – or at least tried to be subtle about it while it was happening. Also, it is not uncommon for individuals to communicate using euphemisms or coded language or to communicate using alternative channels. To overcome these challenges, the analysis and review process must be undertaken with these realities in mind.
Read More
The Need for Speed and Secrecy – eDiscovery Investigations Series, Part 2
“Come, Watson, come!” he cried. “The game is afoot. Not a word! Into your clothes and come!” – Arthur Conan Doyle, The Adventure of the Abbey Grange.
The majority of eDiscovery work takes place in the context of litigation, but a significant amount of it takes place instead in the context of investigations. Although the available ESI and the available eDiscovery technologies are the same, the realities of handling investigations are different in some ways worth discussing.
Read More
When the Game is Afoot – eDiscovery Investigations Series, Part 1
“Come, Watson, come!” he cried. “The game is afoot. Not a word! Into your clothes and come!” – Arthur Conan Doyle, The Adventure of the Abbey Grange.
The majority of eDiscovery work takes place in the context of litigation, but a significant amount of it takes place instead in the context of investigations. Although the available ESI and the available eDiscovery technologies are the same, the realities of handling investigations are different in some ways worth discussing.
Read More
Custodian Interview Tools and Logistics – Conducting Effective Custodian Interviews, Part 3
In planning how to format a script and execute custodian interviews, determining the optimal approach for a particular project requires consideration of the specifics, including: how many potential custodians there are, where they all are, how much time is available, how much money is available, etc. All approaches are one of three types. The first is conversational, where one attorney conducts all of the interviews personally, makes notes as they go, and manually compiles the results for matters of small-to-moderate size. The second is formal, where formats for the script and answer recordation for more complex matters include pre-formatted text documents, spreadsheets, and PDF forms. The last is technology-assisted, for the largest, most-distributed, or most-urgent matters. This survey-style approach can be accomplished using paper forms, electronic forms, or web-based questionnaires.
Read More
New White Papers / Practice Guides
Whether you prefer email, text or carrier pigeons, we’re always available.
Discovery starts with listening.