Case Study: Relativity Assisted Review
One Lawyer
The client, a large energy company, was involved in litigation over an environmental matter. Only one lawyer was assigned to the case, overseen by one independent consultant - an especially small team, given the 778GB of data collected for the case. In an effort to save costs, both sides agreed on a list of keywords to help identify relevant documents.
De-duplication culled the client’s original count of 6.6 million documents down to 3 million documents. The list of keywords culled the document count significantly further, down to only 157,000 documents.
However, with overly inclusive keywords like “environment” being used, it was evident there were still many non-responsive documents left in the collection to sort out, and that meant a lot of work remained for only one person to review.
We proposed using Assisted Review. It is a great option for any matter that has a high volume of documents.
We calculated that if the one lawyer had to review all 157,000 documents linearly - strictly to identify relevant documents - the review would take 1,570 hours to complete.
With the lawyer open to alternative processes, the decision was made. The team would use Assisted Review.
The Assisted Review Process
To begin the workflow, our team used the overly inclusive keyword-responsive document set of 157,000 documents.
We generally don’t use keywords in addition to Assisted Review, but in this scenario, it actually fit the bill. It’s important to have a decent amount of both non-relevant and relevant documents to train the software, and after applying the keywords, it still looked like we were left with a good mix of relevant and non-relevant documents.
The lawyer completed three review rounds. Reviewing a random sample of 1,000 documents during each round to train the software on responsive versus non-responsive. For each round, six to seven per cent of the documents were identified as responsive.
From there, the lawyer tested Relativity’s accuracy by conducting a Quality Assurance (QA) round. For this fourth round our team took a statistical sample of documents based on a 95 per cent confidence level and a two per cent margin of error.
This resulted in a sample of 2,226 documents, and the results were consistent, as six to seven per cent of the documents in the sample were coded as responsive.
In addition to the consistent results, the lawyer overturned only 174 of the 2,226 documents in the QA round. In other words, he disagreed with the software’s coding decisions only 7.8 per cent of the time, further demonstrating the computer’s accuracy.
He also made an interesting discovery while reviewing the overturns.
As he double-checked the overturns to see what led the computer to make an incorrect decision, he actually ended up agreeing with the software the majority of the time. He realised his initial decisions were wrong.
With this realisation and the already low overturn rate in mind, the lawyer felt the computer had achieved a consistent level of accuracy for this case, even after just one QA round - and decided to stop review.
Using the logic it learned from reviewing nearly 5,000 documents - three training rounds and one QA round, Relativity categorised the remaining 152,000 documents. In the end, 27,122 documents were marked responsive, 117,635 were non-responsive, and 300 remained uncategorised.
Because of the limited resources, the legal team had very little time to complete this review. But, they were able to get good results quickly using Assisted Review.
Achieving the Results
Assisted Review saved the client a considerable amount of money.
Without Assisted Review, the client would have likely hired a junior lawyer, generally billing at $200 AUD/hour to manually review all 157,000 keyword-responsive documents. Assuming the junior lawyer reviewed strictly for relevance at a rate of 100 documents per hour, it would have taken 1,570 hours to complete the first-pass review and would have cost the client $314,000 AUD.
However, by using Assisted Review, the client’s senior lawyer was able to complete the first-pass review at an estimated cost of $29,000. A savings of approximately $285,000 AUD for the client.
This matter is a great example of the efficiency and cost savings realised by the use of Assisted Review.
To find out more about how Relativity Asissted Review can help you, get in touch with one of our consultants.
Like what you're reading? Click here to subscribe to our latest news and information.