Having the Acuity to determine Relevance with Predictive Coding

Many commentators have lighted on the paper Crash or Soar – Will the legal community accept “predictive coding?” by Anne Kershaw and Joe Howie, in which they explored whether lawyers will be willing to abide by the results of review accelerators, which they group together with the label “predictive coding”.  The article is based on the results of a survey of eleven legal software companies whose applications or services include review accelerators of some kind. Three of those who took part in the survey are companies which I know well, and happy chance enables me to make a plausible title for this article from FTI Technology’s Acuity, Equivio>Relevance and Recommind’s Predictive Coding.

“Acuity” is sharpness or acuteness. “Relevance” connotes bearing upon, pertinent to, the matter in hand. “Predictive” implies foresight and the ability to anticipate. These are good names, therefore, for products or services whose function is to get you to what matters quickly. The Kershaw/Howie article gets its name from the fact that many lawyers are nervous of reliance on any form of automated review, preferring, or at least claiming, to read every document.

Those who advocate human review must address three points: if predictive coding (I will stick with the Kershaw/Howie label for convenience) can save significant costs without significantly reducing accuracy then the burden falls on its opponents to point to its flaws; consistent accuracy by humans – Monday to Friday, morning till night, across multiple reviewers – is impossible to achieve, at least within reasonable time-frames; and even if you could expect such accuracy, you have no way of verifying it without repeating the exercise with a different set of reviewers, whereas (as Kershaw and Howie observe) “predictive coding is based on human-assisted computer analysis, sets of documents can be examined multiple times using different parameters or sample sets”.

Sedona Conference Commentary on Proportionality in Electronic Discovery

As a shortcut to the Sedona Conference Commentary on Proportionality in Electronic Discovery, I refer you to the excellent  K & L Gates Electronic Discovery Law site  which gives a summary of the Sedona Conference’s eagerly awaited commentary on proportionality together with a link to it (login required). I have not yet read it myself, but the K & L Gates summary sets out what may appear to be statements of the obvious, with the general theme being that the costs of any step should be weighed against the likely benefit of taking that step. If they were in fact obvious, then many of us would be out of a job.

You may care to compare the last point “Technologies to reduce cost and burden should be considered in the proportionality analysis” with paragraph 6(2)of the new UK E-disclosure Practice Direction which says “Technology should be used in order to ensure that document management activities are undertaken efficiently and effectively”.

Are we all saying the same thing, then? Of course, but that it is because it is really too obvious to need saying at all. Why would one not use electronic means to handle electronic documents? There are various stages of denial here: at the bottom we have those who think that electronic documents can simply be ignored, dismissing the whole subject with an airy cry of  “it is disproportionate” whilst in fact having not the first clue as to either side – value and cost – of the proportionality equation;  next up are those who accept that electronic documents exist and must be dealt with, but who treat them as if they were paper – by turning them into paper; beyond them are those who embrace the whole subject with gusto but who, in focusing on the wonders of their technology, overlook the clients’ objective which is not, generally, disclosure / discovery for its own sake. There are shades and variations within and between these categories, few of which are guided by any concept of proportionality.

This is a problem common to every jurisdiction which requires disclosure of electronic documents. I have been to all of them, except Canada. Canada is the source of much thought and practical attention to e-discovery, and the link with Sedona is the Sedona Canada E-Discovery Principles. These were released in early 2008 (in both English and French) and were immediately recognised by federal and provincial courts as an authoritative source of guidance for Canadian practitioners. They were explicitly referenced in the Ontario court rules and practice directives that went into effect in January 2010. Applied Discovery has assembled a distinguished panel discussing the subject under the title Understanding Proportionality . The panel include two Ontario judges and is moderated by  Crystal O’Donnell of Applied Discovery. I am very much looking forward to attending this, and will report back in due course.

Home