London User Group August 18, 2016

Transcription

1 London User Group August 18, 2016 Analytics What is it and how can I use it?

2 Where is Analytics used? Google Analytics Netflix Twitter Pandora itunes Genius Intelligence agencies High frequency trading National security LinkedIn

3 Analytics in the Legal Spectrum Sedona Principles Number 11 use of electronic tools and processes, such as data sampling, searching, or the use of selection criteria, to identify data reasonably likely to contain relevant information. Commentary to Principle 11 - possible to use technology to search for concepts, which can be based on ontologies, taxonomies, or data clustering approaches, for example.

4 Analytics Every Day Uses Finisar Corporation v. Nistica, Inc., No. 13-CV (N.D. Cal. Dec. 12, 2014). Model order for e-discovery searches were required on specific custodians Order that all be produced. Analytics allows review of all , but efficiently

5 Analytics Defensibility in the Courts Master Paul Matthews approved the use of technology-assisted review in the UK case Pyrrho Investments Ltd. V. MWB Property Ltd., where he wrote, There will be greater consistency in using the computer to apply the approach of a senior lawyers towards the initial sample (as refined) to the whole document set. The U.S. Department of Justice approved the use of Relativity Assisted Review when McDermott Will & Emery was called upon to review millions of documents related to the Anheuser-Busch InBev merger. In Irish Bank Resolution Corp. Ltd v Quinn Justice Fullam wrote, I am satisfied that the proposed protocol will be more efficient than manual review in terms of saving costs and saving time.

6 Structured Analytics

7 Threading

8 2/24/99 11:25 a.m. 4/29/99 6:45 p.m. 4/30/99 9:03 p.m. 4/30/99 7:00 p.m. 4/30/99 7:22 p.m. 4/30/99 10:24 p.m. 5/1/99 12:57 a.m.? Barry Pearce Bob Crane & Jeff Harbert Maria Nartey Richard Sage & Mark Elliott

9 Tie the Conversation Together Using Threading With threading: Review only what you need to Understand custodian communication Keep organised for reviewers

10 Threading What is it? Identifies and arranges s that were part of a single thread or conversation. What is it used for? Allows you to: Easily see the order of each in a thread. See which s are inclusive (i.e. have unique content). Identify duplicate spares (i.e. s with the same content). How will it help me? Sort and organise s by thread for more intuitive review. Saves time if only reviewing the non-duplicative inclusive s.

11

12 Near Duplicates

13 Can you spot the difference? Version A Version B

14 Review Near-dupe Documents at the Same Time Save time when tagging highly nearduplicate documents Set apart new information so you don t have to reread content Verify the consistency of coding decisions for Quality Control

15 Textual Near Duplicate Identification What is it? Identifies documents with highly similar text and places them into relational groups. What is it used for? Allows you to: Use near dupe groups in searching or filtering. Conflict check coding decisions amongst near dupes prior to production. How will it help me? Saves time by identifying very similar documents prior to the start of review. You can also use the near dupe groups for review and QC.

16 Language Identification

17 Quickly Identify the Languages within Documents Gain insight into the languages of all documents in your case Batch documents to the correct review teams

18 Language Identification What is it? Determines a document s primary language and up to 2 secondary languages. What is it used for? Allows you to see how many languages are present in your collection, and the percentages of each language by document. How will it help me? Easily filters documents by language and batch out files to native speakers for review. Determines if translation is needed.

19 Conceptual Analytics

20 What is Conceptual Analytics? Relativity Analytics is a mathematical approach to indexing documents. Terminology is understood based on its usage in your documents. No outside word lists Dictionaries, thesauri, etc. Language-agnostic Term co-occurrence, not term location

21 Document Preprocessing The legal department of a large financial services company a long-standing client of kcura reached out to the company for help with a seemingly straightforward project: e-discovery related to an ongoing investigation. The federal government was demanding documents be turned over quickly, meaning the client had a short time frame to produce relevant information. The client initially identified 100,000 documents belonging to three custodians, and a team of contract attorneys set to work. After reviewers started coding the original data set, however, the government agency s lawyers requested documents from a fourth custodian, who they believed was crucial to the case. Nearly all of the original 100,000 documents had been reviewed, and suddenly a fourth custodian shows up with 200,000 documents, said Adam. Bringing in this single custodian tripled the size of the case. kcura needed to develop a plan of attack for their client, and there was no time to waste. A tight schedule for 100,000 documents now had to accommodate 300,000. Given those numbers, kcura and their client quickly determined that Relativity Assisted Review would be the best fit.

22 Document Preprocessing The legal department of a large financial services company a long-standing client of kcura reached out to the company for help with a seemingly straightforward project: e-discovery related to an ongoing investigation. The federal government was demanding documents be turned over quickly, meaning the client had a short time frame to produce relevant information. The client initially identified 100,000 documents belonging to three custodians, and a team of contract attorneys set to work. After reviewers started coding the original data set, however, the government agency s lawyers requested documents from a fourth custodian, who they believed was crucial to the case. Nearly all of the original 100,000 documents had been reviewed, and suddenly a fourth custodian shows up with 200,000 documents, said Adam. Bringing in this single custodian tripled the size of the case. kcura needed to develop a plan of attack for their client, and there was no time to waste. A tight schedule for 100,000 documents now had to accommodate 300,000. Given those numbers, kcura and their client quickly determined that Relativity Assisted Review would be the best fit. legal department relevant information client initially identified custodian tripled size case kcura needed develop plan attack client waste tight schedule documents accommodate kcura client quickly determined relativity assisted review best fit documents belonging custodians team contract attorneys set work reviewers started coding original data large financial services company company help seemingly straightforward project e-discovery related ongoing investigation federal government demanding documents turned quickly meaning client short frame produce long-standing set government agency s lawyers requested documents fourth custodian believed crucial case original documents reviewed suddenly fourth custodian shows documents adam bringing single client kcura reached

23 Value of Concept Search Avoids term mismatch issues Pop vs. soda Football vs. soccer Avoids intentionally confusing use of language Code words Finds documents even if exact language differs Misspellings Synonyms

24 Keyword Expansion

25

26 Expand Your Awareness of Critical Case Language Quickly uncover important terms that would have taken much longer to find by reading through documents Ensure the list of key terms in your review is complete Understand how newly discovered terms rank in relationship to your search term

27 Keyword Expansion What is it? Uses the concept space to allow users to submit terms and returns conceptually related words What is it used for? Investigating the language of the workspace using known keywords How will it help me? Allows you to find code words Assists in expanding the keyword list Familiarise yourself with the language of the case.

28 Concept Searching

29 Example of a Good Concept Search On January 2, 1997, Mr. Allchin wrote concerning "IE and Windows" that Microsoft needed to begin "leveraging Windows from a marketing perspective" if it was to defeat Netscape. Allchin complained that without leveraging Windows from a marketing standpoint: "We do not use our strength which is that we have an installed base of Windows and we have a strong OEM shipment channel for Windows." Allchin emphasized: "I am convinced we have to use Windows this is the one thing they don't have.... We have to be competitive with features, but we need something more Windows integration. If you agree that Windows is a huge asset, then it follows quickly that we are not investing sufficiently in finding ways to tie IE and Windows together." Using Microsoft's code name, Memphis, for the next version of Windows, Allchin concluded that, "Memphis must be a simple upgrade, but most importantly it must be a killer on OEM shipments so that Netscape never gets a chance on these systems." (MS ) Source: Complaint: U.S. v. Microsoft Corp., 2007

30 Example of a Good Concept Search This is the relevant portion to use for the search

31 Clustering

32 Cluster Browser

33 What is Cluster Visualisation? A component of Relativity Analytics that displays your data as a map of conceptually similar documents, allowing you to: Overlay metadata and coding information on top of clusters to uncover patterns and trends in your data Work faster by getting you to the most important documents first Identify potential coding inconsistencies Investigate and explore large, new data sets View relationships between groups of documents*

34 Circle Pack - Parent and Child Hierarchy Displayed

35 Filters Use filters to select the data you want to display in the Heat Map

36 Document List Document List

37 Clustering What is it? Use the power of the conceptual index to identify groups of conceptually related documents. What is it used for? This can be used as a tool for investigation, analysis, review, or QC. How will it help me? Investigate a large unknown dataset Cull out non-relevant documents quickly Speed up a linear review by batching conceptually related documents together

38 Real World Challenges

39 Challenge #1 You have a discovery deadline quickly approaching. Challenge You were on #1 target for your deadline until you were just dropped with 100 GB of data to review. How will you get through this data in time for your deadline?

40 Challenge #1 Solution You have a discovery deadline quickly approaching. You were on target for your deadline until you were just dropped with 100 GB of data to review. How will you get through this data in time for your deadline? 1. Use your existing coded documents as examples in a categorisation set. 2. Create a search to find documents categorised as Responsive with a rank higher than Batch these out to second level review. 4. Create a search to find documents categorised as Not Responsive and rank greater than 80, or where they are not categorised. 5. Batch these out to first level review. The key is prioritisation.

41 Challenge #2 Your attorney received 5 paragraphs from a subject matter expert depicting potential conversations Challenge among three #1people that corporate counsel believes to be important. How will you find these types of conversations between these three custodians?

42 Challenge #2 Solution Your attorney received 5 paragraphs from a subject matter expert depicting potential conversations among three people that corporate counsel believes to be important. How will you find these types of conversations between these three custodians? 1. Create a search to find these three custodians documents. 2. Create a categorisation set. 3. Add each paragraph as an example, using Text Excerpts. 4. Run categorisation against the documents in Search #1.

43 Challenge #3 You need to QC your production to make sure Challenge #1 no privileged documents go out the door. How will you speed up this process to be as efficient as possible?

44 Challenge #3 Solution You need to QC your production to make sure no privileged documents go out the door. How will you speed up this process to be as efficient as possible? 1. Run Textual Near Duplicate Identification against all documents. 2. Run a search using metadata fields to find privileged documents. 3. Include Textual near duplicates on the search and filter down to find inconsistencies.

45 Challenge #4 You ve already coded your own documents, and you just Challenge received a production #1 from the opposing counsel. You ve been data dumped! How will you find the relevant documents that you need?

46 Challenge #4 Solution (Option A) You ve already coded your own documents, and you just received a production from the opposing counsel. You ve been data dumped! How will you find the relevant documents that you need? Option A: 1. Cluster the received production data. 2. Evaluate the clusters to see if there is any junk that can be quickly eliminated from review.

47 Challenge #4 Solution (Option B) You ve already coded your own documents, and you just received a production from the opposing counsel. You ve been data dumped! How will you find the relevant documents that you need? Option B: 1. Use your existing documents as examples in a new categorisation set. a. Designation, Issues, Privilege, Hot Docs, etc. b. Use the production as the Documents to be Categorised search 2. Cluster the received production documents. 3. Use Pivot and compare each cluster with the categorised issues.

48 Additional Resources

49 Additional Resources Training Centre Documentation Analytics Advice Recipes Customer Wins

50 Questions? Comments?