REFINING YOUR E-DISCOVERY REPORTING. How to cut through the noise and produce relevant reporting

Transcription

1 REFINING YOUR E-DISCOVERY REPORTING How to cut through the noise and produce relevant reporting

2 INTRODUCTIONS Scott M. Cohen Daniel Meyers Scott Reents David Smith James A. Sherer Director, ediscovery & Information Governance Winston & Strawn LLP President, Consulting & Information Governance TransPerfect Legal Solutions Senior Attorney, E-Discovery Cleary Gottlieb Steen & Hamilton LLP Chief, E-Discovery Planning & Logistics Securities & Exchange Commission Counsel, Chair Information Governance Practice Baker & Hostetler LLP

3 METRICS IN THE CONTEXT OF E-DISCOVERY

4

5 CASE STUDY Triggering Event - CID received from regulatory agency Scope - documents, drafts, correspondence, messages, e- mails, policies, procedures, manuals, checks, purchase orders, agreements, databases, etc. Sources - servers, desktops, laptops, archives, hard drives, tape drives, floppy drives (??!!)

6 PHASE I: DOCUMENT REQUEST (CID) RECEIVED Goals: Assess scope and likely burden of compliance (in terms of cost, timing and ability to comply) Drive negotiations over scope Estimate and allocate resources Evaluate resolution strategies

7 PHASE I ASSESSMENT STRATEGIES Evaluate the Four Corners of the request. Date range ESI/paper scope Production deadline Production format Determine if any aspects of the request overly broad and/or overly burdensome

8 PHASE I ASSESSMENT STRATEGIES (cont.) Estimate data volume Custodian count Expected volume per custodian Types and storage locations of ESI in scope Make assumptions for time/cost of EDRM phase activities Make assumptions for potential data reduction percentages (i.e. search/filter/cull and review)

9 CID PHASE I ASSUMPTIONS & METRICS (1 of 2) Low Qty High Qty Custodians GB/Custodian Estimated Total Volume (GB) De-duplication removal (%) 50% 75% Post de-duplication GB Total Search reduction (%) 80% 75% Post Search GB Total Review reduction 60% 60% Post Review GB Total

10 CID PHASE I ASSUMPTIONS & METRICS (2 of 2) TASKS Unit Unit Cost Low Cost High Cost Collection Custodian $250 $2,500 $25,000 Filter GB $30 $600 $30,000 Processing GB $200 $1,000 $5,000 Review (10,000 docs/gb) Hour $45 $15,000 $46,875 Production GB $300 $240 $750 Project management (40-60 hrs) Hour $200 $8,000 $12,000 Hosting (GB, 36 months) GB/Month $20 $200 $500 User fees (30 users, 12 months) User/Month $75 $27,000 $27,000 ESTIMATED TOTAL $54,540 $146,625

11 PHASE II: SAMPLE CUSTODIANS TO REFINE INITIAL ESTIMATES Goals: Determine impact of first round of discussions/negotiations with the requesting party Use of results as the basis for further negotiations or for seeking relief from the court

12 PHASE II: SAMPLING STRATEGIES Identify the priority custodians (i.e., the custodians most likely to remain within the scope of the matter) Collect from sources most likely to contain relevant ESI Process and update metrics on surviving data volumes: Impact of date range filtering Search terms analysis De-duplication Domain name analysis

13 PHASE II: SAMPLING STRATEGIES (cont.) Update review and production metrics: Cost and timing of manual review Batching strategies (i.e. by custodian, date range, subject matter, etc.) Costs and timing of TAR Production volume and cost

14 Search Term Analysis Term Documents Unique Documents Responsive % Agent* 52,041 12,654 4% Hydra 19,581 1,834 26% Wolfgang* 11,705 2,234 45% Agent Ten 7, % Monetiz* 4, % France 4, % Strucker* 3, % Cherbeaux* 3, %

15 9000 Unique Search Term Hits Over Time Unique Search Term Hits

16 Jon Smith Unique Hits by New Custodians Jane Doe Sue Ann Deborah Green David Sugar Unique Hits Gene Tool Harry Wield Jason Turn

17 PHASE III: Goals: Ensure reasonableness of processing Identify documents to prioritize / de-prioritize PROCESSING METRICS

18 PHASE III ASSESSMENT STRATEGIES What does the data tell you is missing? Date restrictions Internal policies Missing hardware Collection problems At what point may you enrich the set and add (or remove) data?

19 PHASE III ASSESSMENT STRATEGIES (cont.) What does the data hint at for form? Additional custodians Additional sources Additional subject matter (word count indicators) What does the data hint at for substance? What do metrics, even at the processing stage, indicate about the case?

20 4,500 4,000 3,500 3,000 2,500 2,000 1,500 1, Gap Analysis Custodian A Custodian B Custodian C Custodian D Number of s Jun-15 Jul-15 Aug-15 Sep-15 Oct-15 Nov-15 Dec-15 Jan-16 Feb-16 Mar-16 Apr-16 May-16 Jun-16 Jul-16 Aug-16

21 PHASE IV: REVIEW & PRODUCTION METRICS Goals: Track and manage review progress & efficiency Assess and control quality of the review Assess completeness and accuracy

22 First Level Review: Second Level Review: Privilege Review: Redactions: Quality Control: Total: Review Progress Tracker Total Reviewed Awaiting Review Additional Expected 0% 20% 40% 60% 80% 100%

23 QC Overturn Rates Reviewer A Reviewer B Reviewer C Reviewer D Reviewer E Reviewer F Reviewer G Reviewer H Reviewer I Reviewer J Reviewer K Reviewer L Reviewer M Reviewer N % Major Change % Minor Change 0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

24 120% Review Completeness Recall 100% 80% Recall 60% 40% 20% 0% 0 200, , , ,000 1,000,000 Documents Reviewed

25 METRICS ABOVE AND BEYOND Metrics can be used to evaluate: Merits outlier patterns in priority custodians communications may indicate abnormal activity or potentially wrongful conduct Search terms results in s identify who is communicating with whom on the relevant topics Adversary productions analysis may identify gaps and indicate potential preservation/production deficiencies