CEdMA Certification SIG

Transcription

1 CEdMA Certification SIG Thursday July 8, am 9.00 am Pacific Association of Test Publishers Test Security Committee Update Cathy Donath, The Donath Group

2 CEdMA Anti-Trust Warning CEdMA is a trade association which provides a unique opportunity for competitors to meet and discuss and learn from each other. Because of our status, however, we must exercise caution to assure that we protect CEdMA as an organization, as well as each of our individual members, from unintentional violations of the law. Please take a moment to review and familiarize yourselves with our Antitrust Guidelines on our website if you have any questions.

3 Agenda Introduction Background - Security Committee Security Survey Security Issues Strategies to Enhance Security Education and Awareness Security Initiatives

4 Security Committee - Mission Assist testing organizations to protect and enhance the integrity and the value of their assessments Encourage collaborative efforts aimed at establishing, promoting and disseminating industry-accepted test security practices

5 History Established in 2006 under ATP Test Security Summits: 2008 and 2009 Established committees (formerly CWCs) Identified and prioritized test security issues Collaborative efforts to develop solutions CWCs developed work products Standing committee within ATP

6 Committees History Security Survey 2007 and 2009 Test Administration Matrix Security Incident Matrix Messaging Report Security Plan Guidelines IP Theft Report: Guide to Combating Infringement Sites Enforcement CWC

7 Security Committee Goals Provide an environment where groups can collaborate to address security concerns Enable a free exchange of ideas/concepts Partner with other testing associations Deliver webinars/workshops at conferences

8 Security Concerns Theft of intellectual property is pervasive Common threats/ impacts / risks Impact of security issues Brand is compromised Unqualified individuals being certified Financial impact Resources for security planning Detection, investigation and enforcement Costs to replace and republish exams

9 Security Survey Purpose Determine scope and depth of security concerns Understand what organizations are doing Planning and protection Detection of test fraud Remediation Survey FY09 ATP Members, Test Security Members Vendor outreach / invite clients ICE (NOCA), CLEAR, PTC, CEdMA SIG

10 Survey Key Findings Respondents: 163 Most pervasive security concerns: IP Theft Few organizations pursue legal action Interest in collaborative efforts Practices to manage security risk Security education and awareness

11 Survey - Top 2 Recommendations Adoption of technical best practice guidelines for managing security risk as an end-to-end systems-based approach Promotion of security education and awareness initiatives

12 ATP Security Committee Initiatives Testing professionals collaboration

13 ATPSC 2010 Initiatives Enforcement Messaging Security Practices Security Survey Security Plan Guidelines ATP Website (retire Wiki/new website) ATP 2011 Conference planning

14 Protecting IP Identify threats Copyright Exams Internet / Media Monitoring

15 Security Measures Explore and learn methods used by others Practices: what worked/ what didn t work Effective messaging Proposed actions Share your experience successes Implementing policies and procedures Messages for different audience/mediums

16 Intellectual Property Theft of intellectual property Security breaches Candidates memorizing items Use of unauthorized materials Distribution of IP - exams/items Infringement sites selling exams Brain Dump websites, E-Bay, Facebook Unauthorized training providers / study guides Proxy test taking

17 Exam IP Theft - Candidates buy exams/ items - Infringement sites - Unauthorized study guides - Unauthorized training providers - E-Bay - Candidates share information - Blogs, Forums, Facebook

18 Background - IP Theft CWC Focus on combating exam theft and cheating IP Theft Report Identification (detection) Reaction Prevention

19 IP Theft Security Practices Methods to detect exam infringement Web crawls, Alert mechanisms Hot Line/ Tip lines Confirm infringement Purchase exams/study guides Determination Inappropriate use of trademark/ logo Are questions substantially similar Actions regarding infringement sites Notification to Google AdWords / payment providers ebay: notification/ report a listing

20 IP Rights Enforcement Contact infringement entity Send cease and desist letters Send DMCA (take down notice) web host providers File infringement report Notify ebay of infringement (VeRO program) Contact payment processor, e.g., PayPal Engage infringing entity in live chat session Start a session / Communicate program terms

21 Prevention Copyright exams Pre-screening and applicant tracking Publish multiple exam forms Build item pool to refresh exams Create new content and refresh exams Include non-scored items to pilot items Consider Adaptive Testing Methodologies

22 Security Planning Identify plan and policies Access to data Limit access- test items; item management system Storage and security access Distribution of files Security leak protection measures Enforcement actions Candidates Infringement sites

23 Results Data Analysis Test Scores / Test Results Identify score aberrance and irregular results Use of data analysis to detect aberrant scores Candidates complete exam in unusually short time Candidates with high scores, e.g., >95% Policies regarding score aberrance

24 Test Security Innovations Incorporate stealth or trojan items Items that nearly everyone knows correct answer Included as non-scored with miss-keyed answer Method to flag those who memorized correct answer Pull exam results to flag candidates Do not grant candidate certification Implement policies regarding aberrant scores

25 Security Practices New topics / recommended practices What s the issue / area of concern What are recommended practices / actions Approach depends on organization s situation Include one or many examples Provide example, reference documents Sample correspondence related to proposed action Cease and Desist letters

26 New Content /Topic Areas Copyright exams Investigation methods Enforcement considerations Legal DMCA / Copyright Pre-litigation actions Exam delivery policies Test centers: CompTIA / others International Data Piracy - (example: GMAC) Data forensics Proxy Testing

27 Legal Considerations DMCA - Digital Millennium Copyright Act (DMCA) What it is / How to proceed Reference document on website Copyright exams Why it s needed Where you can find info, e.g., website link Procedures Member offered to submit this information Will compile info with input from colleagues

28 Messaging Security Education & Awareness

29 Security Messages Audience Test takers age/ geography Test center personnel, proctors, trainers Exam developers, subject matter experts Communicating effectively tone, language Medium Candidate agreement, registration application Website, bulletin, blogs Test centers Press release (news)

30 Messages Exam Policies Candidate testing policies Misconduct at test center Exam scoring Indeterminate or aberrant scores Candidate retesting Proxy test taking Consequences of cheating, exam fraud Exam misconduct Legal consequences Certification revocation Provide Tips/Hot Line

31 Messaging

32 Proxy Test Taking

33 Proxy Testing Messaging committee: Suggested messages to candidates Test centers, website, etc. Security Practices: Pursue proxy testing service providers, e.g., purchase services, etc. Identify recommended steps to take

34 Enforcement Initiative Target infringement sites Develop a plan to address and facilitate a collaborative effort of stopping internet test piracy and distribution Engage pilot group Conduct pilot program Security vendor services Survey community interest Areas of vendor services

35 Security Initiative Committees Enforcement Messaging Security Practices Security Survey Security Plan Guidelines

36 Questions? Feedback on Initiatives Discussion Your Experience

37 ATP Security Committee Volunteers/Contributors Ideas generated to benefit organizations Collaborate to develop solutions Resources: LinkedIn Group: Test Security Group ATP website: postings work products; forums ATP conference security track Community sharing

38 Security Committee Cathy Donath, Chair The Donath Group, Inc. Ashok Sarathy, Co-Chair Graduate Management Admission Council Joe Cannata, Secretary Brocade