Collective Intelligence. CAPTCHAs. Eran Hershko

Transcription

1 Collective Intelligence CAPTCHAs Eran Hershko

2 Outline 1) Introduction to CAPTCHA. 2) recaptcha (and Collective Intelligence). 3) How To Break Two CAPTCHAs: EZ- GIMPY & GIMPY. 4) Summery & Future Work.

3

4 CAPTCHA A CAPTCHA is a test that can be automatically generated, which most humans can pass, but most computers can t. Who Uses CAPTCHAs & Why? * e- mail websites- in order to stop spam. * Blogs & forums- in order to stop automatic posting. * Websites that sell tickets- in order to prevent scalpers from buying a lot of tickets. *

5 CAPTCHA EZ- GIMPY Code Completely Automated Public Turing test to tell Computers and Humans Apart

6 Part I Part II CAPTCHA The CAPTCHA achieves two opposite goals: 1) If the CAPTCHA is not broken- there is a way to differentiate humans from computers. 2) If the CAPTCHA is broken- a useful computer vision problem is solved. The CAPTCHA Paradox: A CAPTCHA is a program that can generate and grade tests that it itself can t pass!

7 The Evolution Of CAPTCHA EZ- GIMPY GIMPY ESP-PIX rechaptcha SQUIGL-PIX

8

9 recaptcha Collective Intelligence is a shared or group intelligence that emerges from the collaboration of many individuals. recpatcha uses Collective Intelligence in order to contribute to humanity!

10 Who uses recaptcha? * recaptcha is used by more than 40,000 websites! * Google purchased recaptcha in 2009.

11 How Does It Work? Romeo & Juliet control word Come, come with me, and we will make short work; For, by your leaves, you shall not stay alone Till holy Church incorporate two in one. suspicious word Optical Character Recognition (OCR) I Optical Character Recognition (OCR) II Dictionary Come, come with me, and we will make short work; For, by your leaves, you shall not stay alone Come, come with me, and we will Till holy Ohurch incorporate two in make short work; one. For, by your leaves, you shall not stay alone Till holy Church incorporate two in one. Come, come with me, and we will make short work; For, by your leaves, you shall not stay alone Come, come with me, and we will Till holy Chulch incorporate two in make short work; one. For, by your leaves, you shall not stay alone Till holy Chulch incorporate two in one. Ohurch Church Chulch Chulch

12 How Does It Work? fiery Church fiery Bhurch

13 How Does It Work? fiery Church Church chief Church overlooks Inquiry Church OCR = 1 point = 1/2 point A Suspicious word is Correct if Suspicious word > 2.5 points

14 How Does It Work? fiery Church Church chief Church overlooks Come, come with me, and we will make short work; For, by your leaves, you shall not stay alone Till holy Ohurch incorporate two in one. OCR Come, come with me, and we will make short work; For, by your leaves, you shall not stay alone Till holy Chulch incorporate two in one. OCR The Suspicious word becomes a Control word!

15 How Does It Work? Reject Reject Reject Reject Reject Reject The word is Unreadable!

16 Some Statistics How many humans are required for a word to be considered correct? 3.11% 4.06% * 7.10% 67.87% 17.86% * Including words which are considered unreadable.

17 Why is recaptcha more secure than CAPTCHA? 1) recaptcha uses only words that OCR already failed to decipher. 2) CAPTCHAs generate their own artificial distorted characters. A smart learning algorithm can recognize them. 3) recaptcha has two natural distortions and one artificial: a. The fading of the text in time (natural). b. The noise introduced by the scanning process (natural). c. The added distortion (artificial). Algorithms that succeeded in more than 90% in recognizing CAPTCHA were completely unable to recognize recaptcha!

18 Several Results 50 = 99.1% OCR = 83.5%

19 Several Results * After one year of running: More than 1.2 billion recaptchas were solved! More than 440 million suspicious words were correctly deciphered! recaptcha has successfully achieved its goal in efficiently harnessing Collective Intelligence!

20 Breaking EZ- GIMPY CAPTCHA

21 EZ- GIMPY- How is it done? 1) Choosing a word out of 561 words dictionary. 2) Distorting and blurring its characters. 3) Adding a cluttered and confusing background.

22 The Algorithm This algorithm treats every letter as an individual: The algorithm s steps: Step A & B- Finding individual letters in the image and extracting candidate words. Step C- Choosing the most likely word. requires low computational power requires high computational power

23 Step A Producing a training set: 1) Extracting a letter from a EZ- GIMPY image. 2) Running a Canny edge detection. 3) Sampling 100 points from the letter s interior and exterior edges. 4) Extracting the 2600 (26*100) Shape Contexts. 1) 2) 3) 4)

24 Step A Finding letters in the image: 1) Choosing randomly several sample points from the image. 2) Generating a shape context for each point. 3) Finding the letters from the training set with closest shape contexts. 1) 3)

25 Step B Finding Sequences of letters that form candidate words: For every letter, trying to construct a possible word. There are several constrains: letters must be from left to right, not be too far from each other nor too close and the candidate words must be from the dictionary. profit roll

26 Step C Choosing the most likely word: 1) For each letter, building generalized shape contexts (which assumes many possible deformations in the letters). 2) Giving a score to each letter according to the distance. 3) The answer to EZ- GIMPY is the word with the highest score.

27 Results * This algorithm has a success rate of 83% of the time. collar canvas jewel smile spade soap line here till

28 Breaking GIMPY CAPTCHA

29 GIMPY- How is it done? 1) Choosing words out of 411 words dictionary. 2) Distorting and blurring the characters. 3) Locating the words randomly in the image in 5 pairs (one on the other). 4) Adding a cluttered and confusing background. * The user must recognize 3 words correctly.

30 The Algorithm This algorithm treats every word as a whole and not individual letters: The algorithm s steps: Step A & B- Finding candidate words in the image. Step C- Choosing the most likely words. requires low computational power requires high computational power

31 Step A Finding candidate words in the image: 1) Finding the suspicious places which contain pairs of words. 2) For every pair, conducting edge detection and finding the first two letters and the last two letters, by using shape contexts. 3) Producing a list of the possible candidate words from the dictionary. The result is a list of approximately 4 candidate words. 1) 2)

32 Step B Removing layers of words: 1) Removing the edges of the candidate word from the image and repeating step A (trying to find candidate words). 2) Each pair of words in the image has approximately 16 pairs of candidate words. r o u n d

33 Step C Giving final score: 1) For each pair, producing a synthetic image of the two words overlaid with their estimated locations. 2) Computing the shape contexts of the synthetic image. 3) Every suspicious word in a pair of the original image gets a score according to the distance of its shape contexts from the shape contexts of the synthetic word. 4) The three words with the highest scores are chosen as the answer to the GIMPY CAPTCHA. r c o o u w n d r o w r o u n d

34 Results * This algorithm has a success rate of 33% in guessing the correct three words of GIMPY. * Applying this algorithm on EZ- GIMPY results in a success rate of 92% (The previous algorithm gave only 83%).. true, with, sponge carriage, potato, clock door, farm, important narrow, bulb, right church, tongue, bad sudden, oven, apple

35 Summery 1) EZ GIMPY is successfully broken (92% success). There is still work to be done on GIMPYas a Computer Vision challenge. 2) The reason of recaptcha s success: Solving a recaptcha is an action that people have to do anyway. They feel better when it s for an important cause. 3) The new CAPTCHAs will set new challenges in the Computer Vision field.

36 Future Work The constant battle between Good and Evil The Evil Side: Breaking recaptcha and the new image based CAPTCHAs with a reasonable rate of success. The Good Side: Finding new forms of image based problems that humans can easily solve but computers and computer vision algorithms can t.

37 Questions?

38 References 1) recaptcha: Human-Based Character Recognition via Web Security Measures - Luis von Ahn et al. 2) Recognizing Objects in Adversarial Clutter: Breaking a Visual CAPTCHA - G Mori et al. 3) Shape Matching and Object Recognition Using Shape Context - Serge Belongie et al. 4) Telling Humans And Computers Apart Automatically - Luis von Ahn et al. 5) Breaking recaptcha: A Holistic Approach via Shape Recognition -Paul Baecher et al. 6)

39

40 Related Work From 2011

41 New CAPTCHA Which Uses Empathy