How can search in natural scenes be so efficient? Jeremy M Wolfe (with George Alvarez & Yoana Kuzmova as well as Aude Oliva, Antonio Torralba, et al.

Size: px

Start display at page:

Download "How can search in natural scenes be so efficient? Jeremy M Wolfe (with George Alvarez & Yoana Kuzmova as well as Aude Oliva, Antonio Torralba, et al."

William Malone
5 years ago
Views:

1 How can search in natural scenes be so efficient? Jeremy M Wolfe (with George Alvarez & Yoana Kuzmova as well as Aude Oliva, Antonio Torralba, et al.

2 Things to worry about Will your pedestrian model fail to find this guy because he is too big?

3 Real-world search tasks might be like this Find pedestrians Find a drawer Find a drawer etc What is the set size here?

4 The LabelMe approach Exhaustively label the whole image On each trial, ask about a label. Target present on 50% of trials Use # of labels as an estimate of set size Visit:

5 We can ask about a new item on each trial.

6 Last year s data Maybe Os are learning what this instance of an object looks like Oh, That lamp Maybe Os learn a specific object in a specific setting. Oh, That lamp over there.

7 New data: Was it just that observers knew what the target looked like? Search for Bowl Target was always present. Os clicked on it. 30 trials per scene

9 Second time is still much faster Learning something? First time Second time Order

10 There is very little set size effect msec/item st showing 2nd showing msec/item Set size

11 Why is search for random objects in scenes so efficient? Find the V blueberries A K L A Z N V X T Y M Not so different Very different Hypothesis: Real objects are really different from each other

12 Are objects sparsely represented in the space of the guiding features? Is this why it is easy to find lions among icebergs?

13 No idea (ok, he isn t blue)

14 Let s try a simplified high(ish) dimensional space - 6D So, here is a target It is a Yellow, vertical, straight thing With a Red, oblique, curved part

15 The next slide shows the set (well, most of the set) of distractors that share two features with the target Here are the values for the dimensions % WholeColor: 1=red, 2:green, 3, yellow, 4: blue (cyan) % WholeShape: 1=straight, 2=curve, 3=jagged % WholeOrient: 1=none, 2=vertical, 3=horizontal, 4=oblique % PartColor: 1=red, 2:green, 3, yellow, 4: blue (cyan) % PartShape: 1=straight, 2=curve, 3=jagged % PartOrient: 1=none, 2=vertical, 3=horizontal, 4=oblique n

16 There are a lot

17 Remember, this is the target Try it. It is a Yellow, vertical, straight thing With a Red, oblique, curved part

18 Set Size 10

19 Set Size 20

20 Another set size 20 This is not hard, right?

21 2000 The results Share 4, N 55.6 msec/item Share 3, N 45.8msec/item Share 2, N 35.9msec/item Share 4, Y 27.9 msec/item Share 3, Y 14.0 msec/item Share 2, Y 10.9 msec/item set size

22 So, 6D is quite efficient. Objects live in zillion-d space. Let s search for objects: Lava lamps

23 Actually, we know this is pretty inefficient. But maybe this is just a crowding problem? So, let s do small set size. Crowded and uncrowded.

24 So, look for lava lamps

25 uncrowded

26 Or crowded x Set sizes 1-4

27 Not efficient! crowded 88 msec/item uncrowded 54 msec/item set size

28 Not encouraging for the sparse representation idea crowded 88 msec/item uncrowded 54 msec/item set size

29 This lets us do something interesting with the scene search data st showing 2nd showing Set size First note: The LabelMe estimate of set size is useless

30 Both sets of data st showing 2nd showing crowded uncrowded Set size Plot the scene and object search data together

31 Both sets of data st showing 2nd showing crowded uncrowded Set size Project Scene RTs onto the Object regression line

32 Both sets of data st showing 2nd showing crowded uncrowded Set size Compute the effective set size

33 The effective set size means that if you search for candles 2500 Both sets of data st showing 2nd showing crowded uncrowded Set size something reduces the scene to the equivalent of, say, 6-16 objects

34 The effective set size is determined by an interaction of scene and target. Chair 667 +/- 50 msec Sink 667 +/- 49 msec The fastest (effective set size ~1)

35 The effective set size is determined by an interaction of scene and target. Radio /- 284 msec Stove top /- 49 msec Very slow in the same images (effective set size ~20)

36 The effective set size does NOT seem to depend on the observer Hi Aude I know where you have been looking, too without an eye tracker Avg. RANK across Os For each of 225 images Rank x Avg Rank RANK of AVG RANK of Avg. RT For each of 225 images Everyone does more or less the same thing

37 What matters? Area Size matters Avg Eccentricity Eccentricity? Not so much

38 What else matters? Scene priors? Scene Structure? Salience? Template Specificity? See you next year!

39 Thanks

x 2 + 3, r 4(x) = x2 1

x 2 + 3, r 4(x) = x2 1 Math 121 (Lesieutre); 4.2: Rational functions; September 1, 2017 1. What is a rational function? It s a function of the form p(x), where p(x) and q(x) are both polynomials. In other words, q(x) something