Picking. Lecture 9. Using Back Buffer. Hw notes. Using OpenGL Selection Using Back Buffer

Transcription

1 Picking Lecture 9 Picking Image Processing Using OpenGL Selection Using Back Buffer Hw notes Loop Boundary - ignore edge vertex near boundary rule. use them same edge vertex rule. Make sure to use the second data structure we talked about - based on half-edges Any other questions? Using Back Buffer Need to be in double-buffer mode When the user picks an object scene gets drawn in back buffer - object identifiers in colors Front Back Read the pixel under mouse - R,G,B values gives object number

2 In draw(): if in picking mode 1.disable lighting/textures 2.pick a color for each object ID. 3.draw object In mouseclick() 1. select back buffer 2. draw scene in picking mode 3. read pixels under mouse 4. revert value to integer In draw(): if in picking mode 1.disable lighting/textures gldisable(gl_lighting); 2.pick a color for each object: e.g. unsigned int address=(unsigned int) face; GLubyte *bytes = (GLubyte *) &address;!! glcolor4ub(bytes[0],bytes[1],bytes[2],bytes[3]); 3.draw object In mouseclick() 1. select back buffer 2. draw scene in picking mode 3. read pixels under mouse 4. revert value to integer gldrawbuffer(gl_back) glreadpixels(x,h-y,1,1,gl_red,gl_unsigned_byte,&value[0]) uint* ptr = (uint *)& value; int faceno = ptr[0]; Properties Automatically picks object in front Very easy to implement User never sees false colors - in back buffer (one should never swap buffers) Need to set colors using ubytes(avoid rounding) If static image, can keep a copy of oncedrawn backbuffer instead of re-drawing it with each click. Using OpenGL Selection 3 modes of opengl: 1. Render 2. Select 3. Feedback We ll use selection mode for picking Drawing information returned to application instead of frame buffer Screen stays put

3 Using OpenGL Selection 1) Enter selection mode 2) Designate a small pickable region around the mouse 3) "Draw" the scene also assigning "names" to objects 3) Leave selection mode and enter render mode - info will be returned 4) Process the returned information. In drawscene() glmatrixmode( GL_MODELVIEW ); glclear( GL_COLOR_BUFFER_BIT GL_DEPTH_BUFFER_BIT ); glcolor3f( 1.0, 1.0, 0.0 ); glpushname( OBJ_NAME ); drawobj(); glpopname(); glflush(); In mouseclick() call : checkhits(x,y) glselectbuffer( BUFSIZE, selectbuf ); glrendermode( GL_SELECT ); glinitnames(); glmatrixmode( GL_PROJECTION ); glpushmatrix(); setupcameraview(); glmatrixmode( GL_PROJECTION ); glgetdoublev( GL_PROJECTION_MATRIX, matrix ); glloadidentity(); glgetintegerv( GL_VIEWPORT, viewport ); glupickmatrix( (double)x, (double)( viewport[3]-y), X_PICK_SIZE, Y_PICK_SIZE, viewport); glmultmatrixd( matrix ); drawscene(); glmatrixmode( GL_PROJECTION ) glpopmatrix(); hits = glrendermode( GL_RENDER ); processhits( hits, selectbuf ); In ProcessHits() namestackdepth = *ptr++; zmin = *ptr++; zmax = *ptr++; nearestz = zmin; for( j = 0; j < namestackdepth; j++ ) { currname = *ptr++; } nearestcurrname = currname; //if more than 1 hit for( i = 1; i < hits; i++ ) { namestackdepth = *ptr++; zmin = *ptr++; zmax = *ptr++; for( j = 0; j < namestackdepth; j++ ) { currname = *ptr++; } if (zmin < nearestz) { nearestz = zmin; nearestcurrname = currname; } }

4 Properties Can be more complicated and/or cumbersome If one needs all objects underneath mouse location, the only way to go. One should do some extra comparisons to front-most object. continuous functions => finite number of bits represent by sampling: store values of function at different points reconstruct values in between when needed e.g. An image Can think of it as a function f : R 2! C Representation: Samples s : Z 2!C e.g. A parametric curve Can think of it as a function f : R! R 2 Representation: Samples s : Z!R 2 Image Processing So far the discretized (pixellated) image was satisfactory What happens if you want to modify the image? shrink, scale - esp. non-integer scale factors? Artifacts!

5 Shrinking Aliasing Naïve 1.5x shrinking : drop 21 out of 3? Slightly different frequency What do we get? Lower frequency appears Shrinking Low sampling rate! low frequency Naïve 1.5x shrinking: drop 1 out of 3 But we want (impossible, not enough samples) Aliasing!

6 Aliasing can show up in different forms Moire patterns Frequency analysis Jaggies original samples reconstructed Temporal Aliasing(wagon wheel effect) Need enough samples, to be more precise, sampling frequency should be twice the frequency of the wave Frequency analysis The key to fighting aliasing is to avoid frequencies we cannot represent Big question: Filtering! When can we reconstruct a continuous signal from samples? Frequency analysis! decompose everything into waves! mathematically convenient to use complex waves amplitude A cos( $ t # ") frequency phase Ae i( $ t#" ) % A(cos( $ t # ") # isin( $ t # ") original samples reconstructed

7 Fourier transform Examples of FT pairs Decomposes into freq. components FT[x]( # $ i% t %) & " x(t)e dt $# Inverse transform reconstructs the function IFT[X](t) & 1 2' # " $# X( %)e i% t d% Gaussian Gaussian Examples of FT pairs Comb function comb(t) % " # n% $# &(t $ nt) box sinc FT of a comb is also a comb sinc(x) = sin(!x)/!x

8 Shannon theorem Image transformation (Rough idea, not mathematically precise formulation) Almost any function or signal can be decomposed into a sum of sine waves with different frequencies: f(t) % " a j exp i( ( jt ) ' j ) j amplitude frequency phase continuous original x(t) sampling discrete x[n] discrete transformed y[n] reconstruction often no control (done by scanner, camera, etc.) Can choose no control (display + eye) A function can be reconstructed from samples if the sampling frequency is at least twice the max. frequency in the FT of the function. Continuous reconstructed y(t) General idea Convolution Keep the part of the signal (image) we can represent (low frequencies). Eliminate the part of the signal (image) we cannot represent (high frequencies). Smooth slowly changing signals have lower high frequency components, smoothing eliminates high frequencies, Underlies algorithms used for sampling, filtering, reconstruction Operation on two functions to get a new function

9 Moving Averages method to smooth a 1D function idea: at any point, compute avg of function over a range extending a distance r (= radius) in each direction. continuous: h(x) = 1 2r discrete : c[i] = 1 2r + 1 x+r x r i+r i r g(t)dt b[j] Discrete Convolution Based on moving averages At each point take weighted averages of neighbors given by a different function r (a b)[i] = a[j]b[i j] j= r Steve Marschner a[j] gives the weight for sample that is distance j from index i where convolution is computed Moving Avgs = Box Filter Continuous Convolution same as before moving average, filter provides weights weights are now function values Discrete Continuous Convolution Continuous to Discrete : Sampling a[i] = f(x i) Discrete to Continuous : Reconstruction by discrete-continuous convolution filter is continuous an example we ve seen : Splines! Steve Marschner

10 2D convolution 2D convolution: q[n,m] % # " " # j%$# i% $# h[n $ i,m $ j]p[i, j] Note: not for any h[i,j] we can implement 2D convolution as a sequence of 2 1D convolutions. Convolution is implemented as 4 nested loops. Typically, in formulas the indices of filters run in both directions from -L to L, where L is an integer. Be careful when retrieving the filter values from an array: you have to convert the range [-L..L] to [0..2*L] Why the frequency domain? Gives the frequency decomposition of a signal. Different frequencies give information about image. convolutions become basic multiplications a 2D FT + filter multiply much faster than convolution in spatial domain. Convolution and Fourier Transforms Fourier Transform of a product of two functions x(t), y(t) is a convolution of their FTs. FT(x(t)*y(t)) = X(") Y("), where X(") = FT[x] ", Y(") = FT[y] " Inverse Fourier Transform of a product of X(") and Y(") is the convolution of x(t), y(t). IFT(X(")*Y(")) = x(t) y(t) Sampling and reconstruction Sampling = multiplication by comb original samples samples Reconstruction (ideal) = convolution with sinc reconstructed x (t) xˆ (t) & & comb sinc(t)x(t) sinc(t) * xˆ(t)

11 Samping and reconstruction Frequency domain Sampling = convolution with comb = replication and shift Xˆ (%) & " X( % $ n# s ) original!! samples Reconstruction = multiplication by box (get rid of extra copies) X( %) & " box( %)Xˆ (%)!! n n Shrinking:problem Shrinking by a factor a< 1 in freq. domain becomes stretching by 1/a!!!! below 1/2 sampling above 1/2 sampling Won t be able to reconstruct correctly = won t see the expected image compare to the original samples!!!! reconstructed!!!! reconstructed Some Convolution Filters Box Filter Hat Filter (linear interpolation) Gaussian Filter (a.k.a. normal distribution) f(x) = B-Spline Cubic Filter 1 (2π) e x2 /2 Catmull-Rom Cubic Filter (interpolating) Mitchell-Netravali Cubic Filter Shrinking: solution Theoretical solution: BEFORE shrinking, remove high frequencies, i.e. multiply by a narrow box shrinking!!! "!!!!!! "!!! Now, there is no overlap, can reconstruct (= see the right thing)!!!! reconstructed!!

12 Image shrinking For simplicity, derive everything in 1D; shrinking of a two dimensional image is done in two steps: first in x direction, then in y direction. Problem: after shrinking have too few pixels to represent all pixels of the original. Solution: do local averaging; similar to the continuous case, but instead of integration do summation. Image shrinking sum these pixels length of arrows indicates pixel values (intensities) Sum up all pixel 1/a values that are covered by the box of width 1/a centered at the 0 1/a n/a pixel that we are n-th pixel computing. If box function of width 1 is h(x), then: box function of width 1/a is h(ax) box function of width 1/a centered at n/a is h(a(x-n/a)) = h(ax-n) add a scale factor a, so that we do not get high intensities: p shrinked [n] $ " h(ai # n)p[i] i Image shrinking Shrinking by a factor a < 1: original image, size w pixel centers target size, aw think about pixels of the target as fat pixels ; the size of a fat pixel is 1/a; the the size of a the (rescaled) target image is aw(1/a) = w = size of the original (but the pixel size is different!) Image shrinking p 0 shrinked 1/a n/a [n] $ " h(ai # n)p[i] i sum these pixels 2l=support size of h(x) n-th pixel Can replace the box by other functions (filters) that result in better images. The formula is the same: We have to sum only over pixels for which h(ai - n) is not zero. If h(x) is zero outside [-l,l], The range for i is determined from - l % ai # n % l

13 Image stretching Stretching by the factor of a > 1. Same approach: make images the same size, use tiny pixels of size 1/a. How do we determine values for points between the original pixels? Need to interpolate, that is, find a continuous function coinciding with the original at discrete values. original image stretched image Interpolation Hat functions are linear on integer intervals; when we sum them we get a linear function with values p[n] at integers: Can use other functions instead of hat(x): Just need:! h(0) = 1 f(x) $ " hat(x # i)p[i]! h(n) = 0 for n not outside equal a certain to zero interval around 0! interpolating function: i f(x) $ " h(x # i)p[i] i Interpolation (Discrete - Continuous Convolution) Image stretching Simplest interpolation: linear. How do we write expression for the interpolating function? Use hat functions (one of possible bumps ). 1 Now we only need to resample the interpolating function at tiny pixel intervals 1/a: n p[n] $ " h( # i)p[i] a i Again, the interval over which to sum is determined by the interval [-l,l],on which the h(t) is not zero: n - l % # i % l a -1 1 Put a hat of height p[n] centered at n. Add all hats.

14 Practical filters Image shifts Try to approximate sinc. At the same time, keep short the interval where the filter is not zero. In additin to box and hat, here is a couple of useful filters: Lanczos filter: Mitchell filter: $ sinc(x) sinc(x / 3), if x % 3 h(x) & # " 0 otherwise. 3 2 $ 7 x * 12 x ) 16 3, if x % 1 1 ' 3 2 h(x) & #* 7 3 x ) 12 x * 20 x ) 32 3, if 1 ( x 6 ' 0 otherwise " % 2 Similar to image stretching: Interpolate, then sample at new locations. p shifted [n] $ % h(n # t " i)p[i] i old sample locations: 0,1,2,,i,.. old sample locations: t,1+t,2+t,,i+t,.. Practical filters Lanczos filter, l=3: Mitchell filter, l=2: Implementation summary To implement resizing (or shifts) 1. create a temporary image. If resizing by factor a in x direction and b in y direction, the temporary image should be a*w w by by bh, if the original was w by h. For shifts, use the same size. 2. resize/shift in X direction using formulas from lectures, computing pixels in the temporary image using pixels of the original image. 3. Create a final image of size a*w by b*h, if resizing, w by h if shifitng. Resize/shift in Y direction, computing the pixels of the final image using the pixels of the temporary image.

15 Implementation summary Image Blurring Do all calculations for red, green and blue components of the image separately, that is, new red values are computed from old red values etc. Represent pixels as floats: the results of filtering can be negative or ouside range; truncate only the final result to this range and convert it to integer. If you need a pixel value that falls outside the image, use zero. Write a function that returns a value for any integer pair of arguments (i,j). If (i,j) is inside the image it returns the image value, otherwise a zero. Always access the image using this function. Original Box Gauss Image blurring Sharpening To blur an image, we do local averaging. Simplest case: p filtered box of size 3 new pixel value here is the average of the three old pixel values. [n] # $ h(n " i)p[i] i Note: we need values of h(t) only at integers. A better example: Idea: to sharpen an image, that is, to make edges more apparent, need to increase the high frequency component and decrease low frequency component. Can be achieved by subtracting a scaled blurred version of the image from the original. The operation can be done using a single 2D convolution by a filter like this: '( 1 1 % % ( 2 7 %& ( 1 ( 2 19 ( 2 ( 1$ ( 2 " " ( 1" #

16 Difference filters To compute % x p[n,m] $ p[n # 1,m] " p[n,m] convolve with filter h[0,0] = -1, h[-1,0] =1, h[i,j] = 0 otherwise. To compute % y p[n,m] $ p[n,m # 1] " p[n,m] convolve with filter h[0,0] = -1, h[0,-1] =1, h[i,j] = 0 otherwise. Edge detection Edge detection Idea: an edge is a sharp change in the image. To find edges means to mark with, say, 255, all pixels which are on an edge. For a continuous image, the places where the intensity changes rapidly, the magnitude of the derivative in some direction is large. Directional derivatives can be approximated by differences: df(x, y) f(x & 1,y) $ f(x, y) ' % f(x & 1,y) $ f(x, y) dx 1 To mark locations where the differences are large, compute differences in two directions, square, and threshold: if % x p[n,m] 2 # % y p[n,m] & threshold _ value set p edge [n,m] to 255, otherwise, to zero. 2 Differences can be computed using filtering.

17 Edge detection example diff. filter in x direction square, sum, take square root and threshold diff. filter in y direction Note: two intermediate images have positive (white) and negative (black) values.