MaNGA Technical Note Finding Fiber Bundles in an Image

Transcription

1 MaNGA Technical Note Finding Fiber Bundles in an Image Jeffrey W Percival 05-May-2015 Contents 1 Introduction 2 2 Description of the Problem 3 3 Hexagon Nomenclature and Methods Ranks and Coordinate Systems Numbering Schemes V-Groove Blocks Overview of the Analysis Sobel Operator Hough Transform Blob Detection

2 4.4 Spike Detection Absolute Orientation Problem Motor Stage Alignment Summary 15 1 Introduction The University of Wisconsin Fiber Test Stand Project is an effort to build a highly automated fiber throughput measurement system to be used in the AS3 MaNGA project. MaNGA is the Mapping Nearby Galaxies at APO survey of the Sloan Digital Sky Survey III. See future/manga.php. The items being tested are bundles of fiber optics, arranged at one end (the front end) into a packed hexagonal shape, and at the back end into a linear arrangement suitable for feeding a spectrograph. The test stand consists of a stabilized point source of light at the front end, and a sensitive photometer at the back end. A motorized XY-stage at the front end, holding the bundle, brings each fiber, sequentially, under the point source. At the back end, a motorized XY-stage holding the linear group of fibers is moved around in front of the photometer, searching for the illuminated fiber, and the photometer measures the transmitted light. Before-and-after measurements of the stabilized point source provide a baseline against which the fibers are compared. This measurement acts as an Acceptance Test Procedure (ATP) for the bundles manufactured by the C- Technologies Corporation ( as well as providing calibration data for the survey measurements at the Apache Point Observatory ( At the front end, the front surface of the fiber bundle sits at the focal plane of an Ethernet camera with a sensor of 1624x bit pixels. A beam splitter 2

3 Figure 1: Hexagonal array of polished fibers, and point source illuminating the central fiber sends the image of the point source into the camera as well, so the camera can see the spot falling onto the packed hexagon of polished fibers ends. A light at the back end can illuminate all the fibers at once, allowing the front-end camera to see the whole bundle of fibers at once. See Figure 1. 2 Description of the Problem The tasks necessary for the front end control are: Image the bundle Recognize the orientation of the fiber bundle (center position, rotation, scale) Locate each fiber to sub-pixel precision 3

4 Hexagon Rank Number of Fibers Table 1: Hexagon Ranks and Number of Elements Calculate the deviation of each fiber from its position in an ideal hexagon Calculate the linear transformation between the image coordinates and the motor stage coordinates (zero point, scale, and rotation) Provide precise bundle movement for spot positioning Allow for mapping one fiber numbering scheme to another The scheme for locating the fibers in the image must be robust in the presence of variations in fiber brightness due to differences in opacity and illumination, blemishes in the fiber cores, and perturbations of fiber position due to packing irregularities. We chose a multi-pass set of algorithms performed in a parameter space that is insensitive to these effects. 3 Hexagon Nomenclature and Methods 3.1 Ranks and Coordinate Systems We use the term rank to describe the size of the hexagon. Table 1 shows the number of fibers for each value of rank. Two coordinate systems suggest themselves when describing hexagonal arrays. See Figure 2. It is convenient to switch between them as needed. For 4

5 Figure 2: Cartesian and Hexagonal Coordinate Systems the standard Cartesian coordinate system we use X and Y in units of fiber radius (in packing, radius refers to the total fiber, core + jacket; in the image analysis below, radius refers just to the optical core). The A-B system uses units of fiber diameter. The A-B hexagonal coordinate system is nice because each fiber position can be represented as a pair of integers. As an additional convenience, the fiber numbers falling on the +A axis correspond to the number of fibers interior to that rank. Fiber (4,0) is the rightmost fiber in a rank=4 hexagon, while fiber (-2, 4) is at top center. In the Cartesian system, the Y coordinate is given in multiples of the algebraic irrational 3. The conversion between the two coordinate systems is x = 2 a + b (1) and y = b 3. (2) 5

6 (a) Spiral Numbering (b) C-Tech Numbering Figure 3: Hexagon Numbering Schemes 3.2 Numbering Schemes We use two fiber numbering schemes. The most natural scheme designates the central fiber as 0, and sequentially numbers each layer of fibers as the rank increases. This scheme has the advantage that a fiber s numerical assignment doesn t change as more fibers are added. We call this spiral numbering (Figure 3, left). A different scheme is convenient when manufacturing the bundle. This scheme numbers the fibers sequentially as they are added to the V-Groove Block (see next section). They are sequential in the linear array at the back end. At the front end, the fibers take on a serpentine numbering, moving back and forth across the hexagonal array. We have fiber maps for all ranks and both schemes at html/documents/software.shtml. 3.3 V-Groove Blocks For smaller bundles, the fibers are arranged at the back end in one line, in a V-Groove Block. This is a machined piece with one groove for each fiber, with all the fibers captured by a top piece. For the larger bundles, one block is not sufficient. For those bundles, the fibers are apportioned across several blocks. For a rank=6 (127 fiber) bundle, 4 V-Groove Blocks are used, the 6

7 Spacing Code A B C Block Spacing mm mm mm Table 2: V-Groove Block Spacing Codes Bundle Size Rank=2 (19 fibers) Rank=4 (61 fibers) Rank=6 (127 fibers) V-Groove Blocks A19 A30, A31 A37, A30, B30, C30 Table 3: Bundle Sizes and V-Groove Block Assignments first with 37 fibers, and the other three with 30 fibers each. The fiber maps use colors to designate different V-Groove Blocks. The V-Groove Blocks are named according to their groove spacing and number of grooves. Table 2 lists the V-Groove block spacings encountered in the MaNGA bundles. Table 3 gives the V-Groove Block assignments for MaNGA bundles. 4 Overview of the Analysis The fundamental problem we address is determining the location of each fiber in the camera s image to sub-pixel accuracy. Figure 1 shows that the fibers present themselves as sharply defined illuminated circles. In image space, considering the brightness value v as height in the 3-dimensional space of (x, y, v), the fibers can initially be considered to be flat-topped right cylinders. In such a space, the fiber positions can be measured with a simple brightness-weighted centroid calculation: p = vi p i vi, (3) 7

8 where v i is the brightness value measured in an 8-bit greyscale (0-255), p i is the vector position of pixel i, and p is the measured centroid. Various real-world considerations complicate this simple view. These considerations are: Light leaks These can create illumination smudges in the image, or gradients across the sensor that can bias a brightness-weighted centroid away from the true center of the fiber Fiber Blemishes Fibers can sport irregularities in their imaged surfaces: nicks due to polishing and handling, or irregularly-shaped regions of low or no throughput. These blemishes will weigh in during the centroid calculation. Dark blemishes will push the calculated centroid away from themselves. Additional sources of light The calibration spot can fall on the image, either on a fiber (see Figure 1), between two fibers, or somewhere outside the bundle but still in the field of view. This spot is not as well formed as the illuminated fiber surfaces, and will either bias the centroid calculation (in the opposite sense of a dark blemish) or masquerade as a fiber far out of position. The considerations led us away from using the simple brightness-weighted centroid as a solution. In addition, our experience in doing geometrical calculations in the image domain (for the reasons just described, among others) drove our attention to the many delights of working in transform spaces. To that end, we use the following steps in determining the locations of fibers and the orientation of the bundle: Sobel Transform This is a discrete differentiation operator, essentially a 2- D first derivative of the image. This removes bright regions, brightness gradients, and other artifacts of low spatial frequency. It turns filled circles into rings. See Figure 4(b). Hough Transform This is a feature extraction algorithm that can be made sensitive to the objects we desire (illuminated circles of a given radius) and insensitive to undesired objects (stray spots and circles of undesired 8

9 radii). This essentially uses circular rings to triangulate on their center. See Figure 4(c). Blob Detection This is a form of convolutional sharpening. The Hough transform leaves us with shapes with known properties in the transform space. We use the Laplacian of Gaussian algorithm. We convolve the image with a Gaussian kernel of an appropriate extent, then apply the Laplacian operator. This produces a strong response to features produced by the fibers. See Figure 4(d). Spike Detection Finally, we end up with a transform space consisting of bright spikes representing fibers amid a general confusion of low-level noise. We seek out spikes having a significant height in units of meansubtracted standard deviations. Absolute Orientation Problem (AOP) The Absolute Orientation Problem takes two lists of points, and finds the similarity transformation parameters (rotation, translation, and scaling) that give the least mean squared error (Umeyama, complete reference below) between these two lists. In other words, given the pixel coordinates of the found fibers as imaged through the camera optics onto the sensor, and the AB-space coordinates of an ideal hexagonal bundle (see Figure 2), what is the best transformation between the camera s pixel space and the ideal AB space? Note that this solution allows us to determine the small perturbations of fibers from their ideal locations. The residuals can be computed between the actual, measured fiber positions and their position in an ideal bundle as determined by the best fit solution to all the fibers. 4.1 Sobel Operator The Sobel Operator is a discrete differentiation operator (see wikipedia.org/wiki/sobel_operator) implemented as a two-step 2-D convolution using the kernels G x = (4)

10 Figure 4: (a, upper left) A rank-2 (19 fiber) bundle with the point source falling on the central fiber. (b, upper right) The image after the Sobel (derivative) transform has been applied. (c, lower left) The image after the circular Hough transform has been applied. (d, lower right) The image after convolutional sharpening. and G y = (5) with the final pixel value given by G = G 2 x + G 2 y (6) 4.2 Hough Transform The Hough transform finds circles in the image. See org/wiki/hough_transform. 10

11 If you don t know the radii of the circles, you can find that too, but it s computationally much more expensive. We use our knowledge of the fiber radii as an input, and use the algorithm only for the centers. We use a voting procedure. The voting table is an array of integers the same size as the image, initially zero. We move though the image, examining each pixel. If the pixel is illuminated, then we start casting some votes. (You can use your own criterion for illuminated ; we just use pixels with non-zero counts.) This illuminated pixel could be on the circumference of many circles. Each such circle would be centered one radius away from the pixel being examined. So, we vote for each such circle by incrementing all the pixels in the voting table that are one radius away from the current pixel. If (x, y) are the coordinates of the current pixel, then all pixels (a, b) satisfying (x a) 2 + (y b) 2 = r 2 (7) represent the possible centers of circles containing the pixel (x, y). For each pixel (x, y), sweep through the coordinates a = x r to a = x + r. Then compute b = y ± r 2 (x a) 2 (8) and increment the locations (a, b) in the voting table. See Figure 4(c). Note that this is the voting table, not actually a sensor image any more. The fuzziness of the peaks is due to non-sharp edges, fiber blemishes, and other defects in the sensor image. 4.3 Blob Detection A standard technique for blob detection is the Laplacian of Gaussian. We do a convolution of scale t followed by summing second partial derivatives to produce a string response for structures of scale 2t. See wikipedia.org/wiki/blob_detection. We use a convolutional kernel of K(r, t) = 1 r 2 exp 2πt2 2t 2 (9) 11

12 with t = r 2 0/2, (10) with r 0 chosen here to be 11 pixels. For the Laplacian operator, we make two passes with the differentiation operator given in Equation 4, and then two passes with the operator given in Equation Spike Detection After doing the Sobel, Hough, and convolutional sharpening transformations, we have two kinds of detections: actual fibers, and false positives caused by the echo circles of false votes (see Figure 4(d)). Sometimes the false positives have higher peaks and fluxes than the solutions for the fainter fibers. The false positives arise from the intersection of echo circles. Imagine two intersecting circles, like in a Venn Diagram. At the two crossing points, the circles add to produce a local flux spike. We want to reject these spikes, and will do so by looking at their local structure. The false positives have extended structure from the adjacent circles. They do not fall off in all directions like a Gaussian. To detect this, we will measure the flux in an annulus around the spike; too much flux in the annulus, and we reject the spike. Examining a few images gives us a hueristic: The ratio of the annulus flux to the total flux appears to be much less than about 0.25 for Gaussian-like spikes, and much greater than 0.25 for the false positives. We will parameterize this value (z crit ) so we can change it as needed. Another issue is that even after sharpening, the Hough peaks may be cratered, multi-peaked, and so on. Peak finding sometimes finds two peaks in close proximity, representing the same fiber. We try to exclude these contaminators, and as a final measure when adding a new spike, we check the list to see if the new spike is too near any spike in the list, and leave only the brighter of any pair we find. We use a Keep Clear Distance of 10 pixels for this. We compute the mean µ and standard deviation σ for the image, and the examine each pixel with a value greater than some Nσ. We choose N =

13 For the annular flux, we compute two fluxes, f 1 = f(r 1 ) and f 2 = f(r 2 ) where r 1 = 11 pixels and r 2 = 9 pixels. Our measure of the annular flux is and if z = (f 1 f 2 )/f 1 (11) z > z crit (12) we reject the candidate. 4.5 Absolute Orientation Problem For a complete description of the AOP, see Umeyama, S IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 13, No. 4, April, The AOP applies when you have a set of points represented in two different coordinate systems, with a mapping from one to the other consisting of a scale change, a rotation, and a shift. S = c R(θ) H + S 0 (13) The AOP operates on the two lists of coordinates, and produces the estimate of c, θ, and S 0 that minimizes the mean square error in the transformation. We have two lists: the ideal hexagon fibers (see Figure 2) and the detected spikes. Feeding these lists into the AOP algorithm, we extract c, θ, and S Motor Stage Alignment We assume that the motorized X and Y axes are perpendicular to each other, as are the camera pixel axes. We do not assume, however, that the motorized axes are aligned with the camera pixel axes, have the same scale, or the same zero-points. We use a simple linear transformation to relate displacements on the sensor to movements at the motor stages: M = k R(α) S + M 0 (14) 13

14 where k is the motor constant, given in mm/pixel, α is the angular misalignment, and M 0 is the position of the origin of the sensor (pixel 0) in the motor coordinate system. The Thorlabs APT stages we use provide precise motions in mm, but there is optical power in the imaging system that makes the motor constant k 1. We measure k, α, and M 0 using our stage calibration procedure. We capture two sensor images, changing the motor stage position between them. We have and M 1 = k R(α) S 1 + M 0 (15) M 2 = k R(α) S 2 + M 0 (16) S 1 and S 2 represent the pixel position of some fiducial (e.g. a particular fiber, or the AOP solution for the bundle) at each of the two motor stage positions. The as-yet unknown offset disappears in the difference M = M 2 M 1 (17) = k R(α) ( S 2 S 1 ) (18) = k R(α) S. (19) We estimate the motor constant k with the vector magnitudes as k = S M (20) and the misalignment between the sensor coordinates and motor coordinates as cos(α) = ˆM Ŝ. (21) Finally, we have two estimates of the offset, given by M 0 = M 1 k R(α) S 1 (22) 14

15 and M 0 = M 2 k R(α) S 2 (23) One can measure the packing errors by passing the ideal positions through the forward AOP, or by passing the detected positions through the reverse AOP, and computing the radial deviation from perfect packing. 5 Summary These algorithms succeed in identifying fibers, computing their coordinates, and positioning the motor stage in front of the point source of light. The image reductions are performed by C programs. Each step of the chain is performed by a single-purpose C program, and they can be sequenced using Unix pipelines, or with file operations storing the results of individual steps. We provide a Python interface routine for each C program, allowing easy access to the results in Python. In our Fiber Test Stand implementation, the analysis is managed by a toplevel Python program. 15