Gradient based progressive probabilisti Hough transform C.Galambos, J.Kittler and J.Matas Abstrat: The authors look at the benefits of exploiting gradient information to enhane the progressive probabilisti Hough transform (). It is shown that using the angle information in ontrolling the voting proess and in assigning pixels to a line, the performane an be signifiantly improved. The performane gains are assessed in terms of repeatability of results, a measure that has diret relevane for its use in many appliations. The overall improvement in output quality is shown to be greater than that found for the standard Hough transform using the same information. The improved algorithm gives results very lose to those of the standard Hough transform, but requires signifiantly less omputation. 1 Introdution The Hough transform is a well studied method for extrating geometri primitives. In this paper we look at some refinements to a variant of this method alled the progressive probabilisti Hough transform (). Details of this method have been presented previously [ 1-31. The key differene between the and the probabilisti Hough transform (PHT) [4] on whih it is based is that the aumulator spae is sanned for signifiant peaks as eah vote is ast and lines are removed as they are found. When a line is deteted, all edgels that are assigned to the line are removed from the list of unused edgels. Votes of the edgels that have been added to the aumulator are removed. This sheme ompares well in terms of omputational ost with other adaptive shemes [5-71 whih rely on monitoring peaks in the aumulator spae. It also avoids making assumptions about the distribution of peak sizes, and problems that may our if these onstraints are violated. The only parameter the requires to be set is the false positive threshold (I). The threshold ontrols the fration of aeptable false positives that an be generated by the algorithm. The effet of this parameter on the total number of votes required to proess an image is studied in [2]. In essene, the smaller the value, the longer it will take for the algorithm to terminate but the false-positive line detetion will also be lower. In this paper, we investigate the benefits gained by using gradient diretion in onjuntion with the algorithm. Similar improvements are possible with the [S, 91 and as they have been onsidered elsewhere [lo-121, their 0 IEE, 2001 IEE Proeedings online no. 20010354 DOI: 10.1049/ip-vis:20010354 Paper first reeived 25th April 2000 and in revised form 12th February 200 1 C. Galambos and J. Kittler are with the CVSSP, University of Surrey, Guildford, Surrey GU2 5XH, UK J. Matasis with the CVSSP and Centre for Mahine Pereption, Czeh Tehnial University, Karlovo nimksti 13, 121 35 Praha, Czeh Republi I58 performane benefits will not be examined in detail here. We show that with the use of gradient diretion information, the auray of the an be brought very lose to that of the using the same information, while retaining a signifiant advantage in omputational speed. 2 Modifiations to the algorithm The use of gradient diretion does not signifiantly alter the overall operation of the algorithm (algorithm 1). The differenes will be explained later in this setion. Algorithm 1: Progressive probabilisti Hough-/ runsform Step 1. Vote into the aumulator with a single edgel randomly seleted from the input set. If none, then finish. Step 2. Remove edgel from input set. Step 3. Chek if the highest bin ount in the aumulator (the peak) that was modified by the new edgel is higher than threshold thr(n), defined in Setion 2.1. If not, then go to 1. Step 4. Look along a orridor defined by the peak, and find the longest segment of edges that. is either ontiguous or not exhibiting a gap larger than a given threshold. This forms the segment support. Step 5. Remove the edges in the segment support from input set. Step 6. Unvote from the aumulator all the edgels in the segment support. Step 7. If the segment support is longer than the minimum length add it into the output list. Step 8. Goto 1. In using gradient diretion, we make two key hanges to the original algorithm. The first is to onstrain the range of angles, whih has two effets. It redues the omputation required to proess a new edgel; it also redues the lutter in the Hough spae, or, in other words, inreases the signal to noise ratio. Gradient diretion an be made available from several soures. Most edge detetors provide the information as part of their output. If edge information is not available diretly, it is possible to estimate it, for instane by alulating the moments of the neighbouring edgels. IEE Pvo.-Vis. lmuge Signal Proess., Vol. 148, No. 3, June 2001
The seond hange is to use gradient diretion in the post-proessing part of the. When the orridor orresponding to a peak in the aumulator is searhed, only edgels with a gradient onsistent with the line parameters speified by that peak (within the same range used for voting) are onsidered for assignment to the line. This an signifiantly improve the quality of lines generated, both in edgel assignment at T-juntions, and where there are many losely spaed lines. The key fator influening the improvement gained by using gradient diretion is the amount of unertainty of the gradient diretion at eah edgel. Unfortunately this an be very diffiult to quantify theoretially for the whole image apture and proessing hain. Where it is possible, the results would still only be appliable to a single set of equipment. For the purposes of this paper, we will assume the angular error distribution is unknown but approximately uniform for eah edgel. 2. I Changes to the threshold omputation Constraining the number of angles used when voting hanges the distribution of votes in the aumulator, and so the original analysis of the peak detetion threshold for is no long aurate. It is now easier to onsider the aumulator to be a set of 1-D histograms of possible p values, one histogram for eah angle 6 onsidered (the standard p-h representation of the aumulator spae [ 131 is assumed). When a vote is ast only a subset of these histograms is updated. In setting the deision threshold, we assume that all points are due to noise. It is a worst-ase assumption, but if many lines are present in the image the assumption is almost valid, sine only a fration of points belong to any single line. Let us denote the number of histograms for angles as N(] the number of bins for distane from origin as N, and the number votes for a given angle as VH. We adopt the following model of the voting proess. Every randomly seleted edgel votes one into a subset of the available No histograms, the exat bins depending on the gradient diretion of the edgel and the unertainty in its value. The size of the unertainty, denoted by 7, is related to Vn as follows: An edgel an belong either to no line (a noise point), to a single line, or lie on an intersetion,of lines. In the first ase all votes ast add noise to those histograms. We assume that, for every one of the Vo, histograms voted into, a random bin is inremented. Eah bin in a histogram is equally likely to be inremented with probability 1 /N,. If a point lies on a line, one vote is ast into the bin orresponding to this line and the remaining Vu, votes are assumed to fall into random bins, one in eah of the orresponding histograms. For points on line intersetions, we assume V, - 2 votes fall in random bins. Sine Vo >> 1, V,,X V, - 1 x Vu - 2, we do not (and we annot) distinguish between the three ases and assume that always random Vo bins are inremented. Clearly, the p histograms are not independent, and the ounts in bins with similar 0 and p are not statistially independent either, beause of the osine voting pattern of a single point. Nevertheless, to keep omputation tratable, we will assume that the ount in any single bin in a histogram is an independent random variable with binomial distribution B (Vo, p), where Vo is the number of IEE Pro.-Vis. Image Signul Proess., Vol. 148, No. 3, June 2001 edgels that voted in eah p histogram so far and p = 1 IN, is the probability of seleting a partiular bin with a given 6. In our voting model, the distribution of votes in the N, bins for a given H follows the multidimensional hypergeometri distribution (not multinomial distribution, sine the sampling is without replaement). We adopted the B(V(j, p) simplifiation beause we ould not find a pratial (effiient) testing proedure for the hyper-geometri distribution. The hypothesis that is being tested after every bin inrement is the following: Is the ount C(p, 0) in bin (p, 6) higher than a value likely to our if C(p, 0) was a realisation of a random variable with binomial distribution B( Vo, p)? We would like to set the threshold so that Signifiane level 1 is a user parameter that shall, if the model is aurate, indiate the number of false positives (in ase of no post-proessing) due to noise. If there is more than one peak found in voting for a single edgel, the highest is used. For a binomial distribution, it is easy to ompute the threshold for a given N by evaluating the sum for allj till 1-1 is reahed. This value an be omputed relatively effiiently with the inomplete beta funtion [14]. To further speed the proess of voting, these values an be preomputed and stored in a look-up table. 3 Experiments on syntheti data The experiments presented aim to quantify the performane benefits gained by using gradient diretion with the. The main fators for onsideration are the improvement in the interpretation of. the results and omputational benefits. For onsisteny, the stopping riterion of the algorithm exploiting gradient diretion should be modified to take aount of the redued influene of noise on the aumulator statistis. However, to make lear performane omparisons between the modified and unmodified, no hanges were made to the threshold alulation in the first set of experiments. The syntheti images used for these experiments were 256 pixels squared, eah with 20 lines of random length uniformly distributed between 1 and 100 edgels. An example of suh a syntheti image an be seen in Fig. 1. Eah experiment was repeated 100 times. This enables the omputation of the means and the standard deviations for the measured quantities shown in the graphs. All error bars orrespond to one standard deviation. To make the gradient diretion data in the syntheti image realisti, it was estimated by ounting the moments of all the edgels within a radius of 2.5 grid squares. This estimation also works well on real images, and sometimes better as they often ontain fewer rossing lines. The following riteria were used for determining the error statistis. False positives are all those lines deteted that over less than 80% of any single ground-truth line in the image. False negatives are those lines in the model that are overed by less than 80% by the deteted lines, exluding those ounted as false positives. Fig. 2 shows the number of voting operations used in proessing the images. Though voting for a restrited range 159
~ 1=0.1 Fig. 1 Example of syntheti edge image of angles means that the number of voting operations is no longer diretly proportional to the time required for omputation, these numbers an still be used to ompare the relative performane of the algorithm. The test images ontain 2000 edgels, and hene the full uses 2000 voting operations. This means the results for the were obtained with about one-tenth of the operations required by the. As the range of angles used drops below the unertainty in the gradient diretion information, the number of votes needed to proess the image starts to rise for values of y below 30" (Fig. 3). This ours beause, as the range of bins inremented beomes smaller than the unertainty in the diretion, it beomes inreasingly likely that the bin orresponding to the atual line parameters will not be inremented. Figs. 4 and 5 show the detetion performane results for the as a funtion of the gradient angle onstraint. For 15-14 - 13-12 - 11-10 - 9-5 8- g 7- & 6- T 5-4- 3-2- 1-0- -1 ' 0 45 90 135 Fig. 3 with gradient diretion - false negatives false positives omparison, the results of using the same information are shown in Fig. 3. The performane is measured in terms of average number of false positives (Fig. 4) and false negatives (Fig. 5) (undeteted lines) as ompared with the known ground truth for eah test image. We note that when y has values between 30 and 60, the number of false negatives dips signifiantly for the faster version of the with the false positives threshold 1 set at 0.1. At the same time, the false positive rate is signifiantly redued to a level omparable to the operating at the high 1 of lov9. It is important not to set the orientation threshold too tight, as the false negative rate dramatially inreases as the angle unertainty interval approahes zero. Fortunately, the performane urves are reasonably flat for values of y between 30 and.60, and one an allow a suffiient margin to prevent moving into the degraded performane range due to hanges in the image signal to noise ratio. I OO 45 90 135 Fig. 2 number of edgels that voted, using gradient diretion for aumulation I= 10-9 -1 I 0 45 90 135 Fig. 4 false positives, using gradient diretion info for aitmulation I=0.1 I= 10-9 160 IEE Pro-Vis. Image Signul Proess., Vol. 148, No. 3, June 2001
~ I=0.1 ~ false......, I I 60-3 8 $ 40-2 % U 20 - -1-I Fig. 5 lation ob 45 90 135 I _-- I= 10-9 false negatives, using gradient diretion info for aumu- 45 90 135 Fig. 6 Comparison of results from and the on real images using gradient diretion negatives false positives For values of I smaller than lop4, the results for false positives shown in Fig. 4 are fairly similar for the and the, but the number of false negatives in Fig. 5 shows about a 20% drop over the previous results for values of y between 30 and 60". For small values of 1 the overall results for the, at least on syntheti images, are better than those for the. As an be seen from Fig. 2, this extension has little impat on the number of voting operations required to proess the image. 4 Experiments on real images In the first experiment on real images, we ompare the output of the and the on a real image. This experiment illustrates that the improvements give similar benefits when proessing real image data. These experiments were run with the house edge image as used in [9]. It is worth noting that the with gradient diretion information has been tested suessfully with many other real images. The was run with y of 30" to give as near optimal interpretation as possible. The and the use different stopping rules and hene the number of short lines reovered vary signifiantly. To redue problems with this ausing exessive false positives, only lines of 10 edgels and longer where used in the omparison. The was run with an 1 of lop4 whih has been found to give good performane. Fig. 6 shows the results of the omparison. As indiated by the experiments on syntheti data there is an optimal value for y of about 30". This gives a very lose approximation to the results of the. Evaluating the performane of any feature extration routine on real data taken from a omplex environment is diffiult. The idea of 'orret results' for a line detetor annot be defined without some referene to an intended appliation. This an be learly seen when onsidering how to interpret a urve. Depending on the intended appliation, one may wish either to ignore it, or to approximate it with straight lines. For this reason we disard the abstrat onept of orretness and replae it with repeatability. Repeatability is of diret relevane in traking and objet reognition. It is neessary (but not suffiient) that the line extration routine gives a onsistent interpretation of its input data. A measure of repeatability gives a limit to the best performane you an expet of algorithm that depends on a partiular interpretation. The experiments were arried out on image sequenes aptured from a CCIR601 soure, using the intensity value only. The resolution was halved in both diretions by summing pairs of adjaent pixels, from the orresponding positions in the fields of eah frame. Input images were first proessed by a Derihe [ 151 edge detetor. Subsequently linear non-maximum suppression with four-onnetivity was used to find the edgels, and gradient diretion alulated from the Derihe results. Typial edge images from the sequene, from photos in Figs. 7 and 8, an be seen in Figs. 9 and 10. The differene between onseutive images in a sequene is relatively small, but large enough to generate different line interpretations even in a deterministi algorithm like the. Lines extrated from suessive frame were ompared. Every pair of lines whih have both end points within 5 pixels were onsidered as mathes. Only lines 10 edgels and longer were onsidered in the omparison. Table 1 gives a typial set of results from suh a omparison. y = 40 in all experiments. One of the most striking features of the results shown in Table 1 is the improvement in performane made when edgels with full gradient diretion were used. The on both sets of data showed a 50% improvement in the fration of lines that were suessfully mathed. The improvement to the was greater than that seen in the. This brings the performane of the very near to that of the, within 10%. Table 2 shows the exeution time. The implementation for the was kept idential, where possible, with that used for the. The one area of the that was not optimised was the searh of a peak in the aumulator spae. Even allowing that it ould be speeded up many times, the far outperforms the. IEE Pro.-Vis. Image Signal Proess., Vol. 148, No. 3, June 2001 161
Fig. 7 npiul intage Fig. 8 Typial irnuge 162 IEE Pro.-vis. Image Signul Proess.. Vol. 148, No. 3, June 2001
Fig. 9 Example edge images Fig. 10 Example shelf edge images IEE Pro.-vis. Image Signal Proess., Vol. 148, No. 3, June 2001 163
~ " Table 1: Comparison of the and the Data set Method Range Mathes Std. Dev. Ratio Std. Dev. Lines Std. Dev. 57.42 110.06 111.89 145.79 130.51 207.50 211.60 257.11 6.32 5.63 4.70 4.45 7.26 7.66 7.92 6.81 0.41 0.60 0.71 0.77 0.43 0.61 0.67 0.76 0.044 0.031 0.030 0.026 0.025 0.024 0.024 0.018 144 5.24 187 4.04 160 3.16 192 3.20 310 5.82 341 5.51 320 4.82 342 4.82 Table 2: Exeution times for proessing 100 frames Data Method Range Time 55 590 60 1142 148 1349 150 2297,. 0.7 -..~..... : 'T" :,.....^..!........!.A.. %., : ;..... d p..-t' C_., U :,....... *. i.... E l U)._ E 0.5- - %- (T._ r 3 0.4 -..-.. It is interesting to note that there is a little extra omputational ost in using full " diretion information (the differenes arise beause of the effets a larger aumulator on memory ahing). In the, the inreased aumulator size doubles the ost of sanning for peaks, while in the, beause the san is done during voting, the extra ost is negligible. Figs. 11 and 12 show the effet of using gradient diretion on the stability of the sequene of real images. These results agree well with those given on the syntheti data. The best performane for the image was with y set at 40" and for the the value was around 30. These figures orrespond well with the minimum in 0.3-0.20 320 40 60 80 100 Fig. 12 Fration of lines mathed as a funtion of angle width for the data edgels " edgels false negatives and positives seen in Fig. 6. They also support the results on the syntheti data, shown in Figs. 4 and 5. 5 Disussion C._ 3 0.4 r L I 0.2 I 0 20 40 60 80 100 Fig. 11 Fration of lines mathed as a funtion of angle width for the data " edgels " edgels 164 The results of the experiments learly show an improvement in performane for the. They show that when the is used with gradient diretion information, it has a performane similar to the, even where the uses the same information. From Figs. 4 and 5, it an also be seen that algorithm is robust with respet to the onfidene angle interval. When the range is set too low, or the unertainty inreases the uses more votes to ompensate for this missing information. This is important if the unertainty in the angles varies. It allows this parameter to be set at an optimal value, without fear that the algorithm will ompletely fail if angles beome more noisy than usual for a short while. The results on real images summarised in Figs. 11 and 12 demonstrate that the benefits shown on syntheti data extend to the proessing of real image data. With the added stability in the generated output ahieved by the use of gradient diretion, the beomes an attrative hoie for real-time proessing of edge images. IEE Pr0:Vi.s. Image Signal Proess., Vol. 148, No. 3, June 2001
~ ~~ ~~~ ~ ~, 6 Conlusion The simple modifiations shown here notably improve both the auray and performane of the, when gradient diretion is available. It has also been shown that the relative improvement in the is signifiantly greater than that seen with equivalent modifiations to the. Even where gradient diretion is not diretly available, it is possible to use neighbouring edgels to estimate the required information suessfully. The main disadvantage of using gradient diretion is the addition of an extra parameter that defines the unertainty in the edge1 angles. This, however, an be estimated easily by either tuning to optimise performane or by omparing the angles of the edgels to those of the lines they are finally assigned to. Otherwise, the proposed modifiations are easy to implement, and the improvements are gained without any signifiant drawbaks. 7 Referenes 1 MATAS, J., GALAMBOS, C., and KITTLER, J.: Robust detetion of lines using progressive probablisti Hough transform, Comput. Vis. Image Underst., 2000, 78, (I), pp. 119-137 2 GALAMBOS, C., MATAS, J., and KITTLER, J.: Progressive probabilisti Hough transform for line detetion. Proeedings of IEEE Computer Soiety Conferene on Computer vision andpattern reognition, Los Alamitos, Califomia, June 1999, pp. 554-560 MATAS. J.. GALAMBOS. C.. and KITTLER. J.: Progressive orobabilisti Hough transform. Proeedings of British Mahine Vision Conferene BMVC98, 1998, pp. 256-265 KIRYATI, N., ELDAR. Y.. and BRUCKSTEIN. A.M.: A orobabilisti Hough transform, Pattern Reognit., 1991, 24, (4), pp. 303-316 YLA-JAASKI, A., and KIRYATI, N.: Adaptive termination of voting in the probabilisti irular Hough transform, IEEE Trans. Pattern Anal. Mah. Intell., 1994, 16, (9), pp. 911-915 YLA-JAASKI, A., and KIRYATI, N.: Automati termination rules for probabilisti Hough algorithms. Proeedings of 8th Sandinavian Conferene on Image Analysis, 1993, pp. 121-128 SHAKED, D., YARON, O., and KIRYATI, N.: Deriving stopping rules for the probabilisti Hough transform by sequential-analysis, Comput. Vis. Image Underst., 1996, 63, (3), pp. 512-526 O GORMAN, E, and CLOWES, M.B.: Finding piture edges through ollinearity of feature points, IEEE Trans. Camput., 1976, C-25, (4), pp. 449456 PALMER, P.L., KITTLER, J., and PETROU, N.: Using fous of attention with the Hough transform for aurate line parameter estimation, Pattern Reognit., 1994,21, pp. 1127-1 133 10 ILLINGWORTH, J., and KITTLER, J.: A survey of the Hough transform, Comput. Vis. Graph. Image Proess., 1988, 44, pp. 87-1 16 11 LEAVERS, VF.: Whih Hough transform, CVGIP, Image Underst., 1993, 58, (2), pp. 250-264 12 KALVIAINEN, H., HIRVONEN, P., XU, L., and OJA, E.: Probabilisti and nonprobabilisti Hough transforms-overview and omparisons, Image Vis. Comput., 1995, 13, (4), pp. 239-252 13 DUDA, R.E., and HART, P.E.: Pattem lassifiation and sene analysis (John Wiley, 1973) 14 PRESS, W.H., FLANNERY, B.P., TEUKOLSKY, S.A., and VETTER- LING, W.T.: Numerial reipes in C (Cambridge - University Press, 1992) 15 DERICHE, R.: Using Canny s riteria to derive a reursively implemented optimal edge detetor, Int. 1 Comput. Vis., 1987, 1, pp. 167-187 IEE Pro.-Vis. Image Signal Proess., Vol. 148. Rio. 3, June 2001 165