Automatic Pavement Crack Detection by Multi-Scale Image Fusion

Transcription

1 Autoatic Paveent Crack Detection by Multi-Scale Iage Fuion Haifeng Li, Dezhen Song, Yu Liu, and Binbin Li Abtract Paveent crack detection fro iage i a challenging proble due to intenity inhoogeneity, topology coplexity, low contrat, and noiy texture background. Traditional learningbaed approach ha difficulty in obtain repreentative training aple. We propoe a new unupervied ulti-cale fuion crack detection (MFCD) algorith that doe not require training data. Firt, we develop a windowed inial intenity path baed ethod to extract the candidate crack in the iage at each cale. Second, we find the crack correpondence acro different cale. Finally, we develop a crack evaluation odel baed on a ultivariate tatitical hypothei tet. Our approach uccefully cobine trength fro both the large-cale detection (robut but poor in localization) and the all-cale detection (detailpreerving but enitive to clutter). We analyze and experientally tet the coputational coplexity of our MFCD algorith. We have ipleented the algorith and have it extenively teted on two public dataet. Copared with ix exiting ethod, experiental reult how that our ethod outperfor all counterpart. In particular, it increae the Preciion, Recall and F1-eaure over the tate-of-the-art by 22%, 12% and 19%, repectively, on one public dataet. I. INTRODUCTION Paveent ditree are coon failure of road contruction, where crack are the ajor factor and need to be onitored and evaluated reliably and periodically [1] [3]. Traditional anual crack detection ethod are very tie-conuing, labor-intenive, with low-accuracy, and error prone. It i neceary to develop an autoatic crack detection ethod for the accurate detection of paveent crack. However, autoatic crack detection i challenging due to intenity inhoogeneity, topology coplexity, low contrat, and noiy background. A a reult, local iage-proceing ethod, uch a intenity threholding, and edge detection baed ethod can only obtain a et of dijoint crack fragent with high fale poitive rate. Machine learning approache have been propoed, but the election of paraeter depend on crack variation and iage quality. Moreover, the reult Thi work wa upported in part by National Science Foundation under IIS , NRI , NRI , and NRI , by National Science Foundation of China under grant , by Open Fund Project of Fujian Provincial Key Laboratory of Inforation Proceing and Intelligent Control (Minjiang Univerity) under grant MJUKF201732, and by the Fundaental Reearch Fund for the Central Univeritie under grant B006, and by Chinee Scholarhip Council. H. Li and Y. Liu are with CS Departent, Civil Aviation Univerity of China, Tianjin, , China, and alo with Fujian Provincial Key Laboratory of Inforation Proceing and Intelligent Control, Minjiang Univerity, Fuzhou, , China. Eail: hfli@cauc.edu.cn. D. Song and B. Li are with CSE Departent, Texa A&M Univerity, College Station, TX 7783, USA. Eail: dzong@ce.tau.edu and binbinli@tau.edu. (a) Original iage n 2 1 (b) Generated ulti-cale iage (c) Detected crack at each cale (d) Crack fro ulti-cale fuion Fig. 1. An illutration of our MFCD algorith outline. Given an original iage, we apply Gauian blur to generate ulti-cale iage with different tandard deviation σ 1 < σ 2 <... < σ n repreenting different cale, then detect crack at each cale, and finally fue and filter crack to obtain reult. fro thee learning ethod depend on the quality of anually labeled training data et which are difficult to obtain, becaue crack iage have large variation due to different lighting condition, urface type, and background texture. In fact, crack iage exhibit different characteritic at different cale: at a large cale, crack detection i reliable, but it localization i poor and ay i all detail; at a all cale, detail are preerved, but detection uffer greatly fro clutter in background texture. However, the key challenge i how to effectively cobine the trength of different cale to iprove crack detection perforance. To deal with the challenge, we propoe a new unupervied ulti-cale fuion baed crack detection (MFCD) algorith that doe not require labeled training data (See Fig. 1). MFCD copute the axiu average core of crack at different cale. By extracting crack feature for every probable target cale and evaluating the crack jointly acro cale, MFCD fue and filter crack at all cale. Another contribution of thi paper i our iproved inial intenity path election ethod which ha two advantage. One advantage i that it only relie on tatitical paraeter intead of arbitrary threhold, the other advantage i that the ethod ha no prior training requireent which ake it eay to ue in application. We have ipleented our MFCD algorith and extenively teted it on two public dataet in coparion to ix exiting ethod. Experiental reult how that our ethod conitently outperfor the counterpart. More pecifically, our MFCD algorith increae Preciion, Recall and F1-eaure over the tate-of-the-art by 22%, 12% and 19%, repectively,

2 on one public dataet. The ret of paper i organized a follow: we uarize the related work in Section II before we introduce our crack detection proble in Section III. We detail our algorith deign in Section IV and analyze algorith perforance in V. We tet our algorith in experient in Section VI and conclude our paper in Section VII. II. RELATED WORK Popular iage-baed crack detection ethod can be claified a four type: intenity threholding ethod, edge detection ethod, achine learning technique, and orphological ethod. Intenity threholding ethod [], [5] have been widely tudied due to their iplicity. Thee ethod are enitive to noie, leading to unreliable crack detection reult epecially for field iage with ignificant viual clutter under poor lighting condition. Furtherore, electing the appropriate threhold value i challenging. Edge detection ethod [6], [7] are alo widely adopted. However, the ain drawback i that edge detection ethod can only detect a et of dijoint crack fragent and often fail in low-contrat and high-clutter iage. Machine learning technique [8] have becoe ore popular in recent year which include ethod that build on technique uch a upport vector achine [9], [10], rando foret [11], rando tructured foret [12], and neural network [13], [1]. In thee learning ethod, a paveent iage i often divided into a nuber of ub-iage, each of which i repreented by a vector of feature extracted fro thi ub-iage. Thee ub-iage are then ued for training and claification for crack detection. However, ince the training and claification are conducted at each ub-iage and a local ethod, they cannot exactly egent out crack curve over the whole iage becaue it often only provide a label to the ub-iage a it output. Furtherore, a upervied training tage i needed which require accurately labeled data, a difficult requireent for application with large lighting and cene variation. Morphological ethod [15] [17] exploit the connectivity aong crack pixel and have been uccefully ued in paveent crack detection reearch. However, their perforance are uually dependent upon the paraeter choice [18] which require anual extenive paraeter tuning for each data et. Our ethod i a variation of orphological ethod but focuing on reoving arbitrary threhold election and being elf-adaptive to different iage becaue we eploy tatitic paraeter in the threhold etting. More pecifically, our ethod build on the exiting inial intenity path baed technique which find the bet path between pair of endpoint of potential crack. Gavilan et al. propoe a eed-baed approach [19] by cobining ultiple directional non-iniu uppreion with a yetry deck, where eed are linked by coputing path with the lowet ean pixel intenitie that eet the yetry retriction. Kaul et al. [20] propoe a ethod to detect the ae type of contour-like iage tructure with le prior knowledge about both the topology and the endpoint of the deired curve. To avoid fale detection caued by low intenity loop, Ahaz et al. propoe a two-tage inial intenity path election algorith [17] which firt elect endpoint at the local cale and then elect inial intenity path at the global cale. Built on thi approach and to iprove the accuracy and coputation peed, our new ethod eploy technique uch a crack eed clutering & reoval, windowed path generation, paraeter election, and crack grouping & refineent. Our MFCD algorith i alo inpired by ulti-cale analyi ethod which capture the intrinic geoetrical tructure that i key in huan viual perception. Exiting technique, uch a wavelet tranfor [21], particle filter [22], bealet tranfor [23], are eential different type of ulti-cale ethod. The ain challenge of thee ulti-cale analyi ethod i to elect the right cale for identifying ueful feature or how to cobine the detection at different cale to for output. Mot approache take a iplitic cue cobination: they either accept reult (after threholding) at all cale, or accept reult that appear at the coaret cale. Our ethod fue reult fro different cale uing tatitical hypothei tet. A. Auption III. PROBLEM FORMULATION We aue a crack ha a lower intenity value than that of the background iage. Thi auption can be accepted in ot general cae ince crack alway aborb ore light than other area and often appear a dark curve or tape in the iage. B. Notation Coon notation are defined a follow, I, = 0, 1,..., n 1, the digital iage at the -th cale with I 0 denoting the original iage. All iage are in graycale. x i = [u, v]t, the i-th pixel in I, with (u, v) being iage coordinate. C, the pixel et for the candidate crack in I. C, the detected crack pixel et a the algorith output. C. Proble Definition Our ultiate goal i to extract all crack fro an input paveent iage by ulti-cale crack fuion, a hown in Fig. 1. Thu, our crack detection proble i defined a follow, Definition 1 (Crack detection): Given I 0, generate ulticale iage I, = 0, 1,..., n 1, extract the candidate crack C fro each I, then fue C to obtain C. IV. MULTI-SCALE PAVEMENT CRACK DETECTION Crack in the iage tend to have one or ore particular alient cale. To cobine the trength of all and large cale, we propoe a three-tep ulti-cale crack detection algorith a illutrated in Fig. 1. Firt, we generate the ulticale iage by Gauian blurring with different kernel ize. Second, we find the candidate crack at each cale by utilizing

3 crack eed (a) (b) (c) (d) (e) (f) Fig. 2. An illutration of windowed inial intenity path baed crack detection ethod. (a) Original paveent iage. (b) Crack eed extraction reult. (c) Crack eed clutering and filtering illutration. (d) WMIP generation reult. (e) Path verification reult. (f) Crack region detection by path growing. a windowed inial intenity path election baed ethod. Finally, we deterine the crack correpondence acro different cale and propoe a tatitical crack evaluation odel to copute the average core of each crack at different cale for crack fuion. A. Multi-Scale Iage Generation Uing Gauian blurring technique in coputer viion [2], the iage at the -th cale level, I, can be produced fro the convolution of a variable-cale Gauian, G(u, v, σ ), with an input iage I 0. G(u, v, σ ) = 1 2πσ 2 e (u2 +v 2 )/2σ 2. (1) Thu, a et of iage with different cale are obtained with different σ, which erve a the input to the following tep. B. Candidate Crack Extraction at Each Scale A hown in Fig. 2, we propoe a Windowed Minial Intenity Path (WMIP) ethod to find candidate crack at each cale. 1) Crack eed extraction (Fig. 2(b)): Let g(x) be the intenity value of pixel x, and T e be a threhold. We divide the whole iage I into grid cell, denoted a Yi, i = 1, 2,..., n y. Each cell i a et of pixel. In each grid cell Yi, we elect a pixel x i a crack eed when the following two condition are atified: 1) x i i the darket pixel in Yi, and 2) x i i within the top T e percent darket pixel in I. x i = arg in g(x), x (2).t. x Yi, g(x) < g e, where g e i the pixel intenity value of the top T e percent darket pixel in I. The choice of T e i iportant. A bigger T e lead to ore extracted eed which increae coputation load in the following tep. On the other hand, decreaing value of T e ay reove true crack pixel. Our epirical reult how that a T e value of 0.2% reache a good balance between the coputation load and the reulting fale negative rate. A the output of the tep, let u denote E a the et of crack eed. 2) Crack eed clutering and filtering (ee Fig. 2(c)): With E obtained, we ue DBSCAN algorith [25] to group crack eed into cluter, eanwhile, find the iolated crack eed that need to be reoved. Denote the generated cluter a G n, n = 1, 2,..., n c. Thi clutering tep can help u reduce the earching region while finding the WMIP between crack eed in the following tep. x c x b x i (a) Quadrant definition in crack eed cluter v x j u x a G x W i, j n i Fig. 3. WMIP generation. (b) WMIP generation x j 3) WMIP generation (Fig. 2(d)): For each crack eed x i, we define Di be the et of crack eed which need to be connected with x i. We firt find D i and then eploy a WMIP baed ethod to connect x i with each crack eed in D i. To deterine Di, we define the local coordinate yte of x i at firt, with x i a the origin and two axe parallel with the two axe of I, repectively, a hown in Fig. 3(a). Denote q(x i, x j ) a the quadrant nuber of x j in x i local coordinate yte. 1, u j u i 0 and v j v i > 0, q(x i, x 2, u j u i < 0 and v j v i 0, j) = (3) 3, u j u i 0 and v j v i < 0,, u j u i > 0 and v j v i 0. where (u i, v i ) and (u j, v j ) are the iage coordinate of x i and x j, repectively. For crack eed x i, we connect it with another crack eed x j if 1) they are in the ae cluter, 2) their geoetric ditance i aller than the threhold T p, and 3) x j i the nearet crack pixel to x i in it own quadrant, which ean each crack eed can be connected with other crack eed at ot. Suarizing the condition, for each crack eed x i G n, we can write Di a, { Di := x j x j G n, x i x j < T p, x i x j x i x, } x G n, q(x i, x j) = q(x i, x ). () An exaple i hown in Fig. 3(a), where x i i connected with three crack eed: x j, x a and x b. Thu, D i = {x j, x a, x b }. We do not connect x i and x c becaue their ditance i bigger than T p.

4 For x i and each crack eed x j Di, we ue WMIP to connect the. We conider an iage a a bidirectional weighted graph of pixel. Pixel are vertice. If and only if two pixel x a and x b are adjacent in the iage, we build a bidirectional edge between the, denoted a (x a, x b ). Recall the pixel intenity value of x i g(x). The edge weight fro vertex x a to x b i g(x b ), while edge weight fro x b to x a i g(x a). g(x a) and g(x b ) are often not the ae value. We denote Pi,j a a path connecting x i and x j in the for of a equence of pixel, P i,j := {x i, x k, x k+1,..., x k+n, x j}, (5) uch that x i x k = x k+n x j = x k+ x k++1 = 1, for = 0, 1,..., n 1 becaue they are adjacent pixel. We alo define Pi,j := {P i,j P i,j W i,j } a a et of all poible path in the window et Wi,j where W i,j i the rectangular window with x i = [u i, v i ] T and x j = [u j, v j ] T a the two oppoite vertice, { Wi,j := x b in(u i, u j ) u b ax(u i, u j ), } (6) in(v i, v j ) v b ax(v i, v j ). For each pair of crack eed x i and x j D i, the WMIP i the path iniizing the u of the intenitie of pixel along path (ee Fig. 3(b)). Let Pi,j be the WMIP, then P i,j = arg in P P i,j x a P g(x a), where x a repreent a pixel in a candidate path P. We apply Dijktra algorith [26] to olve the optiization proble in (7). Finally, we obtain the et of WMIP, denoted a P, for all (i, j) pair. ) Path verification (Fig. 2(e)): Not all WMIP in P repreent real crack. To reove fale WMIP fro P, a verification tep i perfored by calculating the ean intenity, (Pi,j ), of each P i,j P, g(x a) (P i,j) = x a P i,j (7) P i,j, (8) where et cardinality operator count the nuber of pixel. If the ean intenity of a WMIP i bigger than a given threhold, g, we conider the cae a fale-poitive detection and dicard the WMIP. Threhold g i et a the intenity value of the top T v percent darket pixel in I. 5) Crack region detection by path growing (Fig. 2(f)): The previou tep ay i oe outide pixel due to it focu on connectivity. Thi tep i to aborb neighboring dark pixel to grow path. The neighboring relationhip i not liited to adjacent pixel, two pixel with ditance aller than a threhold are conidered a neighbor. Define L(x i ) := {x j g(x j ) < g(x i )} be the et of pixel whoe intenitie are le than g(x i ), κ be the total aount of pixel in I, Ci be the i-th pixel et for the candidate crack in I. Initially, we et Ci = P i,j. Denote G i a the new found crack pixel et which i generated fro Ci. G i can be coputed a { } Gi := x a x a x b < r, x b Ci, L(x a) < T r, (9) κ where r i the ditance tolerance for path growing, T r i a threhold. After obtaining G i fro C i, we add G i into C i. C i = C i G i. (10) Thi aggregation proce i repeatedly perfored, until there are no further pixel to add. 6) Crack region grouping: One real crack ay correpond to everal crack region in the iage due to noie. We want to group the detected crack region according to their geoetric ditance between each other. The ditance between two candidate crack pixel et, Ci and Cj, can be coputed a d i,j = in x a x b,.t. x a Ci, x b Cj. (11) For a given threhold T g, if d i,j < T g i atified, thi pair of crack region, Ci and C j, are labeled a the ae group. All crack region ay be divided into everal group. For brevity, we ue one crack to repreent a crack group in the ret of thi paper. 7) Sall crack reoval: If a crack ize (total pixel nuber) i aller than a threhold T, the crack i regarded a noie and dicarded. After detecting candidate crack at each cale, we are ready for ulti-cale fuion. C. Crack Correpondence Matching Now let u find the crack correpondence acro different cale. If two crack at different cale correpond to the ae real crack, they are called a a pair of crack correpondence. The proble of crack atching i defined a follow. Definition 2 (Crack atching): Given C i and C j, for each C i i C i and C j j C j, find C i C i i uch that C i correpond to C j n. and C j n C j j, Then, for each C i i C i and C j j C j, we find the overlapping region with a ditance tolerance a the atched egent between C i i fro the pixel x j b d(x j b, Ci i and C j j to crack C i i ) = in xj b. Let u define the ditance a d(x j b, Ci i ), xi a,.t. x i a C i i. (12) We want to find the overlapping region of C i i and C j j within a ditance tolerance a their correpondence. Initially, we et C i = Cn j =. Then, for each pixel x j b C j j, if the criteria, d(x j b, Ci i ) < T d, i atified, we add x j b into Cn j, where T d i a threhold. Siilarly, for each point x i a C i i, if d(x i a, C j j ) < T d i atified, add x i a into C. i When C i are unchanged, we obtain the correpondence C i and Cn j and C j n. The tep above are applied for each pair of crack at different cale until all crack correpondence are found. In the ret of the paper, we renuber all crack at each cale o that crack with the ae ubcript are a group of correpondence.

5 ean and unit variance. Define C i be the union pixel et of the i-th atched crack acro all cale, thu C i = n 1 =0 C i. (16) (a) Original iage (b) Score Fig.. An exaple output of core coputation. (Bet viewed in color). D. Crack Selection and Verification by Multi-Scale Fuion The key iue here i how to optially integrate all candidate crack C to obtain C. Firt, we deign the odel of average core to evaluate the probability of each detected crack fro a true crack. Then, we develop tatitical hypothei tet to reove fale-poitive crack. We propoe a etric/coring echani to evaluate a pixel probability to be a crack pixel. A dark pixel or a pixel with any dark neighboring pixel ha higher probability to be a crack pixel than otherwie. Thu, the core for each pixel i defined a the aggregation between itelf and fro all neighboring pixel. We odel the probability ditribution of each crack pixel to be a weighted Gauian function. The core of a pixel, x a which i defined a (x a), (x a) = w b 1 2πσ 2 e (ua u 2 b) +(va vb ) 2 2σ p 2, (13) x b Ne(x a ) p where σ p i the tandard deviation of the Gauian ditribution, N e (x a) i the neighboring window with the ize of (3σ p +1) (3σ p + 1) centering a x a, and w b i a weight. We adopt the intenity weight in the core coputation. A pixel intenity weight i inverely proportional to it intenity value, i.e., a pixel with a lower intenity i given a greater intenity weight, w b = 1 L(x b ). (1) κ Fig. illutrate an exaple of core ditribution where we can ee that the crack pixel ha a larger core at thi cale. With the core of pixel defined, we can copute the average core of crack Ci a follow, (x a) ρ(c i ) = x a C i C i. (15) With crack average core defined, we perfor tatitical hypothei tet to reove fale-poitive crack. Fro (13) and (15), we know that ρ(c i ) N(µ, σ 2 ) i a rando variable following Gauian ditribution with ean being µ and variance being σ 2. Hence (ρ(c i ) µ )/σ N(0, 1) i a rando variable following the noral ditribution with zero We define function f(c i ) to evaluate the probability of C i to be a real crack or not. f(c i ) = n 1 =0 ( ρ(c i ) µ σ ) 2. (17) For each candidate crack C i, we et up two hypothee: H 0 : C i i a crack. H 1 : C i i not a crack. Since f(c i ) i the u of quare of ultiple noral ditribution according to (17), f(c i ) follow χ 2 ditribution with n degree of freedo. Define F (x, n ) be the cuulative ditribution function of χ 2 ditribution with n degree of freedo, the probability of a value fro χ 2 ditribution larger than x i P {f(c i ) x} = 1 F (x, n ). (18) By etting the ignificance level a α, we can obtain P {f(c i ) x} = α. (19) Since the function F i continuou and trictly onotonically increaing, cobining (18) and (19), we can obtain x = F 1 (1 α, n ), (20) where F 1 ( ) i the invere function of F ( ). Thu, we reject H 0 if f(c i ) F 1 (1 α, n ). (21) After the tatitical hypothei tet, we reove the falepoitive crack. Cobine all the crack reained to obtain C. V. ALGORITHM ANALYSIS We uarize the propoed WMIP baed candidate crack detection ethod in Algorith 1 to facilitate our analyi. Recall κ i the total aount of pixel in I. For crack eed extraction, ince at ot κ/ 2 cell are obtained, with each cell containing 2 pixel, the crack eed can be detected in O(κ/ 2 2 ) = O(κ) tie. Crack eed clutering take O( E log E ) [25]. Obviouly, E κ/ 2 κ, thu, crack eed cluter ha a tie coplexity O(κ log κ). Since we only find the WMIP within a rectangular window Wi,j whoe diagonal length i aller than T p, the axiu ize of Wi,j i 1 2 T p 2, thu, when proceeding the Dijktra algorith, the nuber of vertex i aller than 1 2 T p 2, and the nuber of edge i no ore than Tp 2. So the tie coplexity of each WMIP generation i O(Tp 2 log T p ). There are at ot κ 2 / path in all becaue E κ/ 2. Thu, the tie log Tp coplexity for all WMIP generation i T 2 p κ 2. Since the length of Pi,j i no ore than the ize of W i,j, we have Pi,j 1 2 T p 2. Therefore, we can copute (Pi,j ) in O(T p 2 )

6 tie. Crack region detection by path growing take O(κ log κ) tie. Coputing d i,j between two et with ize Ci and C j need O( Ci C j ) tie. Uually, the total nuber of crack region i very all and can be conidered a contant. Thu, the tie coplexity of crack grouping i alo O( Ci C j ). To uarize, ince Ci C j < κ2, the ot coputationally expenive tep in Algorith 1 i WMIP generation uing Dijktra algorith. Thu, the coputational coplexity of our WMIP baed crack detection algorith i O( T 2 p log Tp κ 2 ). Theore 1: The coputational coplexity of the propoed WMIP baed crack detection algorith i O( T 2 p log Tp κ 2 ). Algorith 1: WMIP baed Crack Detection input : I output: C 1 Detect all crack eed fro I to generate E ; O(κ) 2 Cluter and filter E uing DBSCAN; O(κ log κ) log Tp 3 foreach x i E and x j E do O( T 2 p κ 2 ) if x j D i then 5 Copute Pi,j by olving (7) uing Dijktra algorith; O(Tp 2 log T p ) 6 Copute ean intenity (Pi,j ) uing (8); O(T p 2 ) 7 if (Pi,j ) < g then 8 Dicard Pi,j ; O(1) 9 foreach Pi,j P do O(κ log κ) 10 repeat O(κ log κ) 11 Copute Gi uing (9); O(n i) 12 Ci = C i G i ; O(1) 13 until Ci i unchanged; 1 foreach Ci and Cj do O( C i C j ) 15 Copute d i,j uing (11); O( Ci C j ) 16 if d i,j < T g then 17 Merge Ci and Cj into the ae group; O( Ci + C j ) 18 foreach Ci C do O( Ci ) 19 if Ci < T then 20 Reove Ci fro C ; O( Ci ) 21 return C ; O(1) We preent our MFCD algorith in Algorith 2, and facilitate the coputational coplexity analyi a follow. Matching two crack C i i and C j j take in O( C i i Cj j ) tie. Generally, the total nuber of crack in I i all and can be conidered a contant. Thu, crack atching acro two cale ha a tie coplexity O( C i i Cj j ). Coputing all crack correpondence acro each pair of adjacent cale take O(n C i i Cj j ) tie. When coputing the average core of any crack Ci, the core of C i pixel in C i need to be calculated firtly, a hown in Eq. (15). The neighboring window for each pixel i (3σ p +1) (3σ p +1). Thu, the overall coputation of average core of Ci take O((3σ p + 1) 2 Ci ) tie. Conidering σ p i a contant, coputing ρ(ci ) ha a tie coplexity O( Ci ). Since the total nuber of crack pixel in each I i aller than κ, erging crack acro n cale take in O(n κ) tie. Define the total nuber of C i be n. Conidering C i κ, the tie coplexity of hypothei tet for all erged crack i O(nκ). Thu, the crack election and verification for each two cale take in O(ax(n i i, nj j )) tie. Since both C i and n are uch aller than κ, the coputational coplexity of our MFCD T algorith i O(n 2 p log Tp κ 2 ). T Theore 2: Our MFCD algorith run in O(n 2 p log Tp κ 2 ) tie. Algorith 2: MFCD Algorith input : I, = 0, 1,..., n 1 output: C log Tp 1 foreach I T do O(n 2 p κ 2 ) log Tp 2 Extract C uing WMIP algorith; O( T 2 p κ 2 ) 3 foreach C i and C j do O(n C i i Cj j ) foreach C i i C i and C j j C j do O( C i i Cj j ) 5 Match C i i and C j j ; O( Ci i Cj j ) 6 foreach C do O(n Ci ) 7 foreach Ci C do O( Ci ) 8 Copute ρ(ci ) uing (15); O( C i ) 9 Merge atched crack uing (16); O(n κ) 10 foreach C i do O(nκ) 11 Copute f(c i ) uing (17); O(n ) 12 if f(c i ) F 1 (1 α, n ) then 13 Add C i into C ; O( C i ) 1 return C ; O(1) VI. EXPERIMENTS We have ipleented our algorith uing MATLAB under a PC with an operating yte of Window 8, which ha an Intel(R) Core TM 2 i5-200u CPU@1.60GHz and GB eory. We evaluate the perforance of our MFCD ethod on two public dataet and copare it with ix tate-of-the-art algorith. A. Paraeter Setting It i iportant to deterine the value of σ in ulti-cale iage generation in Section IV-A. We have to find a trade-off between efficiency and copletene. Define σ = kσ, k N. Firt, we fix σ = 1, and can adjut the nuber of cale fro 1 to 5. By etting k = 0, 1,..., n, we obtain n + 1 iage with different cale. We elect 10 repreentative iage for the experient. We find that when the nuber of cale exceed 3, the detected crack are alot unchanged. However, the cot of coputation increae with thi nuber. Therefore, we ue 3 cale iage. Then, by fixing k = 0, 1, 2, we find σ = 0.8

7 TABLE I PARAMETER SETTING Paraeter Value Decription σ 0.8,1.6 tandard deviation of Gauian kernel 8 ize of grid cell T e 0.2% threhold for crack eed extraction T p 32 threhold for path generation T v 1% threhold for path verification T r 2% threhold for path growing r ditance tolerance for path growing T g 6 threhold for crack region grouping T 6 threhold for all crack reove T d 8 ditance tolerance for crack atching σ p 1 tandard deviation in pixel core coputation α 0.05 ignificance level experientally to be the bet choice. Thu, the 3 cale iage are the original iage, σ = 0.8, and σ = 1.6. Referring to [27], the ize of grid cell,, i et a 8. Other paraeter are et according to the experient epirically a hown in Tab. I. B. Dataet, Counterpart, and Metric Two public dataet are teted in our experient. AigleRN dataet [17]. It contain 38 French paveent iage with ground truth. The reolution i pixel. They have been pre-proceed to itigate the influence of non-unifor lighting condition. CFD dataet [12]. It i copoed of 118 iage with a reolution of pixel, which can generally repreent urban road urface condition in Beijing, China. Each iage ha hand labeled ground truth. The iage contain noie uch a hadow, oil pot, and water tain. The ix exiting ethod which we copare our algorith to are: Canny [28]. Canny i a traditional edge detection ethod. MS [18]. Markov Segentation (MS) i a crack detection ethod baed on a ulti-cale extraction and a Markov egentation. GC [29]. GC i a geodeic contour ethod with autoatic election of point of interet baed on auto-correlation. FFA [30]. Free For Aniotropy (FFA) i baed on the etiation of inial intenity path at each pixel in four direction, and the pixel i recognized a a crack if the path cot greatly varie with the direction. MPS [17]. Minial Path Selection (MPS) i a crack detection algorith baed on the original inial intenity path election. CrackForet [12]. CrackForet i a road crack detection fraework baed on rando tructured foret, by learning the inherent tructured inforation of crack. To evaluate the perforance of different crack detection algorith quantitatively, three etric, including Preciion, Recall and F1-eaure, are eployed. Thee three etric can be coputed baed on true poitive (TP), fale negative (FN), and fale poitive (FP), Preciion = TP TP+FP Recall = TP TP+FN Preciion Recall F1-eaure = 2 Preciion + Recall (22) (23) (2) Since acquiring a high quality ground truth i difficult for real iage, we allow a tolerance argin in eauring the coincidence between the detected crack and the ground truth. A coparion, according to the experient etting in [17] and [12], we aue that TP pixel are included within a 2 and 5 pixel vicinity of the ground truth on AigleRN and CFD, repectively. C. Coputation Speed Tet To validate the coputation coplexity analyi in theore 2, we perfor the peed tet on our MFCD algorith. According to theore 2, the coputation coplexity of MFCD algorith i related to four paraeter: n, κ, and T p. We tet the peed of MFCD algorith with different etting of the four paraeter. Each tie, We fix three paraeter of the and change the reaining one to ee the trend of run tie. The reult are hown in Fig. 5, where for each etting, 10 paveent iage fro CFD dataet are carried out for averaged perforance. Fro Fig. 5 we can ee that thee tiing reult are conitent with the theoretical analyi in general. run tie() run tie() n (a) (c) run tie() run tie() #10 5 (b) T p Fig. 5. Coputation peed tet reult of our MFCD algorith. Each data point in thi figure i an average of reult fro ten paveent iage. D. Reult on AigleRN Dataet Fig. 6 illutrate two typical aple detection on AigleRN. The final detection reult (row 5) are obtained by fuing the (d)

8 Value Preciion Recall F1-eaure Canny MS GC FFA MPS WMIP MFCD Method Fig. 8. Averaged value of Preciion, Recall, and F1-eaure for all the iage in AigleRN. The value of MS, GC, FFA, MPS are fro reference [17]. Here, WMIP indicate WMIP baed crack detection ethod on I 0. TABLE II CRACK DETECTION RESULTS EVALUATION ON CFD DATASET Fig. 6. Repreentative aple reult of crack detection uing our propoed ethod. The firt row lit the original iage, row 2- are detection reult fro cale 1-3, the lat row i the final detection reult. reult fro different cale (row 2-). It i clear that ulticale fuion iprove the overall perforance. Value Preciion Recall F1-eaure Iage No. Fig. 7. Crack detection reult of our propoed MFCD algorith on AigleRN dataet. The crack detection reult of our algorith i hown in Fig. 7. The uary tatitic for coparion are preented in Fig. 8, and repreentative aple reult are hown in Fig. 9. It i clear that our MFCD ethod outperfor the counterpart. Method Preciion Recall F1-eaure Canny 12.23% 22.15% 15.76% FFA 78.56% 68.3% 73.15% CrackForet 82.28% 89.% 85.71% MFCD 89.90% 89.7% 88.0% Traditional edge detection ethod Canny i not uitable for paveent crack detection due to it overly high enitivity. MS i too enitive to the background texture. GC can only find a all part of crack. FFA output the reult a a thicker line, iplying high FP rate. A for MPS, for the iage with eriou noie, the value of FP and FN becoe an iue. Furtherore, to validate whether the ulti-cale fuion can iprove crack detection, we copare the ingle cale crack detection reult on I 0 with MFCD ethod. It i clear that the ulti-cale fuion indeed iprove the perforance of inglecale detection. E. Reult on CFD Dataet Our algorith alo ucceed on CFD dataet. Fig. 10 preent oe repreentative iage and their detection reult uing our MFCD algorith, where we can ee that the iage in CFD are quite noiy. The perforance of our MFCD algorith degrade when the contrat between crack and background i low (a hown in the 2nd row in Fig. 10), or there are dark region, e.g. oil pot, in the paveent iage (a hown in the 5th and 9th row in Fig. 10). The crack detection reult on the CFD dataet are uarized in Tab. II where the reult of Canny, FFA and CrackForet are fro [12]. Again, our algorith outperfor all counterpart.

9 Fig. 9. Exaple reult of different algorith on AiGleRN (fro left to right: original iage, ground truth, Canny, MS, GC, FFA, MPS, WMIP baed ethod on I 0, and MFCD). The whole et of original iage and detection reult are available on the following web page: edu/bridge/dataet.htl. VII. CONCLUSION AND FUTURE WORK We developed a new MFCD ethod for detecting the crack fro paveent iage. Firt, we propoed a WMIP baed ethod to detect the crack at each ingle cale. Second, we atched the correponding crack acro different cale. Finally, a crack evaluation odel wa built, and the crack wa elected a the detected crack if it paed tatitical hypothei T tet. Our MFCD algorith take in O(n 2 p log Tp κ 2 ) tie. We have ipleented the propoed algorith and experiental reult are conitent with our coplexity analyi. We teted MFCD algorith on two public dataet. Experiental reult howed that 1) The propoed MFCD algorith iprove the crack detection perforance copared with the ingle cale WMIP baed ethod. 2) Our MFCD algorith outperfor ix exiting ethod. In the future, we will conider fuing the caera input with ground penetration radar and laer ranger finder input. We will tudy how to perfor crack detection uing ulti-odal input. ACKNOWLEDGMENT We would like to thank Guiu Robot Co. Ltd. for their feedback. We are alo grateful to C. Chou, H. Cheng, S. Yeh, A. Kingery, A. Angert, T. Sun, D. Wang, Y. Sun, Y. Yu, J. Gong, and M. Moin for their input and contribution to the Networked Robot Laboratory at Texa A&M Univerity. REFERENCES [1] H. La, R. Li, B. Baily, et al., Mechatronic yte deign for an autonoou robotic yte for high-efficiency bridge deck inpection and evaluation, IEEE/ASME Tran. on Mechatronic, vol. 18, no. 6, pp , Dec [2] H. La, R. Li, B. Baily, et al., Autonoou robotic yte for highefficiency non-detructive bridge deck inpection and evaluation, in Proc. Int. Conf. on Autoation Science and Engineering., 2013, pp [3] H. La, N.Gucunki, S. Kee, and L. Nguyen, Data analyi and viualization for the bridge deck inpection and evaluation robotic yte, Springer Journal of Viualization in Engineering, no. 3:6, pp.1-16, Feb [] Y. Hu, and C. Zhao, Alocal binary pattern baed ethod for paveent crack detection, Journal of Pattern Recognition Reearch, vol. 5, no. 1, pp.10-17, Sep [5] J. Tang, and Y. Gu, Autoatic crack detection and egentation uing a hybrid algorith for road ditre analyi, in Proc. IEEE Int. Conf. Syt., Man, Cybern., 2013, pp [6] R. Li, H. La, W. Sheng, and Z. Shan, Developing a crack inpection robot for bridge aintenance, in IEEE International Conference on Robotic and Autoation., 2011, pp

10 Fig. 10. Soe experiental reult uing our MFCD algorith on CFD dataet (fro left to right: original iage, ground truth, and reult fro MFCD). vol. 1, no. 1, pp , March [9] Y. Hu, C. Zhao, and H. Wang, Autoatic paveent crack detection uing texture and hape decriptor, IETE Technical Review, vol. 27, no. 5, pp , Sep [10] M. Jahanhahi, S. Mari, C. Padgett, and G. Sukhate, An innovative ethodology for detection and quantification of crack through incorporation of depth perception, Machine Viion Application, vol. 2, no. 2, pp , [11] P. Praanna, K. J. Dana, N. Gucunki, et al., Autoatic crack detection on concrete bridge, IEEE Tranaction on Autoation Science and Engineering, vol. 13, no. 2, pp , April [12] Y. Shi, L. Cui, Z. Qi, F. Meng,and Z. Chen, Autoatic road crack detection uing rando tructured foret, IEEE Tranaction on Intelligent Tranportation Syte, vol. 17, no. 12, pp.33-35, Dec [13] A. Cord, and S. Chabon, Autoatic road defect detection by textural pattern recognition baed on AdaBoot, Coputer-Aided Civil Infratructure Engineering, vol. 27, no., pp.2-29, April [1] B. Lee, Y. Ki, S. Yi, and J. Ki, Autoated iage proceing technique for detecting and analyzing concrete urface crack, Structure Infratructure Engineering, vol. 9, no. 6, pp , [15] H. La, N. Gucunki, K. Dana, and S. Kee, Developent of an autonoou bridge deck inpection robotic yte, Journal of Field Robotic, in pre, Apr [16] D. Zhang, Q. Li, Y. Chen, et al., An efficient and reliable coare-tofine approach for aphalt paveent crack detection, Iage and Viion Coputing, vol. 2017, no. 57, pp , [17] R. Ahaz, S. Chabon, J. Idier, and V. Baltazart, Autoatic crack detection on two-dienional paveent iage: an algorith baed on inial path election, IEEE Tranaction on Intelligent Tranportation Syte, vol. 17, no. 10, pp , Oct [18] S. Chabon, and J. M. Moliard, Autoatic road paveent aeent with iage proceing: Review and coparion, International Journal of Geophyic, vol. 2011, pp.1-20, [19] M. Gavilan, D. Balcone, O. Marco, et al., Adaptive road crack detection yte by paveent claification, Senor, vol. 11, no. 10, pp , Oct [20] V. Kaul, A. Yezzi, and Y. Tai, Detecting curve with unknown endpoint and arbitrary topology uing inial path, IEEE Tranaction on Pattern Analyi and Machine Intelligence, vol. 3, no. 10, pp , Oct [21] Y. Jeon, J. Yun, D. Choi, and S. Ki, Defect Detection algorith for corner crack in teel billet uing dicrete wavelet tranfor, in Proc. Int. Conf. Control, Auto., Syt., 2009, pp [22] R. G. Lin, and S. N. Givigi, Autoatic crack detection and eaureent baed on iage analyi, IEEE Tranaction on Intruentation and Meaureent, vol. 65, no. 3, pp , March [23] L. Ying, and E. Salari, Bealet tranfor-baed technique for paveent crack detection and claification, Coputer-Aided Civil and Infratructure Engineering, vol. 25, no. 8, pp ,nov [2] Rafael C. Gonzalez and Richard E. Wood, Digital Iage Proceing (th Edition). London, U.K.: Pearon, [25] M. Eter, H. P. Kriegel, J. Sander, et al, A denity-baed algorith for dicovering cluter in large patial databae with noie, Kdd, vol. 96, no. 3, pp , [26] E. W. Dijktra, A note on two proble in connexion with graph, Nueriche Matheatik, vol. 1, no. 1, pp , [27] Y. Huang, and B. Xu, Autoatic inpection of paveent cracking ditre, Journal of Electronic Iaging, vol. 15, no. 1, pp.1-6, Mar [28] J. Canny, A coputational approach to edge detection, IEEE Tran. on Pattern Anal. Mach. Intell., no. 6, pp , [29] S. Chabon, Detection of point of interet for geodeic contour: Application on road iage for crack detection, in Proc. Int. Conf. Coput. VISAPP, 2011, pp. 1-. [30] T. S. Nguyen, S. Begot, F. Duculty, and M. Avila, Free-for aniotropy: A new ethod for crack detection on paveent urface iage, in Proc. Int. Conf. Iage Proce., 2011, pp [7] R. Li, M. Hung, and W. Sheng, A robotic crack inpection and apping yte for bridge deck aintenance, IEEE Tranaction on Autoation Science and Engineering, vol. 11, no. 2, pp , April 201. [8] H. Oliveira, and P. L. Correia, Autoatic road crack detection and characterization, IEEE Tranaction on Intelligent Tranportation Syte,

11 Haifeng Li received the Ph.D. degree in control theory and control engineering fro Nankai Univerity, Tianjin, China, in He i an aociate profeor with Departent of Coputer Science and Technology, Civil Aviation Univerity of China, Tianjin, China. He ha authored or coauthored over 30 technical article. Hi reearch interet include coputer viion, iage proceing, robotic ening, ultienor fuion, robot localization and navigation. Yu Liu received the B.S. degree fro the Departent of Coputer Science and Technology fro Taiyuan Univerity of Technology, Taiyuan, China, in He i currently working toward the Mater degree in coputer cience and technology with Civil Aviation Univerity of China, Tianjin, China. Hi current reearch interet include the area of coputer viion, iage proceing and undertanding. Dezhen Song (S 02-M 0-SM 09) received the Ph.D. degree in indutrial engineering fro Univerity of California, Berkeley, CA, US, in 200. He i a Profeor with Departent of Coputer Science and Engineering, Texa A&M Univerity, College Station, TX, USA. Hi reearch interet include networked robotic, ditributed ening, coputer viion, urveillance, and tochatic odeling. Dr. Song received the Kayaori Bet Paper Award of the 2005 IEEE International Conference on Robotic and Autoation (with J. Yi and S. Ding). He received NSF Faculty Early Career Developent (CAREER) Award in Fro 2008 to 2012, Song wa an aociate editor of IEEE Tranaction on Robotic. Fro 2010 to 201, Song wa an Aociate Editor of IEEE Tranaction on Autoation Science and Engineering. He i currently a Senior Editor for IEEE Robotic and Autoation Letter (RA-L), a new flaghip journal fro IEEE Robotic and Autoation Society. He i alo an author and a Multiedia Editor for Springer Handbook for Robotic. Binbin Li received the B.S. degree fro the Departent of Electrical Engineering and Autoation fro Harbin Intitute of Technology, Harbin, China, in He i currently working toward the Ph.D. degree in coputer engineering with Texa A&M Univerity, College Station, TX, USA. Hi current reearch interet include the area of robot viion, viual tracking and recognition.