ON THE QUALITY OF AUTOMATIC RELATIVE ORIENTATION PROCEDURES

Transcription

1 ON THE QUALITY OF AUTOMATIC RELATIVE ORIENTATION PROCEDURES Thomas Läbe, Timo Dickscheid ad Wolfgag Förster Istitute of Geodesy ad Geoiformatio, Departmet of Photogrammetry, Uiversity of Bo Commissio III/1 KEY WORDS: Sesor orietatio, comparative aalysis, accuracy assessmet, matchig ABSTRACT: This paper presets a empirical ivestigatio ito the quality of automatic relative orietatio procedures. The results of a i-house developed automatic orietatio software called AURELO (Läbe ad Förster, 006) are evaluated. For this evaluatio a recetly proposed cosistecy measure for two sets of orietatio parameters (Dickscheid et al., 008) ad the ratio of two covariaces matrices is used. Thus we evaluate the cosistecy of budle block adjustmets ad the precisio level achievable. We use differet sets of orietatio results related to the same set of images but computed uder differig coditios. As referece datasets results o a much higher image resolutio ad groud truth data from artificial images redered with computer graphics software are used. Six differet effects are aalysed: varyig results due to radom procedures i AURELO, computatios o differet image pyramid levels ad with or without with oly two or three observatios, the effect of replacig the used SIFT operator with a approximatio of SIFT features, called SURF, repetitive patters i the scee ad remaiig o-liear distortios. These experimets show uder which coditios the budle adjustmet results reflect the true errors ad thus give valuable hits for the use of automatic relative orietatio procedures ad possible improvemets of the software. 1 INTRODUCTION Orietig images is oe of the basic tasks i photogrammetry. For aerial images the full automatio of this task ca be cosidered as solved, but for close rage photogrammetry full automatic orietatio still is a vital research area (cf. the work of Pollefeys (Siha ad Pollefeys, 004), Nister (Egels et al., 006), Zisserma ad Schaffalitzky (Schaffalitzky ad Zisserma, 00)). Due to errors itroduced while solvig the recogitio task, special ivestigatios ito the factors ifluecig the accuracy which are ot relevat i classical o-automatic budle adjustmet have to be cosidered, e.g. texture, repetitive structures, rotatio of the images, use of all or less i the adjustmet. Depedig o the used algorithms, there are various parameters which have a ifluece o the accuracy of the result. This paper ivestigates the quality of automatic budle orietatio software, especially the effect of various factors oto the quality of the result. As there is o geeral method for automatic mesuratio of cotrol, we ivestigate the relative orietatio of multiple images istead. The fidigs ca be geeralized for a wide rage of scearios ad applicatios. METHOD FOR QUALITY CHECK To evaluate the result of a automatic orietatio procedure we eed to compare the orietatio parameters betwee two datasets directly. Automatic methods may be stochastic or determiistic, depedig o whether they call a radom umber geerator, e. g. for a RANSAC procedure, or ot. I case the algorithm cotais a stochastic compoet, we caot use the 3D object, as differet rus may choose differet ad we ivestigate i relative orietatio procedures without a subsequet absolute orietatio. Therefore we follow the approach of (Dickscheid et al., 008) which compares the orietatio parameters directly. The measures may be used (1) for evaluatig the variatios due to the radomess of the algorithm ad () for comparig a budle adjustmet result with a referece dataset or with each other. A referece dataset i this cotext is a dataset with superior accuracy ad thus may be a result of just the same algorithm, e.g. o 37 a image pyramid level with a higher resolutio. The approach itroduces a so-called cosistecy measure c: The two parameter sets should differ by a spatial similarity trasformatio. This trasformatio is estimated ad the square root of the variace factor of the fial estimatio is the cosistecy measure c = Ω/R with the Mahalaobis distace Ω of the two datasets i a commo coordiate system ad with the redudacy R, which is 6N 7 for sets with N images. The full possibly sigular covariace matrices of the orietatio parameters of both datasets are rigorously take ito accout i this cogruece test. The advatage of this approach is that the compariso is summarized i a sigle scalar value. A value of c = 1 meas that the differeces of the two sets are (o average) cosistet with the precisio give by the orietatio procedures. If c < 1, the accuracies are too pessimistic, if c > 1, the accuracies are too optimistic. Note that the reliability of the cosistecy measure depeds o the redudacy i the estimatio of the trasformatio parameters. Therefore we expect reliable cosistecy estimates for datasets with more tha three or four images. We give the cosistecy measure for every experimet we describe i this paper. I additio to c, we use p ad r max from (Dickscheid et al., 008): The two values compare two covariace matrices ad both deped o the geeralized eigevalues λ i. We compare the two covariace matrices of the two datasets after applyig the estimated spatial similarity trasformatio. Thus both covariace matrices are i the same coordiate system ad ca be compared. p is the average ratio of the covariace matrices. If p = 1, the covariace matrices are idetical, if p > 1 there is a (average) deviatio i precisio. Note that, as p is a metric, p >= 1 ad p is symmetrically. Thus p gives o iformatio about which covariace matrix is smaller, i other words, which accuracies are better. The secod value r max is the maximum ratio of the stadard deviatios: r max max i λ i ad thus idicates the maximum differece betwee the matrices. Note that r max is ot symmetric: if r max < 1 the the precisio of the secod covari-

2 The Iteratioal Archives of the Photogrammetry, Remote Sesig ad Spatial Iformatio Scieces. Vol. XXXVII. Part B3b. Beijig 008 ace matrix is better, otherwise it is worse for some fuctio of the parameters. If c 6= 1 the covariace matrices are ot cosistet with the differeces of the two datasets. Whe arguemetig with p ad rmax i this case it is reasoable ot oly to compare the give covariace matrices, but to also take the observed differeces ito accout. Thus pb = b c p ad b rmax = b c rmax are also used i these cases. These values represet the actual occurred accuracies. b rmax is maximum stadard deviatio of arbitrary fuctio of parameters assumig the estimate b c for c to be realistic. 3 ORIENTATION SOFTWARE For our ivestigatios we use a i-house developed orietatio software for the wide-baselie case of calibrated cameras called AURELO. The followig paragraph gives a brief overview of the software. Details ca be foud i (La be ad Fo rster, 006). The first step i AURELO is the extractio of feature o all give images. The SIFT-descriptor (Lowe, 004) or the SURF-descriptor (Bay et al., 006) ca be used to describe ad match the of a image pair. The SURF descriptor is a fast adaptio of the SIFT-descriptor. We use the implemetatio of D. Lowe for the SIFT descriptor 1 ad the implemetatio of the ETH Zurich, Switzerlad, for SURF. Because the descriptors are rotatio- ad scale-ivariat, a large umber of possible cofiguratios ca be hadled. No prior iformatio about the overlappig parts or the sequece of the images eed to be give. Therefore a matchig is doe betwee all possible image pairs. Relative orietatios for each pair of images are computed with the help of a RANSAC procedure based o the 5-poit solutio proposed by D. Nister (Nister, 004). It should be oted that, as RANSAC uses a radom umber geerator, multiple rus of the software may lead to differet results. The best pairwise relative orietatios are liked together to geerate a iput for a cocludig budle adjustmet. Thus the resultig orietatio parameters are give with a full covariace matrix. Note that by default all are used i this adjustmet, idepedet of the umber of observatios per object poit dataset A # img. 6 B 6 C 4 D1 1 D 9 E F 3 5 descriptio etrace of a large buildig with stairs, medium texture outdoor stairs with rich texture, matches i backgroud which are far away from cameras poster ad box o a table, differet perspectives ad rotatios of the camera facade with rich texture, oe strip with early parallel viewig directios ad high overlap (ca. 90%) same cofiguratio as D1, but with less overlap (ca.75%) idoor scee with a regular grid o the wall redered images with 1 Megapixel image size: two walls with texture o it. The texture varies betwee artificial repetitive patters ad real texture. Table 1: Overview of the datasets used i the experimets. I the tables we give additioal results as give by the orietatio procedure: the umber of object ad image of the fial budle block adjustmet. the outlier rate of the matches: Due to the relative orietatio of a image pair computed i the RANSAC loop, outliers i the matched of this image pair are detected. The average rate of these outliers over all image pairs is give i the tables. σx0 : average precisio of the image coordiates i pixel. As all image observatios curretly are used with the same weight ad thus the weight matrix for the observatios is the uit matrix ad σapriori = 1, σx0 = σ b0, with σ b0 beig the square root of the variace factor of the budle adjustmet. σxo : The average precisio of the projectio cetres over all N images (usig σ bp equa0 ad the iverse of the ormal /N. Note + σy + σz = σx tio matrix): σx 0 that all datasets have a arbitrary scale, because o absolute orietatio has bee doe. The uit of the object coordiate system is defied by the distace from the first to the secod camera. All datasets of oe ivestigatio have bee scaled to the same scale to esure that the values ca be compared to each other. σωφκ : average precisio of the orietatio agles i go. σobj, σobj,m AX : average ad maximal precisio of the object due to the budle adjustmet result. Also scaled to the same scale i oe table. EXPERIMENTAL RESULTS Descriptio of the Datasets ad Experimets We use a wide rage of datasets i our experimets because may effects are highly depedet o the geometric cofiguratio ad o the amout of texture i the images. Due to the limited space, oly a part of the results ca be preseted i this paper, but we verified the observed effects i a umber of further, upublished experimets. The datasets we used for the data here are summarized i Table 1, see also Figure 1. Two cameras were used to take the images: Niko D70s with a 0mm les (6 Megapixel images, datasets A,B,D) ad HP photosmart 435 (3 Megapixel, datasets C,E). 4. Cosistecy over Multiple Rus Due to the fact that a radom procedure for the geeratio of approximate orietatio values is used i AURELO, the user gets differet results whe startig the program multiple times with idetical parameter settigs ad iput images. The variatios of the orietatio parameters should be small compared to their accuracies. We calibrated the cameras ad corrected the oliear distortio by rectifyig the images. All experimets are doe with the same settigs for the cotrol parameters, uless otherwise oted. The stadard parameters imply that the feature extractio is doe with the operator by D. Lowe. To test the ifluece of the radom procedure, we ru 5 datasets yieldig 10-4 samples. The results of these tests ca be foud i Table. The variatios of the orietatio parameters over multiple rus are show by their stadard deviatio X0 ad ωφκ with XX 1 X0 = Xs X N (S 1) Figure 1: Sample images of datasets B,E,F (from left to right). ad 1 lowe/key surf ωφκ XX 1 = N (S 1) 38 s s ωs ω φs φ κs κ!

3 The Iteratioal Archives of the Photogrammetry, Remote Sesig ad Spatial Iformatio Scieces. Vol. XXXVII. Part B3b. Beijig 008 dataset plev #rus R ɛ X0 ɛ ωφκ [go] σ X0 σ ωφκ [go] ɛ X0 / σ X0 ɛ ωφκ / σ ωφκ max. c mea c A B C D D Table : Compariso of multiple rus of AURELO with idetical parameter settigs. Plev: pyramid level o which the orietatio has bee computed (0: origial images, 1: first pyramid level,...). R: redudacy for the similarity trasformatio betwee the samples, R = 6N 7 with N images. ɛ X0,ɛ ωφκ : stadard deviatio of the projectio cetres ad the orietatio agles calculated over the multiple rus. σ X0, σ ωφκ : average precisio for projectio cetres ad orietatio agles due to the covariace matrix of the orietatio parameters over all samples. Max. c, mea c: Maximal ad mea cosistecy measure of all cosistecy measures betwee all samples (pairwise). computed over S multiple rus. Note that all results of a dataset are trasformed to the same datum i order allow a compariso. The ratio of the stadard deviatios ɛ X0, ɛ ωφκ ad the mea accuracies σ X0, σ ωφκ should be small, at least less tha 1.0. (These two values ca be combied to oe value c s, see (Dickscheid et al., 008).) This is fulfilled for most of the tests, except for dataset B o pyramid level 0 ad dataset C. These uexpected results ca be explaied with the weak treatmet of outliers i AURELO: The decisio about ilier ad outlier is doe oly with the epipolar geometry calculated from the approximate orietatio values. These values are computed i the RANSAC loop with oly 5 homologous ad thus a wrog decisio cocerig outlier/ilier is possible. No robust adjustmet is used, so that small outliers still may preset i the fial result. This result thus reveals a deficiecy of the algorithm: it does ot always yield reliable results due to its radomess, especially o a high precisio level. This eeds a clarificatio of the causes, which e. g. could be the o-robustess of the fial budle adjustmet. 4.3 Orietatio Parameters o Differet Pyramid Levels A importat factor for the computatio time ad memory cosumptio of the orietatio procedure is the size of the iput images. Therefore we ivestigate the decrease of accuracy whe calculatig the whole orietatio procedure o a higher image pyramid level istead o the origial image. All thresholds used i AURELO remai costat durig the tests. Table 3 shows the results of two datasets which ca be regarded represetative for the coclusios draw. All pyramid levels where we got a useful result are listed. I these two datasets the umber of images which could be coected by the orietatio procedure early remais the same over all levels. If a dataset icludes parts with low overlap of the images, a decrease i the umber of images ca occur i lower pyramid levels as i the two give examples. The umber of object decreases from pyramid level to pyramid level with the same factor as the umber of image (ot listed i the table). Thus, as to be expected due to the idetical geometry, the distributio of the umber of observatios per object poit is idepedet of the pyramid level. The cosistecy measure betwee the first ad all other image pyramid levels is ot higher tha 1.8, the cosistecy values betwee all the levels (ot listed i the table) are i the same rage. Thus the chage of the orietatio parameters from level to level are i the order of the assumed precisio or slightly higher. There is o evidece ot to trust the iteral precisio values, especially whe comparig them to each other. The evaluatio of the chage i precisio whe chagig the pyramid level eeds to be compared with the theoretical expectatio: I higher levels we expect a loss i precisio for two factors: loss of ad. less accurate due to the reduced resolutio. We thus expect the stadard deviatio of the orietatio parameters to icrease with the same factor as the measuremet accuracy (give with the same pixel size!) ad to decrease with the square root of the ratio of the umber of object-/image. So a reasoable theoretical factor betwee two image pyramid levels l ad m is f oσ = (σ 0,l /σ 0,m) N m/n l ) with N l, N m i pyramid levels l ad m. This factor ca be compared to the average ratio p of the two covariaces matrices of the orietatio parameters. The table shows that the two values are early the same. This validates the use of p for practical cases. Without takig ito accout the type of poit operator, oe could assume a factor of i the measuremet accuracy betwee level l ad l +1 ad a factor of 1/4 i the umber of observatios. Thus a reasoable theoretical factor betwee levels l ad l + 1 is 4 = 4. The table shows that there are sometimes sigificat differeces to that value. The icrease i the stadard deviatios is smaller betwee the origial image ad the first pyramid image (max..1). This is also the smallest value. This pheomeo ca be observed with other datasets, too. From a practical poit of view this is a importat fidig, because it allows the use of the first image pyramid level without a big loss i accuracy. The large value of p=10.1 of dataset D1 from level 1 to level may be explaied by a loss of texture: A large area of the image has the same texture. If o level this texture is ot visible ay more, the is a high loss i the umber of extracted image. 4.4 Ifluece of Twofold ad Threefold Poits o the Result To reduce the computatio time of the budle adjustmet, we may cosider to reduce the umber of observatios used for the recostructio. Oe simple possibility would be to igore object with a small umber of observatios, especially object with oly two observatios. The cosistecy of twofold ca t be tested very well, because the secod observatio may lie at every positio o the epipolar lie iduced by the first observatio. Thus there may be udetected false matches, especially whe repeated patters occur alog the epipolar lies. These false matches may lead to ustable or at least to less accurate solutios i the budle adjustmet. Thus, if the object are used i further applicatios, twofold may be deleted after the adjustmet or ot used at all i the adjustmet. This is of course oly possible if there are eough other -fold, >, available. To test the ifluece of -fold, we have computed the budle adjustmet with all ad with all but the - ad 3- fold, respectively. The results of three datasets are show i Table 4. First we wat to compare datasets D1 ad D, as they cosists of images of the same object. The orietatio parame-

4 The Iteratioal Archives of the Photogrammetry, Remote Sesig ad Spatial Iformatio Scieces. Vol. XXXVII. Part B3b. Beijig 008 dataset pyramid level #images i result [%] outliers of matches [%] # object σ x σ XO σ ωφκ c f oσ p r max rich texture (D1) medium texture (A) Table 3: Quality of the orietatio o differet image pyramid levels. Cosistecy measure c: Measure betwee image pyramid level 0 (origial images) ad all other levels. p, r max: average/maximal ratio of stadard deviatios betwee level l ad level l 1. f oσ: theoretical factor for the decrease of accuracy, f oσ = (σ 0,l /σ 0,l 1 ) N l 1 /N l ), l =pyramid level, N l object o level l. dataset used # object # image σ x σ XO σ ωφκ σ OBJ σ OBJ,MAX c p r max D1 all without -fold without,3-fold D all without -fold without,3-fold B all without -fold without,3-fold Table 4: Quality of the Orietatio without with two or three observatios. All orietatios have bee doe o the origial images. Cosistecy measure: Measure betwee orietatio with all ad with less. p, r max: average ad maximal factor betwee covariace matrices of orietatio with all ad with less. ters of D1 ad D do ot vary sigificatly with respect to their accuracy: the cosistecy measure is below 1.5. The accuracy of the image coordiates (σ x ) remais costat: All lie o a facade with very good texture ad thus the observatios of - fold ad 3-fold have the same accuracy as the other. The umber of object ad image ad hece the computatio time for the budle adjustmet decreases sigificatly, e.g. by a factor of 4 whe comparig all ad without -fold of D. The average ratio of the covariace matrices betwee all ad without -fold is 1. (dataset D1) ad 1.4 (dataset D) which may be a acceptable accuracy loss i most applicatios. The ratio for without,3-fold of D1 may be also acceptable, but for D the value shows a loss i accuracy. Here the maximal ratio has a large value (14.8). This i a idicatio that without - ad 3-fold some images could ot be orieted with good accuracy. As the images cosist of oe strip ad D has less overlap, the orietatio of the images at the begiig ad ed of the strip are difficult without 3-fold. We coclude from this observatio that for sigle strips a geeral deletio of - ad 3-fold tie may be problematic, for circular arragemets of images this problem would ot occur. Dataset B shows a differet behaviour. The cosistecy measures betwee the orietatio with all ad without -fold or - ad 3-fold is 10.6 ad 7.9, thus the orietatio parameters chage with respect to their high accuracy level. The average accuracy of the image becomes slightly better (σ x decreases from 0.8 to 0.5 ad 0.4). These two aspects allow the assumptio that most outliers were amog the - ad 3-fold. A visualizatio of the object (ot show here) idicates that may twofold which are far away ad lie o twigs of a tree have bee deleted. There are remaiig which are very far away, explaiig the high maximal value for the stadard deviatio of the object. Dataset B has also 40 show high cosistecy values for multiple rus (see Table ). Here agai we ca draw the coclusio that for this dataset a better outlier detectio has to be implemeted. The results of the orietatio without twofold may be more accurate due to outliers eve though the average ratio p of the covariace matrices is Usig Differet Feature Extractors If the iput images are large, e.g. greater tha 5 Megapixel, the time for poit extractio is a importat part of the overall computatio time. Therefore we itegrated SURF as a alterative to the SIFT operator by D.Lowe. SURF speeds up also the matchig by a factor of, because the legth of the descriptor of a feature poit is oly 64 compared to 18 whe usig the origial operator. Both operators double the image size before extractig the. Both operators were used with their stadard parameters. Table 5 summarizes a test which shows the ifluece of the operators. The examples show that SURF delivers less i all cases, sometimes oly 1/3 of the umber of Lowe. The matched SURF also cotai more outliers tha the Lowe i all cases. The accuracy of the is worse i datasets A ad B whe usig SURF, so that the average accuracy decreases by a factor of about 4 i these two datasets. This is theoretically the same loss which occurs whe usig the ext image pyramid level (see chapter 4.3). This loss i accuracy must be compared to the computatio time: I our tests SURF is faster with a factor of about 4-8, 4-5 o the smaller images of datasets A ad E. As the same factor (4-5) ca be expected whe calculatig the Lowe SIFT- o the ext pyramid level, a alterative for these two datasets with the same loss of accuracy to SURF would be to use the ext pyramid level with the SIFT operator by D. Lowe. However, there is still the speedup by a factor of durig

5 The Iteratioal Archives of the Photogrammetry, Remote Sesig ad Spatial Iformatio Scieces. Vol. XXXVII. Part B3b. Beijig 008 dataset Features #images i result # object outliers of matches [%] σ x σ XO σ ωφκ c p computatio time [sec] B (plevel=0) Lowe SURF A (plevel=1) Lowe SURF E (plevel=0) Lowe SURF Table 5: Quality of the Orietatio calculated with differet poit extractors. Cosistecy measure: Measure betwee orietatios with the two poit extractors. Computatio time: computatio time for feature poit extractios i secods. matchig which may be advatageous whe choosig SURF. Dataset E (repetitive patters) shows a differet behaviour: The measuremet accuracy of SURF is slightly better, the umber of object is oly 13 % smaller. I cotrast, the average distace of the covariace matrices of the orietatio parameters icreases by a factor of.4 whe usig SURF. This may be explaied by the fact that the distributio of the observatios over the images is better whe usig the Lowe SIFT operator. But the cosistecy measure betwee these two orietatios is 10, the orietatio parameters show big differeces that caot be explaied by gaussia oise i the image coordiates. So far the source of this differece is ukow. Further ivestigatios have to be doe to fid out which result is more reliable. 4.6 Results with with Repetitive Patters ad Groud Truth To guaratee that there is o ukow bias i the results of AU- RELO which is ot eve visible whe comparig results o differet image pyramid levels, a compariso with groud truth data is ecessary. Therefore we redered artificial images with a OpeGL program. We combied this experimet with a ivestigatio of differet matchig strategies ad their performace with repetitive patters. Therefore the scee (dataset F) shows two walls (Figure 1). The walls had bee textured partly with a artificial repetitive patter ad partly with rich texture from real images. We varied the amout of the repetitive patters. Table 6 shows experimets with two differet matchig criteria: The first experimets are doe with the stadard criterio: The ratio of the feature vector distace betwee the secod best ad the best match must be below a certai threshold, e.g. 70% i these tests. If there are multiple matches due to repetitive patters, o of the matches will be used i geeral. The secod part of the Table shows experimets with threshold matchig: The distace of the feature vector must be below a defied threshold. Here multiple matches will be used i the RANSAC loop to compute the relative orietatio of a image pair. Thus the relative orietatio is able to use matches of repetitive patters, but must detect the false matches as outliers. I all cases listed the orietatio was able to coect all five images. Tests were also doe with 100% repetitive patters. The orietatio failed completely i this case or give a coectio of two images with very bad accuracies. Best Matchig: The cosistecy measure c of the experimets with best matchig is below 1.8. This shows that there is o bias betwee the groud truth data ad the results of AURELO that is ot i the order of the stadard deviatios of the result. Thus the accuracies give represet the true accuracy situatio whe usig Lowe features ad best matchig i this image setup (large overlap betwee the images, o textured backgroud that is very far away). We ca coclude that the orietatio procedure is able to cope with up to 40% repetitive patters with early the same accuracy ( p betwee 1.6 ad 1.8). As the repetitive patters are equally distributed i the scee, there are eough reliable 41 poit correspodeces left. Due to the matchig criteria o additioal remaiig outliers for the budle adjustmet are to be expected. Eve with 80% repetitive patters the accuracy loss is oly about a factor of compared to the experimets without repetitive patters. I real world scees repetitive patters may cause more problems, because they are ofte ot equally distributed. Threshold Matchig: The experimets with threshold matchig show worse results: Although σ XO ad σ ωφκ have higher values, the errors are larger tha idicated by the accuracies: The cosistecy measure is always greater tha.9 ad oe experimet is clearly wrog: With 40% repetitive patters the cosistecy measure is 840. Tests show that multiple rus i this case show differet results, ofte better oes. As expected, the outlier rate of the matches is higher tha for the best matchig: at least 59% outliers. The results show, that this outlier rate is too high for the curret algorithm to provide reliable results. As p is always much higher tha i the experimets with best matchig, we do ot recommed the use of threshold matchig. With the curret parameter settigs ad simple outlier detectio methods o advatage ca be observed whe there is a high amout of repetitive patters i the observed scee. The same experimets were carried out with the SURF operator. The results of the SURF operator were always worse tha with the Lowe operator. But the SURF results become better with more repetitive patters: The compariso with 80% repetitive patters shows that p SURF / p Lowe is 1.5. Agai, threshold matchig is always worse tha with best matchig whe usig SURF. Figure : Quality of the orietatio with images with o-liear distortio. X-Axis: Amout of maximal distortio i the image i pixel. Y-Axis: cosistecy measure c ad average ratio of covariace matrices betwee dataset without distortio ad dataset with distortio. Used images: dataset A, Niko D70s camera with 0mm les. 4.7 Results with Differet Amouts of Distortios I may practical applicatios, especially whe usig cosumer cameras, the questio whether to correct the o-liear distortios of the images arises. We used existig lookup tables for distortio ad scaled the distortio offsets. These scaled distortio lookup tables were the applied to the already rectified (ad

6 The Iteratioal Archives of the Photogrammetry, Remote Sesig ad Spatial Iformatio Scieces. Vol. XXXVII. Part B3b. Beijig 008 Matchig rep. parts % # object # image outliers i matches [%] σ x σ XO σ ωφκ c p p Best Threshold Table 6: Quality of the Orietatio accordig to repetitive patters. Dataset used: dataset F (simulated data). The texture cosists of repetitive parts (percetage give i colum ) ad o repetitive parts. Lowe features with best matchig (ratio betwee best ad secod best match as threshold) ad threshold matchig used. The cosistecy measure c is calculated betwee the groud truth data ad the experimet, the average ratio of the covariaces p ad p = cp are calculated betwee the best matchig result with 0% repetitive patters ad all other experimets. therefore declared distortio-free) images ad the impact o the orietatio results was observed. With the help of the distortio scalig we set the maximal distortio successively from 1 to 70 pixel. The result for a Niko D70s camera (dataset A) is visualized i Figure. Here eve with 70 pixel distortio i the images AURELO was able to coect all images. The Figure shows the cosistecy measure ad the average ratio of the covariace matrices with respect to the maximal amout of distortio. The cosistecy icreases approximately liear with the amout of distortio. Eve with small distortios less or equal to 5 pixels the cosistecy measure has values from to 6. That meas that (with low ad with high distortios) the accuracies of the orietatio parameters do ot reflect the shift of the orietatio parameters due to o-liear distortio. For distortios equal or less tha 5 pixels the average ratio p betwee the covariaces is up to 1., so the covariaces i priciple do ot chage. The cosequece for practical applicatios is that (at least with this image setup) it is very difficult to draw coclusios about the remaiig o-liear distortio i the images ad their effect o the accuracy with the help of the statistical results of a budle adjustmet that does ot use ay calibratio parameters. As the loss of accuracy ca be large, e.g. i this experimet for 5 pixels distortio p 5 = c 5 p 5 = 7. ad for 5 pixels distortio, the actual distortio of the used les, p 5 = c 5 p 5 = 80, the o-liear distortios of the camera should always be used if available. 5 SUMMARY AND OUTLOOK The experieces with the tests showed that the cosistecy measure i cojuctio with the average ad maximal ratio of covariace matrices is a useful tool to bechmark differet results of automated relative orietatio procedures. To summarize our fidigs, the followig geeralized coclusios ca be draw: 4 Orietatio procedures which use radom umber geerators, especially RANSAC algorithms, may produce sigificatly differet results whe started multiple times. This is dataset depedet. The accuracy loss whe usig the first image pyramid level istead of the origial images is smaller tha the decrease whe usig the subsequet smaller levels. If there is eough overlap, with oly two or eve with oly three observatios ca be left out with oly a very small loss i accuracy. The SURF feature extractio implemetatio used here rus sigificatly faster tha the classical SIFT operator, but yields much worse results. If performace is ot a mai issue, we thus recommed usig SIFT. If accuracy is o issue, we recommed to use SURF, which is approximately a factor less accurate i stadard deviatio. Matchig SIFT descriptors with a threshold delivers worse results tha matchig with the ratio of best to secod best match. This is at least true whe o other sophisticated outlier detectio is used. Eve for images with large areas of repetitive patters threshold matchig has o advatages. Umodeled o-liear distortio has a sigificat ifluece o the result. A big amout of the errors is ot reflected i the estimated accuracies of the orietatio parameters. Not all iflueces o the quality were take uder cosideratio i this paper, e.g. the ifluece of the geometry (rotated images), the reflectace properties of the object, the lightig coditios ad so o. Nevertheless the fidigs ca help to produce more reliable results ad improve the orietatio procedures. For AURELO the eed for the implemetatio of a more sophisticated outlier detectio method ad/or the use of a robust budle adjustmet became clearly visible. The compariso with other orietatio procedures is also oe objective for our future work. REFERENCES Bay, H., Tuytelaars, T. ad Gool, L. v., 006. Surf: Speeded up robust features. I: Proceedigs of the 9. ECCV. Dickscheid, T., Förster, W. ad Läbe, T., 008. Bechmarkig automatic budle adjustmet results. I: XXI. ISPRS cogress, Beijig, submitted, accepted. Egels, C., Stewéius, H. ad Nistér, D., 006. Budle adjustmet rules. I: Photogrammetric Computer Visio (PCV). Läbe, T. ad Förster, W., 006. Automatic relative orietatio of images. I: Proceedigs of the 5th Turkish-Germa Joit Geodetic Days, Berli, Germay. Lowe, D., 004. Distictive image features from scale-ivariat key. I: Iteratioal Joural of Computer Visio, Vol. 0, pp Nister, D., 004. A efficiet solutio to the five-poit relative pose problem. I: IEEE PAMI, Vol. 0(6), pp Schaffalitzky, F. ad Zisserma, A., 00. Multi-view matchig for uordered image sets. I: Proceedigs of the 7th Europea Coferece o Computer Visio, Lodo, UK, pp Siha, S. ad Pollefeys, M., 004. Camera etwork calibratio from dyamic silhouettes. I: Proc. of IEEE Cof. o Computer Visio ad Patter Recogitio.