arxiv: v2 [cs.ro] 11 Apr 2018

Transcription

1 c 28 IEEE. Pesonal use of this mateial is pemitted. Pemission fom IEEE must be obtained fo all othe uses, in any cuent o futue media, including epinting / epublishing this mateial fo advetising o pomotional puposes, ceating new collective woks, fo esale o edistibution to seves o lists, o euse of any copyighted component of this wok in othe woks Aticle submitted to the Intenational Confeence on Intelligent Robots and Systems (IROS) 28. axiv:84.256v2 [cs.ro] Ap 28

2 OpenSeSLAM2.: An Open Souce Toolbox fo Visual Place Recognition Unde Changing Conditions Ben Talbot, Souav Gag 2, and Michael Milfod 2 Abstact Visually ecognising a tavesed oute egadless of whethe seen duing the day o night, in clea o inclement conditions, o in summe o winte is an impotant capability fo navigating obots. Since SeSLAM was intoduced in 22, a lage body of wok has followed exploing how obotic systems can use the algoithm to meet the challenges posed by navigation in changing envionmental conditions. The following pape descibes OpenSeSLAM2., a fully open souce toolbox fo visual place ecognition unde changing conditions. Beyond the benefits of open access to the souce code, OpenSeSLAM2. povides a numbe of tools to facilitate exploation of the visual place ecognition poblem and inteactive paamete tuning. Using the new open souce platfom, it is shown fo the fist time how compehensive paamete chaacteisations povide new insights into many of the system components peviously pesented in ad hoc ways and povide uses with a guide to what system component options should be used unde what cicumstances and why. I. INTRODUCTION Fo navigating obots to become commonplace, stong and obust solutions to the visual place ecognition poblem the ability to ecognise a peviously visited place must be established. While pesenting esults that demonstate place ecognition fo a specific scenaio with hand chosen paametes is impotant, it is also impotant to develop chaacteisations of the elationship between paamete values and pefomance unde a wide ange of configuations. Fo place ecognition systems to be safely deployed on live obotic systems, paameteisation must be based on a moe igoous undestanding than expeience-based intuition. SeSLAM, a seuence-based method fo visual place ecognition, has been followed by a wide body of liteatue since fist being intoduced by Milfod & Wyeth in 22 []. The ensuing liteatue to date has mostly focused on impovements to the algoithm s pefomance [2] [7], efficiency [8] [2], and a wide ange of othe contibutions [3] [9], with minimal focus on how to maximise geneal pefomance of the existing algoithm o even the influence exhibited on system pefomance by each paamete. To maximise the pefomance of any futue visual place ecognition system which builds on SeSLAM, it is cucial that it is fist undestood how to maximise the pefomance of the undelying algoithm based on the use s desied deployment scenaio. The authos ae with the School of Electical Engineeing and Compute Science, Queensland Univesity of Technology (QUT), Bisbane, Austalia b.talbot@ut.edu.au 2 The authos ae with the ARC Cente of Excellence fo Robotic Vision, Queensland Univesity of Technology (QUT), Bisbane, Austalia. michael.milfod@ut.edu.au Fig. : Snapshots fom one of the many visual intefaces in the OpenSeSLAM2. toolbox. The snapshots show the ability of the toolbox to dynamically econfigue theshold paametes post execution, and investigate individual seuence matches fom both image and matix centic pespectives. The following pape pesents OpenSeSLAM2., shown in Figue and available at openseslam2, as a highly configuable open souce toolbox fo using SeSLAM. The toolbox facilitates detailed and dynamic exploation of esults, as well as chaacteising the effect of paametes on pefomance. Using the novel OpenSeSLAM2. softwae, the pape pesents the following contibutions: a compaative summay of a numbe of diffeent seuence seach and match selection techniues that have only peviously been pesented in isolation; detailed pefomance chaacteisations fo numeous vaiations to the oiginal SeSLAM algoithm; and an open souce toolbox fo the obotics community, boasting the ability to gaphically configue paametes, dynamically econfigue paametes, auto-optimise thesholds to maximise pefomance, pofile pefomance via pecision-ecall, and sweep paamete values. The emainde of the pape poceeds as follows. Section II povides a bief discussion of visual place ecognition appoaches, with a focus on pefomance chaacteisations and existing open souce softwae. The algoithm implemented in OpenSeSLAM2. is then discussed in Section III, including all implemented vaiations to the appoach. In addition, the toolbox is demonstated with detailed pefomance chaacteisation tests which have thei expeimental design and esults outlined in Sections IV and V espectively. To conclude, the majo insights povided by the paamete chaacteisations ae pesented.

3 II. BACKGROUND OpenSeSLAM2. appoaches the visual place ecognition poblem using the SeSLAM method fist intoduced by Milfod et al. []. SeSLAM is one of many methods available fo woking with the visual place ecognition poblem, and all appoaches exhibit thei own stengths and weaknesses. The ensuing section exploes the types and instances of visual place ecognition methods, discusses published pefomance chaacteisations of existing systems, and lists the existing elevant open souce softwae packages. A. Visual Place Recognition Appoaches Visual place ecognition has eceived significant attention in obotics, with a ange of appoaches pesented by the community. Lowy et al. [2] pesent a discussion of the concepts behind the poblem, and detailed suvey of the visual place ecognition landscape. Instead, the discussion hee focuses on only the coe points of diffeence between visual place ecognition methods. Appeaance-only based methods like FAB-MAP [2] employ local point featues like SIFT [22], ORB [23], etc. in a bag of visual wods famewok fo matching places. Howeve, the featues lack obustness towads exteme peceptual changes. Pefoming visual place ecognition unde changing envionmental conditions typically involves image tansfomation based on, fo example, illumination invaiance [24], shadow emoval [25], appeaance pediction [26], convolutional filteing [8], etc. Futhe, seuence-based methods like SeSLAM [], SMART [3], and seveal othes [8], [], [27], [28] seuentially integate place hypotheses fo achieving state-of-the-at pefomance. B. Pefomance Chaacteisations Seuence-based visual place ecognition techniues [], [8], [], [27], [28], including SeSLAM, typically depend on paamete tuning. The paametes ae diectly elated to the diffeent aspects of the challenges faced in ecognising places unde exteme appeaance vaiations. As a esult, the liteatue has focused on ovecoming the challenges posed by appeaance vaiations. Howeve, the poposed solutions geneally deal with only one of the aspects of the poblem. Examples include gaphbased seach stategy [27], [28] and use of odomety [3] to deal with velocity vaiability between the compaed taveses, deep-leant featues [2], [8] fo obust place epesentation, adapting place neighbouhood ange fo impoved local seach [4], etc. The tageted focus in tun limits the chaacteisation of pefomance [4], [26], [29], [3] to meely suppoting the eseach contibution athe than holistically examining paamete influence. The liteatue does not cuently povide exhaustive pefomance chaacteisations fo most seuence-based techniues, with the pefomance of SeSLAM unde vaiations to seuence seaching and matching selection pooly undestood. C. Existing Open Souce Softwae Ove the yeas the obotics community has eleased a numbe of open souce softwae packages fo visual place ecognition and elated technologies. A numbe of tools exist fo visual place ecognition including a libay fo bag of wods image convesion [3], a c++ vesion of FAB-MAP named openfabmap [32]; an implementation of gaph-based matching of image seuences [27]; and tools fo elocalisation boosted visual place ecognition [33]. Moe geneal tools in elation to visual place ecognition include OpenRatSLAM [34] fo the visual appeaance based topological SLAM system RatSLAM [35]; LSD-SLAM fo eal-time monocula SLAM [36]; and eleases of both ORB- SLAM [23] and ORB-SLAM2 [37]. Fo the SeSLAM method, an ealie open souce implentation of OpenSeSLAM [3] was eleased in 23 (with othe implementations following with vaious tweaks [8]). The fist vesion of OpenSeSLAM is a thin wappe aound an implementation of SeSLAM, which is missing a numbe of featues key to analysing pefomance and optimising paamete selection. Missing featues include a gaphical inteface fo inteactively configuing paametes, viewing algoithm pogess, exploing diffeence matices and thei enhanced vesions, and checking matched seuences; the ability to dynamically econfigue match selection thesholds post opeation; auto-optimisation of thesholds to maximise pefomance against standad metics; tools fo ceating pecision ecall plots; the option to expot matching videos; and an explicit batch opeation mode. The OpenSeSLAM2. toolbox povides the highlighted missing featues allowing uses to igoously investigate, chaacteise, and optimise paamete selection. III. OPENSEQSLAM2. The OpenSeSLAM2. toolbox uses an appoach to the visual place ecognition poblem with the same high level stuctue as the oiginal SeSLAM solution [], but adds in a numbe of configuable vaiations to steps of the pocess. Pevious wok has intoduced a numbe of diffeent methods in isolation, albeit without any analysis of thei elative pefomance. The visual place ecognition poblem aims to match the images fom a uey tavesal to a efeence set of images. Fo the following section, it is assumed thee ae n images in the efeence tavesal and m images fo the uey. The SeSLAM pocess begins with an image pepocessing stage whee all images ae tansfomed into downsampled patch-based epesentations. Next, a diffeence matix is constucted between the two sets of tavesal images, and the local contast within the matix enhanced. Thidly, a local seuence seach is used to popose match candidates fo each of the uey dataset images. Finally, match selection is pefomed to eliminate weak match candidates. Each of the fou phases, and the configuation paametes contained in the OpenSeSLAM2. toolbox, ae outlined below.

4 Quey Image Quey Image (a) Oiginal Image (b) Gayscale Image Refeence Image Refeence Image (c) Copped and Resized Image (d) Enhanced Contast Image Fig. 2: The odeed pe-pocessing steps pefomed on input images by the OpenSeSLAM2. toolbox. All paametes, including cop location, downsized dimensions, aspect atio, and local contast enhancement citeia ae configuable. A. Image Pe-pocessing The toolbox pefoms the standad image pe-pocessing steps, visually outlined in Figue 2. Each image is conveted to geyscale, copped to a egion of inteest, and downsampled to a esolution consisting of only thousands of pixels. Finally, patch nomalisation is pefomed to enhance the local contast of the image. The toolbox adds in the ability to adjust all image pe-pocessing paametes, with visual intefaces to help guide value selection. B. Diffeence Matix Constuction Next, a diffeence matix of size n m is constucted by compaing each patch nomalised uey image to each patch nomalised efeence image. The compaison is pefomed via sum of absolute diffeences, nomalised to the dimensions of the image. Befoe the diffeence matix can be used to seach fo place matches, it is fist enhanced to facilitate finding local best matches. The enhancement pefomed local neighbouhood nomalisation of the diffeence vecto is analogous to a D vesion of patch nomalisation. Fo each uey image thee is a diffeence vecto D, whee each element coesponds to the diffeence value fo a given efeence image. Each element is nomalised within a window of size R nom to poduce a contast-enhanced image diffeence vecto ˆD, whee ˆD = D X σ, () X is the mean within the window, and σ is the standad deviation in the window. The effect of local contast enhancement on a diffeence matix section is shown in Figue 3. The toolbox allows configuation of the nomalisation window, and povides tools to diminish the dominance of outlies in diffeence matix visualisation. (a) Befoe local nomalisation (b) Afte local nomalisation Fig. 3: Local neighbouhood nomalisation of the diffeence vectos fo each uey image, whee the dake colous epesent a lowe diffeence scoe coesponding to a stonge match between efeence and uey image. C. Local Seuence Recognition With the diffeence matix enhanced to emphasise the best matches in local neighbouhoods, the next step in the SeSLAM pocess is to seach localised seuences fo potential matches. Seach is pefomed fo evey pai of efeence image and uey image. In each pai, a seach is done fo the linea seuence of image pais that poduce the best scoe unde a seuence evaluation metic. The metic is detemined by the seach method being used. Pevious wok has intoduced a numbe of diffeent methods fo finding seuences, but without any compaative analysis on what methods might be best. Hee fo the fist time, the OpenSeSLAM2. toolbox pesents thee diffeent methods of seach fo local seuence ecognition in the same famewok. The methods ae efeed to as tajectoybased [], cone-based [38], and hybid [2]. All methods constain thei seach to a seuence length d s, and limit the angle of thei seach fo seuences. The angle of seach is also often efeed to as the velocity though the diffeence matix, with limits v min and v max. Figue 4a shows the geneic seuence seach stuctue, with the afoementioned paametes highlighted. Whee the methods diffe, is in the seuences that they test and how they evaluate thei likelihood of epesenting a match. OpenSeSLAM2. splits the seach between looking ahead and behind the uey image (ceating the fowad and backwad facing seach cones seen in Figue 4a), but othe seach placements ae possible. The explicit details of each seach method ae shown in Figues 4b-4d and Algoithms -3 espectively. Tajectoy-based seach, used in the fist SeSLAM elease, tests seuences of length d s in steps of v step looking fo the minimum scoing tajectoy fo each, image pai (whee scoe is the sum of contast-enhanced diffeence matix scoes). The method has a fixed numbe of seuences to seach fo each image pai, and depending on v step, can be the slowest of the thee seach methods. Cone-based seach takes a diffeent appoach by using the unenhanced diffeence matix to seach fo global best matches in the local neighbouhood. Fo each, image pai, the numbe of global best matches in the cones of seach ae

5 d s v min v max (a) Geneal seach configuation v step (b) Tajectoy-based method Algoithm Tajectoy-based seach algoithm : v steps linspace(v min, v max, v step) 2: fo all, [..n], [..m] do 3: s best 4: fo v v steps do 5: s tajectoyscoe(,, v, d s) 6: if s < s best then 7: s best s 8: end if 9: end fo : s matix(, ) s best : end fo (c) Cone-based method (d) Hybid method Fig. 4: Visual desciptions fo each of the seach methods suppoted by OpenSeSLAM2.. counted. The scoe fo each image pai is given as a scoe between and, denoting the pevalence of best matches within the seach cones (n matches /d s, o 3/d s in the case of Figue 4c). Lastly, the hybid seach method combines the coe pinciple of the tajectoy method with the local seach fo global best matches of the cone method. Analogous to the fist method, a seach fo global best matches is pefomed in the local cones, with tajectoies dawn though each best match found. The best scoing tajectoy is then given as the scoe fo the image pai. D. Match Selection As a esult of the seuence seach pocess, a matix of seach scoes s matix denoting the best scoe found fo each, image paiing exists to guide match selection. Each uey image has a vecto of scoes epesenting the peceived likelihood that each efeence image could be a match. Match poposals ae deived by conveting the best scoe matix into a vecto of scoes fo each uey image s best efeence image match. While the entie matches vecto could be taken as matches (i.e. evey uey image is given a match), the appoach typically leads to vey poo pecision esults. Instead, a final step is pefomed to eject poo match poposals by only taking those that pass a given theshold metic. The OpenSeSLAM2. toolbox povides two match selection methods: scoe thesholding, and windowed uniueness thesholding. The methods have been published peviously, but neve had thei utility justified. ) Scoe Thesholding: The fist method is staightfowad; all poposed matches with a seuence scoe below a theshold λ ae emoved fom the match list. It is impotant to note that choosing a theshold can be difficult given that changing seuence length d s and seach method can Algoithm 2 Cone-based seach algoithm : fo all, [..n], [..m] do 2: s 3: fo all c cellsincones(, ) do 4: if isglobalbest(c) then 5: s s + /d s 6: end if 7: end fo 8: s matix(, ) s 9: end fo Algoithm 3 Hybid seach algoithm : fo all, [..n], [..m] do 2: s best 3: fo all c cellsincones(, ) do 4: if isglobalbest(c) then 5: v velocity(,, c) 6: s tajectoyscoe(,, v, d s) 7: if s < s best then 8: s best s 9: end if : end if : end fo 2: s matix(, ) s best 3: end fo significantly change the scoe ange. To assist in choosing appopiate thesholds, the toolbox includes an autooptimisation mode that can choose the theshold which achieves the geatest pecision, ecall, o F scoe. 2) Windowed Uniueness Thesholding: The second mode looks at the uniueness of a match poposal in its local neighbouhood, athe than the seuence scoe fo the poposal. To get a uniueness scoe, the vecto of scoes fo each uey image ae evaluated. The uniueness of a match poposal u is the pecent that the best match s scoe outpefoms the next best match s scoe, with the next best having to be outside an exclusion window of width R window fom the best match (i.e. µ /µ 2 in Figue 5). Afte a uniueness scoe has been given fo each uey image s match poposal, all poposals that do not have a uniueness scoe above µ ae discaded. The uniueness theshold is significantly easie to configue befoe unning SeSLAM, with uniueness scoes all typically falling in the same ange egadless of paamete values. Auto-optimisation mode is also available in the toolbox fo µ selection. IV. EXPERIMENTAL EVALUATION DESIGN The OpenSeSLAM2. toolbox was used to investigate and evaluate the influence of a numbe of paametes and

6 Refeence Match Scoe µ 2 µ Refeence Image Numbe R window Fig. 5: Calculation of the uniueness scoe µ fom the efeence match scoes fo image. vaiations in the SeSLAM pocess. Sweeps of selected paametes wee pefomed, fo multiple datasets, unde vaious configuations. The tests wee selected to povide insights into the undelying behaviou of SeSLAM, whee much pevious wok has elied on ad hoc handcafting and heuistics. The toolbox povides a batch opeation mode out of the box, allowing uses to easily examine the effects of specific paametes and configuations on pefomance with thei own datasets. A. Datasets The paamete sweeps ae pefomed on two benchmak datasets widely used in the liteatue [2], [8], [9], [7], [8], [2], [27], [3]: Nodland and Eynsham. ) Nodland: The dataset, fist intoduced in [3], compises of footage of a 728km tain jouney captued acoss fou diffeent seasons. The dataset does not exhibit any viewpoint vaiation between taveses but pesents sevee changes in appeaance due to seasonal vaiations. The expeiments pefomed below used the two taveses with the most diffeence in conditions, that is, winte vesus summe. As in [3], the taveses wee subsampled such that fames wee at fps and poposed matches wee accepted if they wee within fame of the gound tuth. 2) Eynsham: The dataset is a lage 7km oad-based dataset used by [2] in the FAB-MAP studies. The dataset consists of two 35km taveses with panoamic images captued at 7m intevals by a Ladybug 2 camea. In the expeiments below, the second tavese was matched to the fist tavese. In accodance with the oiginal FAB-MAP study ([2]), the 4m GPS-deived gound tuth povided with the Eynsham dataset was used fo evaluating pefomance. B. Paametes Paamete sweeps wee ochestated to exploe fou aeas of the SeSLAM algoithm. The sweeps exploed the enhancement of the diffeence matix, length and method of seuence seach though the matix, and method fo selecting fom match poposals. ) Size of column nomalisation window: The sweep vaied the width of the window R nom used when enhancing the local contast of the diffeence matix (see Section III- B fo paamete details). By vaying the width, the sweep exploed how limiting the window of nomalisation affects maximum pefomance. The window width was vaied fom 2 to the length of tavese, with the basic tajectoy seach used (see Section III-C), and match selection optimised fo the best F scoe. 2) Quantity of infomation used in seaching: Next, the length of seuences used in the seach pocess (see d s in Section III-C) was swept. By inceasing the numbe of images in the seach, the sweep investigated whethe the intuition held fo SeSLAM that moe infomation automatically wields bette esults. The length was vaied fom 2 to 4 fames, with the basic tajectoy seach used, and match selection optimised fo the best F scoe. 3) Method of seaching fo match candidates: The elative pefomance of the thee seach methods descibed in Section III-C was exploed by sweeping the match selection theshold fo each method. By focusing on the pefomance cuve, and maximum F scoe, the sweep pobed whethe a best pefoming seach method could be established. The sweep was pefomed acoss the ange of valid basic match selection thesholds, fo each of the seach methods with a seach length of fames. 4) Method of match selection fom candidates: Lastly, the effectiveness of each of the two seach methods descibed in Section III-D was evaluated by sweeping thei coesponding selection paametes. The shapes of the pefomance cuves, and the maximum F scoe, wee used to glean insights into which method pefoms the best and is most esistant to theshold selection eo. The sweep was pefomed acoss the ange of valid method thesholds, with the basic tajectoy seach used fo each test. V. EXPERIMENTAL RESULTS AND DISCUSSION The following section outlines the esults fo each of the fou aeas whee paamete sweeps wee pefomed. Plots fo each aea ae pesented, with comments on thei defining featues and identified tends discussed. Fo a numbe of the esults plots (Figues 6, 8, 9, and ), the valid value anges do not match. Conseuently, the plots ae aligned by plotting the dependent axis as a pecent of the tested value ange. A. Nomalisation width Nomalisation width was vaied between a minimum value of 2, and the maximum value eual to the numbe of efeence images n in the dataset. The esults ae shown in Figue 6, with the F scoes plotted fo each of the datasets. As can be seen, both datasets pesent an initial peak befoe dopping off and gadually inceasing as moe of the dataset is included in the nomalisation. The initial peak fo both datasets falls at aound 2% of the dataset length, which could be useful in situations whee maximising pefomance fo minimum compute cost is desied. Despite both peaking initially, the two dataset esults diffe in that Nodland neve etuns to its oiginal peak, wheeas Eynsham uickly supasses it and continues to gow. Reaching nea-peak pefomance uickly, at a vey small nomalization window, has implications fo designing computationally efficient and low latency systems. A small

7 F Scoe Nodland Eynsham Nomalisation Width (% in tested value ange) F Scoe tajectoy cone hybid Match Selection Theshold (% in tested value ange) Fig. 6: Effects of vaying column nomalisation width R nom fo the Nodland and Eynsham datasets. F Scoe Nodland Eynsham Seuence Length (# images) Fig. 7: Effects of vaying seuence length d s fo the Nodland and Eynsham datasets. Fig. 8: Chaacteistic pefomance cuves fo each seach method with the Nodland dataset. F Scoe tajectoy cone hybid Match Selection Theshold (% in tested value ange) Fig. 9: Chaacteistic pefomance cuves fo each seach method with the Eynsham dataset. nomalization window euies a shote tempoal histoy, which is especially elevant when pefoming initial global localisation at statup o afte a kidnap event. The shot window also is useful because it inceases the applicability of the algoithm to highly fagmented envionments whee using a long nomalization window ove multiple diffeent sections of the envionment is undesiable. B. Seuence Length Next, the esults of vaying seuence length between 2 and 4 fames fo both datasets is pesented. The esults ae shown in Figue 7, with both datasets pesenting simila esults. In both cases, pefomance impoves as seuence length inceases with diminishing pefomance etuns the geate the seuence length. The esult of vaying seuence length matches what intuition would suggest; the moe infomation included in the seach the geate the pefomance. It also highlights that the pefomance gains diminish as the amount of infomation povided to the seach inceases. Fo both datasets, a point of compomise (maximising pefomance while minimising infomation) exists fo a seuence length of between and 2. C. Seach Method Thidly, esults fom vaying thesholds fo each of the thee seach methods ae pesented fo both datasets. Fig- ue 8 shows the esults fo the Nodland dataset, with the tajectoy-based seach pefoming best and cone-based seach pefoming wost. Contastingly, the esults fo the Eynsham dataset in Figue 9 show cone-based seach as the best pefome and tajectoy-based as the weakest. The invesion in method pefomance can be explained by looking into the diffeence matices fo each dataset. The Nodland matix has no stong diagonal its global best matches ae not along the gound tuth diagonal due to the exteme condition vaiance until the matix has its local contast enhanced. Theefoe the cone method will stuggle to find coect matches because the global matches do not eflect the gound tuth. In contast, the global best matches fo the Eynsham matix do closely follow the gound tuth due to the lack of condition vaiance. In undestanding the vaying pefomance of the tajectoy-based seach method, the explanation is moe staightfowad. The Nodland dataset has a staight diagonal gound tuth, which fits linea tajectoies, wheeas the Eynsham dataset has a jagged diagonal gound tuth to which linea tajectoies cannot easily be fit. Inteestingly, all pefomance cuves in the Nodland dataset decline afte a peak wheeas the Eynsham cuves settle at the peak. The esults suggest that the ability of a match selection to be too geedy (i.e. sacifice pecision) is a function of the dataset, athe than the seach method.

8 scoe thesholding uniueness atio 36.2% 9.7% scoe thesholding uniueness atio 44.8% 7.7% F Scoe F Scoe Match Selection Theshold (% in tested value ange) Match Selection Theshold (% in tested value ange) Fig. : Chaacteistic pefomance cuves fo matching methods with the Nodland dataset. Pefomance bas denote % of value ange fo which pefomance > 9% of peak. Fig. : Chaacteistic pefomance cuves fo matching methods with the Eynsham dataset. Pefomance bas denote % of value ange fo which pefomance > 9% of peak. Theefoe, it is concluded the most appopiate seuence matching method (fom those pesented in the liteatue) is highly dependent on the chaacteistics of the envionment, and can be chosen accoding to the above obsevations. D. Match Selection Method Lastly, esults fom vaying the match selection method ae pesented. Fo both the Nodland and Eynsham datasets, in Figues and espectively, the maximum pefomance fo the methods ae simila. Whee the methods diffe is in the sensitivity of the method to theshold section. The pefomance bas in the figues emphasise the diffeence in sensitivity, with the scoe thesholding poducing a 4 6 wide stable pefomance paamete ange. The scoe thesholding method is much moe toleant to theshold selection, wheeas the windowed uniueness method has a small peak aound the maximum value with pefomance apidly deteioating as µ selection moves away fom the peak. Conseuently, the scoe thesholding method appeas eually effective fom a pefomance point of view, and moe stable unde paamete tweaking. VI. DISCUSSION AND CONCLUSIONS In summay, the pape has intoduced the OpenSeS- LAM2. toolbox as novel open souce softwae fo compehensively chaacteising the pefomance elationship between the key SeSLAM system components and paametes to maximise visual place ecognition pefomance. The toolbox s batch paamete sweep mode was used to investigate the effects of vaying column nomalisation width, seuence length, seuence seach method, and match candidate selection method. Fom the esults, conducted on two established datasets, the following conclusions wee obseved: A column nomalisation width of 2% of the dataset size pesents a pefomance peak, befoe lage width inceases ae euied to futhe incease pefomance. The optimal pefomance of a shot window has positive implications fo initial global localisation, highly fagmented envionments, and low latency systems. A seuence length of between and 2 fames maximises pefomance while not unnecessaily inceasing the algoithm s computation costs. Significantly, the optimal opeating ange manages to maximise pefomance while minimising computational cost. The hybid seuence seach method has the most stable pefomance, with tajectoy-based and cone-based seach pefomance dependent on the shape of the dataset gound tuth and condition vaiance espectively. Results suggest the tajectoy-based seach methods will pefom bette in envionments with high condition vaiance and consistent dataset velocity, wheeas cone-based methods pefom best fo visually simila taveses with inconsistent dataset velocity. The scoe thesholding method is 4 6 less sensitive to theshold selection than the windowed uniueness method, with simila peak pefomance. The esult suggest scoe thesholding should be pefeed by default. Although deep leaning-based techniues have come to the foe ecently [2], [7], [5], [7] [2]; thee is still a significant ole fo taditional visual place ecognition techniues, especially in applications whee compute is constained, sufficient taining data is not always available, o in combination with challenging envionmental conditions such as in autonomous undegound mining vehicles [39]. Seuence-based techniues ae also elevant even when deepleant image epesentations ae involved, because they can be added on top of single image based etieval to futhe impove pefomance. The open souce toolbox enhances the ability of the eseach community to wok with SeSLAM by poviding automatic theshold optimisation, dynamic paamete econfiguation, batch opeation, pefomance chaacteisation, and othe assisting featues. This pape fo the fist time compehensively chaacteised many of the citical paametes of the SeSLAM system, poviding an evaluation of thei utility and pefomance which builds on top of mostly ad hoc fome implementations. By poviding fo the fist time a compehensive open souce softwae package that

9 enables a complete chaacteization of one of the most widely used and benchmaked-against techniues SeSLAM, the aim is to futhe facilitate eseach in the gowing topical aea of visual place ecognition. REFERENCES [] M. J. Milfod and G. F. Wyeth, Seslam: Visual oute-based navigation fo sunny summe days and stomy winte nights, in Robotics and Automation (ICRA), 22 IEEE Intenational Confeence on. IEEE, 22, pp [2] Z. Chen, O. Lam, A. Jacobson, and M. Milfod, Convolutional neual netwok-based place ecognition, axiv pepint axiv:4.59, 24. [3] E. Peppeell, P. I. Coke, and M. J. Milfod, All-envionment visual place ecognition with smat, in Robotics and Automation (ICRA), 24 IEEE Intenational Confeence on. IEEE, 24, pp [4] S. Gag, A. Jacobson, S. Kuma, and M. Milfod, Impoving condition-and envionment-invaiant place ecognition with semantic place categoization, axiv pepint axiv:76.744, 27. [5] M. Milfod, Vision-based place ecognition: how low can you go? The Intenational Jounal of Robotics Reseach, vol. 32, no. 7, pp , 23. [6] P. Neubet, N. Sündehauf, and P. Potzel, Supepixel-based appeaance change pediction fo long-tem navigation acoss seasons, Robotics and Autonomous Systems, vol. 69, pp. 5 27, 25. [7] M. Milfod, C. Shen, S. Lowy, N. Suendehauf, S. Shiazi, G. Lin, F. Liu, E. Peppeell, C. Lema, B. Upcoft, et al., Seuence seaching with deep-leant depth fo condition-and viewpoint-invaiant outebased place ecognition, in Poceedings of the IEEE Confeence on Compute Vision and Patten Recognition Wokshops, 25, pp [8] S. M. Siam and H. Zhang, Fast-seslam: A fast appeaance based place ecognition algoithm, in Robotics and Automation (ICRA), 27 IEEE Intenational Confeence on. IEEE, 27, pp [9] R. Aoyo, P. F. Alcantailla, L. M. Begasa, and E. Romea, Towads life-long visual localization using an efficient matching of binay seuences fom images, in Robotics and Automation (ICRA), 25 IEEE Intenational Confeence on. IEEE, 25, pp [] Y. Wang, X. Hu, J. Lian, L. Zhang, and X. Kong, Impoved se slam fo eal-time place ecognition and navigation eo coection, in Intelligent Human-Machine Systems and Cybenetics (IHMSC), 25 7th Intenational Confeence on, vol.. IEEE, 25, pp [] M. Milfod, W. Scheie, E. Vig, A. Glove, O. Baumann, J. Mattingley, and D. Cox, Condition-invaiant, top-down visual place ecognition, in Robotics and Automation (ICRA), 24 IEEE Intenational Confeence on. IEEE, 24, pp [2] Z. Chen, S. Lowy, A. Jacobson, Z. Ge, and M. Milfod, Distance metic leaning fo featue-agnostic place ecognition, in Intelligent Robots and Systems (IROS), 25 IEEE/RSJ Intenational Confeence on. IEEE, 25, pp [3] P. Hansen and B. Bowning, Visual place ecognition using hmm seuence matching, in Intelligent Robots and Systems (IROS 24), 24 IEEE/RSJ Intenational Confeence on. IEEE, 24, pp [4] M. Milfod, H. Kim, S. Leutenegge, and A. Davison, Towads visual slam with event-based cameas, in The Poblem of Mobile Sensos Wokshop in conjunction with RSS, 25. [5] D. Bai, C. Wang, B. Zhang, X. Yi, and X. Yang, Cnn featue boosted seslam fo eal-time loop closue detection, axiv pepint axiv:74.56, 27. [6] N. Sündehauf, S. Shiazi, A. Jacobson, F. Dayoub, E. Peppeell, B. Upcoft, and M. Milfod, Place ecognition with convnet landmaks: Viewpoint-obust, condition-obust, taining-fee, Poceedings of Robotics: Science and Systems XII, 25. [7] P. Neubet and P. Potzel, Local egion detecto+ cnn based landmaks fo pactical place ecognition in changing envionments, in Mobile Robots (ECMR), 25 Euopean Confeence on. IEEE, 25, pp. 6. [8] N. Sündehauf, S. Shiazi, F. Dayoub, B. Upcoft, and M. Milfod, On the pefomance of convnet featues fo place ecognition, in Intelligent Robots and Systems (IROS), 25 IEEE/RSJ Intenational Confeence on. IEEE, 25, pp [9] Z. Chen, F. Maffa, I. Sa, and M. Chli, Only look once, mining distinctive landmaks fom convnet fo visual place ecognition, in Intelligent Robots and Systems (IROS), 27 IEEE/RSJ Intenational Confeence on. IEEE, 27, pp [2] S. Lowy, N. Sündehauf, P. Newman, J. J. Leonad, D. Cox, P. Coke, and M. J. Milfod, Visual place ecognition: A suvey, IEEE Tansactions on Robotics, vol. 32, no., pp. 9, 26. [2] M. Cummins and P. Newman, Fab-map: Pobabilistic localization and mapping in the space of appeaance, The Intenational Jounal of Robotics Reseach, vol. 27, no. 6, pp , 28. [22] D. G. Lowe, Distinctive image featues fom scale-invaiant keypoints, Intenational jounal of compute vision, vol. 6, no. 2, pp. 9, 24. [23] R. Mu-Atal, J. M. M. Montiel, and J. D. Tados, Ob-slam: a vesatile and accuate monocula slam system, IEEE Tansactions on Robotics, vol. 3, no. 5, pp , 25. [24] C. McManus, W. Chuchill, W. Madden, A. D. Stewat, and P. Newman, Shady dealings: Robust, long-tem visual localisation using illumination invaiance, in Robotics and Automation (ICRA), 24 IEEE Intenational Confeence on. IEEE, 24, pp [25] P. Coke, R. Paul, W. Chuchill, and P. Newman, Dealing with shadows: Captuing intinsic scene appeaance fo image-based outdoo localisation, in Intelligent Robots and Systems (IROS), 23 IEEE/RSJ Intenational Confeence on. IEEE, 23, pp [26] P. Neubet, N. Sundehauf, and P. Potzel, Appeaance change pediction fo long-tem navigation acoss seasons, in Mobile Robots (ECMR), 23 Euopean Confeence on. IEEE, 23, pp [27] O. Vysotska and C. Stachniss, Lazy data association fo image seuences matching unde substantial appeaance changes, IEEE Robotics and Automation Lettes, vol., no., pp , 26. [28] T. Nasee, L. Spinello, W. Bugad, and C. Stachniss, Robust visual obot localization acoss seasons using netwok flows. in AAAI, 24, pp [29] J. Skinne, S. Gag, N. Sündehauf, P. Coke, B. Upcoft, and M. Milfod, High-fidelity simulation fo evaluating obotic vision pefomance, in Intelligent Robots and Systems (IROS), 26 IEEE/RSJ Intenational Confeence on. IEEE, 26, pp [3] N. Sündehauf, P. Neubet, and P. Potzel, Ae we thee yet? challenging seslam on a 3 km jouney acoss all fou seasons, in Poc. of Wokshop on Long-Tem Autonomy, IEEE Intenational Confeence on Robotics and Automation (ICRA). Citesee, 23, p. 23. [3] D. Gálvez-López and J. D. Tados, Bags of binay wods fo fast place ecognition in image seuences, IEEE Tansactions on Robotics, vol. 28, no. 5, pp , 22. [32] A. Glove, W. Madden, M. Waen, S. Reid, M. Milfod, and G. Wyeth, Openfabmap: An open souce toolbox fo appeaancebased loop closue detection, in Robotics and automation (ICRA), 22 IEEE intenational confeence on. IEEE, 22, pp [33] O. Vysotska and C. Stachniss, Relocalization unde substantial appeaance changes using hashing. [34] D. Ball, S. Heath, J. Wiles, G. Wyeth, P. Coke, and M. Milfod, Openatslam: an open souce bain-based slam system, Autonomous Robots, vol. 34, no. 3, pp , 23. [35] M. J. Milfod, G. F. Wyeth, and D. Passe, Ratslam: a hippocampal model fo simultaneous localization and mapping, in Robotics and Automation, 24. Poceedings. ICRA IEEE Intenational Confeence on, vol.. IEEE, 24, pp [36] J. Engel, T. Schöps, and D. Cemes, Lsd-slam: Lage-scale diect monocula slam, in Euopean Confeence on Compute Vision. Spinge, 24, pp [37] R. Mu-Atal and J. D. Tadós, Ob-slam2: An open-souce slam system fo monocula, steeo, and gb-d cameas, IEEE Tansactions on Robotics, vol. 33, no. 5, pp , 27. [38] M. Milfod, J. Fin, J. Beattie, A. Jacobson, E. Peppeell, E. Mason, M. Kimlin, and M. Dunbabin, Automated sensoy data alignment fo envionmental and epidemal change monitoing, in Austalasian Confeence on Robotics and Automation 24. Austalian Robotic and Automation Association, 24, pp.. [39] F. Zeng, A. Jacobson, D. Smith, N. Boswell, T. Peynot, and M. Milfod, Enhancing Undegound Visual Place Recognition with Shannon Entopy Saliency, in Austalas. Conf. Robot. Autom. ACRA, Sydney, Austalia, 27. [Online]. Available: http: //