Stable Road Lane Model Based on Clothoids C Gakstatter*, S Thomas**, Dr P Heinemann*, Prof Gudrun Klinker*** *Audi Eletronis Venture GmbH, **Leibniz Universität Hannover, ***Tehnishe Universität Münhen Abstrat In the following, the main onepts of a road lane model are presented that keeps trak of an arbitrar number of lane borders Information from an existing lane detetion devie, a grosensor, and map data are merged and filtered to reate a road model with a desired number of road lines The model is based on lothoids and ontinuousl provides positions, angles, and urvatures of the border lines of the vehile s own lane as well as of several neighboring lanes Partiularl on urban roads, in situations with upoming turning lanes, or when the lane detetion sstem fails to detet road lines, the model an still provide plausible information This information signifiantl simplifies the situation analsis in further algorithms that rel on a lane detetion sstem and require detailed information on urrent road lanes Introdution Various advaned driver assistane sstems are based on vision-based lane detetion sstems to determine harateristis of the urrent road and its lanes Examples are lane departure warning or the lane keeping assistant that use the lane detetion to support the driver to keep the lane If the driver is distrated or areless, the sstem warns the driver b vibration in the steering wheel or b direting him bak to the middle of his lane Diffiulties arise when the lane detetion sstem fails to reognize road lines or interprets objets as road lines that are not atuall part of the urrent road Espeiall in urban regions sstems reah their limits Vision-based approahes like in [] use edge-oriented methods for a robust detetion of road lines and approximate the ourse of the lanes b lothoids These methods still fail if line markings are ompletel missing or overed b other vehiles Subsequent sstems that are based on lane detetion have to deal with these inauraies and reat properl Current driver assistane sstems that rel on a lane detetion method are mainl foused on highwa situations If the sstem needs onl information on the own lane or if the vehile drives on a highwa with lear road markings, approahes as desribed in [] an be used It presents a multi sensor approah that fuses image measurements and map data to improve the traking of road boarders in highwa senarios Another method as shown in [3] uses six ues and a partile filter to ahieve robust lane traking This sstem is robust against dramati lane hanges and disontinuous hanges in road harateristis B using the Distillation Algorithm to trak the vehiles pose relative to the road and their width, the sstem seems to be stable in situations that are ritial for vision-based lane trakers A omparison of lane position detetion and traking tehniques is presented in [4] One of the methods listed is desribed in detail in [5] This approah uses a road model based on lothoids and traks them b a linear vehile dnamis model This road model, however, enounters problems when edge detetion fails in omplex situations If detailed information on the urrent road is needed and when driving on urban roads, it beomes hallenging to guess the road geometr in situations when line detetion fails This happens for example if road lines are oluded b other vehiles, or if the urvatures are ver high Therefore, algorithms are needed to deide whih deteted road lines ontain useful information, whih lines should be ignored, and to determine the position of road lines that were not deteted at all The advantage of our sstem is that additional information on the urrent road is provided ontinuousl, in partiular with regard to urban roads, at intersetions, and exit ramps The road lane model presented in this paper provides a stable estimate of the urrent road geometr at an time It selets suitable road lines from a vision-based lane detetion sstem and guesses the harateristis of missing road lines b neighboring lines and a grosensor as illustrated in Fig For eah line, a Kalman filter is used to keep trak of the lateral distane to the vehile, the heading diretion, and the lothoid parameters
Fig Road lane model sstem overview Road Lane Model The road model in its urrent implementation keeps trak of eight road lines, four on the right side and four on the left side of the vehile This allows providing information on at least four lanes at an time More lines ould be modeled easil, but due to their high lateral offset, the are unlikel to be reognized b lane detetion sstems The benefit of modeling parallel road lines individuall beomes obvious on urban streets, lose to exit ramps, or at the beginning or ending of turning lanes as depited in Fig Fig Road lane model in seleted situations Our implementation inludes Kalman filtering of the line parameters as well as a sensor fusion of the data from lane detetion with the aw rate of the grosensor (setion ) At eah time step, suitable road lines from the lane detetion sstem are seleted and integrated into the urrent model (setion 3) If no line is found at a ertain position, we estimate the line parameters b the inner neighboring line Sine a parallel urve is needed at this step, we perform an optimization of lothoid parameters to obtain a nearl parallel lothoid even at high urvatures (setion 4) In addition, the aw rate sensor ombined with the urrent map data ontinuousl provides information on the relative movement between the vehile and the road (setion 5) Basis of Clothoids Clothoids are a speial tpe of urve that is used for road onstrution to avoid abrupt hanges of the steering angle when driving from straight to irular road setion and vie versa The are defined b their begin urvature, a onstant urvature hange rate and their total length l The urrent urvature of a lothoid after length l equals (l ) = + l
The tangent angle τ at a length l desribes the hange in orientation and is obtained b integration over l : τ(l ) = l + 5 l = + ( l ) l Appliation of the Extended Kalman Filter The Kalman filter is a set of mathematial equations [6] It offers a onvenient wa to estimate the exat state of a tehnial sstem b drawing onlusions from defetive observations For the lane model, we use a distint Kalman filter for eah potential road line to estimate eight different parameters The lothoid parameters,, and l, the lateral offset d, and the heading angle ψ are determined b the lane detetion sstem The heading angle represents the angle between the vehile axis to the road The grosensor ombined with map data provides additional information on the aw rate γ and the lateral aeleration of the line a Therefore, the state vetor for eah line that we want to keep trak of omprises the following parameters: ( d v a ψ γ l) T ˆ = xk All parameters (exept the lothoid parameter) are given in relation of the own vehile Estimations on the urrent aeleration provide additional information for the veloit v at the next time step and also the urrent veloit helps to estimate the next lateral offset d Similarl, the aw rate provides information for the next aw angle ψ and the urvature hange of the lothoid is ruial to determine the urvature at the beginning of the lothoid These onnetions are onsidered in the proess funtion of the disrete Kalman filter that estimates the next state after a time step t b the parameter estimates of the urrent state Due to the heading angle that we have to onsider when alulating the next lothoid parameters and l, the proess step of the Kalman filter is not linear and an extended Kalman filter has to be applied In omparison to a standard Kalman filter, the extended Kalman filter provides estimates even on non-linear proesses The proess funtion used for the road lane model relates the urrent state to the next state as follows: f ( xˆ, u d ) = + t v + 5 t a v + t a a ψ + t γ + γ + ψ t v os l t v osψ k k 3 Seletion of Deteted Lines When road lines are deteted b the lane detetion sstem, a deision algorithm determines if the line fits into a ertain position of the urrent road model The main requirements are a small distane to an existing road line in the model, and suitable distanes to the vehile s enter and the inner neighboring road line, whereas the suitable distane is derived from the urrentl observed lane width To allow for upoming additional lanes and ending lanes as depited in Fig, the distanes to the neighboring line at the beginning or at the end of the line an be smaller 4 Clothoid Parameters Estimation b Neighboring Lines As mentioned in setion, the values for a and γ are ontinuousl provided b the grosensor and map data, but d, ψ,,, and l are onl available if the line is deteted b the lane detetion sstem So if a line is not deteted or its urrent position and shape do not meet the riteria mentioned in setion 3, we estimate the lothoid s
parameters b the parameters of the inner neighboring line For the lateral offset and aw angle, we restrit the values of the missing line to an appropriate range based on the values of the neighboring line However, we have to determine the lothoid parameters,, and l in a wa that the lothoid desribes a parallel urve Sine two lothoids with a urvature hange other than zero annot be parallel [7], we an onl state the expeted lothoid parameters and the expeted hange in orientation of the desired lothoid in a distane r The variable denotes the urvature at the end of the lothoid and both urvature values are the reiproal of the related radius: ~ = + r, ~ = + r ~, l = l + r τ, ~ τ τ = All four onditions should be onsidered to obtain a suitable lothoid, partiularl in ase of high urvatures If we neglet one of the parameters, the resulting lothoid ma onsiderabl differ from the desired parallel urve as demonstrated in Fig 3 Sine a lothoid has onl three parameters, we used a least-squares method to determine an optimal set of lothoid parameters After performing onl a single step of the Gauss-Newton algorithm, the values onverge and we observed that optimal parameters impl onl minimal hanges in length l and angle τ This result orresponds to the observation made in [8], that different urvature parameters an lead to similar urve shapes Therefore, the results of the numerial optimization an be approximated b leaving the expeted length l and hange in orientation τ unhanged and b adjusting the values and To ensure the expeted hange in orientation at the expeted length, the sum of the begin and end urvature has to be ~ τ +, optimized = l, optimized ~ Sine this differs from the sum of the expeted urvature values, we add half of the resulting differene to and : = ~ ~ τ + ( ~ + ~ ~ ), = ~ ~ τ + ( ~ + ~, optimized ~ ) l l, optimized Fig 3 Clothoid parameter estimation for a parallel line to a lothoid with =-, =6, l=6 An example of a lothoid with optimized urvature parameters is shown in Fig 3 In partiular for lothoids that hange between positive and negative urvature or with high urvature in general, the optimization improves the resulting shape of the lothoid signifiantl 5 Parameter Estimation b Grosensor The data fusion with the grosensor beomes important if not a single line is identified b the lane detetion sstem In ombination with the urrent speed v, aeleration a, and the expeted road urvature r, provided b map data, information on the aw rate γ and the lateral aeleration a relative to the road are available:
s r, γ s r ψ ψ ψ γ = γ v osψ a = ( v os ) v os + a sin In both equations, the differene between the measured aw rate γ s and the expeted aw rate r v*osψ due to the urrent road urvature is needed The result is the aw rate relative to the urrent road geometr For the lateral aeleration, the relative aw rate needs to be multiplied with the urrent veloit and the heading angle ψ should be onsidered to get the aeleration in the diretion perpendiular to the street The effets on the road model when onl the lateral aeleration is available and road lines are not deteted are analzed in detail in the following setion 3 Results We tested the road lane model in different situations on urban roads, rural streets, and highwas to ensure the stabilit of the model in situations when vision-based lane detetion fails Eah of the evaluations depited in Fig 4 to Fig 7 shows the distanes of road lines to the vehile and a omparison of the urrent aw rate values The red graphs in the upper images show the estimated lateral offsets d of the inner four road lines, while the blak graphs represent the lateral offsets measured b the vision-based lane detetion sstem If a line is not deteted or ontains no useful information for the urrent road model, its blak graph swithes to zero If all graphs of one olor shift up or down as in Fig 5 at t=4s, this indiates a lane hange to the right or left respetivel In the figures 5 to 7 we purposel ignore all deteted lines for a few seonds to observe the proess of the road model in omparison to the atual road lines that are still depited in gre olor That means the model estimates the urrent positions of road lines solel based on the measured aw rate and map data Furthermore, the lower graphs show a omparison of the measured aw rate in red olor and the expeted aw rate due to the urrent road urvature in blak Negative values are due to a right turn, positive values follow from a turn to the left It beomes obvious that differenes between these two values affet the lateral offsets in the road lane model In Fig 4 we have a ommon inner it situation that demonstrates the diffiulties of vision-based lane detetion on urban roads and partiularl in tight urves Suitable lines are not onstantl deteted, so that the model an onl rel on single, oasionall observed lines or on the measured aw rate and map data In the ase shown, we have a tight right urve with turn rates up to - deg/s Still, the model estimates the lateral offsets qualitativel orret b ontinuousl observing the aw rate and seleting temporaril deteted lines at t=64s and t=67s When several road lines are deteted again after t=7s, all lateral offsets will be adjusted The next example evaluated in Fig 5 represents a highwa senario Due to the good visibilit of road markings, the lane detetion sstem will usuall not fail Also the map data on highwas is based on the more preise Advaned Driver Assistane Sstems (ADAS) qualit and provides appropriate values for the urrent road urvature So even if lane detetion fails as simulated from t=3s to t=48s, we an keep trak of the atual distanes between t=3s and t=4s and reognize the lane hange to the right around t=4s However, due to the high speed (approximatel 3 km/h), the grosensor is sensitive to slight steering orretions as an be seen in the aw rate image at t=45s This auses a positive hange of the lateral offsets in the road model to the extent that it erroneousl assumes another lane hange to the left On urban roads, when driving at an eas rate (approximatel 5 km/h), this problem beomes less eminent As illustrated in Fig 6, we an detet a lane hange without the mentioned effet The model follows the atual positions of road lines for seonds with a final deviation of not even m This result demonstrates the potential qualities of the model to trak road markings independentl from measured lines for a ertain time period To ahieve this performane, the qualit of the map data and the grosensor is ruial Fig 7 presents the effet of poor digitalized map data Although the vehile did not perform an lane hange, the model wrongl estimates lateral offsets that impl several lane hanges
Fig 4 Line positions and aw rates (lane detetion failure) Fig 5 Line positions and aw rates (highwa) Fig 6 Line positions and aw rates (ADAS qualit) Fig 7 Line positions and aw rates (non-adas qualit) 4 Conlusion In this paper we presented the main onepts of a road lane model that keeps trak of a number of parallel lanes b fusing a vision-based lane detetion sstem with a grosensor, veloit, and map data The model is based on lothoids and delivers positions, angles, and urvatures for a desirable number of parallel lines It is robust against missing lines or erroneous detetions b the vision-based sstem Suitable road lines are seleted b their length and distanes to neighboring lines Missing lines are estimated b numeriall optimized lothoids that desribe parallel urves even at high urvatures If no lines are deteted, the model an still provide plausible information b traking the ego motion and omparing it to map data The better this map data and the aw rate values, the longer the model delivers plausible information In further steps, the road geometr of map data an be improved b additional foresighted sensors Also the aw rate parameters ma be adapted on highwa senarios to be able to deal with high veloities
Referenes [] Goldbek, J, Huertgen, B, Lane detetion and traking b video sensors, Proeedings of IEEE Intelligent Transp Sstems, pp 74-99, 999 [] Cramer, H, et al, A new approah for traking lanes b fusing image measurements with map data, IEEE Intelligent Vehiles Smposium, pp 67-6, 4 [3] Apostoloff, N, Zelinsk, A, Robust vision based lane traking using multiple ues and partile filtering, IEEE Intelligent Vehiles Smposium, pp 558-563, 3 [4] MCall, J, Trivedi, M, Video-based lane estimation and traking for driver assistane: surve, sstem, and evaluation, IEEE Intelligent Transp Sstems, pp -37, 6 [5] Dikmanns, E, Msliwets, B, Reursive 3-D Road and Relative Ego-State Reognition, IEEE Trans on Pattern Analsis and Mahine Intell, vol 4, no, pp 99-3, 99 [6] Welh, G, Bishop, G, An Introdution to the Kalman Filter, SIGGRAPH, Annual Conferene on Computer Graphis, ourse 8, [7] Bäumker, M, Rehenverfahren der Ingenieurvermessung [8] Swartz, D, Clothoid Road Geometr Unsuitable for Sensor Fusion, IEEE Intelligent Vehiles Smposium, pp 484-488, 3 Christina Gakstatter Sahsstraße 8 858 Gaimersheim, German hristinagakstatter@audide Dr Patrik Heinemann Sahsstraße 8 858 Gaimersheim, German patrikheinemann@audide Sven Thomas Institut für Informationsverarbeitung Fakultät für Elektrotehnik und Informatik Leibniz Universität Hannover Appelstraße 9a, 367 Hannover, German thomas-sven@gmxde Prof Gudrun Klinker Boltzmannstraße 3 85748 Garhing, German klinker@intumde Kewords: Driver assistane, road model, lane model, lothoids, Kalman filter, sensor data fusion