Traffic Sign Localization and Classification Methods: An Overview Ivan Filković University of Zagreb Faculty of Electrical Engineering and Computing Department of Electronics, Microelectronics, Computer and Intelligent Systems ivan.filkovic@fer.hr 24.10.2014.
Content Introduction Motivation Traffic Signs Traffic Sign Recognition Traffic Sign Localization State of the Art Integral Channel Features Detector Traffic Sign Classification State of the Art Convolutional Neural Networks Conclusion
Introduction Motivation Motivation Applications Advance Driver Assistance Systems - ADAS Systems for mapping and assessing the state of traffic infrastructure Autonomous vehicles Project VISTA: within the system for detection, tracking and recognition of traffic signs Existing commercial systems Possibility of improving performance Removing restriction like driving in non-urban areas (e.g. only on motorways) Increase detectable and recognizable subset of traffic signs (supported only speed limit signs)
Introduction Traffic Signs Traffic Signs Simple objects, limited by shape and color Noticeable to humans (intense colors, regular forms and reflective surfaces) Problems Changing lighting conditions Partial occlusion Changing weather conditions Different traffic signs perspective Visually similar subsets Standardization
Introduction Traffic Signs Standardization Vienna Convention on Road Signs and Signals (1968) Minor differences between countries 62 signatory countries (Croatia; most European countries) Problem Inability to develop a unified traffic sign recognition system
Introduction Traffic Sign Recognition Traffic Sign Recognition = Traffic Sign Localization + Traffic Sign Classification
Introduction Traffic Sign Recognition Traffic Sign Localization Are there traffic signs in image? Location?
Introduction Traffic Sign Recognition Traffic Sign Classification Determining traffic sign type?
Traffic Sign Localization Traffic Sign Localization Approaches State of the art is based on machine learning techniques Traffic sign localization methods Feature extraction/selection Learning algorithm Problems Localization inaccuracy False alarms Computationally expensive Real time requirement
Traffic Sign Localization State of the Art [Liu, 2014] et. al. presented traffic sign localization framework Novel ideas: 1. Multi-block normalized locale binary pattern (MN-LBP), Tilted MN-LBP features 2. Coarse-to-fine classifier, Split-flow cascade (SFC) 3. Derivate of AdaBoost, Common-finder AdaBoost (CF.AdaBoost)
Traffic Sign Localization State of the Art MN-LBP and Tilted MN-LBP features Express the different patterns found on different traffic signs Similar to classical LBP and MB-LBP Pixel sum values of eight rectangles around the center are compared with an average value to obtain a binary sequence
Traffic Sign Localization State of the Art Split-flow cascade Coarse-to-fine tree structure classifier (improves detection time) Incorporates features shared among multiple classes of traffic signs Branching nodes have same structure as Viola and Jones cascade
Traffic Sign Localization State of the Art Common-finder AdaBoost Finds shared features among different feature sets (efficient SFC-tree) Boosting algorithm selects good features for each training set Sets are then intersected to find same features Shared features are once again trained
Traffic Sign Localization State of the Art [Mathias, 2013] et. al. applied Integral Channel Features detector Detector builds on HOG features and boosted decision trees Very high speed and state-of-the-art performance
Traffic Sign Localization Integral Channel Features Detector Integral Channel Features Detector Sliding window over multiple scales Main focus is on choice of features Diverse information from multiple features Haar-like features Local sums Histograms Integral Channel Features are computed using linear and non-linear transformations from different image channels Learning algorithm is standard boosting
Traffic Sign Localization Integral Channel Features Detector Integral Channel Features Detector Large number of first-order and second-order candidate features are generated randomly First-order features Sum of rectangular regions over different channels Six HOG orientation bins Magnitude of gradient Three LUV color channels Second-order features Weighted sums of first-order features Boosting variant called soft cascade Simple threshold is used after evaluation of every weak classifier Values are set by pruning algorithm which assumes that examples with small weights can be removed early
Traffic Sign Localization Integral Channel Features Detector Integral Channel Features Detector Weak classifiers Decision trees with depth of two Optimization Precomputed features are cached in the beginning of training
Traffic Sign Classification Traffic Sign Classification Approaches Based on machine learning Generic approach Applicable to various object classification Difficult multi-class classification problem Problems Large number of traffic sign classes Wast amount of good quality examples Real time requirement
Traffic Sign Classification State of the Art [Ciresan, 2011] et. al. Committee of Convolutional Neural Networks (CNNs) and a Multi-layer Perceptron (MLP) Preprocessing Examples are resized to 48 x 48 pixels Histogram equalization CNNs architecture Feed-forward architecture Initial weights: sampled from uniform distribution Sub-sampling layers: max-pooling Seven hidden layers 43 neurons in the output layer (one for each class) Neuron Activation function: scaled hyperbolic tangent Output represent class probability Learning algorithm Stochastic gradient descent MPL trained on pre-calculated features (HOG3)
Traffic Sign Classification Convolutional Neural Networks Convolutional Neural Networks Inspired by simple and complex cells in the primary visual cortex of a brain Differ in training procedure and in implementation of convolutional and sub-sampling layers Convolutional layers Number of maps (M) Size of maps (mx,y ) Kernel sizes (kx,y ) Skipping factors (s x,y ) Sub-sampling layers Max-pooling Average-pooling Output Downscaling the output maps of the last convolutional layer to 1 pixel per map Combining the output of last convolutional layer into feature vector
Traffic Sign Classification Convolutional Neural Networks Convolutional Neural Networks mx,y n = mn 1 x,y kx,y n sx,y n + 1 + 1
Conclusion Conclusion Relevant research in traffic sign localization and classification was presented Development of new methods improved results of traffic sign classification and localization Accuracy and real time performance are not satisfying for ADAS application
References Liu, C. and Chang, F. and Chen, Z. (2014) Rapid Multiclass Traffic Sign Detection in High-Resolution Images Intelligent Transportation Systems, IEEE Transactions on PP99, 1 10. Mathias, M. and Timofte, R. and Benenson, R. and Van Gool, L. (2013) Traffic sign recognition - How far are we from the solution? The International Joint Conference on Neural Networks Ciresan, D. and Meier, U. and Masci, J. and Schmidhuber, J. (2011) A committee of neural networks for traffic sign classification The International Joint Conference on Neural Networks
The End