International Journal of Computer Engineering and Applications, Volume XII, Special Issue, March 18, ISSN

Transcription

1 USABILITY OF MAPREDUCE NEURAL NETWORK FRAMEWORK IN IMAGE RECOGNITION 1 Dr. Sitalakshmi Venkatraman 2 Dr. Siddhivinayak Kulkarni 1 Department of Information Technology, Melbourne Polytechnic, Prahran, Australia 2 Department of Computer Engineering, Maharashtra Institute of Technology Pune, India ABSTRACT SitaVenkat@melbournepolytechnic.edu.au siddhivinayak.kulkarni@mitcoe.edu.in Image recognition has wide applications in the context of computer vision such as medicine, smart digital libraries, sports, crime detection, accessibility for the visually impaired, targeted advertising, multimedia, and other advanced research. Recent multimedia digital advances have resulted in large image collections forming major subset of Internet of Things (IoT). They are located in distributed l data centers and efficient and accurate image recognition becomes a major challenge. It becomes increasingly important to develop new content-based image retrieval (CBIR) and identification techniques for real-time image recognition. This paper presents the usability of a novel MapReduce neural network framework in certain image processing applications. We apply Map and Reduce functions for efficient image retrieval, classification and identification. Experimental results show the desired level of accuracy achieved in image detection within the context of two main real-life scenarios, one using natural language queries and the other using template image matching. Keywords-Image recognition, CBIR, artificial neural networks, MapReduce, natural language query, template matching Dr. Sitalakshmi Venkatraman, Dr. Siddhivinayak Kulkarni 1

2 USABILITY OF MAPREDUCE NEURAL NETWORK FRAMEWORK IN IMAGE RECOGNITION 1. INTRODUCTION Image recognition involves machine vision technologies that combine a camera with artificial intelligence algorithms to identify various objects, living beings, written scripts, locations, signs, and actions through images [1][2][3]. Image recognition helps people to perform a large number of automated image classification and aids in visual tasks such as navigation. Image recognition can be used to label the content of images in digital libraries, guide autonomous robots or people for locomotion, and perform image search which have various discipline-specific applications. In image recognition, content-based image retrieval (CBIR) techniques are used to retrieve specific images based on feature extraction from large databases [4][5][6]. Various features such as colour, texture, object or shape that are called primitive features and logical features such as the identity of the objects in the image are usually combined with abstract attributes such as the significance or relevance depending on the context of the image [7][8]. With exponential growth in Internet of Things (IoT), mountains of images are available in the digital media and this poses a major challenge to achieve the desired efficiency and accuracy for real-time image recognition [9][10]. In this paper, we present the usability of a MapReduce neural network framework for processing from very large image databases that overcomes the current limitations of CBIR techniques due to their computationally complexity and inefficiency for real-time applications. CBIR systems predominantly make use query images as templates for image retrieval and recognition [11][12][13]. However, since template images may not be available for all applications, there are inherent usability issues [14][15]. User-centric flexible image recognition is warranted. We demonstrate the usability of the proposed MapReduce neural network framework to two contrast scenarios: 1. using natural language based user queries in the absence of template images, and 2. using visual signs as template images for visually impaired persons. In scenario 1, our technique overcomes the usability limitations of current CBIR systems as users are can use natural language for their queries. We combine primitive features of image colours such as red, blue and green, along with content types such as low, medium, high and very high in natural language queries. The computational intensive processing is overcome by adopting parallelism in our MapReduce neural network framework [16][17][6]. In scenario 2, our method makes use of many emergency exit signs and symbols as templates available in public places such as hospitals, offices and malls. Such signs and symbols are usually available in a fixed format, for example, image showing a running person, a door, an arrow pointing to a direction, the word Exit, etc. Such a visual template based CBIR system is useful for visually impaired persons, as well as serve as direction aids to people who are unfamiliar with a particular location [18]19[][20]. The rest of paper is organized as follows. Section 2 describes the proposed use of a novel MapReduce neural network framework as a CBIR technique. Section 3 presents two scenarios with experimental results of the usability of our CBIR technique. Finally, conclusions and future work of this research are given in Section 4. I.USE OF MAPREDUCE NEURAL NETWORK FRAMEWORK FOR CBIR The limitations of traditional methods of image retrieval have led to new CBIR techniques such as QBIC [21], Virage [22], Photobook [23] and Netra [24] that combine similarity measures of Dr. Sitalakshmi Venkatraman, Dr. Siddhivinayak Kulkarni 2

3 different feature classes. However, they are in-efficient in searching from very large and diverse image dataset as well as in processing multiple natural language queries from users [25][26][27]. We describe the usability of a hybrid technique for CBIR based on fuzzy logic and neural networks within a MapReduce distributed framework [9]. The MapReduce framework consists of a master Job Tracker and a slave Task Tracker in each cluster nodes as illustrated in Figure 1. This facilitates parallel processing in a cluster environment for the MapReduce search tasks that are assigned by the job tracker to the task tracker present in each of the nodes. Natural language queries from the client are processed in parallel to retrieve matched images from large databases. We combine the primitive features of colour s as well as fuzzy terms of colour content such as 'no colour', 'very low', 'low', 'medium', 'high', and 'very high' for colour image classification using neural network to train multiple user queries. Figure 1. MapReduce Architecture Supervised learning neural network can efficiently learn the colours and content types using neural network ensembles (NNE) to produce more accurate outcomes as they combine different networks such as multilayer perceptrons, radial basis functions neural networks, and probabilistic neural networks [28][29]. Figure 2 shows a typical NNE training model using MapReduce architecture for image classification. Dr. Sitalakshmi Venkatraman, Dr. Siddhivinayak Kulkarni 3

4 USABILITY OF MAPREDUCE NEURAL NETWORK FRAMEWORK IN IMAGE RECOGNITION Figure 2. NNE Training Model for Image Classification In the MapReduce architecture, the Map function inputs a list input key and associated values and produces a list of intermediate <key, value> pairs, and the Reduce function performs the merge operation on all intermediate pairs for the same key and outputs the results. Figure 3 shows the MapReduce implementation of the CBIR framework used for processing images with <key, value> pairs as shown below: Map: < Feature1, File1 > List < Feature2, File2 > Reduce: < Feature2, List < File2 >> List < Feature3, File3 > Figure 3. Illustration of MapReduce Implementation for CBIR Dr. Sitalakshmi Venkatraman, Dr. Siddhivinayak Kulkarni 4

5 I. EXPERIMENTAL RESULTS FOR TWO SCENARIOS We present two contrast scenarios with experimental results showing the usability of our MapReduce neural network framework. The experiment was conducted using thousands of images collected from public domains A. Scenario 1: Using Natural Language Queries This scenario typically involves retrieving images from larger datasets based on natural language queries posed by the user. In the absence of template images for image recognition, the queries are processed based on the colour and content parameters specified by the user in natural language [30][31][32]. Such query-based image retrieval is typically required in smart digital libraries. Our experiment involves matching those images that satisfy user queries from the large data collection of images. For example, the output of the features of the experiment with natural language query, where we can see the main colours as orange, black, grey and yellow are shown in Figure 4. Image Colour % Fuzzy Logic Terms Used Red 0.52 Orange 44.2 Yellow 14.5 Green 2.07 Cyan 0.11 Blue 0.92 Purple 0.02 Magenta Pink Black White Grey Very Low Medium Low Very Low No Colour Very Low No Colour No Colour Very Low Low No Colour Low Figure 4. Features extracted using natural language query Feature set F O for the colour Orange in an image is given by the following formula: F O = N O N P where, N O represents the number of Orange colour pixels in the image. and N P represents the total number of pixels in the image [33][34. These features extracted were stored in text files for each image. These are in the form of pairs <Feature#, File#> of the Map function Dr. Sitalakshmi Venkatraman, Dr. Siddhivinayak Kulkarni 5

6 USABILITY OF MAPREDUCE NEURAL NETWORK FRAMEWORK IN IMAGE RECOGNITION of the MapReduce framework. The user query could have a combination of colours from the set {red, orange, yellow, green, cyan, blue, purple, magenta, pink} and content type in natural language terms from the set {no colour, very low, low, medium, high, very high}. A neural network ensemble (NNE) based fusion of classes are performed and the image classification is done using the Reduce function of the MapReduce framework. Figure 5 provides the top four images retrieved in descending order of percentage relevance (P) values when the user query was 'medium' orange. Figure 5. CBIR results for user query 'medium' orange Image Classification and Retrieved for 'medium' orange B. Scenario 2: Using Image Templates In this scenario, image recognition is done by matching with a database of template images. For instance, by using visual signs as template images for detecting emergency signs, visually impaired persons could navigate easily in public places [35][36][37]. We consider emergency exit signs as input images forming the templates. The image recognition platform recognises various forms of emergency exit signs and gives an acoustic signal. If an arrow is present on the sign, the direction of the sign is also indicated which aids in the ease of navigation. The dataflow diagram for the image recognition via a mobile phone with camera is shown in Figure 6. P = 57.6% P = 40.28% P = 41.57% P = 31.42% Figure 5. Dataflow diagram for template image matching The mobile device captures an image through the camera and the MapReduce functions are made use in the image matching algorithm to recognises exit signs and finally an acoustic signal is given as the output to the user. This works in real-time, simplifying the mobility for the visually challenged and giving them a user-friendly navigation tool. Figure 7 shows an outcome of an exit sign successfully detected. Dr. Sitalakshmi Venkatraman, Dr. Siddhivinayak Kulkarni 6

7 CONCLUSIONS Image recognition plays an important role in a variety of disciplines, such as health, security, arts, and more recently in everyday life due to advances in mobile technologies and computing. In addition to efficiently retrieving the images, this paper focused on the usability. A novel MapReduce Neural Network framework for CBIR was adopted with Fuzzy logic in neural network ensembles for processing natural language queries as well template matching. The classification of images according to fuzzy content was successfully performed and preliminary results were demonstrated in the paper. ACKNOWLEDGMENT The authors wish to thank student groups in developing the software in different stages, which helped to conduct the initial experimental study as part of this ongoing research work. Dr. Sitalakshmi Venkatraman, Dr. Siddhivinayak Kulkarni 7

8 USABILITY OF MAPREDUCE NEURAL NETWORK FRAMEWORK IN IMAGE RECOGNITION REFERENCES [1] M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele and P. B. Verma, and S. Kulkarni, Fuzzy Logic Based Interpretation and Fusion of Colour Queries, Journal of Fuzzy Sets and Systems, 147(1), pp , [2] R.Fernando, and S. Kulkarni, Hybrid technique for colour image classification and efficient retrieval based on fuzzy logic and neural networks, International Joint Conference on Neural Networks (IJCNN),: June [3] S. Kulkarni, Natural Language based Fuzzy Queries and Fuzzy Mapping of Feature Database for Image Retrieval. Journal of Information Technology and Applications. 4(1), 11-20, [4] B. Verma, and S. Kulkarni, Fuzzy Logic Based Interpretation and Fusion of Colour Queries, Journal of Fuzzy Sets and Systems, 147(1), pp , [5] A. Gupta, Visual Information Retrieval: A Virage Perspective, Technical Report Revision 4, Virage Inc, San Diego, CA 92121, [6] A. Pentland, R. Picard and S. Sclaroff, Photobook: Content-based Manipulation of Image Databases, International Journal of Computer Vision, Vol. 3, pp , [7] A. Parulekar, R. Datta, J. Li and J. Wang. J, Large-scale Satellite Image Browsing using Automatic Semantic Categorization and Content-based Retrieval, Proceedings of IEEE International Workshop on Semantic Knowledge in Computer Vision, , [8] Gao Li-chun and Xu Ye-qiang. Image retrieval based on relevance feedback using blocks weighted dominant colors in MPEG-7. Journal of Computer Applications.vol.31(6), pp , [9] Venkatraman, S. and Kulkarni, S. MapReduce Neural Network Framework for Efficient Content Based Image Retrieval from Large Datasets in the Cloud, Proceedings of 12th International Conference on Hybrid Intelligent Systems, ICHIS 2012, Pune, [10] M., Islam, S. Venkatraman, and M. Alazab, Stochastic Model Based Approach for Biometric Identification, Proceedings of IEEE s International Joint Conferences on Computer, Information, and Systems Sciences, and Engineering (CISSE-IETA 09), 4-12 December, University of Bridgeport, USA (2009), Technological Developments in Networking, Education and Automation, Springer, Netherlands,, ISBN , pp , 2010, [11] S. Chatzichristofis and Y. Boutalis, Content based Radiology Image Retrieval using a Fuzzy Rule based Scalable Composite Descriptor, Journal of Multimedia Tools and Applications, 46 (2-3), pp , [12] A. Torralba, R. Fergus, and W. T. Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(11):1958{1970, [13] R. Datta, J. Li and J. Wang, Content-Based Image Retrieval -A Survey on the Approaches and Trends of the New Age, Proceedings of ACM International Workshop on Multimedia Information Retrieval, ACM Multimedia, Singapore, pp , [14] C. Shahabi, and Y.Chen, Soft Query in Image Retrieval Systems, Proceedings of the SPIE Internet Imaging (EI14), Electronic Imaging, Science and Technology, San Jose, California, pp , [15] P. Ciaccia, D. Montesi, W. Penzo, and A. Trombetta. Fuzzy Query Languages for Multimedia Data. Book Chapter in Design and Management of Multimedia Information Systems: Opportunities and Challenges, M.R. Syed Ed., Idea Group Publishing, Hershey, PA, USA, [16] Y. Chen, and J. Wang, A Region based Fuzzy Feature Matching Approach to Content based Image Retrieval, IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 24, Number 9, pp , [17] S. Yu, and M. Dunham, A Graph-based Fuzzy Linguistic Metadata Schema for Describing Spatial Relationships, Proceedings of International Symposium on Visual Information Communication, Article no. 14, [18] S. Venkatraman, User-centric Ontology for Smart Holistic Health Information Systems, Journal Series in Multidisciplinary Research (IJSMR), Archives of Medical and Health Sciences, 3, 2017, pp , [19] H. Lee, and S. Yoo, Applying Neural Network to Combining the Heterogeneous Features in Contentbased Image Retrieval, Proceedings of SPIE Applications of Artificial Neural Networks in Image Processing, 4305(13), pp , [20] J. Dean and S. Ghemawat. Mapreduce: simplified data processing on large clusters. Commun. ACM, 51(1), pp , [21] X. Kanglin and W. Xiaoling, Application of the Fuzzy Logic in Content-based Image Retrieval, Journal of Computer Science & Technology, 5(1), pp , [22] M., Alazab, M. Islam, and S. Venkatraman, Towards Automatic Image Segmentation Using Optimised Region Growing Technique, Proceedings of 22nd Australasian Joint Conference on Artificial Dr. Sitalakshmi Venkatraman, Dr. Siddhivinayak Kulkarni 8

9 Intelligence (AI 09,), 1-4 December, Melbourne, Australia, Lecture Notes in Computer Science, Springer Berlin/Heidelberg, ISBN , Vol. 5866, pp , [23] P. Muneesawang, and L. Guan, Automatic Machine Interactions for CBIR using Self Organized Tree Map Architecture, IEEE Transactions on Neural Networks, 13(4), pp , [24] M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele and P. Yanker, Query By Image and Video Content: The QBIC System, IEEE Computer, Vol. 28, Number 9, pp , September [25] A. Gupta, Visual Information Retrieval: A Virage Perspective, Technical Report Revision 4, Virage Inc, San Diego, CA 92121, 1996, URL: [26] A. Pentland, R. Picard and S. Sclaroff, Photobook: Content-based Manipulation of Image Databases, International Journal of Computer Vision, 3, pp , [27] W. Ma and B. Manjunath, NETRA: A Toolbox for Navigating Large Image Databases, Journal of ACM Multimedia Systems, 7(3}, pp , [28] C. Zhang and X. Chen, Region-Based Image Clustering and Retrieval Using Multiple Instance Learning, Proceedings of 4th International Conference on Image and Video Retrieval, Singapore, pp , [29] Venkatraman, S. and Kulkarni, S. (2010), Risk-Based Neuro-Grid Architecture for Multimodal Biometrics, Proceedings of IEEE s International Conference on Systems, Computing Sciences and Software Engineering (CISSE-SCSS 09), 4-12 December, University of Bridgeport, USA (2009), Innovations in Computing Sciences and Software Engineering, Springer, Netherlands, ISBN , pp [30] M. Paterno, F. Lim, and W. Leow, Fuzzy Semantic Labelling for Image Retrieval, International Conference on Multimedia and Expo, Vol. 2, pp , [31] S. Kulkarni, Natural Language based Fuzzy Queries and Fuzzy Mapping of Feature Database for Image Retrieval. Journal of Information Technology and Applications. 4(1), 11-20, [32] R.Fernando, and S. Kulkarni, Hybrid technique for colour image classification and efficient retrieval based on fuzzy logic and neural networks, International Joint Conference on Neural Networks (IJCNN),: June [33] S.Vidivelli and S.Sathiya devi. Wavelet based integrated color image retrieval. IEEE-International Conference on Recent Trends in Information Technology, ICRTIT, [34] S. Kulkarni, T. Joshi, K. Biyani, S. Bhide 2016, Framework For Authentication System Based On Keystroke Dynamics For Online Banking, International Journal of Computer Engineering and Applications, Vol. 10, No. 3, 1-15 [35] E. Oja, J. Laaksonen, M. Koskela, and S. Brandt, Self Organising Maps for Content Based Image Database Retrieval, In Oja, E. and Kaski, S., editors, Kohonen Maps, Amsterdam, pp , [36] Y. Yang and J. Laaksonen, Partial Relevance in Interactive Facial Image Retrieval, Proceedings of 3rd International Conference on Advances in Pattern Recognition, Bath, UK, pp , [37] S. Venkatraman, and S. Kulkarni, The Impact of Biometric Systems on Communities: Perspectives and Challenges, Proceedings of Australian Conference for Knowledge Management and Intelligent Decision Support, 8-10 December, Australia., Dr. Sitalakshmi Venkatraman, Dr. Siddhivinayak Kulkarni 9