Intelligent Systems Reference Library

Transcription

1 Intelligent Systems Reference Library Volume 145 Series editors Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland Lakhmi C. Jain, University of Canberra, Canberra, Australia; Bournemouth University, UK; KES International, UK URL:

2 The aim of this series is to publish a Reference Library, including novel advances and developments in all aspects of Intelligent Systems in an easily accessible and well structured form. The series includes reference works, handbooks, compendia, textbooks, well-structured monographs, dictionaries, and encyclopedias. It contains well integrated knowledge and current information in the field of Intelligent Systems. The series covers the theory, applications, and design methods of Intelligent Systems. Virtually all disciplines such as engineering, computer science, avionics, business, e-commerce, environment, healthcare, physics and life science are included. The list of topics spans all the areas of modern intelligent systems such as: Ambient intelligence, Computational intelligence, Social intelligence, Computational neuroscience, Artificial life, Virtual society, Cognitive systems, DNA and immunity-based systems, e-learning and teaching, Human-centred computing and Machine ethics, Intelligent control, Intelligent data analysis, Knowledge-based paradigms, Knowledge management, Intelligent agents, Intelligent decision making, Intelligent network security, Interactive entertainment, Learning paradigms, Recommender systems, Robotics and Mechatronics including human-machine teaming, Self-organizing and adaptive systems, Soft computing including Neural systems, Fuzzy systems, Evolutionary computing and the Fusion of these paradigms, Perception and Vision, Web intelligence and Multimedia. More information about this series at

3 Halina Kwaśnicka Lakhmi C. Jain Editors Bridging the Semantic Gap in Image and Video Analysis 123

4 Editors Halina Kwaśnicka Department of Computational Intelligence Wrocław University of Science and Technology Wrocław Poland Lakhmi C. Jain Faculty of Education, Science, Technology and Mathematics University of Canberra Canberra, ACT Australia ISSN ISSN (electronic) Intelligent Systems Reference Library ISBN ISBN (ebook) Library of Congress Control Number: Springer International Publishing AG, part of Springer Nature 2018 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Printed on acid-free paper This Springer imprint is published by the registered company Springer International Publishing AG part of Springer Nature The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

5 Preface Tremendous advances in intelligent techniques and wide applications of vision systems have resulted in an exponential increase in research in the field of image interpretation, also called as semantic analysis of images. Semantic gap is a challenging area for research. The semantic gap is a difference between low-level features extracted from the image and the high-level semantic meanings that people can recognize on the image. The scene understanding is the highest level of image processing. The result strongly depends on the lower-level vision techniques, such as features selection, segmentation, objects recognition, and so on. The chapters in the book deal with different stages of image processing influenced on bridging the semantic gap. Chapter 1 introduces the semantic gap in image and video analysis. Chapter 2 reports and compares a selected set of feature extractors. These extractors are chosen due to their use in a wide number of smart machine vision systems. Chapter 3 is on promising conformity of segmentation results with semantic image interpretation. Relations between semantics-based image annotation and SIMSER features are investigated. Chapter 4 is also on segmentation. It describes the active partitions technique which is a generalization of known active contour approach. Chapter 5 reports 3D object recognition techniques and object model. Chapter 6 demonstrates that structured video annotations can be efficiently queried manually or programmatically and can be used in scene interpretation, video understanding, and content-based video retrieval. Chapter 7 presents an overview of deep learning in semantic gap at different levels of image processing. The book is directed to the researchers, practitioners, students, and professors in the field of semantic image processing, multimedia processing, and deep learning applied to semantic gap. The book was meant to indicate the direction of research v

6 vi Preface for all who seek an answer to the question: how to overcome the semantic gap in images and video analysis. We wish to express our gratitude to the authors and reviewers for their contributions. The assistance offered by the Springer-Verlag is acknowledged. Wrocław, Poland Canberra, Australia Halina Kwaśnicka Lakhmi C. Jain

7 Contents 1 Semantic Gap in Image and Video Analysis: An Introduction... 1 Halina Kwaśnicka and Lakhmi C. Jain 1.1 Introduction Chapters Included in the Book Conclusion... 5 References Low-Level Feature Detectors and Descriptors for Smart Image and Video Analysis: A Comparative Study... 7 D. Avola, L. Cinque, G. L. Foresti, N. Martinel, D. Pannone and C. Piciarelli 2.1 Introduction Low-Level Feature Detectors and Descriptors SIFT, SURF, ORB, and A-KAZE Extractors PHOG, WGCH, and Haralick Extractors Low-Level Feature Comparison and Discussion Behavior and Robustness Matching Process Conclusions References Scale-Insensitive MSER Features: A Promising Tool for Meaningful Segmentation of Images Andrzej Śluzek 3.1 Introduction Summary of MSER and SIMSER Features MSER Features SIMSER Features Segmentation Using MSER Blobs vii

8 viii Contents 3.3 SIMSER-Based Image Segmentation Image Smoothing in SIMSER Detection SIMSER Detection in Color Images Concluding Remarks References Active Partitions in Localization of Semantically Important Image Structures Arkadiusz Tomczyk 4.1 Introduction Background Active Contours Knowledge Active Partitions Representation Partition Evolution Example Global Analysis Local Analysis Summary References Model-Based 3D Object Recognition in RGB-D Images Maciej Stefańczyk and Włodzimierz Kasprzak 5.1 Introduction Knowledge Representation Hierarchy Related Work Proposed RGB-D Data Hierarchy D Object Modelling Geometric Primitives Complex Objects System Framework Solution Principles Knowledge-Based Framework Semantic Net Bayesian Net The Basic Control System Implementation Model Structure Object Instances... 88

9 Contents ix 5.6 Testing Scenarios Data Acquisition Data Preprocessing and Extension Segmentation Hypothesis Generation Hypothesis Update and Verification Method Vulnerabilities Conclusions References Ontology-Based Structured Video Annotation for Content-Based Video Retrieval via Spatiotemporal Reasoning Leslie F. Sikos 6.1 The Limitations of Video Metadata and Feature Descriptors Core Video Metadata Standards Feature Extraction for Concept Mapping Machine Learning in Video Content Analysis The Semantic Gap Semantic Enrichment of Audiovisual Contents Video Semantics Spatiotemporal Video Annotation Using Formal Knowledge Representation Vocabularies and Ontologies Semantic Enrichment of Videos with Linked Data Spatiotemporal Annotation in Action Ontology-Based Video Scene Interpretation Video Event Recognition via Reasoning Over Temporal DL Axioms Video Event Recognition Using SWRL Rules Handling the Uncertainty of Concept Depiction with Fuzzy Axioms Utilizing Video Semantics: From Intelligent Video Indexing to Hypervideo Applications Content-Based Video Indexing and Retrieval Video Semantics in Hypervideo Applications Summary References Deep Learning A New Era in Bridging the Semantic Gap Urszula Markowska-Kaczmar and Halina Kwaśnicka 7.1 Introduction Content Based Image Retrieval Deep Learning Deep Learning Architecture

10 x Contents Visual Attention Models Based on Deep Learning Embedding Semantic Features Deep Learning in Bridging the Semantic Gap Feature Level Knowledge Level Inference Level Conclusions References About the Editors Index