Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute

Size: px
Start display at page:

Download "Yiannis Kompatsiaris Multimedia Knowledge Laboratory CERTH - Informatics and Telematics Institute"

Transcription

1 Convergence of multimedia and knowledge technologies acemedia, BOEMIE, MESH, X-Media, K-Space, VITALAS and VICTORY Practitioner Day CIVR 2007 Yiannis Kompatsiaris CERTH -

2 Introduction Content - applications A common view Multimedia Ontologies Analysis Reasoning Retrieval Common issues Dissemination Conclusions Outline 2 2

3 TITLE DIRECTO R TAKE SCENE Multimedia Content Content adaptation and distribution - Multiple Terminal & Networks Networks Storage & Devices Segmentation KA Analysis Labeling Web 2.0 photo - video applications Cross-media analysis Context Reasoning Metadata Generation & Representation Hybrid / Content-based retrieval recommendations and personalization Semantic technology in Markets 3 3

4 Need for annotation + metadata The value of information depends on how easily it can be found, retrieved, accessed, filtered or managed in an active, personalized way 4 4

5 Content - Applications Content Knowledge Extraction Applications Personal Sports Semantic Desktop Retrieval Industrial Fashion News 3D News Commercial Personalization Mobile 5 5

6 Content - Applications Content Knowledge Extraction Applications Personal Retrieval Commercial Personalization Mobile 6 6

7 Content - Applications Content distribution and adaptation Sharing ACE concept Actionable content 7 7

8 Content - Applications Content Knowledge Extraction Applications Retrieval News Personalization Mobile 8 8

9 Content - Applications News Syndication Multi-National & Local news providers 9 9

10 Content - Applications Content Knowledge Extraction Applications Semantic Desktop Retrieval Industrial 10 10

11 Content - Applications Large-Scale content Process support Industry content 11 11

12 Content - Applications Content Knowledge Extraction Applications Retrieval 3D 12 12

13 Content - Applications From raw data to semantics Maximise automation of the shape knowledge lifecycle From semantics to model raw data geometric model structural model Extracting semantic content Embedding semantic content conceptual sketch semantically structured model shape facets and semantic mapping semantic model geometric model 13 13

14 Content - Applications Content Knowledge Extraction Applications Sports Retrieval 14 14

15 Content - Applications OTHER ONTOLOGIES EVENTS DATABASE SEMANTICS EXTRACTION FROM VISUAL CONTENT MAP ANNOTATION INTERFACE FROM NON-VISUAL CONTENT MAPS DATABASE Content Collection (crawlers, spiders, etc.) INITIAL ONTOLOGY SEMANTICS EXTRACTION RESULTS FROM FUSED CONTENT ONTOLOGY EVOLUTION MULTIMEDIA CONTENT EVOLVED ONTOLOGY Automatic semantic annotation of digital maps POPULATION & ENRICHMENT COORDINATION INTERMEDIATE ONTOLOGY 15 15

16 Content - Applications Content Knowledge Extraction Applications Personal Sports Retrieval News 16 16

17 Content - Applications Integration K-Space R&D Fellowships Dissemination Network of Excellence Emphasis on integration of research activities 17 17

18 Content - Applications Content Knowledge Extraction Applications Retrieval Fashion News Large-Scale Real use cases News Personalization 18 18

19 Content - Applications Content Knowledge Extraction Applications Retrieval 3D P2P and Mobile Mobile 19 19

20 Knowledge Extraction A common view Single Modality Analysis Additional Analysis Information Semantic Analysis Manual Annotation - Models Knowledge Infrastructure (Multimedia Ontology) Implicit Knowledge Signal Level Explicit Knowledge Logic - Semantics & Hybrid Level 20 20

21 Feature extraction Text, Image analysis Segmentation, SVMs Evidence generation Vehicle, Single Building Modality Analysis Knowledge Extraction Single Modality Analysis A common view Classifiers fusion Global vs. Local Modalities fusion Additional Context Ambulance Analysis Information Additional Analysis Information Reasoning Fusion of annotations Consistency checking Higher-level concepts/events Emergency Semantic scene Analysis Semantic Analysis Manual Annotation - ManualModels Multimedia Annotation content annotation - Models tools Training (Statistical) Modeling Knowledge Infrastructure (Multimedia Ontology) Knowledge Infrastructure (Multimedia Domain Ontology) Multimedia content Annotations Algorithms - Features Context 21 21

22 Use of ontologies Metadata representation Annotation Interoperability Reasoning Extracting higher-level annotations Consistency checking Fusion Ontology-driven analysis Retrieval Personalization 22 22

23 Multimedia Ontologies Multimedia content structure acemedia(mpeg-7, RDF), content) Multimodality MESH(OWL), BOEMIE (OWL-DL) K-Space, X-Media (COMM, OWL, DOLCE) Fuzziness K-Space, X-Media (Fuzzy-OWL) Changing knowledge BOEMIE (evolution) X-Media (versioning, reasons of change) Specific domains 23 23

24 COMM: Core Ontology of MultiMedia K-Space, X-Media Instead of translating MPEG-7 1-to-1 into an ontology, COMM provides 5 multimedia design patterns which formalize basic notions of multimedia annotation Digital data pattern Decomposition pattern Content annotation pattern Media annotation pattern Semantic annotation pattern Usage of DOLCE as modeling basis and consideration of common requirements for multimedia annotation COMM already covers large parts of MPEG-7 (some additional patterns may be required for complete coverage) DOLCE DigitalData MultimediaData DOLCE Description StructuredDataDescription MPEG 7Descriptor Decomposition pattern OutputRole ProcessingRole OutputSegmentRole plays DigitalData MultimediaData defines Method Algorithm InputRole InputSegmentRole plays AnnotationRole satisfies Situation SegmentDecomposition SegmentationAlgorithm Content annotation pattern setting defines ProcessingRole OutputRole Method Algorithm InputRole AnnotatedDataRole satisfies plays Situation Annotation setting setting plays 24 24

25 Multimedia Information Objects MESH Example: Analysis of a Multimedia Web Document Linguistic-IO / Visual-IO Format Format: JPG, UTF-8 Lang: DE orderedby instanceof realizedby MultimediaFile text, image 1 2 Segmentation-Tool interpretedby hasdecomposition about MatchTeam Decomposition spatio-temporal-region about TextSegment 1 2 hassegment hascontent Image Text hascontent hassegment 25 25

26 Multimedia Content Annotation M-Ontomat-Annotizer (acemedia, K-Space) 26 26

27 Multimedia Content Analysis MPEG-7 widely used for LL features Segmentation and feature extraction tools (acemedia, K-Space) Well-known classifiers applied and developed SVMs, EM, HMM Bio-inspired approaches Increasing use of context (acemedia, K-Space) Spatial, Frequency, EXIF Fusion Classifiers (K-Space, MESH: global+local) Modalities X-Media (Text+Image+1D data) MESH (Text+Speech+Video) acemedia (Text+Video) Mostly statistical and machine learning (implicit) based but also Hybrid (implicit + explicit, K-Space) 27 27

28 Context and Reasoning for Analysis acemedia sky <RDF /> sea person/bear KAA rock rock/beach sea, sky beach scene classification beach/rock beach scene person face person/face detection other analysis methods Creation of contextual information multimedia reasoning Use of contextual information From metadata layer spatio-temporal relations Domain knowledge Reduction of label sets Merging of segments 28 28

29 Classification Results acemedia Segment s hypothesis set Natural-Person: Sailing-Boat: Sand: Building: Pavement: Road: Body-Of-Water: Cliff: Cloud: Mountain: Sea: Sky: Stone: Waterfall: Wave: Dried-Plant: Dried-Plant-Snowed: Foliage: Grass: Tree: Trunk: Snow: Sunset: Car: Ground: Lamp-Post: Statue:

30 Semantic Region Merging K-Space RSST Sky Building Semantic RSST Ground Sea 30 30

31 Cross Media Knowledge Acquisition X-Media Cross Media approaches: Result level: combining results obtained from different systems on different types of media Extractor level: using system results from different types of media as annotation or background knowledge Feature level: using features coming from different media. Cross Media Framework: Multimedia Document Processing: Extract single & cross media features Feature Processing: Find optimal representation of feature space Cross media data models Create knowledge models for all concepts Cross media dependency models Integrate background knowledge & exploit causality information 31 31

32 Reasoning Support of imprecision - uncertainty Logic-based approaches Extensions of formal theories (X-Media, K-Space) Ad-hoc solutions based on crisp reasoners (acemedia) Statistical approaches (X-Media) Used for: Fusion Consistency checking Higher-level results 32

33 Ad-hoc Fuzzy Reasoning acemedia 1. Classical (crisp) DL reasoning applied on assertions, leaving out fuzzy degrees image1 region3 region1 2. Axioms revisited by external module to update appropriately the degrees according to fuzzy interpretation semantics: region2 Remarks Annotations (fuzzy ABox) considered: fuzzy, positive, concept assertions crisp role assertions Prior knowledge (TBox) considered: Crisp inclusion and equivalence axioms 33 33

34 Fuzzy OWL K-Space, X-Media Automatically Derived Multimedia Annotation is often imprecise or errorprone Model this uncertainty Extend OWL with fuzzy A-Box region4 shows an object which is a ball with a fuzzy degree of 0.8 and a pumpkin with a fuzzy degree of

35 Abduction as non-standard retrieval: Acquire what should be added to a knowledge base (Σ,Δ) to make a query (set of assertions) Γ true TBox ABox Interpretation as abduction: let Γ be the analysis produced concept and role assertions, Γ1 : by default assertions, Γ2 : ones that need to be explained Σ includes apart from DL axioms, rules to answer the Γ queries in a backward-chaining way. Multiple explanations may be obtained. Consistency checking eliminates not valid answers Preference measure according to the number of new assertions required ΒΟΕΜΙΕ Σ Possible explanations Δ 1 : adds 2 new individuals and infers Pole_Vault Δ 2 : adds 1 new individual and infers Pole_Vault Γ 2 Δ 3 : adds 1 new individual and infers High_Jump (neglects though pole 1 :Pole) 35 35

36 VITALAS: Retrieval VITALAS, Victory Adapting the search space to the user profile and providing interactive functionalities to control the results The system validation will be performed on professional collections, up to 10,000 hours of video (television archives INA/IRT) and still images (Belga) The textual annotation would have different interpretation regarding the usage context. VICTORY: 3D and multimedia distributed content (MultiPedia) into Peer-to- Peer (P2P) and mobile Ρ2Ρ networks 3D content (and context) analysis, personalization, 3D object watermarking techniques 36 36

37 acemedia applications Standalone acemedia PC applications Web-based 37 37

38 Common (Open) Issues Evaluation Annotated content Ontologies Fusion in analysis Uncertainty in reasoning Large-Scale Generic vs. Specific approaches Multiple domains support 38 38

39 Dissemination Activities SMART: Semantic MultimediA Research and Technology, networking cluster SAMT: International Conference on Semantics and digital Media Technologies (EWIMT) 2007: 5-7 December 2007, Genova, Italy SSMS: Summer School on Multimedia Semantics 2007: Glasgow, UK, July 15-21, 2007 Special issues, sessions, workshops, books 39 39

40 Conclusions Semantic analysis of multimedia is already providing results Fundamental and applied research in Logic-based + signal approaches Implicit + explicit (knowledge) approaches Different applications and requirements Ongoing research in all areas Future direction: analysis+reasoning for social (Web 2.0) applications 40 40

41 Many thanks to the projects! 41 41

42 Thank you! CERTH-ITI /

Multimedia Reasoning with f-shin

Multimedia Reasoning with f-shin Multimedia Reasoning with f-shin N. Simou, Th. Athanasiadis, V. Tzouvaras, S. Kollias Image Video and Multimedia Systems Laboratory, School of Electrical and Computer Engineering, National Technical University

More information

Knowledge-Driven Video Information Retrieval with LOD

Knowledge-Driven Video Information Retrieval with LOD Knowledge-Driven Video Information Retrieval with LOD Leslie F. Sikos, Ph.D., Flinders University ESAIR 15, 23 October 2015 Melbourne, VIC, Australia Knowledge-Driven Video IR Outline Video Retrieval Challenges

More information

Ontology as Knowledge Base for Spatial Data Harmonization

Ontology as Knowledge Base for Spatial Data Harmonization Ontology as Knowledge Base for Spatial Data Harmonization Otakar Cerba, Karel Charvat University of West Bohemia, Plzen, Czech Republic Help Service Remote Sensing, Benesov, Czech Republic 1 Objectives

More information

Using a Multimedia Ontology Infrastructure for Semantic Annotation of Multimedia Content

Using a Multimedia Ontology Infrastructure for Semantic Annotation of Multimedia Content Using a Multimedia Ontology Infrastructure for Semantic Annotation of Multimedia Content Thanos Athanasiadis 1, Vassilis Tzouvaras 1, Kosmas Petridis 2, Frederic Precioso 2, Yannis Avrithis 1 and Yiannis

More information

Search Computing: Business Areas, Research and Socio-Economic Challenges

Search Computing: Business Areas, Research and Socio-Economic Challenges Search Computing: Business Areas, Research and Socio-Economic Challenges Yiannis Kompatsiaris, Spiros Nikolopoulos, CERTH--ITI NEM SUMMIT Torino-Italy, 28th September 2011 Media Search Cluster Search Computing

More information

Empowering People with Knowledge the Next Frontier for Web Search. Wei-Ying Ma Assistant Managing Director Microsoft Research Asia

Empowering People with Knowledge the Next Frontier for Web Search. Wei-Ying Ma Assistant Managing Director Microsoft Research Asia Empowering People with Knowledge the Next Frontier for Web Search Wei-Ying Ma Assistant Managing Director Microsoft Research Asia Important Trends for Web Search Organizing all information Addressing user

More information

Image Processing (IP)

Image Processing (IP) Image Processing Pattern Recognition Computer Vision Xiaojun Qi Utah State University Image Processing (IP) Manipulate and analyze digital images (pictorial information) by computer. Applications: The

More information

IMOTION. Heiko Schuldt, University of Basel, Switzerland

IMOTION. Heiko Schuldt, University of Basel, Switzerland IMOTION Heiko Schuldt, University of Basel, Switzerland heiko.schuldt@unibas.ch IMOTION at a Glance Project Title Intelligent Multimodal Augmented Video Motion Retrieval System (IMOTION) Project Start

More information

Leveraging flickr images for object detection

Leveraging flickr images for object detection Leveraging flickr images for object detection Elisavet Chatzilari Spiros Nikolopoulos Yiannis Kompatsiaris Outline Introduction to object detection Our proposal Experiments Current research 2 Introduction

More information

The HMatch 2.0 Suite for Ontology Matchmaking

The HMatch 2.0 Suite for Ontology Matchmaking The HMatch 2.0 Suite for Ontology Matchmaking S. Castano, A. Ferrara, D. Lorusso, and S. Montanelli Università degli Studi di Milano DICo - Via Comelico, 39, 20135 Milano - Italy {castano,ferrara,lorusso,montanelli}@dico.unimi.it

More information

Knowledge Acquisition from Multimedia Content using an Evolution Framework

Knowledge Acquisition from Multimedia Content using an Evolution Framework Knowledge Acquisition from Multimedia Content using an Evolution Framework D. Kosmopoulos, S. Petridis, I. Pratikakis, V. Gatos, S. Perantonis, V. Karkaletsis, G. Paliouras Computational Intelligence Laboratory,

More information

Semantic Representation of Multimedia Content

Semantic Representation of Multimedia Content Semantic Representation of Multimedia Content Kalliopi Dalakleidi 1, Stamatia Dasiopoulou 2( ), Giorgos Stoilos 1( )( ), Vassilis Tzouvaras 1, Giorgos Stamou 1, and Yiannis Kompatsiaris 2 1 School of Electrical

More information

An ontology for the Business Process Modelling Notation

An ontology for the Business Process Modelling Notation An ontology for the Business Process Modelling Notation Marco Rospocher Fondazione Bruno Kessler, Data and Knowledge Management Unit Trento, Italy rospocher@fbk.eu :: http://dkm.fbk.eu/rospocher joint

More information

6/2/10. Generating Semantic Annotations MOTIVATION. Semantic Web. Where are we? Agenda

6/2/10. Generating Semantic Annotations MOTIVATION. Semantic Web. Where are we? Agenda Where are we? Semantic Web Generating Semantic Annotations # Title 1 Introduction 2 Semantic Web Architecture 3 Resource Description Framework (RDF) 4 Web of data 5 Generating Semantic Annotations 6 Storage

More information

A Survey of Context Modelling and Reasoning Techniques

A Survey of Context Modelling and Reasoning Techniques Formal Information A Survey of Context Modelling and Reasoning Techniques Bettini, Brdiczka, Henricksen, Indulska, Nicklas, Ranganathan, Riboni Pervasive and Mobile Computing 2008 (submitted), 2010 (published)

More information

Intelligent Systems Reference Library

Intelligent Systems Reference Library Intelligent Systems Reference Library Volume 145 Series editors Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland e-mail: kacprzyk@ibspan.waw.pl Lakhmi C. Jain, University of Canberra, Canberra,

More information

<is web> 1 st Face to Face Meeting of the W3C XG Multimedia Semantics XG MMSEM. Thomas Franz. ISWeb - Information Systems & Semantic Web

<is web> 1 st Face to Face Meeting of the W3C XG Multimedia Semantics XG MMSEM. Thomas Franz. ISWeb - Information Systems & Semantic Web University of Koblenz-Landau, Germany 1 st Face to Face Meeting of the W3C XG Multimedia Semantics ISWeb @ XG MMSEM ISWeb Interests by Books Ontologies Text & Multimedia P2P Webservices Projects Ontologies

More information

Extending the Facets concept by applying NLP tools to catalog records of scientific literature

Extending the Facets concept by applying NLP tools to catalog records of scientific literature Extending the Facets concept by applying NLP tools to catalog records of scientific literature *E. Picchi, *M. Sassi, **S. Biagioni, **S. Giannini *Institute of Computational Linguistics **Institute of

More information

Are clickthrough data reliable as image annotations?

Are clickthrough data reliable as image annotations? Video & Image Indexing and Retrieval in the Large Scale Are clickthrough data reliable as image annotations? Theodora Tsikrika (CWI) Christos Diou (AUTH, CERTH-ITI) Arjen P. de Vries (CWI, TU Delft) (

More information

COMP718: Ontologies and Knowledge Bases

COMP718: Ontologies and Knowledge Bases 1/35 COMP718: Ontologies and Knowledge Bases Lecture 9: Ontology/Conceptual Model based Data Access Maria Keet email: keet@ukzn.ac.za home: http://www.meteck.org School of Mathematics, Statistics, and

More information

Local Closed World Reasoning with OWL 2

Local Closed World Reasoning with OWL 2 Local Closed World Reasoning with OWL 2 JIST 2011 Tutorial Jeff Z. Pan Department of Computing Science University of Aberdeen, UK Agenda 1. Brief introduction to Ontology and OWL 2 (10m) 2. Open vs. Closed

More information

MESH. Multimedia Semantic Syndication for Enhanced News Services. Project Overview

MESH. Multimedia Semantic Syndication for Enhanced News Services. Project Overview MESH Multimedia Semantic Syndication for Enhanced News Services Project Overview Presentation Structure 2 Project Summary Project Motivation Problem Description Work Description Expected Result The MESH

More information

High-level Interpretation of the Visual Environment

High-level Interpretation of the Visual Environment High-level Interpretation of the Visual Environment Bernd Neumann Department of Informatics Hamburg University Germany kogs-www.informatik.uni-hamburg.de neumann@informatik.uni-hamburg.de Presentation

More information

Simplified Approach for Representing Part-Whole Relations in OWL-DL Ontologies

Simplified Approach for Representing Part-Whole Relations in OWL-DL Ontologies Simplified Approach for Representing Part-Whole Relations in OWL-DL Ontologies Pace University IEEE BigDataSecurity, 2015 Aug. 24, 2015 Outline Ontology and Knowledge Representation 1 Ontology and Knowledge

More information

PRISM: Concept-preserving Social Image Search Results Summarization

PRISM: Concept-preserving Social Image Search Results Summarization PRISM: Concept-preserving Social Image Search Results Summarization Boon-Siew Seah Sourav S Bhowmick Aixin Sun Nanyang Technological University Singapore Outline 1 Introduction 2 Related studies 3 Search

More information

PROJECT PERIODIC REPORT

PROJECT PERIODIC REPORT PROJECT PERIODIC REPORT Grant Agreement number: 257403 Project acronym: CUBIST Project title: Combining and Uniting Business Intelligence and Semantic Technologies Funding Scheme: STREP Date of latest

More information

Semantic Web. Ontology Pattern. Gerd Gröner, Matthias Thimm. Institute for Web Science and Technologies (WeST) University of Koblenz-Landau

Semantic Web. Ontology Pattern. Gerd Gröner, Matthias Thimm. Institute for Web Science and Technologies (WeST) University of Koblenz-Landau Semantic Web Ontology Pattern Gerd Gröner, Matthias Thimm {groener,thimm}@uni-koblenz.de Institute for Web Science and Technologies (WeST) University of Koblenz-Landau July 18, 2013 Gerd Gröner, Matthias

More information

Introducing fuzzy quantification in OWL 2 ontologies

Introducing fuzzy quantification in OWL 2 ontologies Introducing fuzzy quantification in OWL 2 ontologies Francesca Alessandra Lisi and Corrado Mencar Dipartimento di Informatica, Centro Interdipartimentale di Logica e Applicazioni Università degli Studi

More information

Towards a simplification of COMM-based multimedia annotations

Towards a simplification of COMM-based multimedia annotations Towards a simplification of COMM-based multimedia annotations Miroslav VACURA a, Raphaël TRONCY b a University of Economics, W. Churchill Sq.4, 13067, Prague 3, Czech Republic b CWI, Kruislaan 413, 1098

More information

Multimedia Data Management M

Multimedia Data Management M ALMA MATER STUDIORUM - UNIVERSITÀ DI BOLOGNA Multimedia Data Management M Second cycle degree programme (LM) in Computer Engineering University of Bologna Semantic Multimedia Data Annotation Home page:

More information

AKnowledgeEngineeringApproachfor Complex Violence Identification in Movies

AKnowledgeEngineeringApproachfor Complex Violence Identification in Movies AKnowledgeEngineeringApproachfor Complex Violence Identification in Movies Thanassis Perperis 1 and Sofia Tsekeridou 2 1 University of Athens, Greece a.perperis@di.uoa.gr 2 Athens Information Technology,

More information

Query Decomposition: A Multiple Neighborhood Approach to Relevance Feedback Processing in Content-based Image Retrieval

Query Decomposition: A Multiple Neighborhood Approach to Relevance Feedback Processing in Content-based Image Retrieval Query Decomposition: A Multiple Neighborhood Approach to Relevance Feedback Processing in Content-based Image Retrieval Kien A. Hua, Ning Yu, Danzhou Liu School of Electrical Engineering and Computer Science

More information

Session 2 A virtual Observatory for TerraSAR-X data

Session 2 A virtual Observatory for TerraSAR-X data Session 2 A virtual Observatory for TerraSAR-X data 3rd User Community Workshop Chania, 12-14 June 2013 Presenter: Mihai Datcu and Daniela Espinoza Molina (DLR) Motivation ENVISAT provided measurements

More information

Development of an Ontology-Based Portal for Digital Archive Services

Development of an Ontology-Based Portal for Digital Archive Services Development of an Ontology-Based Portal for Digital Archive Services Ching-Long Yeh Department of Computer Science and Engineering Tatung University 40 Chungshan N. Rd. 3rd Sec. Taipei, 104, Taiwan chingyeh@cse.ttu.edu.tw

More information

Description Logics and OWL

Description Logics and OWL Description Logics and OWL Based on slides from Ian Horrocks University of Manchester (now in Oxford) Where are we? OWL Reasoning DL Extensions Scalability OWL OWL in practice PL/FOL XML RDF(S)/SPARQL

More information

Flexible Tools for the Semantic Web

Flexible Tools for the Semantic Web Flexible Tools for the Semantic Web (instead of Jans Aasman from Franz Inc.) Software Systems Group (STS) Hamburg University of Technology (TUHH) Hamburg-Harburg, Germany (and GmbH & Co. KG) 1 Flexible

More information

^ Springer. Computational Intelligence. A Methodological Introduction. Rudolf Kruse Christian Borgelt. Matthias Steinbrecher Pascal Held

^ Springer. Computational Intelligence. A Methodological Introduction. Rudolf Kruse Christian Borgelt. Matthias Steinbrecher Pascal Held Rudolf Kruse Christian Borgelt Frank Klawonn Christian Moewes Matthias Steinbrecher Pascal Held Computational Intelligence A Methodological Introduction ^ Springer Contents 1 Introduction 1 1.1 Intelligent

More information

Towards the Semantic Desktop. Dr. Øyvind Hanssen University Library of Tromsø

Towards the Semantic Desktop. Dr. Øyvind Hanssen University Library of Tromsø Towards the Semantic Desktop Dr. Øyvind Hanssen University Library of Tromsø Agenda Background Enabling trends and technologies Desktop computing and The Semantic Web Online Social Networking and P2P Computing

More information

Citizen Sensing: Opportunities and Challenges in Mining Social Signals and Perceptions

Citizen Sensing: Opportunities and Challenges in Mining Social Signals and Perceptions Wright State University CORE Scholar Kno.e.sis Publications The Ohio Center of Excellence in Knowledge- Enabled Computing (Kno.e.sis) 7-19-2011 Citizen Sensing: Opportunities and Challenges in Mining Social

More information

Semi-automated ontology learning: the BOEMIE approach

Semi-automated ontology learning: the BOEMIE approach Semi-automated ontology learning: the BOEMIE approach Petasis George 1, Karkaletsis Vangelis 1, Krithara Anastasia 1, Paliouras Georgios 1, Spyropoulos D. Constantine 1 1 National Center of Scientific

More information

Performance Evaluation of Semantic Registries: OWLJessKB and instancestore

Performance Evaluation of Semantic Registries: OWLJessKB and instancestore Service Oriented Computing and Applications manuscript No. (will be inserted by the editor) Performance Evaluation of Semantic Registries: OWLJessKB and instancestore Simone A. Ludwig 1, Omer F. Rana 2

More information

Joint Inference in Image Databases via Dense Correspondence. Michael Rubinstein MIT CSAIL (while interning at Microsoft Research)

Joint Inference in Image Databases via Dense Correspondence. Michael Rubinstein MIT CSAIL (while interning at Microsoft Research) Joint Inference in Image Databases via Dense Correspondence Michael Rubinstein MIT CSAIL (while interning at Microsoft Research) My work Throughout the year (and my PhD thesis): Temporal Video Analysis

More information

The LICHEN Framework: A new toolbox for the exploitation of corpora

The LICHEN Framework: A new toolbox for the exploitation of corpora The LICHEN Framework: A new toolbox for the exploitation of corpora Lisa Lena Opas-Hänninen, Tapio Seppänen, Ilkka Juuso and Matti Hosio (University of Oulu, Finland) Background cultural inheritance is

More information

TRENTINOMEDIA: Exploiting NLP and Background Knowledge to Browse a Large Multimedia News Store

TRENTINOMEDIA: Exploiting NLP and Background Knowledge to Browse a Large Multimedia News Store TRENTINOMEDIA: Exploiting NLP and Background Knowledge to Browse a Large Multimedia News Store Roldano Cattoni 1, Francesco Corcoglioniti 1,2, Christian Girardi 1, Bernardo Magnini 1, Luciano Serafini

More information

COMM: Designing a Well-Founded Multimedia Ontology for the Web

COMM: Designing a Well-Founded Multimedia Ontology for the Web COMM: Designing a Well-Founded Multimedia Ontology for the Web Richard Arndt 1, Raphaël Troncy 2, and Steffen Staab 1, Lynda Hardman 2,, and Miroslav Vacura 3 1 ISWeb, University of Koblenz-Landau, Germany

More information

Reducing Consumer Uncertainty

Reducing Consumer Uncertainty Spatial Analytics Reducing Consumer Uncertainty Towards an Ontology for Geospatial User-centric Metadata Introduction Cooperative Research Centre for Spatial Information (CRCSI) in Australia Communicate

More information

An Evaluation of Geo-Ontology Representation Languages for Supporting Web Retrieval of Geographical Information

An Evaluation of Geo-Ontology Representation Languages for Supporting Web Retrieval of Geographical Information An Evaluation of Geo-Ontology Representation Languages for Supporting Web Retrieval of Geographical Information P. Smart, A.I. Abdelmoty and C.B. Jones School of Computer Science, Cardiff University, Cardiff,

More information

Semantic Annotation, Search and Analysis

Semantic Annotation, Search and Analysis Semantic Annotation, Search and Analysis Borislav Popov, Ontotext Ontology A machine readable conceptual model a common vocabulary for sharing information machine-interpretable definitions of concepts in

More information

IJREAT International Journal of Research in Engineering & Advanced Technology, Volume 1, Issue 5, Oct-Nov, 2013 ISSN:

IJREAT International Journal of Research in Engineering & Advanced Technology, Volume 1, Issue 5, Oct-Nov, 2013 ISSN: Semi Automatic Annotation Exploitation Similarity of Pics in i Personal Photo Albums P. Subashree Kasi Thangam 1 and R. Rosy Angel 2 1 Assistant Professor, Department of Computer Science Engineering College,

More information

Semantics Modeling and Representation. Wendy Hui Wang CS Department Stevens Institute of Technology

Semantics Modeling and Representation. Wendy Hui Wang CS Department Stevens Institute of Technology Semantics Modeling and Representation Wendy Hui Wang CS Department Stevens Institute of Technology hwang@cs.stevens.edu 1 Consider the following data: 011500 18.66 0 0 62 46.271020111 25.220010 011500

More information

Semantic Annotation of Images and Videos for Multimedia Analysis

Semantic Annotation of Images and Videos for Multimedia Analysis Semantic Annotation of Images and Videos for Multimedia Analysis Stephan Bloehdorn 1, Kosmas Petridis 2, Carsten Saathoff 3, Nikos Simou 4, Vassilis Tzouvaras 4, Yannis Avrithis 4, Siegfried Handschuh

More information

Jie Bao, Paul Smart, Dave Braines, Nigel Shadbolt

Jie Bao, Paul Smart, Dave Braines, Nigel Shadbolt Jie Bao, Paul Smart, Dave Braines, Nigel Shadbolt Advent of Web 2.0 supports greater user participation in the creation of Web content Good way to generate lots of online content e.g. Wikipedia ~3 million

More information

The Fraunhofer IDMT at ImageCLEF 2011 Photo Annotation Task

The Fraunhofer IDMT at ImageCLEF 2011 Photo Annotation Task The Fraunhofer IDMT at ImageCLEF 2011 Photo Annotation Task Karolin Nagel, Stefanie Nowak, Uwe Kühhirt and Kay Wolter Fraunhofer Institute for Digital Media Technology (IDMT) Ehrenbergstr. 31, 98693 Ilmenau,

More information

Ontology Engineering for Product Development

Ontology Engineering for Product Development Ontology Engineering for Product Development Henson Graves Lockheed Martin Aeronautics Company Fort Worth Texas, USA henson.graves@lmco.com Abstract. This analysis is to identify requirements for a Description

More information

Visualisation : Lecture 1. So what is visualisation? Visualisation

Visualisation : Lecture 1. So what is visualisation? Visualisation So what is visualisation? UG4 / M.Sc. Course 2006 toby.breckon@ed.ac.uk Computer Vision Lab. Institute for Perception, Action & Behaviour Introducing 1 Application of interactive 3D computer graphics to

More information

Specification of the Architecture

Specification of the Architecture Specification of the Architecture Version 1.0 Thomas Tikwinski (FHG/IAIS), Carsten Rosche (FHG/IAIS), George Paliouras (NCSR), Alfio Ferrara (UniMi), Atila Kaya (TUHH), Vasileios Papastathis (CERTH) Distribution:

More information

Two interrelated objectives of the ARIADNE project, are the. Training for Innovation: Data and Multimedia Visualization

Two interrelated objectives of the ARIADNE project, are the. Training for Innovation: Data and Multimedia Visualization Training for Innovation: Data and Multimedia Visualization Matteo Dellepiane and Roberto Scopigno CNR-ISTI Two interrelated objectives of the ARIADNE project, are the design of new services (or the integration

More information

An Ontology-Based Methodology for Integrating i* Variants

An Ontology-Based Methodology for Integrating i* Variants An Ontology-Based Methodology for Integrating i* Variants Karen Najera 1,2, Alicia Martinez 2, Anna Perini 3, and Hugo Estrada 1,2 1 Fund of Information and Documentation for the Industry, Mexico D.F,

More information

Columbia University High-Level Feature Detection: Parts-based Concept Detectors

Columbia University High-Level Feature Detection: Parts-based Concept Detectors TRECVID 2005 Workshop Columbia University High-Level Feature Detection: Parts-based Concept Detectors Dong-Qing Zhang, Shih-Fu Chang, Winston Hsu, Lexin Xie, Eric Zavesky Digital Video and Multimedia Lab

More information

2 Which Methodology for Building Ontologies? 2.1 A Work Still in Progress Many approaches (for a complete survey, the reader can refer to the OntoWeb

2 Which Methodology for Building Ontologies? 2.1 A Work Still in Progress Many approaches (for a complete survey, the reader can refer to the OntoWeb Semantic Commitment for Designing Ontologies: A Proposal Bruno Bachimont 1,Antoine Isaac 1;2, Raphaël Troncy 1;3 1 Institut National de l'audiovisuel, Direction de la Recherche 4, Av. de l'europe - 94366

More information

VERGE: A Video Interactive Retrieval Engine

VERGE: A Video Interactive Retrieval Engine VERGE: A Video Interactive Retrieval Engine Stefanos Vrochidis, Anastasia Moumtzidou, Paul King, Anastasios Dimou, Vasileios Mezaris and Ioannis Kompatsiaris Informatics and Telematics Institute 6th Km

More information

Introduction. Multimedia Applications and Data Management Requirements 1.1 HETEROGENEITY

Introduction. Multimedia Applications and Data Management Requirements 1.1 HETEROGENEITY 1 Introduction Multimedia Applications and Data Management Requirements Among countless others, applications of multimedia databases include personal and public photo/media collections, personal information

More information

Improving Semantic Interoperability of Distributed Geospatial Web Services:

Improving Semantic Interoperability of Distributed Geospatial Web Services: Improving Semantic Interoperability of Distributed Geospatial Web Services: G-MAP SEMANTIC MAPPING SYSTEM Mohamed Bakillah (Ph.D Candidate) Mir Abolfazl Mostafavi, Center for Research in Geomatic Laval

More information

ELL 788 Computational Perception & Cognition July November 2015

ELL 788 Computational Perception & Cognition July November 2015 ELL 788 Computational Perception & Cognition July November 2015 Module 6 Role of context in object detection Objects and cognition Ambiguous objects Unfavorable viewing condition Context helps in object

More information

MOWIS: A system for building Multimedia Ontologies from Web Information Sources

MOWIS: A system for building Multimedia Ontologies from Web Information Sources MOWIS: A system for building Multimedia Ontologies from Web Information Sources Vincenzo Moscato, Antonio Penta, Fabio Persia, Antonio Picariello University of Naples Dipartimento di Informatica e Sistemistica

More information

Aggregation for searching complex information spaces. Mounia Lalmas

Aggregation for searching complex information spaces. Mounia Lalmas Aggregation for searching complex information spaces Mounia Lalmas mounia@acm.org Outline Document Retrieval Focused Retrieval Aggregated Retrieval Complexity of the information space (s) INEX - INitiative

More information

A Consumer Photo Tagging Ontology

A Consumer Photo Tagging Ontology A Consumer Photo Tagging Ontology Concepts and Annotations Stefanie Nowak Semantic Audiovisual Systems Fraunhofer IDMT Corfu, 29.09.2009 stefanie.nowak@idmt.fraunhofer.de slide 1 Outline Introduction Related

More information

FIBO Metadata in Ontology Mapping

FIBO Metadata in Ontology Mapping FIBO Metadata in Ontology Mapping For Open Ontology Repository OOR Metadata Workshop VIII 02 July 2013 Copyright 2010 EDM Council Inc. 1 Overview The Financial Industry Business Ontology Introduction FIBO

More information

Web Services Annotation and Reasoning

Web Services Annotation and Reasoning Web Services Annotation and Reasoning Mikhail Roshchin, PhD Student Peter Graubmann, Evelyn Pfeuffer CT SE 2, Siemens AG roshchin@gmail.com Motivation _ Current Problems Software Applications work with

More information

The Semantic Web Explained

The Semantic Web Explained The Semantic Web Explained The Semantic Web is a new area of research and development in the field of computer science, aimed at making it easier for computers to process the huge amount of information

More information

Toward a Standard Rule Language for Semantic Integration of the DoD Enterprise

Toward a Standard Rule Language for Semantic Integration of the DoD Enterprise 1 W3C Workshop on Rule Languages for Interoperability Toward a Standard Rule Language for Semantic Integration of the DoD Enterprise A MITRE Sponsored Research Effort Suzette Stoutenburg 28 April 2005

More information

DL User Interfaces. Giuseppe Santucci Dipartimento di Informatica e Sistemistica Università di Roma La Sapienza

DL User Interfaces. Giuseppe Santucci Dipartimento di Informatica e Sistemistica Università di Roma La Sapienza DL User Interfaces Giuseppe Santucci Dipartimento di Informatica e Sistemistica Università di Roma La Sapienza Delos work on DL interfaces Delos Cluster 4: User interfaces and visualization Cluster s goals:

More information

Enhanced retrieval using semantic technologies:

Enhanced retrieval using semantic technologies: Enhanced retrieval using semantic technologies: Ontology based retrieval as a new search paradigm? - Considerations based on new projects at the Bavarian State Library Dr. Berthold Gillitzer 28. Mai 2008

More information

Semantic Web Methoden, Werkzeuge und Anwendungen

Semantic Web Methoden, Werkzeuge und Anwendungen Semantic Web Methoden, Werkzeuge und Anwendungen Rudi Studer & Peter Haase Institut, Universität t Karlsruhe (TH) & FZI Forschungszentrum Informatik & Ontoprise GmbH Universität t Kassel 26. Januar 2006

More information

Tree structured CRF models for interactive image labeling

Tree structured CRF models for interactive image labeling Tree structured CRF models for interactive image labeling Thomas Mensink 1,2 Gabriela Csurka 1 Jakob Verbeek 2 1 Xerox Research Centre Europe, Grenoble, France 2 INRIA, Grenoble, France To appear at CVPR

More information

Toward Human-Computer Information Retrieval

Toward Human-Computer Information Retrieval Toward Human-Computer Information Retrieval Gary Marchionini University of North Carolina at Chapel Hill march@ils.unc.edu Samuel Lazerow Memorial Lecture The Information School University of Washington

More information

0.1 Upper ontologies and ontology matching

0.1 Upper ontologies and ontology matching 0.1 Upper ontologies and ontology matching 0.1.1 Upper ontologies Basics What are upper ontologies? 0.1 Upper ontologies and ontology matching Upper ontologies (sometimes also called top-level or foundational

More information

Object Recognition I. Linda Shapiro EE/CSE 576

Object Recognition I. Linda Shapiro EE/CSE 576 Object Recognition I Linda Shapiro EE/CSE 576 1 Low- to High-Level low-level edge image mid-level high-level consistent line clusters Building Recognition 2 High-Level Computer Vision Detection of classes

More information

A Logic-based Approach To Multimedia Interpretation Dissertation Presentation Atila Kaya

A Logic-based Approach To Multimedia Interpretation Dissertation Presentation Atila Kaya A Logic-based Approach To Multimedia Interpretation Dissertation Presentation Atila Kaya Chairman: Prof. von Estorff Reviewers: Prof. Möller Prof. Neumann Prof. Grigat Hamburg, 28 th of February 2011 1

More information

T chnology chnology Ma turity turity for fo Adaptiv Adaptiv Massively Massiv ely Pa P ra r llel llel Computing F rst rst Wo W rksho p 2009

T chnology chnology Ma turity turity for fo Adaptiv Adaptiv Massively Massiv ely Pa P ra r llel llel Computing F rst rst Wo W rksho p 2009 Technology Maturity for Adaptive Massively Parallel Computing First Workshop 2009 March h2 3, 2009 Portland, OR, USA AMP Computing Workshop 2009 Massive Data Computing Pradeep K. Dubey Senior Principal

More information

Knowledge Integration Environment

Knowledge Integration Environment Knowledge Integration Environment Aka Knowledge is Everything D.Sottara, PhD OMG Technical Meeting Spring 2013, Reston, VA Outline Part I The Consolidated Past : Drools 5.x Drools Expert Object-Oriented,

More information

Offering Access to Personalized Interactive Video

Offering Access to Personalized Interactive Video Offering Access to Personalized Interactive Video 1 Offering Access to Personalized Interactive Video Giorgos Andreou, Phivos Mylonas, Manolis Wallace and Stefanos Kollias Image, Video and Multimedia Systems

More information

Towards an Ontology Visualization Tool for Indexing DICOM Structured Reporting Documents

Towards an Ontology Visualization Tool for Indexing DICOM Structured Reporting Documents Towards an Ontology Visualization Tool for Indexing DICOM Structured Reporting Documents Sonia MHIRI sonia.mhiri@math-info.univ-paris5.fr Sylvie DESPRES sylvie.despres@lipn.univ-paris13.fr CRIP5 University

More information

<is web> Information Systems & Semantic Web University of Koblenz Landau, Germany

<is web> Information Systems & Semantic Web University of Koblenz Landau, Germany Information Systems University of Koblenz Landau, Germany Semantic Multimedia Management - Multimedia on the - Thomas Franz,, Richard Arndt A Common Scenario... Do you have any recent paper about semantic

More information

SKOS. COMP62342 Sean Bechhofer

SKOS. COMP62342 Sean Bechhofer SKOS COMP62342 Sean Bechhofer sean.bechhofer@manchester.ac.uk Ontologies Metadata Resources marked-up with descriptions of their content. No good unless everyone speaks the same language; Terminologies

More information

Multimedia Information Retrieval The case of video

Multimedia Information Retrieval The case of video Multimedia Information Retrieval The case of video Outline Overview Problems Solutions Trends and Directions Multimedia Information Retrieval Motivation With the explosive growth of digital media data,

More information

Modeling Actuations in BCI-O: A Context-based Integration of SOSA and IoT-O

Modeling Actuations in BCI-O: A Context-based Integration of SOSA and IoT-O Modeling Actuations in BCI-O: A Context-based Integration of SOSA and IoT-O https://w3id.org/bci-ontology# Semantic Models for BCI Data Analytics Sergio José Rodríguez Méndez Pervasive Embedded Technology

More information

Workshop W14 - Audio Gets Smart: Semantic Audio Analysis & Metadata Standards

Workshop W14 - Audio Gets Smart: Semantic Audio Analysis & Metadata Standards Workshop W14 - Audio Gets Smart: Semantic Audio Analysis & Metadata Standards Jürgen Herre for Integrated Circuits (FhG-IIS) Erlangen, Germany Jürgen Herre, hrr@iis.fhg.de Page 1 Overview Extracting meaning

More information

Object Class Recognition using Images of Abstract Regions

Object Class Recognition using Images of Abstract Regions Object Class Recognition using Images of Abstract Regions Yi Li, Jeff A. Bilmes, and Linda G. Shapiro Department of Computer Science and Engineering Department of Electrical Engineering University of Washington

More information

Semantic reasoning for dynamic knowledge bases. Lionel Médini M2IA Knowledge Dynamics 2018

Semantic reasoning for dynamic knowledge bases. Lionel Médini M2IA Knowledge Dynamics 2018 Semantic reasoning for dynamic knowledge bases Lionel Médini M2IA Knowledge Dynamics 2018 1 Outline Summary Logics Semantic Web Languages Reasoning Web-based reasoning techniques Reasoning using SemWeb

More information

Computer vision is the analysis of digital images by a computer for such applications as:

Computer vision is the analysis of digital images by a computer for such applications as: Introduction Computer vision is the analysis of digital images by a computer for such applications as: Industrial: part localization and inspection, robotics Medical: disease classification, screening,

More information

H1 Spring C. A service-oriented architecture is frequently deployed in practice without a service registry

H1 Spring C. A service-oriented architecture is frequently deployed in practice without a service registry 1. (12 points) Identify all of the following statements that are true about the basics of services. A. Screen scraping may not be effective for large desktops but works perfectly on mobile phones, because

More information

Mir Abolfazl Mostafavi Centre for research in geomatics, Laval University Québec, Canada

Mir Abolfazl Mostafavi Centre for research in geomatics, Laval University Québec, Canada Mir Abolfazl Mostafavi Centre for research in geomatics, Laval University Québec, Canada Mohamed Bakillah and Steve H.L. Liang Department of Geomatics Engineering University of Calgary, Alberta, Canada

More information

Thematic Mapping with Remote Sensing Satellite Networks

Thematic Mapping with Remote Sensing Satellite Networks Thematic Mapping with Remote Sensing Satellite Networks College of Engineering and Computer Science The Australian National University outline satellite networks implications for analytical methods candidate

More information

Semantic Video Indexing

Semantic Video Indexing Semantic Video Indexing T-61.6030 Multimedia Retrieval Stevan Keraudy stevan.keraudy@tkk.fi Helsinki University of Technology March 14, 2008 What is it? Query by keyword or tag is common Semantic Video

More information

Ontologies SKOS. COMP62342 Sean Bechhofer

Ontologies SKOS. COMP62342 Sean Bechhofer Ontologies SKOS COMP62342 Sean Bechhofer sean.bechhofer@manchester.ac.uk Metadata Resources marked-up with descriptions of their content. No good unless everyone speaks the same language; Terminologies

More information

Automatic Categorization of Image Regions using Dominant Color based Vector Quantization

Automatic Categorization of Image Regions using Dominant Color based Vector Quantization Automatic Categorization of Image Regions using Dominant Color based Vector Quantization Md Monirul Islam, Dengsheng Zhang, Guojun Lu Gippsland School of Information Technology, Monash University Churchill

More information

SEMANTIC AUGMENTATION OF GEOSPATIAL CONCEPTS:

SEMANTIC AUGMENTATION OF GEOSPATIAL CONCEPTS: SEMANTIC AUGMENTATION OF GEOSPATIAL CONCEPTS: THE MULTI-VIEW AUGMENTED CONCEPT TO IMPROVE SEMANTIC INTEROPERABILITY BETWEEN MULTIPLES GEOSPATIAL DATABASES Mohamed Bakillah Mir Abolfazl Mostafavi, Center

More information

Mapping provenance in ontologies

Mapping provenance in ontologies Mapping provenance in ontologies František Jahoda FNSPE Czech Technical University Institute of Computer Science Academy of Sciences of the Czech Republic What is Provenance? from the French provenir,

More information

Fuzzy Ontology Models Based on Fuzzy Linguistic Variable for Knowledge Management and Information Retrieval

Fuzzy Ontology Models Based on Fuzzy Linguistic Variable for Knowledge Management and Information Retrieval uzzy Ontology Models Based on uzzy Linguistic Variable for Knowledge Management and Information Retrieval Jun Zhai, Yiduo Liang, Jiatao Jiang and Yi Yu School of Economics and Managemen, Dalian Maritime

More information