CISC 7610 Lecture 4 Approaches to multimedia databases

Transcription

1 CISC 7610 Lecture 4 Approaches to multimedia databases Topics: Metadata Loose vs tight coupling of data and metadata Object (oriented) databases Graph databases Object-relational mapping

2 Homework 1 Entity-relationship diagram RecordingDate GPS Lat Tape GPS Lon Program Description CameraPerson 1 N Description Contains Contains N Name StartTime N StartTime EndTime StartTime EndTime N EndTime Description Shot FirstAired Contains N Story Title Reporter

3 Metadata is data about data Information on creation, content, and interaction Holistic or time-related Multimedia file types typically include facilities for embedding metadata

4 Camera files have EXIF metadata Exchangeable image file format (EXIF) Recorded by digital cameras, phones, scanners Somewhat easy to break when editing file For video and still images For a video:

5 A still image from the same camera

6 MP3s have ID3 metadata Mainly targeted at professional releases Info: Title, artist, album, track number Album art, lyrics De facto, not recognized by any standardization bodies

7 MP3s have ID3 metadata Mainly targeted at professional releases Info: Title, artist, album, track number Album art, lyrics De facto, not recognized by any standardization bodies

8 Holistic (Object-level) metadata Automatic metadata from creation: EXIF Metadata about the content Frame rate, resolution, quality, codecs, equipment Topics, tags, participants Mostly human-generated Will discuss machine-generated later in course Usage and interaction metadata Comments, thumbs ups, skips, shares

9 Time-related metadata Automatic metadata from creation (theoretically) Metadata about the content Location, view direction, camera/lens parameters, scene changes Closed captions, events/actions, scene descriptions Usage and interaction metadata Comments, thumbs ups, skips, shares

10 Time-related metadata

11 What type of metadata are these? RecordingDate GPS Lat Tape GPS Lon Program Description CameraPerson 1 N Description Contains Contains N Name StartTime N StartTime EndTime StartTime EndTime N EndTime Description Shot FirstAired Contains N Story Title Reporter

12 But where does the actual video go? RecordingDate GPS Lat Tape GPS Lon Program Description CameraPerson 1 N Description Contains Contains N Name StartTime N StartTime EndTime StartTime EndTime N EndTime Description Shot FirstAired Contains N Story Title Reporter

13 Simplest solution: on the filesystem Simple key-value store (filename data) Stores some metadata (size, create/modify dates) Fast access, can be distributed But: hard to query, hard to add metadata

14 Metadata is easy to store in a relational database But media data is hard to store in one No notion of constituent parts Can store it in BLOBs (Binary Large Objects) but can't do much with them Streams, tracks, frames Difficult to extend functionality to include mediaspecific operations The point of RDBMS is run-time querying But BLOBs are not useful in this context

15 RDBMS strengths and weaknesses Strengths Extremely flexible queries Database can have arbitrary schema Some overhead for accesses (esp with locking) Can distribute reads across multiple servers Weaknesses Difficult to build hierarchical models Fixed schema Can't perform media-specific operations Difficult to support distributed writes

16 Loose vs tight coupling of DB and multimedia storage Loose coupling Tight coupling

17 Loose vs tight coupling of DB and multimedia storage Loose coupling Tight coupling DBMS for metadata, filesystem for media Simpler to implement, maintain, optimize, improve Difficult to maintain integrity and consistency DMBS for metadata and media Maintains integrity and consistency between metadata and media We will strive for tightly coupled systems

18 Tight coupling attempt 1: Object (oriented) databases Application using the database The database, conversely Is procedural Deals with one item of data at a time Creates complex data structures Is declarative Deals with tuples in sets instead of individually Holds data in flat tables Combining them exposes an impedance mismatch Can we create a database that stores data in a way more similar to the application?

19 Object databases Persistent store for objects Preserves object type and structure (nesting) Preserves references between objects Easy to combine data and metadata Little querying, mainly reference-following Lookups performed in code, not in query language Example: VelocityDB for C#

20 Object databases strengths and weaknesses Stengths Stores objects directly Easy to extend datatypes Arbitrary and modify-able schema Stores metadata and data together Can define media-specific operations Weaknesses Hard to distribute load across machines Hard to query Mixes code and schema Difficult to administer

21 Tight coupling attempt 2: Graph databases One speed advantage of object databases over RDBMS, comes from reference following Why compute joins that you know the answer to? Precompute joins and store as a graph of relationships: a graph database Examples: Facebook TAO, Neo4j

22 Facebook's social graph Everyone's facebook feed is extremely customized Facebook data is a densely connected graph Can't cache pages, hard to cache anything Hard to partition into disconnected shards Were writing Memcache and MySQL code directly But in 2009 started building a graph database abstraction on top of them Described in post here and USENIX 2013 paper (Bronson et al, 2013)

23 The Associations and Objects (TAO)

24 TAO operations Insert objects and associations Point queries: find specific (id1, type, id2) triples (associations) Usually to check if two objects are connected or to fetch association data Range queries: find outgoing (id1, type) associations Include time stamps for temporal locality Order associations by time to find most recent comments, friends liking something, etc Count queries: find total number of outgoing (id1, type) associations Can keep track of this as associations are created and deleted too

25 Neo4j Example: Relational model

26 Neo4j Example: Graph model

27 Neo4j Example: Converting relational to graph-based Each row in a entity table is a node Each entity table becomes a label on nodes Columns on those tables become node properties Foreign keys become relationships to the corresponding nodes in the other table Join tables become relationships, columns on those tables become relationship properties

28 Neo4j Example: Relational vs graph schema Relational Graph-based

29 Graph database Strengths and weaknesses Strengths Joins are precomputed Easy to modify schema Fast Scalable Weaknesses Harder to execute queries not embodied in relationships

30 Between object and relational: Object-relational mapping (ORM) Layer between RDBMS and OO program Translates certain OO calls into queries Translates results tuples into objects Almost all languages have an ORM

31 Object-relational mapping Objects to be represented

32 Object-relational mapping Object database approach

33 Object-relational mapping Object-relational approach

34 ORM in pseudo-code for homework1 The name and description of every shot used in Program 1, in the order in which they appear p1 = Program.find('id': 1) shots = p1.shots('order by': 'starttime') for shot in shots: print shot.name, shot.description How many queries does this result in?

35 ORM in pseudo-code for homework1 The latitude and longitude at which every shot used in Story 3 was filmed, in the order in which they appear s3 = Story.find('id': 3) shots = s3.shots for shot in sorted(shots, key='starttime'): print shot.gpslat, shot.gpslon

36 ORM in pseudo-code for homework1 The tape number and start and end time on the tape of every shot used in Program 2, in the order in which they appear p2 = Program.find('id': 2) shots = p2.shots for shot in sorted(shots, key='starttime'): tape = shot.tape print tape.number, tape.starttime, tape.endtime

37 ORM in pseudo-code for homework1 The dates when programs containing Shot 1 first aired s1 = Shots.find('id': 1) for program in s1.programs: print program.dateaired

38 ORM Strengths and weaknesses Strengths Easier to extend with new operations than RDBMS Can be distributed across many computers (for reads) Arbitrary schema Weaknesses May require many queries Still can't store large files/objects Fixed schema

39 Summary Awkward to fit multimedia data directly into relational databases Object-relational impedance mismatch makes it difficult to use RDBMS in programs anyway Object-relational mappings provide an imperfect bridge between the two Object databases remove the impedance mismatch But make it awkward to query Graph databases extract associations from objects Or pre-compute joins in a relational model