MBrowser v1.8 (Multimedia Browser) Quick Reference

Size: px

Start display at page:

Download "MBrowser v1.8 (Multimedia Browser) Quick Reference"

Mary Walsh
5 years ago
Views:

MBrowser v1.8 (Multimedia Browser) Quick Reference by S. KIRANYAZ MBrowser is the primary and only MUVIS application that is used for multimedia browsing and retrieval within a MUVIS database.

1 MBrowser v1.8 (Multimedia Browser) Quick Reference by S. KIRANYAZ MBrowser is the primary and only MUVIS application that is used for multimedia browsing and retrieval within a MUVIS database. It is a powerful multimedia terminal, which furthermore supports database visual/aural rendering, retrieval via query by example, Progressive Query, video summarisation, etc.. 1 Installation Simply run Setup MBrowser v1.8.exe file. Once installed the same file can be used to uninstall the program. Q: Which operating systems MBrowser can be installed? A: Like the other MUVIS applications, it is developed and tested for the following OSs: Windows 9x Windows ME Windows 2000 Windows XP Q: Are there any pre-requisite programs that should be installed before MBrowser installation? A: Yes. It is strongly recommended to install MPEG-4 codecs such as DivX and XviD. &

2 2 MBrowser in Run Time When you run, MBrowser UI appears on the screen with no current action. User should initiate the actions. Q: What are the actions that MBrowser can perform? A: In basic terms, MUVIS multimedia (MM) items such as audio/video clips, MUVIS databases, images and frames are rendered and used for retrieval using MBrowser. You can load a MM item simply by drag-n drop into the main UI or you can use <File> menu to load them. Q: What types of databases MBrowser supports? A: Any type of MUVIS databases with such extensions: vdbs, hdbs & idbs. Q: What happens when I load a database? A: According to the rendering mode, 12 MM items of the database that are randomly chosen within the database are displayed on the left side of the UI. If the database is a hybrid database, the 12 Audio/Video clips are initially displayed. Q: How can I change the Hybrid database display mode from video to images or vice versa? A: From <Display Functions> <Hybrid Database Options> sub-menu. Q: How can play/view a database MM item? A: Just left-click on it OR chose playback/clip from the right click menu. Q: How can I query a database MM item? A: There are 2 ways of initiating a Query Process; the first one is to start playback or view it as explained and then you can press Query button to query it immediately (if the database is indexed by some features). The second way is to use the right-click menu to start query immediately. Q: If I stop playback or pause playback or choose another item from the database, is Query affected? A: No. Essentially it is recommended to stop or pause playback if you query a video/audio file to give more CPU power to Query process. But there is no harm to do anything else the query will be performed anyway. Q: I loaded a database and it gives a Warning message saying that the database contains no features and therefore, the query is disabled. Why? A: At least (theoretically) one MM item in the database should have one feature vector extracted in order to establish a query. In other words the database should be indexed by some feature vectors in order to enable the query process. If you query a video or audio clip then that clip should have at least one feature indexed along with the clip. Q: How can I see whether a database does contain features and the type of the features? A: There are various ways of checking the database features: Load a database into MBrowser and from <Query> <General Query Parameters..> menu you can see the visual (if exists) and audio (if exists) features indexed into the database. The second way is to load the database file by a text editor and check for the features all of which either visual or audio tag preceding.

3 Q: How many Query modes are supported by MBrowser? A: This basically depends on the types of the features and MM items existed within the MUVIS database. Once a database is loaded into Mbrowser, from <Query> <General Query Parameters..> menu you can see the visual (if exists) and audio (if exists) features indexed into the database. So if visual feature(s) are present, you can perform Visual Query and similarly if Aural feature(s) are present, you can perform Audio Query. Visual queries can be performed over video clips, key-frames and images. However audio queries can only be performed over audio clips (with or without video). Q: How can I change the query mode between Visual & Aural Queries? A: First choose <General Query Parameters..> menu to get Query Parameters dialog. In this dialog there is a switch button at the top saying Visual Query is Enabled. You can change the Query mode from this switch. Q: How can I change query parameters? A: As explained above, choose <General Query Parameters..> menu to get Query Parameters dialog. In this dialog you can enable/disable the features used for query and you can give weights for each sub-feature (if enabled) using the slider bar or typing the number (<1.0) into the edit box. The similarity distance obtained from that sub-feature vectors will then be multiplied by that weight number to calculate total similarity distance. Q: What is Progressive Query (PQ)? A: PQ in one of the important features from MUVIS v1.6. It is the default Query type in MBrowser (so you do not need to change anything) and it enables the user to get the progressive query results periodically. You can set any period from Query Parameters dialog but make sure that to enable Progressive Query beforehand. If you do not enable it then the query type will be Normal Query where the results (the final) will arrive after the query process is completed. Q: Can I stop or browse through the Progressive Query retrieval results? A: Yes. By turning the Query button in the main window of MBrowser one can browse among the query retrieval results. Also you can use directional keys (up & down) for this. In order to stop an ongoing PQ simply press STOP button (same as Query button). It will stop as soon as next periodic sub-query is completed. Q: What is HCT Browsing? A: HCT stands for Hierarchical Cellular Tree. It is a novel indexing structure that is further optimized for MUVIS v1.8. If a MUVIS database is indexed by HCT then there are two potential use of it: within PQ for faster retrievals and for HCT Browsing. PQ over HCT is the default query method it is automatically used when a query is performed. HCT Browsing on the other hand is the alternative browsing method for random and sequential browsing. Q: How can I do HCT Browsing? A: Choose <Display Functions> <Database Browsing Options> <HCT Browsing>. Similar to query types there are two different HCT Browsing modes available: Visual and aural, both of which uses the appropriate HCT indexing structure if present. If not, the query browsing will automatically be set as the default browsing mode (random browsing). When the HCT Browsing is selected then the HCT Browsing Control Window will pop up and you can browse through the HCT body using this window.

Q: How can I use HCT Browsing Window? A: As shown in Figure 1 the HCT Browsing Control Window provides buttons to navigate among levels and cells within the same level.

4 Q: How can I use HCT Browsing Window? A: As shown in Figure 1 the HCT Browsing Control Window provides buttons to navigate among levels and cells within the same level. The hierarchical structure of HCT can be used to give an overview to the user about what lies under the current level so that if well supported via a user friendly GUI, HCT Browsing can turn out to be a guided tour of the database items. When the user initiates it, it is designed to show the items in the cell at the top level, so it is the first clue about what the database content or more specifically a brief summary of it. The next logical step for you is to choose an interesting item in this cell and starts the tour downwards. So this is the first functionality that HCT Browsing provides: to choose an item in the upper level and trace down to see the cell it represents (as a nucleus item of that cell). As long as the item chosen belongs to the current cell, the level down option will lead to the lower-level cell that is represented by that item. Otherwise, the first cell in the lower level will be shown by default. In the ground level the level down option is naturally disabled. The opposite functionality, the level up will yield the upper level cell, which is the owner (cell) of the nucleus item of the host cell at the current level and it works at all levels except the top level. This is also a useful functionality to browse similar cells of a particular cell. Therefore, the (slight) variations of a particular content can be visited using both of the level functionalities. In addition to such inter-level navigation options, HCT Browsing provides inter-cellular trips within a certain level. You can visit the cells sequentially, in a forward or backward direction and one cell at a time. This is especially useful when you do not have any particular target content in mind and may be just looking for interesting items. So in such an indefinite or open-ended exploration task, navigating through consecutive cells at a certain level will summarize the overall database content and the amount of summarization obviously depends on the height of the navigation performed, that is, simply the current level number. Nucleus Item Level Controls Cell Controls Cell Items HCT Info Item Navigator Buttons Cell MST Info Prev Next Figure 1: HCT Browsing Contol Window

5 Q: When I query an image/key-frame, I saw a curve in the left side, what is it for? A: The features of images are displayed in graphical form for comparison. The upper black graphic area is for the queried image features. The lower black graphical area can be used to sketch the feature vector of any image on the database side. Just right click menu on an image and choose <Show Features...>. You can switch to any other feature by choosing an available feature from the list that is under the graphical displays. Q: Is graphical feature display for only images? A: Yes. Since it is not suitable for video and audio, each of which have multiple frames and so multiple features. Q: Once the query is finished, what exactly is displayed on the UI screen? A: The descending order of query results with the similarity distances reported under each MM item. So the best match is the upper left one. The second best match is the upper middle one and so on.. Q: There are only 12 best matches displayed then? A: No. Once a query is performed, then the arrow buttons on the right down corner of the UI screen becomes active. You can see the 2 nd best 12 results, 3 rd best 12 results, etc. by pressing Next button. Q: How do I know which button is used for what? A: Bring the cursor on top of them and you will see pop-up of a data tip reporting the button function. The same is true for the MM items of the database. Q: Sometimes I am loosing refresh on the UI screen, what should I do to refresh the screen? A: You can manually refresh the screen simply by clicking anywhere in the UI window. Q: How can I do video summarisation? A: There are several ways of doing it: If you want to see only the Key-Frames (KFs) of the video clip, you can activate the KF animation of a video clip by pressing right-click menu and choosing the <KF Animation>. You can then use stop button to stop it or pause button to pause it. If you pause it you can KF next and KF prev buttons to see the KFs manually. If you want to see KFs with the video same time, just right-click menu and choose <Playback with KFs>. If you want to see KFs all in once, then from main menu, choose <View> <KFs & SFs Window>. A new dialog called Key-Frames Display Dialog and Scene Detection will emerge. Load a clip and you will get all the KFs shown in the UI window. You can also do Scene Detection to extract to Scene Frames (SFs) out of KFs. Recommendation: KFs & SFs window may also be used for random-access rendering in the video. Choose a start KF by left-click onto it you will see an left-arrow will be put under it and an end KF again by left-click onto another a right-arrow is placed under it. Then by pressing the display button in the middle of the UI screen, you can playback the partial video that is in between the two arrows. Once the playback is initiated, you can use buttons to pause and stop video clip. Q: What parameters exist to extract SFs? A: None except the mode of SF extraction. You can do automatic extraction with no parameters, or you can give the number of SFs or a threshold value.

Q: Where will the SFs displayed? A: Right under the KFs. There are 2 SF extraction methods and so you will see 2 lines of SFs displayed. Q: But only 7 frames per line are present in the UI, Why?

6 Q: Where will the SFs displayed? A: Right under the KFs. There are 2 SF extraction methods and so you will see 2 lines of SFs displayed. Q: But only 7 frames per line are present in the UI, Why? A: You can see more SFs by pressing Next arrow button on the right button corner of the KFs & SFs window. Q: How can I make full-screen display of video clips? A: There are 2 ways of doing this: You can double-click the video window OR you can choose <Display Functions> <Full Screen Display..> menu and then start playback. Q: How can I see database images in their original sizes? A: You can use right click menu on an image and choose <Normal Display> to see the image on its original size. Q: How can I see Segmentation Masks (SMs) within MBrowser A: You can use right click menu on an image or a video clip and choose <Show SMs Dialog> to see the SMs as shown below. Similarly you use the Main Menu from <View> <Seg. Masks Dialog> to see SMs of any MM item active within MBrowser. For video clips, you need to pause the video to navigate back and forth among its Key-Frames using Key-Frame buttons (in a Key shape) below the slider bar.

7 Q: How can I see Shot Boundaries (SBs) and Key-Frames (KFs) from a particular SBD algorithm of a video clip? A: You can use right click menu on a video clip and choose <Show SBD Dialog> or similarly you use the Main Menu from <View> <SBD Dialog> to see SBs and KFs of any video clip active within MBrowser, as one typical example shown below.

DbsEditor v1.8 (Database Editor) QUICK REFERENCE

DbsEditor v1.8 (Database Editor) QUICK REFERENCE By S. KIRANYAZ This application is a dialog-based program, which is designed to perform several off-line tasks of the MUVIS multimedia databases. DbsEditor