MediaQ. Figure 1 MediaQ Home Screen

Transcription

1 MediaQ 1 Introduction MediaQ is an online media management framework that provide services to collect, organize and share user-generated mobile videos/images with automatically tagged geo-spatial metadata. It could be used in disaster response situation such as earthquake or fire place. The government and any helping association unit can get the first in-place look to help make the accurate decision. Considering the broadcast of network in crowded place, after user generates video/photo, MediaQ always upload the small-size metadata first which contain capture time, camera location, trajectory and viewing direction. Then as anyone interested in that video/photo, we use Google Cloud Message to manage the content query from other user and send notification to the content owner. Therefore, the goal of our final project is to develop such feature together with photography uploading in Android phones. Figure 1 MediaQ Home Screen

2 2 Basic Functions 2.1 Record a video and take a picture Users can record a video and take a picture like other camera app. While doing it, our app saves the metadata such as GPS, Compass data, Internet information and device information in the background side. Figure 2. Screen shot of camera 2.2 Automatic metadata uploading When users create content by recording a video or taking a picture, metadata associated with it is automatically uploaded to the server. Content itself is only uploaded when user triggers it to avoid unwanted network/data usage for uploading large files (metadata file size is very small, at most a few kbs). We stored the device id, user Id, and location information and so on to both show video or photo icons in the MediaQ website and identify the file holder later on. 2.3 Upload videos and pictures When users push the upload button, the picture and video contents are uploaded to the server including their metadata. 2.4 View Tranjectory While recording a video and taking a picture, our app saves the path described by an android device moving by using GPS information. Users can see the path on google maps. 2.5 Tasks Inquiries and Pending Tasks Users can ask some tasks to the server. The server can receive the user s requests and give some tasks like quests to users. After receiving the tasks, user can upload the videos and pictures taken from the specific place which the server gave.

3 2.6 Push notifications Since metadata is automatically uploaded, server knows where users recorded videos/photos, even if they didn t upload the content. The server can send out push notifications to the clients to request that they upload specific media. 2.7 Minor functions Users can view list of recorded videos, change the mood, preview the videos, see where they were taken, upload them, delete them, view the trajectory on google maps, and etc, Figure 3. Pending Tasks Figure 4. Video preview/upload/delete

4 3 Details 3.1 Overall Application Figure 5. MediaQ Architecture There are three partition of this overall system shown in Figure 5. Android application, GCM server and data server. Android application upload metadata to data server and also communicate with GCM server. A notification sent by GCM server notice user to upload the recorded video/photo content. For more specific, the MediaQ application not only upload the metadata, the client also register their device ID on server. Then the server keep track all client's ID and use it to distinguish each device. So when user want to participate with video which is not uploaded yet, the server is able to sent the notification to the content owner through Google cloud message, asking them upload the real data.

5 3.2 Analyzing a Json file ( The data sending to the server ) { "format_version": "0.4", "video_id": "1lwpl9ky2c21o_2016_3_23_Videotake_ mp4", "owner_properties": { "tid_type": " ", "id": "geovideo@example.com" }, "device_properties": { "SIM_id": "1lwpl9ky2c21o", "OS": "android", "OS_version": "5.0", "firmware_version": "5.0", "model_name": "SM-N900K", "camera_direction": 180 }, "UploadType": "request", "TaskId": " c e", "TaskInquiryId": "f001dd75-49ec-4b", "TaskLat": "null", "TaskLong": "null", "UserId": "2832bc0e-cfc8-48", "Username": "user190", "TaskType": "video", "keywords": [ ], "LocationType": "outdoor", "LevelOfVideoPrivacy": "Public", "Mood": 2, "UploadDate": "03/23/2016/08:13:07/PM/UTC-07:00", "RecordDate": "03/23/2016/08:12:59/PM/UTC-07:00", "sensor_data": [ { "location_array_timestamp_lat_long_accuracy": [ // request: task inquiries, personal: personal upload // GPS data [ , 03/23/2016/08:12:59/PM/UTC-07:00, , , , 0.0], [ , 03/23/2016/08:12:59/PM/UTC-07:00, , , 17.08, ] ] }, { "sensor_array_timestamp_x_y_z": [ // Compass data [ ,03/23/2016/08:12:59/PM/UTC-07:00,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0], [ ,03/23/2016/08:13:00/PM/UTC-07:00, , , , , , , ,92.8,-121.7,-30.7, , , ], [ ,03/23/2016/08:13:00/PM/UTC-07:00, , , , , , , ,92.8,-121.1,-31.4, , , ], [ ,03/23/2016/08:13:00/PM/UTC-07:00, , , , , , , ,92.8,-119.9,-31.4, , , ] ] },{ "wifi_array_timestamp_bssid_level_frequency": [ [ ,"00:24:6c:b4:1e:19",-58,"5745"] ] } ] }

6 3.2.1 Location_array_timestamp_lat_long_accruacy (1) Timestamp (2) Recording date (3) Latitude (4) Longitude (5) Altitude (6) Accuracy Sensor_array_timestamp_x_y_z (1) Timestamp (2) Recording date (3) Orientation z (manipulated azimuth) (4) Orientation x (manipulated pitch) (5) Orientation y (manipulated roll) (6) Light (7) Accelerometer x (8) Accelerometer y (9) Accelerometer z (10) MagneticField x (11) MagneticField y (12) MagneticField z (13) Gyroscope x (14) Gyroscope y (15) Gyroscope z 3.3 Push notifications to request uploading of media contents Figure 6. flow of getting GCM Registration ID

7 To identify each intent, we retrieve GCM registration ID by communicate with GCM server as Figure 6 shown. In the beginning we get the google API key at Google Developers Console. With the API key, MediaQ application communicate with google cloud message and get Registration ID. Finally, the client share RegID with server and post notification to GCM. 4 The role of files 4.1 Server upload [_pictures].php Our application support both picture and video recording and uploading. After received notification from GCM, we deployed this API to upload the video/photo content to MediaQ Database server. uploadstatus [_picture].php To ensure we know the file content or metadata is correctly stored in the server, especially all the metadata is uploaded successfully on the server, we request the file status through uploadstatus[_picture].php and it returns us both the metadata length, and content length of the requested file. If both length matches the local file lengths, we marked this file as completed in the client application. Otherwise, if the metadata size is less the local file, we uploaded metadata. This is a service to help client applications know the file status in the server, and client will try to upload metadata whenever it can when server does not acquire the metadata because of some interruption of internet or server breakdown at the moment that user just takes videos or photos. GCMDeviceRegistration.php On the data server, there are not only stored media content and metadata, but also server services run on it. One of these is GCM device registration. For every intent, Google will assign a unique ID which is depends on the Application. In this php process, we collect any identical information from device, user and registration ID getting from google server and produce them to store on MediaQ server for later GCM notification usage. 4.2 Client (Android) DashboardActivity First screen shown as Figure 7 is the page user sees after logging in. In this page, user can access Task Inquiries, Pending Tasks and videos/photos collection. While this page load, it immediately retrieves registration token from GCM server and sending device token to MediaQ data server, to register for push notifications service.

8 Figure 7. Dashboard page GeoVidDashboardActivity As Figure 8 shown, the dashboard support user to access that content record page and navigates to list of videos/photos. This dashboard is also automatically switched to right after user just taken a video/photos. More specific, the video taken content also contain the trajectory and the direction of camera. So it s geo-spatial media content.

9 Figure 8. Data Collection including Video and Photo GeoVidActivity This is the activity that show a list of video/photo contents. It displays a list of contents, each with its information, screenshot picture, taken date and time and the upload status. Each element connects to its video/photo. When a user selects an item, that correspond video/photo is ready for uploading by one click on it and choose upload. GeoVidDialog This dialog menu activity will be shown when user selects one of videos/photos in GeoVidActivity. GeoVidActivity retrieve the content id and popup an dialog. From here, user can upload, change mood, preview, etc.

10 Figure 9. Dialog to make change of any video in the list GeoRecorderActivity This activity allows the user to take a picture for a new video/photo content. It utilizes the Android phone s camera to take a picture or video and saves it onto the photo s local storage prior to sending the metadata to the MediaQ data server. After finishing, the activity flow return to GeoVidActivity. GeoVideo This service Encapsulates the Android Camera APIs, and the recording of metadata (GPS, sensors) along with recording video. After the the video/photo taken, the metadata will be produced and will be upload to MediaQ database server after it by calling UploadService. UploadService This service is used to upload metadata and/or content to the server. In our application the metadata is immediately upload after the video/photo content has been taken and the content will be asked to upload when the user is notified by an inquiry and upload the video/photo content. And these uploading process is served with this activity. We upload files chuck by

11 chuck, for each chuck, the UploadService retries at maximal 10 times to avoid some sudden internet interruption stopping uploading. MetadataTimedUploadService This service is used to try uploading metadata for files which haven t had metadata uploaded yet. In case of failure, retries after 2 minutes, for a maximum of 10 times. GcmBroadcastReceiver This service is used to receive GCM messages (i.e. push notifications from server). Launches GcmMessageHandler. The application periodically subscribes the GCM server to check if any new inquiry has been sent which the subject content owner is the self device. If the server responds with new video/photo content inquiry and this device and user is the content owner, the user will be notified via Google cloud messaging and the notification will inform the detail information of the desire video/photo content, file name as helping user to upload assigned content. GcmMessageHandler Handles GCM messages by showing notification in the system notification bar with the message that was received from the server. If the server responds with new video/photo content inquiry and this device and user is the content owner, there will be an message shown at notification bar and clearly inform the asked file to be upload. Figure 10. Receive GCM Notification

12 5 Class diagram Figure 11. The flow of process

13 6 Future works 6.1 GCM notification For now, the GCM notification only show which file is requested to upload. In the future, it might directly link to the uploading activity of specific file for user to upload the file. 6.2 GeoRecorderActivity For now, when recording a video and taking a picture, we get rotation sensor data from the android device. While rotating the device to record a video for landscape or portrait mode, some lags occur. It might be fixed by using orientation sensor data instead of rotation sensor data. 6.3 Compass data In the server side, only an azimuth value(orientation[0]) is used among the 15 compass data. We will think about how remaining data will use. 6.4 Resolution Currently, the android device sends the photo and video contents with the highest resolution to the server. We will analyze what are the pros and cons on the resolution and make buttons which users can choose the resolution.