Visual System Implementation Shamal AL-Dohuki and Ye Zhao
Shamal AL-Dohuki Ph.D. candidate in the Department of Computer Science at Kent State University, Ohio, USA. Software Development Lead of TrajAnalytics My current research interest includes: Implementing visual analytics of big urban data Urban data management and visualization Visual query of trajectory data Semantic data query and analytics
Overview Exploratory visualization systems for urban trajectory data Efficient user interaction Instant visual feedback. Support practitioners, researchers, and decision-makers to store, query and visualize the data
Overview Cont. The system should integrate scalable data management visualization with powerful computational capability. and interactive Powerful computing platform. Easy access gateway. Scalable data storage and management. Exploratory visualization.
Web based Visual System Web browser based visual analytics is the trend Cost effective and fit for multiple platforms Easy installation and maintenance Secure with easy account management and access control Built on client-server scheme Data uploading and remote management We focus on web-based visual system in this tutorial
Web System Challenges Web Services are web accessible applications and application components that exchange data, share tasks, and automate processes over the Internet. To build a web service for visual exploration of trajectory data, researchers face some general issues: How to transfer data between the client and the server and in what format? How to visualize trajectory data on the client side? What views and interactions are needed? What server side and database supports are needed?
Client and Server Scheme Datasets are stored and controlled by the remote data server. Users send requests and receive information from the server via a client. Some functions are performed on both server and client side. There is a trade-off between performing functions on the database server and on the client side.
An Example Web-based Software System TrajAnalytics is a web based software for visually exploring urban trajectories http://vis.cs.kent.edu/trajanalytics/
TrajAnalytics Features A cloud-based software which can be accessed through Web browsers. Users can Load raw trajectory data Perform map matching and aggregation Conduct extensive visual analysis at any time A downloadable version is available for users to use it in local machines Localhost is used
TrajAnalytics Demo
Visual System Implementation In next slides, we describe the details of implementing web based visualization system for urban trajectory datasets Mostly based on our work in TrajAnalytics design and implementation
Web Based System Server: Spatial Database with Spatial Indexing. PostgreSQL, MySQL, MongoDB Client: HTML, PHP with HTTP, accessing with any internet browser. Data Transfer: jquery AJAX request to PHP and PostgreSQL. The data will be restored in the.json object and will be transferred to the client side. Web interface: JavaScript with multiple libraries. Leaflet.js, D3.js, and more.
PostgreSQL Database PostgreSQL is an object-relational database system. It has the features of a traditional proprietary database system with enhancements to be found in next-generation DBMS systems. PostgreSQL has most features that are present in large proprietary DBMSs, like transactions, sub selects, triggers, views, foreign key referential integrity, and sophisticated locking.
PostGIS for Spatial Datasets PostgreSQL also offers some special features like user-defined types, inheritance, rules, and multi-version concurrency control to reduce lock contention. PostGIS is a spatial database extender for PostgreSQL. It adds support for geographic objects allowing location queries to be run in SQL. SELECT superhero.name FROM city, superhero WHERE ST_Contains(city.geom, superhero.geom) AND city.name = 'Gotham';
Network Communication Challenging issues: How to connect the client with the database (PostgreSQL). How to extract data from the database and transfer the data to the client side. PHP and the Apache server are used for this purpose. Trajectory data in the database are read by PHP through the Apache server. The data will be restored in the.json object and will be transferred via the internet to the client side.
System Structure PostgreSQL Database PHP Middleware Apache Server Clients Internet Connecting with the database by using PHP and Apache server
Web Servers There are many types of web servers available. Apache server is the most common server around, very easy to set up, and compatible with all major operating systems. On Linux: Installation and configuration of Apache webserver can be done in one step.» sudo apt-get install apache2
Web Servers On Mac OS: Apache is installed by default. All you need to do is turn it on. Go to Terminal and run the following command to turn on your already pre-installed Apache web server.» sudo apachectl start On Windows: there are several install wizards that bundle things like Apache, MySQL, and PHP together to make our lives easier. One of them is XAMPP. Note: XAMPP is available for Linux and Mac OS X too. (Recommended)
XAMPP and PHP XAMPP bundle tools like Apache, MySQL, and PHP together. PHP stands for Hypertext Preprocessor. PHP is a server side scripting language, like Microsoft s Active server page (ASP). PHP is open source software and widely used for making dynamic and interactive Web pages. PHP uses pg_connect to speak to the PostgreSQL database on the server.
XAMPP Control XAMPP Control Panel: Bundle Apache, MySQL, and PHP together
Connect to Database To connect to PostgreSQL database using PHP, a pg_connect function with connection strings that consists of host name, name of the database, user and password was performed. The following codes describe how to connect to PostgreSQL using PHP: <?php $db_connection = pg_connect("host=localhost dbname=dbname user=username password=password"); $result = pg_query($db_connection, "SELECT lastname FROM employees");?>
Data Queries and Transfer In a web based visual exploration of trajectory data, jquery AJAX request to PHP and PostgreSQL could be used. The data will be restored as a JSON object and will be transferred to the client side.
Data Queries and Transfer Cont. Ajax (also AJAX) short for "Asynchronous JavaScript And XML. Set of Web development techniques using many Web technologies on the client side to create asynchronous Web applications. With Ajax, Web applications can send and retrieve data from a server asynchronously (in the background) without interfering with the display and behavior of the existing page. Modern implementations commonly utilize JSON instead of XML due to the advantages of JSON being native to JavaScript.
Data Queries and Transfer Cont. Normal HTTP request VS. Ajax Request
Data Queries Example Example: Using AJAX to do Region Query Temporal Constraint Spatial Constraint
Data Queries Example Cont. Connect to the Database Insert Your Query here The received response is JSON object query.php
Data Queries Results The parsed JSON object is visualized by using different visualization methods TrajAnalytics: A Free Software for Visually Exploring Urban Trajectories http://vis.cs.kent.edu/trajanalytics/
Data Queries: Demo
Map Visualization and Tools The common visualization method used for urban trajectory data is static and animated maps. This method provides some interaction techniques as well. Creating a good web map is not an easy task but over the years we ve seen plenty of amazing geovisualization examples. All of them have been developed with the use of one of the following tools, APIs or libraries.
CARTO CARTO previously known as CartoDB is the best platform for complex and dynamic geospatial data visualization and analysis. example: Flights around the world
ArcGIS ArcGIS Online: Esri has monopolized the field of GIS and is also one of the most popular geo visualization platforms. example: 5 years of drought
Data Lens Here Data Lens is a set of APIs that provide a seamless developer experience for creation & deployment cool data visualizations. example: Billboard analytics tool demo
Mapbox Mapbox is a geo-visualization platform that gives easy to use set of tool for creating beautiful web and mobile maps. example: 1.5 Million Walks, Runs, and Bike Rides from RunKeeper mapped on Mapbox
Google Maps Google Maps offers a set of APIs for different mapping purposes. It offers access to Google s mapping data that includes StreetView among others. example: GeoGuessr StreetView based game?
D3 D3.js is a JavaScript library that offers powerful data visualization features for producing dynamic, interactive data visualizations in web browsers. example: Earth wind visualization
OpenLayers OpenLayers is an open source JavaScript library for displaying map data in web browsers. It provides an API for building web-based geographic app. Similar to Google Maps but for free. example: Bike sharing map
Leaflet Leaflet.js API an open source JavaScript library for mobilefriendly interactive maps. It offers really powerful mapping features that commonly used for the best and most beautiful maps. example: Travel times map HERE Isoline + Leaftet (Recommended)
Leaflet Cont. Leaflet is the leading open-source JavaScript library for mobile-friendly interactive maps. It works efficiently across all major desktop and mobile platforms. It has a beautiful, easy to use and well-documented API and a simple, readable source code that is a joy to contribute to.
Leaflet Cont. Example: Create a map in the 'map' div, add tiles of our choice, and then add a marker with some text in a popup:
Leaflet Cont. Example: Create a map in the 'map' div, add tiles of our choice, and then add a marker with some text in a popup
Leaflet Cont. For more map tiles, check the link below: https://leaflet-extras.github.io/leaflet-providers/preview/ This page shows mini maps for all the layers available in Leaflet-providers.
Leaflet Cont. Besides tiles layers, you can easily add other things to your map, including markers, polylines, polygons, circles, and popups. Markers can be used to visualize POI on the map. Polylines can be used to visualize trajectories and road network on the map. Polygons can be used to visualize zip code regions on the map. Circles can be used to visualize pick-up and drop-off locations on the map. Popups can be used to show some semantic information on the map.
Leaflet Cont. To add a marker: Adding a circle is the same (except for specifying the radius in meters as a second argument), but lets you control how it looks by passing options as the last argument when creating the object:
Leaflet Cont. Adding a polygon is as easy: Popups are usually used when you want to attach some information to a particular object on a map. Leaflet has a very handy shortcut for this:
GeoJson GeoJSON is a format for encoding a variety of geographic data structures. GeoJSON is becoming a very popular data format among many GIS technologies and services. A GeoJSON object may represent a geometry, a feature, or a collection of features. For instance, the road network that extracted from OSM. GeoJSON uses the JSON standard. Leaflet Map is quite good at handling it.
GeoJson Cont. In this example, we check the "party" property and style our polygons accordingly:
Heatmap Leaflet Heatmap is a plugin to create a heatmap layer for Leaflet map.
Heatmap Cont.
Markers Marker Clustering plugin for Leaflet Provides Beautiful Animated Marker Clustering functionality for Leaflet, a JS library for interactive maps. Check this Example
Mapbox Bloomberg New Energy Finance relaunched their Energy Asset maps to visualize everything from power plant capacity to distribution of gas pipelines and biofuel production. Using Mapbox.js with the Leaflet Marker Cluster plugin, the map reveals more information as you zoom in. Drilling down on wind versus hydro versus nuclear power with Bloomberg New Energy Finance maps.
Data Attributes Visualization Urban trajectory data consists of time/location/other properties (e.g. Speed). The common visualization method used for location is static and animated maps. Creating charts and infographics for other attributes can be time-consuming. Many APIs have been implemented to make this process easier. D3.js, Chart.js, Google Charts, Highcharts, Plotly and others. Next, we will show some of the recommended tools that can make the tedious task of making beautiful charts easier:
D3.js D3.js is a JavaScript library that uses HTML, SVG, and CSS to render some amazing diagrams and charts from a variety of data sources including Urban Trajectory Data.
Chart.js Although armed with only six chart types, open source library Chart.js is the perfect data visualization tool for hobbies and small projects. (Recommended)
Leaflet DVF Leaflet Data Visualization Framework (DVF) The Leaflet DVF is an extension to the Leaflet JavaScript mapping library. The primary goal of the framework is to simplify data visualization on the map. Allow the user to draw different types of glyphs on the map.
Interactions Interactive visualizations can add an impressive animation to plain and boring datasets. It allows users to select specific data points to visualize the story in the way they choose. One of the most powerful ways to accomplish this is through JavaScript. Created by Joanna S. Kao
Interactions Cont. Example: Building an interactive visualization of geospatial data by Adil Moujahid
Interactions Cont. A web service that allows users to search for businesses within a radius of a specific point that has received reviews within a specified date range. The interaction is provided for both map visualization and information visualization. Interaction is linked to both views. SpaceTime Reviews Data from Yelp Open Dataset
Interaction Example Leaflet.js - Event Handling The Leaflet JavaScript program can respond to various events generated by the user. Example: Add Marker to the map when user click on the map
Interaction Example Leaflet.draw To do spatial (region) query over the map canvas, there is a need to provide users with the ability to draw different shapes (polygons, bounding box, circle, line) on the map to define the spatial constraints.
Interaction Example Leaflet.draw is designed to help users to: Draw shapes on your map with easy to use drawing tools. Edit and delete vectors and markers. Super customizable: Customize the styles of each shape to fit in with your maps theme. Pick and choose the which tools you want to use. Roll your own by simply using the drawing and editing handlers. Event based system allows you to perform any necessary actions when shapes are created, edited or deleted.
Interaction Example Cont. Leaflet.draw is very simple to drop into you Leaflet application. The following example will add both the draw and edit toolbars to a map:
Interaction Example Cont. Once you have successfully added the Leaflet.draw plugin to your map, you will want to respond to the different actions users can trigger. For instance, do a point query