Performing Intelligent Mobile Searches in the Cloud using Semantic Technologies

Transcription

1 2010 IEEE International Conference on Granular Computing Performing Intelligent Mobile Searches in the Cloud using Semantic Technologies Vishnu S. Pendyala Department of Computer Engineering Santa Clara University Santa Clara, CA, USA JoAnne Holliday Department of Computer Engineering Santa Clara University Santa Clara, CA, USA Abstract The paper presents a novel approach to do intelligent mobile searches using semantic, mobile, and cloud computing technologies. A working prototype of a Restaurant Search mobile system is presented for demonstration of the concept. The prototype built is intelligent in understanding and assisting the user choose a restaurant that gives maximum satisfaction to the user under the given constraints. Keywords Intelligent Mobile Search, Semantic Querying, Ontology, Cloud Computing I. INTRODUCTION From the time IBM Chairman Thomas Watson said in 1943 that there s a world market for maybe five computers to today s web as the ubiquitous computer [1], computing has come a long way. Bandwidth availability, inexpensive commodity hardware, and service oriented architectures (SOA) have played a wide role in enabling Cloud Computing. The synergy of hundreds of thousands of servers working in clusters hidden behind search engines such as Google s, demonstrated the advantage of deploying similar configurations that can be leased. This discovery of the potential of large infrastructures led to Cloud Computing for a wider use. Computation is increasingly becoming mobile with the proliferation of the relatively inexpensive and networked mobile devices. Mobile devices will continue to be the only computing devices accessible to populations in developing countries.. The global market for mobile devices is bound to grow in leaps and bounds, as more and more echelons of the global society get added to the economic mainstream. The Ubiquitous Computer that the cloud represents, needs to scale to this need very quickly. Any paradigm shift in computing cannot afford to overlook mobility aspect to be successful. As the demand for computationally intensive applications such as for providing augmented reality, on-the-fly decision making, and learning grows, there will be increasing demand on the cloud to provide real-time, scalable compute resources. Energy critical mobile devices will have to depend on Cloud Computing [13] for machine cycles. Computation offloading seems to be the way to go, to give mobile devices access to applications that can revolutionize quality of life. Response time is extremely critical for many such applications. The cloud computing paradigm opens up a plethora of opportunities for mobile applications. As the web starts to be used more and more as the ubiquitous computer [1] with abundant computing power and resources, clients can afford to get thinner and thinner, consuming less power and be able to run on mobile devices easily. Applications themselves will need to demonstrate intelligence as the dependence on the web for many high end needs increases. Web 3.0 has already been a step in this direction. Semantic technologies already provide for reasoning and deduction. A mobile user has limited computing and battery power in the device in his hands. It is also likely that the mobile user would search for something like a restaurant, while on the go. So, the user does not have a whole lot of time to browse through the search results. It is a noted trend that ever-exploding information available on the web creates a paucity of attention, requiring customization and personalization. These are two different techniques to address the issue [2]. The system presented here is envisaged to use both these techniques, though the prototype implemented has more to do with personalization. The intelligence of the system lies in personalizing the results quickly and efficiently, so that the user gets results that are as close as possible to what he wants. The key function of the system is to support or facilitate decision making for people on the go, who are searching for resources such as shops or restaurants. To achieve this function, it is also necessary to integrate data and knowledge from diverse sources including reviews of the resources sought and other pertinent information available on the web. This is where intelligence comes into picture. The SmartWeb project [4], which envisions mobile broadband access to the semantic web served as a good introduction for this project. A number of papers, couple of which [5], [6] are listed in the references section, published as part of this project provide good insights into the problem of mobile access to intelligent information systems. Ample information about the semantic technologies is presented in [9]. The architecture of the semantic portion of the system presented here is largely inspired by [9]. Information exchange between applications and their integration is presented in [10] and gives insights into what may be involved when the /10 $ IEEE DOI /GrC

2 prototype presented here scales-up to an industrial strength system. The prototype uses Google s Android emulator [8] to test the proof of concept. A. Our Contributions and Paper Organization This paper addresses a significant problem the new developments facilitate: to provide access to intelligent solutions in the hands of the mobile user. It is our hope that the system presented in this paper serves as a classic example of the coming together of mobile access and cloud computing, using semantic technologies. As far as we know, there is no other work that combines all three paradigms and discusses the implementation details of a working prototype, using state-of the-art tools. This paper is organized as follows. Section II presents a model of the system at a conceptual level. Section III describes the prototype. Implementation of the prototype is presented in Section IV. Results, including what makes the prototype intelligent and what differentiates it from general search systems are presented in Section V. The paper concludes with a summary and future research directions in Section VI. II. INTELLIGENT MOBILE SEARCH MODEL The problem statement, motivation and architecture of the model is best understood by considering a scenario. A. The Problem Scenario It is 1pm. The group of friends driving on highway I-5 to Los Angeles is extremely hungry. Each one in the group has at least one unique criteria of choosing a restaurant. But all of them agree that the restaurant has to serve Indian cuisine and a good one at that. One of them is a pure vegetarian. Another prefers North Indian food. Yet another is from Southern India, so needs rice in the lunch. One has ulcers in stomach, so cannot eat spicy food and the other is from the southern city of Hyderabad, so prefers hot and spicy food. Each one can compromise on her criteria to some extent, but not completely. The group wished there was an assistant, who was conversant with all these criteria and the location information to find the right restaurant in a 20-mile radius. B. The Solution Enter the Mobile Intelligent Search system, which functions like the assistant and finds the restaurant in a few seconds. The system, designed to be a Client / server mobile application, where the server runs in the cloud, would sift through customer reviews and combine the result with the pre-defined categories to narrow down the search if there are too many restaurants meeting the criteria, or broaden the search if too few matches are found. So, if the user is looking for Indian cuisine with dishes like rice and North Indian bread, but there weren't any Indian restaurants near the location, the system should be smart enough to suggest "Singaporean", "Jamaican", "Malaysian" or other South Asian or Asian Fusion restaurants, which have similar flavors and dishes, as alternatives. That is, the system can infer that "Indian", Jamaican, "Singaporean", "Malaysian" or other South Asian or Asian Fusion cuisines are quite similar. The system is also able to understand that dishes like Indian naan, Malaysian paratha, etc fall in the same category of flat bread. These approximations and assumptions can be improvised as the application evolves. That is the machine learning part that is planned for future work. The system is able to identify places that might serve certain types of food that might not be apparent from the names of the restaurants or the cuisine they serve. For example, a Caribbean restaurant serving a wide variety of curries and other Indian-like food, which is quite popular with the local Indian ethnicity, might be the best choice in the present context. The system is smart enough to draw this inference. Obviously, such a decision cannot merely be based on a simple search of some database fields. It needs to be based on reasoning with information and knowledge that the system contains. Figure 1 shows a typical system flow. The various steps in the system are as follows: Mobile Client Authentication: User unlocks the screen of the mobile phone, logging-in, if required and chooses the search application. Criteria Feed: The user lists the criteria. For a restaurant search, this would be the cuisine information that she expects the restaurant to meet. Query Generation: The system generates a semantic query based on the above criteria. This can be done either on the client or the server, depending on how simplified the inputs are. For the prototype, this is done on the server to keep the design simple. Server Access: The mobile client talks to the server listening on a TCP port somewhere in the cloud and establishes a connection and passes the query to the server. Inferencing and Information Retrieval: The server queries the ontology of resources, such as restaurants, using built-in inferencing based on the criteria and produces the results. The results are sent across to the mobile client. Results: The mobile client displays the results in a user-friendly GUI.Prototype 382

3 start < southindianfood rdf:id="southindianfood"> Client Authentication Input Criteria <southindianfood rdf:id="southindianmadrasfood"> <hasdish> <dish rdf:id="idli"> Query Generation Server Information Retrieval Ontology Android APIs, Java.net.* Inference Engine Rule Base Client Display Result Java, SPARQL Stop Figure 1. System Flow A key component in the flow is the ontology. Data and knowledge are extracted and modeled as an integrated ontology using authoring tools. The ontology so generated forms the basis for the decision support module of the search application. The decision support module can be queried at the semantic level by the application in the cloud using semantic query languages. III. IMPLEMENTATION OF THE PROTOTYPE The prototype is implemented based on the above model. Figure 2 shows the architecture of the prototype and the tools used at each stage of the application. The tools are described below: Protégé 3.4 is used for authoring the ontology, checking its consistency, and running sample queries to test sanity. Pellet 1.5 and Jena s built-in inference engine is used for reasoning. Java, specifically java.net.* APIs are used for rest of the computation in the Cloud. Android APIs are used for mobile access to the application in the cloud. A. Ontology Authoring Once the implementation technologies are decided, the next step is to author the ontology. At a conceptual level, the ontology, in this case, for restaurants and cuisines, comprises of subject-predicate-object statements (triples) describing the domain knowledge.examples: Figure 2. A Simple Architecture Diagram Jena, Pellet Manual Process / Protege The authoring tool - protégé, generates the above RDF and OWL syntactical entities in this case. Reasoners, which perform the essential intelligent task of inferencing, work only when the ontology confirms to OWL-DL. A number of "OWL Full: OWL DL does not support subclasses of RDF(S) classes:" errors had to be fixed in the process of converting the ontology to OWL DL. The online validator [11] helped determine the remaining errors and once those were addressed, the ontology confirmed to OWL-DL, ready to be inferenced. The RDF Query language, SPARQL (a recursive acronym that stands for SPARQL Protocol and RDF Query Language) is used to query the ontology. To sanity test the ontology in addition to the protégé tests, a few SPARQL queries one of which is shown in Table I were run using protégé and Jena as well. temasektastes servescuisine southindianfood SouthIndianFood hasdish idli SingaporeanFood hasdish fishheadcurry These statements are modeled in Web Ontology Language (OWL) serialized using Resource Description Framework (RDF) syntax as: <restaurant rdf:id="temasektastes"> <servescuisine> 383

4 Search for restaurants which serve Hyderabad Cuisine and include Dum Biryani on the menu PREFIX ont4:< SELECT?restaurant?cuisine FROM <ont4> WHERE {?restaurant ont4:servescuisine?cuisine.?cuisine ont4:hasdish ont4:dumbiryani } Results: restaurant cuisine ====================================== ont4:charminarchathouse ont4:hyderabadmoghlaifood Table I. Sample SPARQL Query and Result B. Implementation of the Server The server is implemented in java, using Jena and Pellet. ModelFactory.createOntologyModel, QueryFactory, PelletQueryExecution are the key classes in the code. The ontology, which is stored as a file is read as an inputstream by the read method in the model class. The server then opens a TCP port and listens on the port for connections from the client. Once the client connects, the cuisine of choice is received from the client and a SPARQL query is constructed from it. The query is executed using the getresultvars method of the query class and results returned to the client in a tabular format. The relevant code snippets with explanations given as comments are shown below: // ontology that will be used String ont = "C:\\Documents and Settings\\vishnu\\My Documents\\intSrch\\rest_ontology.owl"; // create an empty ontology model using Pellet spec OntModel model = ModelFactory.createOntologyModel( PelletReasonerFactory.THE_SPEC ); // read the file into the ontology model InputStream in = FileManager.get().open(ont); model.read( in, null ); // open socket and wait for client while (true) { System.out.println("Waiting for client..."); try { client = server.accept(); } // get inputs from client cin = client.getinputstream(); // construct SPARQL query, querystr from inputs and execute Query query = QueryFactory.create(queryStr); QueryExecution qexec = new PelletQueryExecution(query, model); C. Implementation of the Client Android provides a very interesting computing paradigm using XML and Java. The phone GUI can be programmed by either changing the XML or by manipulating views in the code, using java classes. However, it is highly preferable that the client be thin. Any external jar or class files needed for the application are to be converted into the Android specific dex format and the process is not devoid of issues. For this project, both XML files and java source was created and modified to program the Android GUI [8]. Here s a code snippet: public void oncreate(bundle savedinstancestate){ super.oncreate(savedinstancestate); setcontentview(r.layout.main); Spinner s = (Spinner) findviewbyid(r.id.spinner); ArrayAdapter adapter = ArrayAdapter.createFromResource( this, R.array.cuisine, android.r.layout.simple_spinner_item); adapter.setdropdownviewresource(android.r.layo ut.simple_spinner_dropdown_item); s.setadapter(adapter); //setcontentview(r.layout.main); s.setonitemselectedlistener (new OnItemSelectedListener() { int n = 0; public void onitemselected(adapterview<?> parent, View v, int position, long id) { if (n == 0) { n++; } else { performquery(parent.getselecteditem().tostring ()); }} public void onnothingselected(adapterview parent) { } }); } The method performquery above, connects with the server, passes input parameters, gets the results from the server and displays them. Here is the manifest file [8], AndroidManifest.xml: <?xml version="1.0" encoding="utf-8"?> <manifest xmlns:android=" res/android" package="org.coen296" android:versioncode="1" android:versionname="1.0"> <uses-permission android:name="android.permission.internet" /> <application android:icon="@drawable/icon" android:label="@string/app_name"> 384

5 <activity android:name=".restaurantsearch" <intent-filter> <action android:name="android.intent.action.main" /> <category android:name="android.intent.category.launcher " /> </intent-filter> </activity> </application> <uses-sdk android:minsdkversion="3" /> </manifest> IV. RESULTS A. Sample Execution The sequence of steps to run the query and see the results is given in the following self-explanatory screen-shots. alone, as it does not have a built-in reasoner. But when used with Jena and Pellet, they worked as expected implying the semantic underpinnings of the application. This is because the reasoner in Pellet did the inferencing and matching The query used for testing the semantic aspects is given in Table II. The expected results for the query should include restaurants which serve idli as a dish, because that is an inferred dish that an ordinary query processor cannot directly figure out from the given ontology. PREFIX ont4:< #> SELECT?restaurant?dish FROM <ont4> WHERE {?restaurant ont4:servescuisine?cuisine.?cuisine ont4:hasdish?dish.?dish ont4:matches ont4:southindianmadrasfood } Table II. Query Requiring Semantic Detail Results from protégé (Does not return the two inferred restaurants which serve idli, that jena with pellet does, as shown below): Figure 3. (a) RestSearch Application is chosen (b) Cuisine and other options are selected Results from Jena without using pellet (Does not return the 2 inferred restaurants which serve idli, that jena with pellet does, as shown below): restaurant dish ================================= ont4:angmokiostall ont4:chickenrice ont4:temasektastes ont4:chickenrice (c) Results displayed after query is executed in the cloud B. Semantic aspects included in the system: From the problem description given at the beginning of this report, it is quite clear that an ordinary search application will not be able to return the results that the user is looking forward to. The queries used in the application did not yield the expected results on protégé using pellet, possibly because of a bug in the tool or a usage problem, which caused semantic reasoning to not work. Nor did they run as expected with Jena Results from Jena using pellet (Also returns the 2 inferred restaurants which serve idli, proof that the application is indeed semantic): 385

6 V. CONCLUSION AND FUTURE DIRECTIONS The research work presented in this paper successfully demonstrated the potential of combining mobile access with cloud computing and semantic technologies. The project also demonstrated building intelligent applications using semantic tools rather than the conventional AI languages such as Prolog. Though the prototype implemented a thin vertical slice of the envisaged intelligent architecture, the prototype served as a good proof of concept for future research. A Few Future Directions under Consideration: Enhance the ontology by adding more classes, properties, and instances and restrictions on them. Include location and GPS information into the problem statement, increasing the scope and usefulness of the project. Adding many more semantic rules to the project and actually implementing a rule base. Deploying the server part of the project in a true cloud computing environment such as Amazon EC2. Adding a middleware component, possibly using webservices. Addressing the issue of response times for mobile access. [12] floats an idea in this direction that we are considering to further evaluate. Evolving formalism using probabilistic models for the application and incorporating a learning component. Library. IEEE Computer Society, [13] Kumar, K., & Lu, Y. Cloud Computing for Mobile Users: Can Offloading Computation Save Energy? IEEE Computer, 43(4), Eventually, we hope that the work will open-up a plethora of applications in various domains using the model presented here. REFERENCES [1] Vishnu S. Pendyala, Simon S.Y.Shim, Web as the Ubiquitous Computer, IEEE Computer, vol. 42, no. 9, pp , Sept. 2009, doi: /mc [2] Bin Hu, Philip Moore, Hsiai-Hwa Chen, A Semantic Context Model for Location-Based Cooperative Mobile Computing. ICC 2007: [3] Hochul Jeon; Taehwan Kim; Joongmin Choi, Mobile Semantic Search Using Personal Preference Filtering, Fourth International Conference on Networked Computing and Advanced Information Management, NCM '08. Volume 2, 2-4 Sept Page(s): [4] SmartWeb Project [Online] [5] Daniel Sonntag, Ralf Engel, Gerd Herzog, Alexander Pfalzgraf, Norbert Pfleger, Massimo Romanelli, Norbert Reithinger (DFKI), SmartWeb Handheld Multimodal Interaction with Ontological Knowledge Bases and Semantic Web Services, Proceedings of International Workshop on AI for Human Computing (AI4HC) in conjunction with (IJCAI) 2007, Hyderabad (Indien). [6] Wolfgang Wahlster (DFKI), Mobile Interfaces to Intelligent Information Services: Two Converging Megatrends, Proceedings of Mobile the mega medium?" a MINDS symposium, Berlin [7] Wiki Portal [Online], [8] Google [Online], [9] Vipul Kashyap, et al, The Semantic Web - Semantics for Data and Services on the Web. Springer, st Edition [10] Vishnu S. Pendyala, Simon S.Y. Shim, Jerry Z. Gao, An XML Based Framework for Enterprise Application Integration, IEEE International Conference on E-Commerce Technology (CEC'03), 2003, pp.128. [11] Ontology Validator [Online] [12] Mahadev Satyanarayanan, Victor Bahl, Ramon Caceres, Nigel Davies, "The Case for VM-based Cloudlets in Mobile Computing," IEEE Pervasive Computing, 14 Aug IEEE computer Society Digital 386