Yet Another Parallel Hypothesis Search for Inverse Entailment Hiroyuki Nishiyama and Hayato Ohwada Faculty of Sci. and Tech. Tokyo University of Scien

Transcription

1 Yet Another Parallel Hypothesis Search for Inverse Entailment Hiroyuki Nishiyama an Hayato Ohwaa Faculty of Sci. an Tech. Tokyo University of Science, 2641 Yamazaki, Noa-shi, CHIBA, , Japan Abstract. In this stuy, we esign an implement a powerful Inuctive Logic Programming (ILP) system to conuct a parallel hypothesis search for inverse entailment. One of the most important parts of ILP is speeing up the hypothesis search, an a number of parallel hypothesis exploration methos have been propose. Recently, the Map-Reuce algorithm has been use for large-scale istribute computing, but it is ifficult to apply such clou technology to the ILP hypothesis search problem. By contrast, we esigne a metho that can ynamically istribute tasks for the hypothesis search, an implemente a network system in which moules autonomously cooperate with each other. We also conucte a parallel experiment on a large number of CPUs. Results confirm that the hypothesis search time is shortene accoring to the number of computers use, without reucing the optimality of the generate hypothesis. 1 INTRODUCTION Recently, various analytical approaches to a large amount of ata have been evelope. For example, the Map-Reuce moel consists of a large number of workers an one master, an uses on-line istribute-computing for quick analysis. However, Map-Reuce has limite efficiency for Inuctive Logic Programming (ILP) [2]. For example, searching for relationships in information is not efficient istribute processing in the Map-Reuce scheme, because new relationships are foun after searching for relationships in backgroun knowlege to generate rules. Therefore, the Map-Reuce application of ILP for large ata is not realistic because ILP requires much learning time. To solve this problem, various stuies have been conucte in an effort to spee up ILP [1, 3, 4]. Although the problems were partially solve, the process was not sufficiently spe up base on the number of processors provie, an the quality of the generate rules was not optimal, resulting in ifficulty in its use as a practical tool. Consiering these issues, we esigne an implemente a new parallel processing system for ILP. With this system, the worker itself has an autonomous function, unlike Map-Reuce. When a worker has no task (e.g., immeiately after start-up or immeiately after completion of a task), the worker accepts a 86

2 ^ E Fig. 1. System configuration of the ILP parallel computation system ivie task from another worker an starts the task. When the workloa (the number that the worker must search for relationships in information) reaches a fixe quantity, the worker requests other workers to process the ivie task. After the request for the first task issue by a master is implemente for one worker, autonomous process istribution is starte among workers, an all existing workers are engage (saturation of the task). Because the ivie task continues to be repeate among workers, all workers finally complete all processing at approximately the same time, an the master receives the processing result (generate rules). Our parallel processing system has the following features. 1: The master oes not nee to consier the ivision of the process in avance, as the master of the Map-Reuce scheme oes. 2: All workers work until the en (no free time). The first feature inicates that the propose system oes not require preivision processing of the master, which is use in conventional Map-Reuce. The secon feature means that speeup can be achieve. In our stuy, we implement the propose parallel ILP system an emonstrate its effectiveness by conucting parallel-learning experiments using rug iscovery ata. 2 SYSTEM DESIGN 2.1 System Configuration Figure 1 shows our ILP parallel-processing system consists of four moules. We esigne each moule as follows. Server moule for negotiation between moules The server moule meiates negotiation messages (e.g., request, accept, an commit) regaring the task share by workers. This moule broacasts to connecte workers an the master. This function is use uring negotiation to fin an acceptor of the ivie task. 87

3 Master Moule The master moule sens a task (main task) request to the worker moule (worker) an collects results (generate rules) from the worker moule when all ivie tasks of the worker moules are finishe. By connecting with the server moule, this moule can negotiate with worker moules an collect the negotiations between worker moules to recognize the processing situation of all worker moules. Worker Moule The worker moule manages an ILP moule an negotiates with other worker moules via the server moule. When this moule has no task, it accepts the share task from other worker moules, transfers the task ata to its own ILP moule, an starts learning. In aition, this moule sens a request message for the share task to other workers when the ILP moule nees to ivie the task. ILP Moule Basically, the ILP moule has the same functions as in traitional ILP. Receiving task ata from the worker moule, the ILP moule starts the learning process. It searches for generating rules; thus, the amount of search processing that this moule must perform increases. If the amount of search processing excees a threshol, this moule ivies the search processing an generates a ivie task. It then sens a request for the ivie task to the worker moule. In aition, when all search processing (its own learning process) is finishe an the requesting task exists (waiting for an accept response), this moule cancels the request of the task via the worker moule an starts the task itself. When all search processing is finishe an the requesting task no longer exists, this moule eclares the en of the task to the worker moules. As a result, this moule starts other ivie tasks from other worker moules. 2.2 Communication Protocol Between Moules In our system, a master moule communicates with worker moules an worker moules communicate with each other via the network (see Appenix Fig. 2,3). We prepare two communication protocols. One is the Negotiations Protocol between Moules to fin a worker moule that can execute a task (main task or ivie task). The other is the Task Content Transmission Protocol to sen task contents to a worker. The etails of each protocol are as follows. Protocol 1. Negotiations Protocol between Moules This protocol communicates via the server moule. All messages are broacast to all worker moules an a master moule. This protocol is use for negotiation messages (e.g., request, accept, an commit) to fin a worker moule that can execute a task. Protocol 2. Task Content Transmission Protocol After a worker moule is etermine by protocol 1, protocol 2 is use for transmitting the task contents to the moule. Communication between mo- 88

4 ules is carrie out by peer-to-peer communication (not via the server moule). As a result, all worker an ILP moules perform a task by repeately requesting share tasks (i.e., saturation of the ivie task). In the saturation state, all worker moules sen request messages regaring the share task to other worker moules an await an accept message. Conversely, each worker moule is reay to receive a request task from all other worker moules. Thus, a worker moule that has finishe its own task sens an accept message to another worker moule an receives a new share task (see Appenix Fig. 4,5). Share tasks are requeste an receive until a learning rule is generate. As a result, worker an ILP moules operate without spare time. 3 IMPLEMENTATION AND EXPERIMENT In the present stuy, we implemente a rule generation engine for ILP [2] an each moule escribe above using the Java programming language. In aition, in this implementation experiment, we temporarily set the threshol (number of searches to be conucte) at 200 for the ivision request in the ILP moule. We use two 6-CPU computers an two 4-CPU computers in our parallel processing experiment. 3.1 Speeup Experiment We use two 6CPU computers to conuct an experiment to measure speeup an use mile-scale ata on rug iscovery analysis as training ata. One by one, we increase the number of CPUs from 1 to 12, an performe 12 experiments. In our system, a worker moule (an an ILP moule) use one CPU. The experiment results are presente in Table 1. We obtaine a speeup of 7.4 using a 12-CPU unit for this exercise. However, this problem was smallscale an i not provie enough speeup. 3.2 Large-Scale Parallel Experiment Using four computers, we performe application experiments on a large-scale example of a rug iscovery problem. In this experiment, we use 1 CPU, 12 CPUs (two 6-CPU computers), an 20 CPUs (two 6-CPU computers + two 4- CPU computers) in a total of three experiments. Table 2 shows the experiment results. The results are confirme that speeup correspone to the number of CPUs use. For this rug-iscovery problem, it conventionally took at least 15h using a parallel-processing system; however, with our propose metho, one learning process require less than 1h. Thus, experiments can be easily reconucte by ajusting the parameters an can achieve better research results. 89

5 Table 1. Parallel experiments using 12 CPUs (two 6CPU computers) The Number of CPU Processing time(sec.) Speeup Table 2. Parallel experiments using 20 CPUs (two 6-CPU computers an two 4-CPU computers) The Number of CPU Processing time(sec.) Speeup DISCUSSION For the above parallel processing experiments, we measure the communication time an processing steps between workers to investigate how the workers (worker moule an ILP moule) epen on each CPU. For this measurement, we use six workers an one 6-CPU computer. In this reconucte experiment, we implemente measuring functions on each moule. Thus, the processing time is longer than in the experiment results using 6 CPUs. Table 1 shows the results. We measure the time require for learning an for receiving the task ata of each worker. Table 3 lists the processing time of each worker. The en time for all tasks is sec. Table 3. Learning time an receiving time of each worker in the parallel-processing experiment using six workers (6 CPUs) (in sec.) Worker ID Average Learning time Receiving time Total operation time

6 Table 4. Learning time an receiving time of each worker in the parallel-processing experiment using six workers (6 CPUs) (%) Worker ID Average Learning time Receiving time Total operation time Table 3 inicates the processing time of each worker. We confirme that each worker spent an average of 94% of the time on learning an 3% on receiving from other workers (see Appenix Fig. 6). In aition, learning cannot start until receiving is complete; however, sening to other workers is performe concurrently with learning. In a previous stuy [3], much time was necessary for sening an receiving between workers. In the present stuy, we successfully shortene the time require for sening an receiving, because the implementation of the propose system use the object communication function an serialize function of Java API. 5 CONCLUSIONS In the present stuy, we esigne an implemente a parallel system of Inuctive Logic Programming (ILP) to realize a parallel hypothesis search for inverse entailment. The hypothesis generation of ILP requires search processing, an the result of the hypothesis generation influences the next hypothesis generation. Thus, ILP cannot be parallelize using the Map-Reuce metho that was evelope base on clou technology. To aress these issues, we esigne a metho that can ynamically istribute tasks for search processing, an implemente a network system in which moules autonomously cooperate with each other. In aition, we enable parallel experimentation using a large number of workers (CPUs). Results confirme that we shortene the processing time epening on the number of computers use, without reucing the quality of the rule generate. References 1. Anreas Fijelan, Wayne Luk an Stephen Muggleton: Customisable Multi- Processor Acceleration of Inuctive Logic Programming, Latest Avances in Inuctive Logic Programming, pp , Fumio Mizoguchi an Hayato Ohwaa: Constraine Relative Least General Generalization for Inucing Constraint Logic Programs, New Generation Computing 13, pp , Hayato Ohwaa, Hiroyuki Nishiyama an Fumio Mizoguchi: Concurrent Execution of Optimal Hypothesis Search for Inverse Entailment, Inuctive Logic Programming, Lecture Notes in Computer Science Vol. 1866, pp , Ashwin Srinivasan: Parallel ILP for istribute-memory architectures, Machine Learning, Vol. 74, Issue 3, pp ,

7 Appenix A: Image of communication protocol Figures 2,3,4,5 show flow of communication protocol with moules. ^ Z E Fig. 2. Flow of communication from a master moule to worker moules to fin a worker that commits to a task using the Negotiations Protocol between Moules. The master moule sens the request message of the task to all worker moules. If there is no learning task, the worker moule that receives the request message sens an accept message to the source of the request (master moule). The master moule that receives the accept messages chooses one worker moule an sens a commit message to that worker moule. D ^ ^ E Fig. 3. Flow of communication to transmit task ata to a worker moule using the Task Content Transmission Protocol. 92

8 > Z ^ E Fig. 4. Flow of communication to fin a worker that commits a task between worker moules using the Negotiations Protocol between Moules. ^ ^ E Fig. 5. Flow of communication to transmit task ata between worker moules using the Task Content Transmission Protocol. 93

9 Appenix B: Operational status of each worker t t t t t te / e e e e Fig. 6. Operational status of each worker in the parallel-processing experiments using six workers (6CPUs). The vertical axis represents time, an the horizontal axis represents the operational status of each worker. Task ID refers to the number of main tasks requeste by the master moule. In this learning experiment, the master moule sent 15 requests for the task. 94