Resource Allocation in a Middleware for Streaming Data

Transcription

1 5 Middleware 2004 Companion Resource Allocation in a Middleware for Streaming Liang Chen Department of Computer Science and Engineering Ohio State University, Columbus OH chenlia@cse.ohio state.edu Gagan Agrawal Department of Computer Science and Engineering Ohio State University, Columbus OH agrawal@cse.ohio state.edu ABSTRACT Increasingly, a number of applications rely on, or can potentially benefit from, analysis and monitoring of data streams. To support processing of streaming data in a grid environment, we have been developing a middleware system called (Grid-based AdapTive Execution on Streams). Our target applications are those involving high volume data streams and requiring distributed processing of data arising from a distributed set of sources. This paper addresses the problem of resource allocation in the system. Though resource discovery and resource allocation have been active topics in grid community, the pipelined processing and real-time constraint required by distributed streaming applications pose new challenges. We present a resource allocation algorithm that is based on minimal spanning trees. We evaluate the algorithm experimentally and demonstrate that it results in configurations that are very close to optimal, and significantly better than most other possible configurations. 1. INTRODUCTION Increasingly, a number of applications across computer sciences and other science and engineering disciplines rely on, or can potentially benefit from, analysis and monitoring of data streams. Inthe stream model of processing, data arrives continuously and needs to be processed in real-time, i.e., the processing rate must match the arrival rate. There are two trends contributing to the emergence of this model. First, scientific simulations and increasing numbers of high precision data collection instruments (e.g. sensors attached to satellites and medical imaging modalities) are generating data continuously, and at a high rate. The second is the rapid improvements in the technologies for Wide Area Networking (WAN), as evidenced, for example, by the National Lambda Rail (NLR) proposal and the interconnectivity between the TeraGrid and Extensible Terascale Facility (ETF) sites. As a result, often the data can be transmitted faster than it can be stored or accessed from disks within a cluster. The important characteristics that apply across a number of streambased applications are: 1) the data arrives continuously, 24 hours a day and 7 days a week, 2) the volume of data is enormous, typically Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. 2nd Workshop on Middleware for Grid Computing Toronto, Canada Copyright 2004 ACM $5.00. tens or hundreds of gigabytes a day, and the desired analysis could also require large computations, 3) often, this data arrives at a distributed set of locations, and all data cannot be communicated to a single site, 4) it is often not feasible to store all data for processing at a later time, thereby, requiring analysis in real-time. As an example, we consider the hadron collider (LHC) being setup in CERN (located at Geneva). It is expected to generate hundreds of petabytes of data per year by the year 2007, and exabytes by the year 2012 [2]. It is planned that this data will be distributed to around 10 Tier 1 centers, and then onto around 50 Tier 2 centers. As the data is continuous or streaming in nature, and because of the very high volume of data, the initial analysis or filtering of data will need to be done at real-time. The storage capacities will require that the data is filtered by a factor of to. Thus, it is important that crucial information is extracted by real-time analysis on continuous streams. We view the problem of flexible and adaptive processing of distributed data streams as a grid computing problem. We believe that a distributed and networked collection of computing resources can be used for analysis or processing of these data streams. Computing resources close to the source of a data stream can be used for initial processing of the data stream, thereby reducing the volume of data that needs to be communicated. Other computing resources can be used for more expensive and/or centralized processing of data from all sources. Because of the real-time requirements, there is a need for adapting the processing in such a distributed environment, and achieving the best accuracy of the results within the real-time constraint. In view of the above, we have been developing a grid middleware system called (Grid-based AdapTive Execution on Streams) [6]. The three important aspects of this system are as follows. First, it is designed to use the existing grid standards and tools to the extent possible. Specifically, our system is built on the Open s Architecture (OGSA) model and uses the initial version of GT 3.0. Second, the system offers a high-level interface that allows the users to specify the algorithm(s) and the steps involved in processing data streams. The users need not be concerned with the details like discovering and allocating grid resources, registering their own data stream s web services and deploying the web services. Third significant aspect of our system is that it flexibly achieves the best accuracy that is possible while maintaining the real-time constraint on the analysis. In this paper, we consider the problem of resource allocation for an application using the system. Though resource discovery and resource allocation have been active topics in grid community, the pipelined processing and real-time constraint required by distributed streaming applications pose new challenges. We present a resource allocation algorithm that is based on minimal spanning

2 Middleware for Grid Computing 6 Application Developer Web Server Configuration info. the place where the configuration file is Fetch the configuration info. Launcher Application User parameters are, rate of sampling, i.e., what fraction of dataitems are actually processed, and size of summary structure at an intermediate stage, which means how much information is retained after a processing stage. The middleware automatically adjusts the values of these parameters to meet the real-time constraint on processing. Application Repository Codes for stages Codes for stages Codes for stages Retrieve the codes Deployer Grid Resource manager 4. Enable easy deployment of the application. This is done by supporting a Launcher and a Deployer. The system is responsible for initiating the different stages of the computation at different resources. Code for stage 1 Streams Code for stage 2 Streams Figure 1: Overall System Architecture Code for stage N trees. We evaluate the algorithm experimentally and demonstrate that it results in configurations that are very close to optimal, and significantly better than most other possible configurations. The rest of this paper is organized as follows. In Section 2, we give an overview of the system. The resource allocation problem and our algorithm are described in Section 3. We evaluate our algorithm in Section 4. We compare our work with related efforts in Section 5 and conclude in Section OVERVIEW OF THE SYSTEM This section describes the major design aspects of our system. 2.1 Key Goals There are four main goals behind the design of the system. 1. Use the existing grid infrastructure to the extent possible. Particularly, our system builds on top of the Open s Architecture (OGSA) [12], and uses its reference implementation, Globus 3.0. The Globus support allows the system to do automatic resource discovery and matching between the resources and the requirements. 2. Support distributed processing of one or more data streams, by facilitating applications that comprise a set of stages. For analyzing more than one data stream, at least two stages are required. Each stage accepts data from one or more input streams and outputs zero or more streams. The first stage is applied near sources of individual streams, and the second stage is used for computing the final results. However, based upon the number and types of streams and the available resources, more than two steps could also be required. All intermediate stages take one or more intermediate streams as input and produce one or more output streams. s APIs are designed to facilitate specification of such stages. 3. Enable the application to achieve the best accuracy, while maintaining the real-time constraint. For this, the middleware allows the application developers to expose one or more adjustment parameters at each stage. An adjustment parameter is a tunable parameter whose value can be modified to increase the processing rate, and in most cases, reduce the accuracy of the processing. Examples of such adjustment is also designed to execute applications on heterogeneous resources. The only requirements for executing an application are: 1) support for a Java Virtual Machine (JVM), as the applications are written in Java, 2) availability of Globus 3.0, and 3) a web server that supports the user application repository. Thus, the applications are independent of processors and operating systems on which they are executed. 2.2 System Architecture and Design The overall system architecture is shown in Figure 1. The system distinguishes between an application developer and an application user. An application developer is responsible for dividing an application into stages, choosing adjustment parameters, and implementing the processing at each stage. Moreover, the developer writes an XML file, specifying the configuration information of an application. Such information includes the number of stages and where the stages codes are. After submitting the codes to application repositories, the application developer informs an application user of the URL link to the configuration file. An application user is only responsible for starting and stopping an application. The above design simplifies the task of application developers and users, as they are not responsible for initiating the different stages on different resources. To support the easy deployment and execution, the Launcher and the Deployer are used. The Launcher is in charge of getting configuration files and analyzing them by using an embedded XML parser. To start the application, the user simply passes the XML file s URL link to the Launcher. The Deployer is responsible for the deployment. Specifically, it 1) receives the configuration information from the Launcher, 2) consults with a grid resource manager to find the nodes where the resources required by the individual stages are available, 3) initiates instances of grid services at the nodes, 4) retrieves the stage codes from the application repositories, and 5) uploads the stage specific codes to every instance, thereby customizing it. After the Deployer completes the deployment, the instances of the grid service start to make network connections with each other and execute the stage functionalities. The grid service is an OGSA Grid service [11] that implements the selfadaptation algorithm and is able to contain and execute user-specified codes. 3. RESOURCE ALLOCATION ALGORITHM In this section, we describe our algorithm for resource allocation. Initially, we describe the problem and argue about its complexity. 3.1 Problem Definition and Complexity The resource allocation problem for a application is essentially that of creating a deployment configuration. A deployment configuration of an application comprises the following components: 1. The number of data sources and their location.

3 7 Middleware 2004 Companion Source 1 Source 2 Destination Stage 4 Stage 3 Source 3 Source 4 Source 1 Stage 4 Stage 3 Source 2 Destination Stage 4 Stage 3 Source 3 Figure 2: Two Example Deployment Configurations Instance 3 Source 4 2. The destination, i.e., the node where the final results are needed. 3. The number of stages in the application. 4. The number of instances of each stage. 5. How the instances connect to each other. 6. The node at which each instance is assigned. Figure 2 shows two possible deployment configurations for an application that has 4 data sources and 5 stages. Now, let us consider the problem of creating a deployment configuration for an execution of an application. The components 1, 2, and 3 of the deployment configuration are known when an application is initiated. Therefore, the problem is that of determining components 4, 5, and 6. Determining components 4, 5, and 6 involves both resource discovery and resource allocation. Our focus is on the problem of resource allocation. For resource discovery, we can make use of information services [9] provided by Globus toolkit 3.0 (GT3.0) to collect and aggregate resource information. One possible approach for resource allocation is to enumerate and compare all possible configurations and find the one that will enable the best performance. However, such an exhaustive search algorithm has at least an exponential complexity. Given an application with stages, data sources and available computing nodes for placement of stages instances, the number of possible configurations can be denoted by,where, # $ & ( *,.,. * 1 where, *,and >.,. > denotes the Stirling numbers of the second kind. If there are only 3 stages, the number of all possible configurations is: # $ 7 & ( * # $ 5 & ( * 1 3 $ & ( The above derivation shows that a lower bound on the complex-. Therefore, it is ity of the exhaustive search algorithm is E not practical. Below, we will describe our algorithm that can find a deployment configuration in G I time. 3.2 Algorithm Description The goal of our algorithm is to determine the deployment configuration that gives the application the best chance to achieve the real-time constraint, which still maintaining high accuracy in the processing. In this paper, we only consider static resource allocation, and dynamic resource allocation is a topic for future research. We also assume that except for the final stage in the pipeline, where data from all sources must be combined together, stages in the application could be processed independently for each data source. The main observation in our algorithm design is that data arrival rates at the first one or two stages are typically so high that high network bandwidths are desired at these stages. After data have been processed by these stages, the arrival rates at the following stages typically decrease significantly. Our algorithm is also based on the assumption that computation is typically not the bottleneck in the processing. The algorithm has two main steps. First, we create a key path corresponding to each data source. Second, we merge these key paths to create a layout tree. The algorithm proceeds as follows. We initially construct a weighted graph in which every node is viewed as a vertex, and a network connection between two nodes is viewed as an edge. By node, we mean any computing unit, which could be a cluster or an SMP machine, and is capable of executing multiple processes. The main idea is that communication bandwidth for processes within a node is much higher than the bandwidth between the nodes. The weight of an edge is the negative value of the network connection s bandwidth. Given this formulation, our goal is to place stages so as to minimize the network communication time in the processing. Therefore, for every data source, we construct a Minimum Spanning Tree (MST) by making the data source the initial set and applying the Prim s algorithm to the graph. Note that we prefer to apply the MST algorithm, rather than the shortest path algorithm, because it aggressively minimizes the weight at the top level of the tree. Next, we seek the key path for each data source. This is the path from the data source node to the destination node in the MST corresponding to each data source. We start to mark nodes in the path, i.e., the parent node of the data source node is marked as the second stage, the grandparent node of the data source node is marked as the third stage, etc. Ideally, we will like to have the last stage placed on the destination node. However, this will only happen if the number of tree nodes in a key path is equal to the number of stages being deployed. This may not be true in practice, and the number of stages could be both less than or greater than the number of nodes in the path. We make the following adjustments in such cases. If the length of a key path is longer than the number of stages, we insert some transport stages, called transporters, which simply forward the data they receive. Adding transporters does introduce some overheads. However, we believe that these overheads are nominal compared with the delays caused by choosing another lower bandwidth path, simply because it has exactly number of nodes. Moreover, these transporters are always deployed at the end of a pipeline, where the arrival rates are not high. Towards the end of this section, we will describe an optimization to reduce the number of transporter if possible. When the length of a key path is shorter, the additional stages are deployed at the parent node of the data source node. The

4 I I Middleware for Grid Computing 8 reason again is that higher data arrival rates are typically seen at the beginning of a pipeline. Now, we have a key path corresponding to each data source, and we need to create a layout tree. By default, we will have one instance of each stage for each data source, with the exception of the last stage. As we stated earlier, our goal is to minimize the communication time and computation is typically not a bottleneck. Therefore, we proceed as follows. Consider two different key paths which involve the same node. In general, the node could be executing different stages for different key paths. However, if it executes the same stage, we merge the paths. By repeating this step, we can get a layout tree from the set of key paths. A layout tree determines the components 4 and 5 of a deployment configuration. To decide the component 6, we need to map a vertex in the layout tree to a computing node. We can query the node information service by specifying the resource requirements of the stage that is supposed to be deployed in this node. Thus, we can get a deployment configuration and then call the launcher program to automatically launch the application. We now state the complexity of this algorithm. Recall that the number of data sources is, the number of stages is,andthe number of available nodes is J. The main cost is in the invocation of the Prim s algorithm. Each invocation takes G I time and we need invocations. Thus, the complexity of the algorithm is G I. 3.3 Additional Optimization Although the overheads introduced by transporters are not critical, we will still like to find unnecessary transporters and eliminate them. Assume that a key path is denoted by O P O P Q Q Q O P Q Q Q O S, where the data source is denoted by O, the destination is denoted by O S, O is the ith node in the path, and P is the edge connecting O with O U. If the node O is marked as a transporter, O X has an edge Y in the graph which connects to O Z \ J., and the weight of Y is smaller than that of the edges from P to P X. Then the nodes from O to O Z X are unnecessary transporters. We can eliminate them and replace the edges from P to P Z X with the edge Y. As we will show in our experiments, an optimized configuration does achieve better performance than the non-optimized configuration. 4. EXPERIMENTAL EVALUATION This section presents results from a number of experiments we conducted to evaluate our algorithm. Specifically, we had the following goals: 1) Show that the deployment configuration created and optimized by our algorithm can be as good as the best one among a large number of choices manually enumerated, and 2) Demonstrate that the algorithm-created configuration can outperform most of a large number of possible configuration. One of the challenges in conducting our experiments was to have a setup where network bandwidths can vary significantly and topology can be quite complex, and yet, repeatable and reliable experiments could be conducted. In a LAN environment or within a single organization, the bandwidths are unlikely to vary much, and the topology is usually very simple. In such scenarios, the resource allocation problem usually becomes quite trivial. At the same time, a WAN environment does not allow repeatable experiments. Therefore, to conduct our experiments, we set up an environment in which network bandwidths are simulated by inserting delays between packages and network topologies are created randomly. This allowed us to focus on our goal, which was to demonstrate that our algorithm is effective when network bandwidths can vary significantly and topology is quite complex. Execution Time (ms) 900, , , , , , , , , , , , , , integers integers Size of the data set auto-config opt-config man-config 805, Figure 3: Comparing auto-config, manual-config, and opt-config The experiments we report were conducted using the counting samples application. The classical counting samples problem is as follows [13]. A data stream comprises a set of integers. We are interested in determining the most frequently occurring values and their number of occurrences at any given point in the stream. Since it is not possible to store all values, a summary structure must be maintained to determine the frequently occurring values. Gibbons and Matias have developed an approximate method for answering such queries with limited memory. We implement a distributed version of the counting samples application. In this version, full streams are not forwarded to a central machine, but are instead sent to some machines close to their respective data source. A fixed number of most frequently occurring items at each stream are determined there and forwarded to the central machine, where the final results were computed. This number was 100 in all our experiments. A deployment configuration of this application could be determined manually by comparing a large number of possible configurations. We call this configuration manual-config. The configuration automatically generated by the algorithm is called auto-config. The configuration where we further apply the optimization of removing unnecessary transporters is called opt-config. Based on these three configurations, we conducted three sets of experiments, which are detailed below. Experiment 1: Our first experiment demonstrated auto-config and opt-config are almost as good as a manual-config, which is based on enumerating a significant fraction of all possible choices. We had 4 data sources with fixed locations. We further made 6 clusters available to run the intermediate stage of the application. The results are shown in Figure 3. We considered two different cases, corresponding to 700,000 and 2,000,000 integers being produced by each data source. When each data source created integers, the application using the manual-config is 5.3% faster than that using the auto-config and 3.6% faster than that using opt-config. In the second case, we see that the manual-config is just 1.4% faster than auto-config and 0.8% faster than opt-config. Thus, the results indicate that the performances in the above three scenarios are very close, i.e., our algorithm is effective. Furthermore, the larger the size of the datasets, the differences are smaller.

5 9 Middleware 2004 Companion Execution Time (ms) 1,000, , , , , , , , , ,000 0 opt-config Experiments using the various configurations configurations randomly chosen Figure 4: Comparing auto-config with 120 Other Configurations Execution Time (ms) 1,000, , , , , , , , , ,000 0 Experiments using the various configurations opt-config All man-configs Figure 5: Comparing auto-config with All Possible Configurations allowed exhaustive comparisons. We compared the algorithm generated and optimized configuration, opt-config, against all 64 configurations. The results are shown in Figure 5. opt-config outperforms all possible configurations, and in most cases, by at least a factor of 2. This confirms our observations from the previous 2 experiments. 5. RELATED WORK Resource allocation has been an important topic in the grid community. Most of the initial work has been on static matching of the resource requirements and the available resources [10, 5, 24, 15, 21, 25, 22]. However, none of the these efforts considered pipelined or streaming applications. It should be noted that our approach does not require resource requirements to be explicitly stated by an application, in contrast to Condor s matchmaking [20] or the Aurora system [3]. Much work has been done on resource discovery[16, 17], often using mobile agents or objects to do efficient search. Our focus is on resource allocation, and we assume that one of the existing techniques has been used for resource discovery. Realtor [8] is a protocol for supporting survivability and information assurance by migrating components to safe locations under circumstances of external attack, malfunction, or lack of resources. Our work is distinct in considering resource degradation and application adaptation. In the area of stream processing, the work that is probably the closest to our work is the dquob project [18, 19]. This system enables continuous processing of SQL queries on data streams. Our work is distinct in the following ways. First, we support an API to allow general processing, and not just SQL queries. Second, the processing can be done in a pipeline of stages. Third, our system is built on top of Globus 3.0, and thus, exploits the existing support for resource discovery. stream processing has also received much attention in the database community [14]. Prominent work in this area has been done at Stanford [1], Berkeley [4], Brown and MIT [3], Wisconsin [23], among others. The focus in this community has largely been on centralized processing of a single data stream. Our focus is quite different, as we consider distributed processing of distributed data streams, and use grid resources and standards. Aurora* is a framework for distributed processing of data streams, but only within a single administrative domain [7]. Also, the focus in this work is on scalable communication, and there is no support for adaptation of the processing. Experiment 2: The second experiment was conducted in the same environment as the first one. We randomly selected 120 out of 1296 configurations that were possible and compared them with the opt-config. We did not carry out the extensive evaluation, as the total number of choices was very large. As shown in Figure 4, opt-config outperforms all but one of the randomly selected configurations. The one configuration that performs better than the optimized one is just 2% faster. Almost all of the other configurations resulted in a slow-down by at least a factor of 2, and in many cases, up to a factor of 3. This again shows that our algorithm is effective. Experiment 3: In the environment we had considered for the first two experiments, the total number of possible deployment configurations was very large. This did not allow us to perform exhaustive comparisons. Therefore, we used a different environment, in which the number of data sources was 3 and the number of available clusters was 4. The network topology was randomly created. The number of possible deployment configurations was now 64, which 6. CONCLUSIONS AND FUTURE WORK Resource allocation has been an important problem in grid computing and has been widely studied for many application classes. With scientific instruments and experiments that continuously generate data, and increasing network speeds, we expect processing of data streams to be an important application class for grid computing. This paper has investigated resource allocation for applications that involve processing of distributed data streams. We have presented a heuristic algorithm for this problem, which is based on minimum spanning trees. Our current evaluation has shown that the algorithm is effective in practice. Several issues remain to be considered. First, our current algorithm assumes that networking bandwidths between the initial stages are the major bottleneck. In general, the algorithm should be able to take relative computational and communication costs between different stages as the input, and should then determine the bottleneck. Similarly, we currently assume that all but the final stages can process each data source independently. Again, we can generalize this and consider a tree or a DAG representation of the stages. Another direction will be to consider dynamic resource al-

6 Middleware for Grid Computing 10 location, as available network bandwidths or computational cycles can vary over time. Finally, evaluation of algorithms in realistic but controlled environment remains a challenge. 7. REFERENCES [1] A. Arasu, S. Babu, and J. Widom. An abstract semantics and concrete language for continuous queries over streams and relations. In Proc. of the 9th International Conference on Base Programming Languages (DBPL 03), Sep [2] J. Bunn and H. Newman. -Intensive Grids for High-Energy Physics. In F. Berman, G. Fox, and T. Hey, editors, Grid Computing: Making the Global Infrastructure a Reality. John Wiley and Sons, [3] D. Carney, U. Etintemel, M. Cherniak, C. Corvey, S. Lee, G. Seidman, M. Stonebraker, N. Tatbul, and S. Zodnik. Monitoring Streams - A New Class of Management Applications. In Proceedings of Conference on Very Large Bases (VLDB), pages , [4] S. Chandrasekaran, O. Cooper, A. Deshpande, M. J. Franklin, J. M. Hellerstein, W. Hong, S. Krishnamurthy, S. R. Madden, V. Raman, F. Reiss, and Mehul A. Shah. Telegraphcq: Continuous dataflow processing for an uncertain world. In CIDR, pages , [5] Steve Chapin, Dimitrios Katramatos, John Karpovish, and Andrew Grimshaw. Resource management in Legion. Future Generation Computer System, 15(5-6): , [6] Liang Chen, Kolagatla Reddy, and Gagan Agrawal. : A Grid-Based Middleware for Distributed Processing of Streams. In Proceedings of IEEE Conference on High Performance Distributed Computing (HPDC). IEEE Computer Society Press, [7] M. Cherniack, H. Balakrishnan, M. Balazinska, D. Carney, U. Cetintemel, Y. Zing, and S. Zdonik. Scalable Distributed Stream Processing. In Proceedings of the Conference on Innovative Systems Research (CIDR), January [8] Byung Kyu Choi, Sangig Rho, and Riccardo Bettati. Dynamic resource discovery for applications survivability in distributed real-time systems. In IPDPS, page 122, [9] K. Czajkowski, S. Fitzgerald, I. Foster, and C. Kesselman. Grid information services for distributed resource sharing. In Proceedings of the Tenth IEEE International Symposium on High-Performance Distributed Computing (HPDC-10), August [10] I. Foster, C. Kesselman, and S. Tuecke. The anatomy of the grid: Enabling scalable virtual organizations. International Journal of Supercomputer Applications, 15(3), [11] Ian Foster, Carl Kesselman, J. Nick, and Steven Tuecke. Grid Services for Distributed Systems Integration. IEEE Computer, [12] Ian Foster, Carl Kesselman, Jeffrey M. Nick, and Steven Tuecke. The Physiology of the Grid: An Open s Architecture for Distributed Systems Integration. In Open Infrastructure Working Group, Global Grid Forum, June [13] Phillip B. Gibbons and Yossi Matias. New Sampling-Based Summary Statistics for Improving Approximate Query Answers. In Proc. of the 1998 ACM SIGMOD, pages ACM Press, June [14] L. Golab and M. Ozsu. Issues in data stream management. In SIGMOD Record, Vol. 32, No. 2, pages 5 14, June [15] A. Iamnitchi, I. Foster, and D. Nurmi. A peer-to-peer approach to resource discovery in grid environments. In High Performance Distributed Computing, Edinbourgh, UK, July IEEE. [16] K. Jun, L. Boloni, K. Palacz, and D.C. Marinescu. Agent-based resource discovery. In 9th Heterogeneous Computing Workshop, pages 43 52, May [17] L. Moreau. Agents for the grid: A comparison for web services (part 1: the transport layer). In Proceedings of Second IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID 2002), pages , Berlin, Germany, [18] Beth Plale. Leveraging Runtime Knowledge about Event Rates to Improve Memory Utilization in Wide Area Stream Filtering. In IEEE High Performance Distributed Computing (HPDC), August [19] Beth Plale and Karsten Schwan. Dynamic Querying of Streaming with the dquob System. IEEE Transactions on Parallel and Distributed Systems, 14(3), April [20] Rajesh Raman, Miron Livny, and Marvin Solomon. Matchmaking: Distributed resource management for high throughput computing. In Proceedings of the Seventh IEEE International Symposium on High Performance Distributed Computing, Chicago, IL, July [21] O. F. Rana, D. Bunford-Jones, D. W. Walker, M. Addis, M. Surridge, and K. Hawick. Resource discovery for dynamic clusters in computational grid. In Proceedings of 10th IEEE Heterogeneous Computing Workshop, San Francisco, CA, USA, [22] Douglas Thain, Todd Tannenbaum, and Miron Livny. Distributed computing in practice: The condor experience. Concurrency and Computation: Practice and Experience, [23] S. Viglas and J. Naughton. Rate-based query optimization for streaming information sources. In Proc. of the 2002 ACM SIGMOD Intl. Conf. on Management of, June [24] D. Xu, K. Nahrstedt, and D. Wichadakul. Qos-aware discovery of wide-area distributed services. In Proceedings of 1st IEEE/ACM International Symposium on Cluster Computing and the Grid, pages 92 99, May [25] L. Zhang, S. Deering, D. Estrin, S. Shenker, and D. Zappala. Rsvp: a new resource reservation protocol. IEEE Networks Magazine, 31(9):8 18, September 1993.