Project Report, CS 862 Quasi-Consistency and Caching with Broadcast Disks

Project Report, CS 862 Quasi-Consistency and Caching with Broadcast Disks Rashmi Srinivasa Dec 7, 1999 Abstract Among the concurrency control techniques proposed for transactional clients in broadcast environments, BCC-TI [6] has been demonstrated to have better performance than a simple adaptation of optimistic concurrency control with forward validation. Our work adapts BCC-TI to enforce quasi-consistency [3], and introduces caching. Through simulation experiments, we demonstrate the workload characteristics for which our techniques achieve good performance. 1 Introduction The broadcast model is the most commonly used model for data dissemination in asymmetric communication environments. A few concurrency control techniques have been proposed for transactional clients in a broadcast-based data dissemination environment. A technique based on dynamic adjustment of timestamp intervals, for read-only transactions at clients, was proposed in [6]. This technique was shown to achieve serializability in a more efficient manner than optimistic concurrency control with forward validation adapted to broadcast environments. In this paper, we present a technique that guarantees quasi-consistency [3]. Our technique is an adaptation of the work described in [6]. Further, we introduce caching and demonstrate through simulations that our techniques are beneficial for certain characteristics of workloads. 2 Background and Related Work In many domains, the upstream communication capability from the clients to the server is non-existent or much lower than the downstream communication capability from the server to the clients. An example of such an asymmetric communication environment is a mobile environment. The asymmetry in communication capabilities can be caused by bandwidth limitations of the physical communication medium. Asymmetry can also be caused because there is insufficient capacity at the server or the network to handle simultaneous requests generated by a large number of clients. 2.1 Broadcast Disks Traditional client-server systems follow the pull model, where clients request data from the server. The pull model is unsuitable for asymmetric communication environments because of the model s reliance on significant upstream communication capability and because of the possibility of a large number of requests from clients overwhelming the server and/or the network. Therefore, a push model or a broadcast data dissemination model is more appropriate for asymmetric communication environments. Broadcast data dissemination has been proposed in [4], [5] and [1]. In the broadcast disk model, the server repeatedly broadcasts all the objects in the database. The clients can read the values of the objects as they are broadcast. The model can be that of a flat disk where the frequency of broadcast is the same for all data objects, or of multiple disks where the frequency of broadcast of a data object is proportional to the probability of access of that data object. In [2], the multiple broadcast disks model is extended to allow updates at the server, and the trade-offs in broadcasting periodic invalidates or propagates or both are discussed. 2.2 Transactional Clients The clients in many applications in broadcast environments execute only read-only transactions (queries), and update transactions are executed at the server. Examples of such applications include dissemination of weather information or stock prices. Concurrency control techniques that exploit the semantics of read-only transactions have been proposed in [7] and [6]. A correctness criterion that is weaker than serializability is proposed in [7], wherein different read-only transactions can observe different orders of 1

update transactions. A concurrency control technique called Broadcast Concurrency Control using Timestamp Intervals (BCC-TI) is presented in [6]. BCC-TI operates in the following way. For every update transaction, the server assigns a timestamp to the transaction when it commits. Let WS(U) be the set of data objects that an update transaction U wrote. WS(U) is called the write set of U. Let TS(U) be the timestamp assigned to U when it commits. For every data object d, the server stores the largest timestamp of a committed transaction that has written object d. Let this timestamp be WTS(d) for object d. The server maintains a data structure called the control information table. When an update transaction U commits, the server performs the following steps. 1. Set the timestamp of U to the current time TS(U). 2. Copy TS(U) into WTS(d), for all d in WS(U). 3. Record TS(U) and WS(U) into the control information table. The server repeatedly broadcasts all the objects in the database along with their WTS. Moreover, the server periodically broadcasts its control information table. When a client starts the execution of a read-only transaction (query), the client assigns a timestamp interval of (0, infinity) to the query. Whenever the query reads a data object, its timestamp interval is adjusted to reflect the serialization order induced between the query and committed update transactions. Let LB(Q) and UB(Q) denote the lower and upper bounds of the timestamp interval of a query Q. When Q reads object d, the following steps are performed. 1. Set LB(Q) = max (LB(Q), WTS(d)). 2. If LB(Q) >= UB(Q) then restart Q. A client also reads the control information table that is broadcast by the server. For every update transaction U in the control information table, the client checks the validity of an outstanding query in the following way: 1. If WS(U) intersects with the read set of Q, then UB(Q) = min (UB(Q), TS(U)). 2. If LB(Q) >= UB(Q) then restart Q. BCC-TI has been compared to optimistic concurrency control with forward validation adapted to broadcast environments, and performance improvements in terms of query response time, percentage of deadline misses and restart rate have been demonstrated. 2.3 Quasi-Consistency Forms of consistency that are weaker than serializability, and that are based on data semantics, are presented in [3], in the context of caching data at clients in information retrieval systems to improve response time for user queries. To reduce the overhead involved in keeping cache copies consistent, it may be appropriate in some applications to allow cache copies to diverge from the central server copy in a controlled manner. Quasi-caching is a cache management technique that allows such controlled divergence. Examples of quasi-caching constraints include: 1. arithmetic condition: divergence is limited by difference in values of cache copy and central copy. 2. version condition: divergence is limited by number of changes between cache copy and central copy. 3. delay condition: divergence is limited by time by which cache copy lags behind central copy. 3 Quasi-Consistency in Broadcast Environments We propose a concurrency control technique - Quasi-consistent Timestamp Intervals (Quasi-TI) - for broadcast environments. Quasi-TI is an adaptation of BCC-TI, and is based on the quasi-caching constraints described in section 2.3. Each query specifies information about the amount of inconsistency it can tolerate in the values it reads, in terms of: 1. imprecision of the data values; 2. number of version lags between the read version and the latest version; and 3. amount of time lag between the read version and the latest version. 2

For instance, a query might specify that it can tolerate an imprecision of 3.0 in the values it has read, up to 3 modifications of the values it has read until the time it commits, and up to 20 time units between the time it read the values and the last time one of the values was modified. Quasi-consistency can lead to fewer aborts and restarts than serializability. 3.1 Specifying Quasi-Consistency Limits Every query specifies tolerable limits for the three kinds of quasi-consistency discussed above. The quantity ImpLimit specifies the amount of imprecision in value the query can tolerate per data object. The quantity VersionLagLimit specifies the number of modifications the query can tolerate, to an object it has read. The quantity TimeLagLimit specifies the tolerable difference in time between the time the query read an object and the time the object was modified last. If any of these tolerable limits is exceeded before the query can commit, the query must be aborted and restarted. 3.2 Quasi-TI Algorithm Update transactions execute periodically at the server. When an update transaction U is ready to commit, the server assigns a timestamp TS(U) to U. Let WS(U) be the write set of transaction U. The server records TS(U) and WS(U) in its control information table. The server also maintains WTS(d) - the timestamp of the last committed transaction that wrote to object d - for every object d in the database. The server broadcasts all data objects with their associated timestamps repeatedly. Periodically, the server also broadcasts the control information table and then deletes all entries in the table. The control information table is broadcast at least once every cycle, and possibly many times a cycle. At the client, when a query begins execution, its timestamp interval is set to LB=0 and UB=infinity. The query also carries a running estimate of the number of versions each of its read objects d is lagging by, in versionlag[d]. When a query requests a read, the client adds the read request to the list of data objects to be read this cycle, and notes the id of the requesting query. When a control information table is broadcast, the client processes each outstanding query Q in the following way: 1. For every object d that was read in the past by Q and has since been written by a transaction U, versionlag[d] is incremented by the number of such transactions U. If versionlag[d] now exceeds VersionLagLimit[Q], then UB(Q) is set to min (UB(Q), WTS(U)). If LB(Q) >= UB(Q), then Q is aborted and restarted. 2. For every object d that was read in the past by Q and has since been written by a transaction U, the time difference between TS(U) and the read time of d is calculated. If the difference exceeds TimeLagLimit(Q), then UB(Q) is set to min (UB(Q), WTS(U)). If LB(Q) >= UB(Q), then Q is aborted and restarted. 3. If ImpLimit(Q) is 0, UB(Q) is set to min (UB(Q), TS(U)) for all U that have updated an object read by Q. If LB(Q) >= UB(Q), then Q is aborted and restarted. If ImpLimit(Q) is not zero, the actual imprecision must be checked the next time the object is broadcast. Therefore, a reread request is enqueued for every object d that was read by Q and has been updated by transaction U. When a requested data object d is broadcast, the client reads the value d and the timestamp WTS(d). For every query Q that requested a read of this object, the client does the following: 1. Store the value of the object. 2. Store WTS(d) (the timestamp of the object). 3. Set LB(Q) to max (LB(Q), WTS(d)). If LB(Q) >= UB(Q), then Q is aborted and restarted. For every query Q that requested a reread of this object, the client does the following: 1. Calculate the difference between the previously read value of d and the new value of d. 2. If the difference exceeds ImpLimit(Q), then set UB(Q) to min (UB(Q), WTS(d)). 3. If LB(Q) >= UB(Q), then abort and restart Q. 3

4 Caching Next, we added caching at the clients. We assume that a client knows the static access probabilities for all objects that it will access. Each cache entry contains the value of the object, the timestamp of the object, the version lag that the copy has accumulated, and the time lag that the copy has accumulated. The server s algorithm remains the same. The following are the changes in the client s algorithm. 4.1 Inserting Objects into the Cache When a requested data object is broadcast, the client checks if a copy of the object already exists in cache. If a copy exists, the value and timestamp of the entry are updated, and the version lag and time lag of the entry are reset to zero. If a copy of the object does not exist in the cache, the object is inserted into the cache if there exists an available cache slot. If no slot is available, a victim cache entry is selected based on the probability of access of the objects. If there exists a victim cache entry with a lower access probability than the object that has just arrived, the victim is replaced by the newly-arrived object. 4.2 Updating Cache Entries Every time a control information table is broadcast, the client checks to see if any of the modified objects are in its cache. If such an object d is in the cache, the timelag of the cached copy is set to TS(U) - timestamp of the cache entry, and versionlag is incremented by the number of transactions that have updated d since the last control information table was broadcast. 4.3 Reading Cache Copies When a query Q requests a read, the client first checks if there is a cached copy of the object. If there is a cached copy, the client does the following: 1. If the version lag and timel ag entries of the cached copy are zero, then the client gives the cached value to Q. 2. If ImpLimit(Q) is DONT_CARE, and if the entry s version lag < VersionLagLimit(Q) and if the entry s time lag < TimeLagLimit(Q), then the client gives the cached value and timestamp to Q. Otherwise, a read request is enqueued as in the no-caching case. A query may specify its ImpLimit to be DONT_CARE if it knows the maximum value by which any transaction can change the value of an object d. In such a case, all the query has to do is to specify the right VersionLagLimit to ensure correctness according to its semantics. 4.4 PIT Caching The caching scheme described above (P-caching) uses the probability of access of data objects to decide which objects to cache. In a quasi-consistent caching environment, there is another criterion that is important: the tightness of the consistency constraints of a query. The constraint tightness of a query is inversely proportional to the amount of imprecision it allows in ImpLimit, VersionLagLimit and TimeLagLimit. The idea is that if an object has very tight consistency constraints, then caching it is less useful because queries may have to read the object off the air every time, to obtain sufficient consistency. We introduce a new caching scheme called PIT (Probability-of-access Inverse constraint-tightness) caching. Every cache entry has a PIT value stored with it. The PIT value is calculated as the probability of access of the object divided by the constraint tightness - a function of (ImpLimit, VersionLagLimit, TimeLagLimit) of a query that reads the object. Every time a query reads the object, its constraint tightness is updated to the highest value of tightness required for that object so far. When an uncached object is read off the air, its PIT value is calculated, and a victim cache entry is selected based on PIT values instead of probabilities of access (P-values). 5 Simulation 5.1 Parameters We simulated a flat broadcast disk system, one server and one client. The server broadcasts Server- 4

Objects data objects repeatedly in a cycle. HotObjects of these objects are more frequently updated than the other objects. The probability of update of hot objects is HighContProb. The simulation proceeds in units of bit times, where a bit time is the time required by the server to broadcast one bit of data. We assume that the network delivers data objects in a FIFO order to the clients. Data objects are of size Object- Size. Update transactions are committed at the server periodically, one every UpdateArrivTime Kbit time units. The size of update transactions (the number of data objects updated) is uniformly distributed between MinUpdateSize and MaxUpdateSize data objects. An update can cause the value of a data object to change by at most MaxValChange units. The server broadcasts a control information table periodically, after every CITcycleLength data objects have been broadcast. The client accesses ClientObjects data objects out of the total ServerObjects. ClientHotObj of these objects are read more frequently than the others. The probability of read of hot objects is HighContProb. No attempt was made to match up the hot spots at the server and at the client. Read-only transactions (queries) arrive at the client periodically, one every QueryArrivTime Kbit time units. A query submits a read every ReadThinkTime Kbit time units. The size of queries (the number of data objects read) is uniformly distributed between MinQuerySize and MaxQuerySize data objects. Queries specify their ImpLimit, VersionLagLimit and TimeLagLimit when they arrive. The values for these limits are uniformly distributed between zero and MaxImprecision, MaxVersionLag or Max- TimeLag respectively. In addition, PercNoImp percent of queries specify DONT_CARE as their imprecision. Depending on the experiment, a query reads data objects either off the air or from a local cache. The cost of reading an object off the air is set to 10 Kbit time units, and the cost of reading a cached copy is set to 1 Kbit time unit. The cost of selecting a victim cache entry and replacing it is set to 2 Kbit time units. The size of the cache is CacheSize data objects. A query is allowed to use a cache copy only if it is known that the copy is sufficiently accurate, given the query s quasi-consistency requirements. A query will be aborted if data objects that it has read become too stale before it commits. An aborted query is restarted and executed until it commits. We simulate a restart by introducing a new query into the system when an original query is aborted. 6 Experiments and Results We ran several experiments to find the conditions under which quasi-consistency and caching are useful. Our performance metrics were the average response time for a query, and the number of restarts (out of 5000 queries). 6.1 BCC-TI versus BCC-TI with P-Caching For BCC-TI, serializability is the correctness criterion. In other words, ImpLimit, VersionLagLimit and TimeLagLimit are all zero for all queries. We ran the BCC-TI algorithm with no caching, and then with caching based only on the probability of access. Figure 1 shows the baseline parameters we used. Parameter Value Parameter Value ServerObjects 900 HotObjects 900 ClientObjects 300 ClientHotObj 30 ObjectSize 8000 UpdateArrivTime 1000 QueryArrivTime 64 ReadThinkTime 32 MinUpdateSize 3 MaxUpdateSize 4 MinQuerySize 6 MaxQuerySize 8 CITcycleLength 900 CacheSize 150 5

Parameter Value Parameter Value MaxImprecision 0.0 MaxVersionLag 0 MaxTimeLag 0 HighContProb 90 PercNoImp 0 MaxValChange 10.0 Figure 1: Baseline Parameters for Experiment 1 Figure 2 shows the average response time of a query, given different rates of arrival of update transactions at the server. The graph shows that caching is most useful when updates are rare. The improvement in response time is more than 50% at low update arrival rates. As updates become more frequent, the benefit of caching becomes less, because the frequent updates invalidate the cache frequently. 6 Query response time (Mbittimes) 4 2 No Caching P-Caching 0 0 20 40 60 80 100 Update transaction arrival rate (transactions/mbittime) Figure 2: Effect of Update Arrival Rate on Response Time of BCC-TI Figure 3 shows the average response time of a query, given different cache sizes. As the cache size 6

increases, caching becomes more effective, reducing the response time by more than 50%. 6 Query response time (Mbittimes) 4 2 No Caching P-Caching 0 0 50 100 150 200 250 Cache size (objects) Figure 3: Effect of Cache Size on Response Time of BCC-TI 6.2 BCC-TI versus Quasi-TI BCC-TI enforces serializability, while Quasi-TI enforces only quasi-consistency. Quasi-consistency can reduce the number of aborts because a query s semantics may allow it to ignore updates to previouslyread data, within certain tolerable limits. Reducing the number of aborts reduces the average response time because fewer queries are restarted. Figure 4 shows the baseline parameters used for this experiment. Parameter Value Parameter Value ServerObjects 400 HotObjects 300 ClientObjects 300 ClientHotObj 150 ObjectSize 8000 UpdateArrivTime 25 QueryArrivTime 64 ReadThinkTime 32 MinUpdateSize 19 MaxUpdateSize 21 MinQuerySize 19 MaxQuerySize 21 CITcycleLength 200 CacheSize 150 MaxImprecision 60.0 MaxVersionLag 6 MaxTimeLag 1500 HighContProb 50 PercNoImp 0 MaxValChange 10.0 Figure 4: Baseline Parameters for Experiment 2 Figure 5 shows the number of restarts as the update arrival rate is varied. As updates become more frequent, the number of restarts in BCC-TI increases rapidly, while the number of restarts in Quasi-TI 7

increases less rapidly. 2500 Number of restarts (queries) 2000 1500 1000 500 BCC-TI QUASI-TI 0 0 20 40 60 80 100 Update transaction arrival rate (transactions/mbittime) Figure 5: Effect of Update Arrival Rate on Restarts in Quasi-TI As a consequence, the average response time increases more rapidly for BCC-TI than for Quasi-TI (Figure 6). 6 Query response time (Mbittimes) 5 4 BCC-TI QUASI-TI 3 0 20 40 60 80 100 Update transaction arrival rate (transactions/mbittime) Figure 6: Effect of Update Arrival Rate on Response Time of Quasi-TI Figure 7 shows the number of restarts as the looseness of constraints of queries is varied. The higher the value for looseness, the higher the values of MaxImpLimit, MaxVersionLagLimit and MaxTimeLagLimit. A value of zero for looseness represents the BCC-TI case. As the looseness of consistency con- 8

straints increases, the number of restarts reduces, and Quasi-TI outperforms BCC-TI. 2500 Number of restarts (queries) 2000 1500 1000 500 0 0 5 10 15 20 Looseness of constraints (version lag limit) Figure 7: Effect of Constraint Looseness on Restarts in Quasi-TI As a result, the average response time also reduces as looseness increases, allowing Quasi-TI to outperform BCC-TI (Figure 8). 6 Query response time (Mbittimes) 5 4 3 0 5 10 15 20 Looseness of constraints (version lag limit) Figure 8: Effect of Constraint Looseness on Response Time of Quasi-TI 6.3 Quasi-TI versus Quasi-TI with P-Caching In this experiment, we ran Quasi-TI without caching, and then with P-caching. Caching benefits Quasi-TI if consistency constraints are loose and there are a significant number of queries which don t care 9

about the imprecision as long as the version lag and time lag are within tolerable limits. The table in Figure 9 shows the baseline parameters we used for this experiment. Parameter Value Parameter Value ServerObjects 400 HotObjects 300 ClientObjects 300 ClientHotObj 150 ObjectSize 8000 UpdateArrivTime 25 QueryArrivTime 64 ReadThinkTime 32 MinUpdateSize 19 MaxUpdateSize 21 MinQuerySize 7 MaxQuerySize 9 CITcycleLength 200 CacheSize 75 MaxImprecision 160.0 MaxVersionLag 16 MaxTimeLag 6000 HighContProb 50 PercNoImp 0 MaxValChange 10.0 Figure 9: Baseline Parameters for Experiments 3 and 4 Figure 10 shows the average response time of a query as the percentage of queries with ImpLimit = DONT_CARE is increased. The response time decreases slightly as more and more queries declare their ImpLimit to be DONT_CARE, because more cache copies can be used. 3.0 Query response time (Mbittimes) 2.5 2.0 1.5 No Caching P Caching 1.0 0 20 40 60 80 100 Percentage of DONT_CARE ImpLimit queries (queries) Figure 10: Performance of P-Caching in Quasi-TI 6.4 Quasi-TI versus Quasi-TI with PIT Caching In this final experiment, we ran Quasi-TI without caching, and then with PIT-caching, which takes into account both probability of access and tightness of consistency constraints. The baseline parameters 10

used are listed in the table in Figure 9. Figure 11 shows the average response time of a query as the percentage of queries with ImpLimit = DONT_CARE is increased. Again, the response time decreases slightly as more and more queries declare their ImpLimit to be DONT_CARE, because more cache copies can be used. 4.0 Query response time (Mbittimes) 3.5 3.0 2.5 No Caching PIT Caching 2.0 0 20 40 60 80 100 Percentage of DONT_CARE ImpLimit queries (queries) Figure 11: Performance of PIT Caching in Quasi-TI However, there is no discernible improvement of PIT caching over P-caching. PIT caching would outperform P-caching if there were a significant number of data objects that are frequently accessed but have such tight consistency constraints that caching them would be useless. We conjecture that our workload does not have this property, and therefore PIT caching does not offer any benefits. 7 Conclusions and Future Work We have presented Quasi-TI - an adaptation of the BCC-TI concurrency control technique for asymmetric communication environments. Quasi-TI allows queries to specify semantic consistency constraints, and enforces these constraints. We have simulated BCC-TI and Quasi-TI without caching and with two different caching schemes - P-caching and PIT caching. We have presented simulation results that demonstrate the workload characteristics for which different schemes are useful. BCC-TI benefits from P-caching when updates are rare and the cache size is sufficiently large. Quasi-TI outperforms BCC-TI when updates are frequent and queries have loose consistency constraints. P-caching benefits Quasi-TI only slightly when consistency constraints are loose and queries don t consider the imprecision in object values. PIT caching also benefits Quasi-TI only slightly under the same conditions, but does not offer any benefits over P-caching. We would like to extend this work to multiple broadcast disk environments, and study caching based on broadcast frequency. The effect of prefetching data objects into the client s cache is another interesting issue. We would like to run more extensive simulation tests for different workloads and determine the effect of matching the hot spots at the server to the hot spots at the client. Finally, we would like to explore other caching schemes. 8 References [1] Acharya S., Alonso R., Franklin M. and Zdonik S., Broadcast Disks: Data Management for Asymmet- 11

ric Communication Environments, ACM SIGMOD, 1995 [2] Acharya S., Franklin M. and Zdonik S., Disseminating Updates on Broadcast Disks, Proc. 22nd VLDB Conf., Bombay, India, 1996 [3] Alonso R., Barbara D. and Garcia-Molina H., Data Caching Issues in an Information Retrieval System, ACM TODS 15/3, Sep 1990 [4] Herman G., Gopel G., Lee K. C. and Weinrib A., The Datacycle Architecture for Very High Throughput Database Systems, Proc. ACM SIGMOD Conf., 1987, pp. 97-103 [5] Imielinski T., Viswanathan S. and Badrinath B. R., Energy Efficient Indexing on Air, Proc. ACM SIG- MOD Conf., May 1994 [6] Lee V., Son S. H. and Lam K., On the Performance of Transaction Processing in Broadcast Environments, Int. Conf. on Mobile Data Access (MDA 99), Hong Kong, Dec 1999 [7] Shanmugasundaram J., Nithrakashyap A., Sivasankaran R. and Ramamritham K., Efficient Concurrency Control for Broadcast Environments, ACM SIGMOD, 1999 12