Data Dissemination in Mobile Computing Environments (2)

Size: px

Start display at page:

Download "Data Dissemination in Mobile Computing Environments (2)"

Cory Wilkerson
5 years ago
Views:

1 Data Dissemination in Mobile Computing Environments (2) Sungwon Jung Dept. of Computer Science and Engineering Sogang University Seoul, Korea jungsung@sogang.ac.kr Indexing Techniques for Broadcast Data Tree-based Indexing (1,m) Indexing Distributed Indexing Alphabetic Huffman tree Exponential Indexing 2 1

2 Tree-based Indexing Terminologies A bucket: the smallest logical unit of a broadcast All buckets are of the same size index buckets holding the index and data buckets holding the data Index segment: refers to a set of contiguous index buckets Data segment: refers to a set of data buckets broadcast between successive index segments A bcast: consists of all data segments interleaved with the index information (all index segments) Each bcast is periodically broadcast on the wireless channel 3 Tree-based Indexing Terminologies In order to make all buckets self-identifying, each bucket has the following information: bucket_id: the offset of the bucket from the beginning of the bcast bcast_pointer: the offset to the beginning of the next bcast index_pointer: the offset to the beginning of the next index segment bucket_type: data bucket or index bucket The actual time of broadcast for bucket P from the current bucket the product of (offset-1) and the time necessary to broadcast a single bucket An index bucket is arranged a sequence of (attribute_value, offset) offset is a pointer to the bucket containing the record identified by attribute_value A data bucket is arranged as a sequence of data records 4 2

3 General access protocol for retrieving data The initial probe: the client tunes into the broadcast channel and determines when the next index segment will be broadcast Traversing the index : A sequence of pointers (in the index segment) is accessed to find out when to tune into the broadcast channel to get the required data Retrieving the data the client tunes to the channel when buckets containing the required data arrive, and downloads all the required records 5 Parameters of Concern Tuning time: the amount of time spent by a client listening to the channel determine the power consumed by the client to retrieve the required data Latency: the time elapsed (on the average) from the time a client requests data to the point when all the required data is downloaded by the client 6 3

4 Parameters of Concern Latency: Latency = Probe Wait + Bcast Wait Probe Wait: The average duration for getting to the next index segment when a initial probe is made into the broadcast channel Bcast Wait: the average duration between the point the index segment is encountered and the point when all the required records are downloaded Probe Wait and Bcast Wait work against each other Minimizing probe wait will result in increasing bcast wait, and vice versa Example: To minimize the bcast wait, we can broadcast the index once at the beginning of each bcast The probe wait will be large!!! Why? 7 Latency_opt Provides the lowest latency with a very large tuning time the best latency is obtained when no index is broadcast along with the file For a file of size Data buckets, on the average it takes (Data/2) time to get to the first record with the required attribute value Takes a duration of C, to download all the required records Latency = (Data/2 + C) and Tuning time = (Data/2 + C) 8 4

5 Tune_opt Provides the best tuning time with a large latency the server broadcasts the index at the beginning of each bcast Tuning time : (1+ k + C) k is the # of levels in the multilevel index tree Latency : (Data + Index + C) the probe wait = (Data + Index) /2 the bcast wait = (Data + Index)/2 + C 9 (1, m) Indexing the whole index is broadcast preceding every fraction (1/m) of the file the first bucket of each index segment has a tuple with two fields the first field: the attribute value of the record that was broadcast last the second field: the offset to the beginning of the next bcast 10 5

6 Analysis of (1, m) Indexing Latency: the probe wait: ½ *(Index + Data/m) the bcast wait: ½ *((m*index) + Data) + C Tuning Time: 1 + k + C The first probe is the initial probe that gets a pointer to the next index bucket k probes are required for following the pointer in the index C more probes are required for tuning in for getting the required records Optimum m a formula to compute the optimal m to minimize the latency for the (1,m) indexing the optimum m, denoted by m * is: m * Data Index 11 Distributed Indexing 12 6

Distributed Indexing Nonreplicated Distribution Different index segments are disjoint the probe sequence: the bcast_pointer at bucket 3 will direct the client to the beginning

replication 13 Distributed Indexing Entire Path Replication The offset at data bucket 3 will direct the client to the index bucket I that precedes second_a1 where the client

7 Distributed Indexing Nonreplicated Distribution Different index segments are disjoint the probe sequence: the bcast_pointer at bucket 3 will direct the client to the beginning of the next bcast where I, a3, b8, c23, and bucket 66 will be successively probed the probe wait is quite significant and will offset savings in bcast wait due to the lack of replication 13 Distributed Indexing Entire Path Replication The offset at data bucket 3 will direct the client to the index bucket I that precedes second_a1 where the client makes the successive probes such as first_a3, b8, c23, and bucket 66 The latency suffers from the replication of index information the root was unnecessarily replicated six times!!! 14 7

Distributed Indexing Partial Path Replication The offset at the data bucket 3 will direct the client to second_a1 To make up for the lack of root preceding second_a1, there is a small index called

8 Distributed Indexing Partial Path Replication The offset at the data bucket 3 will direct the client to second_a1 To make up for the lack of root preceding second_a1, there is a small index called control index within second_a1 If second_a1 does not have a branch leading to the required record, then the control index (CI) is used to direct the client to a proper branch in the index tree CI directs the client to i 2 where first_a3, b8, c23, and bucket 66 are successively probed 15 Control Index for Partial Path Replication 16 8

Indexing for Broadcast Data with Skewed Access Frequencies B+ tree index assume uniform popularity patterns for all data Not suitable for the broadcast data with skewed access patterns Need an index

17 Huffman Codes Huffman codes are a widely used and very effective technique for compressing data Suppose we have a 100,000 character data file that we wish to store compactly a b c d e f

9 Indexing for Broadcast Data with Skewed Access Frequencies B+ tree index assume uniform popularity patterns for all data Not suitable for the broadcast data with skewed access patterns Need an index tree that are sensitive to the skewed data access patterns Index pointers of more popular data items are higher up in the tree than others The Huffman Tree have this property! 17 Huffman Codes Huffman codes are a widely used and very effective technique for compressing data Suppose we have a 100,000 character data file that we wish to store compactly a b c d e f Frequencies (in thousands) Fixed-length codeword Variable-length codeword A fixed-length codeword requires 3*100,000 = 300,000 bits to code the entire file A variable-length codeword requires (45*1 + 13*3 + 12*3 + 16*3 + 9*4 + 5*4)*1,000 = 224,000 bits to represent the file, a saving of approximately 25% 18 9

10 An Example of Huffman Tree Construction 19 The problems with Huffman Tree The Huffman Tree is not a search tree User need to know the Huffman codes for each key before they can traverse the tree for a given file 20 10

11 Alphabetic Huffman Tree Alphabetic Huffman trees can function as search trees They preserve leaf ordering on any input sequence used to construct them like B + trees left-to-right scan of the leaves of each tree will show the leaves ordered by their keys A simple comparison-based tree traversal suffices to determine the location of a desired leaf. They are sensitive to a frequency distribution on the input The Hu-Tucker algorithm is used to construct the Alphabetic Huffman trees with binary fan-out 21 An Example of Alphabetic Huffman Tree 22 11

12 A 23 B 4 C 12 D 10 E 17 F 31 G 15 H 21 I 29 J 19 K 7 L 12 M 16 N 14 O A 23 B 4 C 12 D 10 E 17 F 31 G 15 H 21 I 29 J 19 K 7 L 12 M 16 N 14 O Level:

13 H E L C F I N A F I O D G J M A B D E G H J K M N B C K L Alphabetic Huffman Tree Mapping index trees to channeltime space When we have 5 index channels: When we have a single index channel, do the time multiplexing I N O A G I E F G D E - B C D I N O J M N H I - D E

14 Topological Tree Approach Alphabetic Huffman tree approach requires the dedicated index and data channels The drawbacks with Alphabetic Huffman tree approach Lack of flexibility The number of needed channels to allocate index nodes is fixed If we increase the fan-out of the tree to use less number of available index channels, it will not show the effect of the skewness for accessing the data with different access frequencies Waste of channel space Take an extreme case where an index tree is an chain 27 Topological Tree Approach Topological tree Alphabetic Index tree broadcast schedule 28 14

15 Topological Tree Approach Topological Tree Allocating Index by using topological trees to 2 index channels 29 Exponential Indexing Scheme -1 Simple Exponential Index Example: Suppose that a client issues a query for item NOK right before item DELL (i.e., bucket 1) is broadcast 30 15

16 Exponential Indexing Scheme -2 Generalized Exponential Index r = index base, I = chunk size Example: (r=2, I=2) Suppose that the client makes a query for item NOK right before the bucket containing item DELL is broadcast. 31 Cache Invalidation Methods for Mobile Clients Data is being updated at the servers the replicated copies at mobile clients should be kept consistently How to manage cached data by considering the disconnection of mobile clients? No caching in the Mobile Clients: Each query goes directly to the server Caching data in the Mobile Clients: Have to worry about how to inform Mobile Clients that their cached items have changed 32 16

Cache Invalidation Report IR i = {[j,t j ] j D and t j is the timestamp of the last update of j such that T i w t j T i } T i = i L, w = time window, TS of IR i = T i w = 30, IR 5 = {TS=50, (a, 25),

17 Cache Invalidation Report IR i = {[j,t j ] j D and t j is the timestamp of the last update of j such that T i w t j T i } T i = i L, w = time window, TS of IR i = T i w = 30, IR 5 = {TS=50, (a, 25), (b,35), (c, 28), (d, 45) } L=10 Update = Invalidation Report T 1 =10 T 2 =20 T 3 =30 T 4 =40 T 5 =50 Time MC1: Disconnected From Server at time 35 MC1: Reconnected at time 43 Cache { TS=30, (a, 25), (b,15), (c,28) } Query = {a, b, c} 33 Protocol for Broadcast Timestamps if (T i T l > w) {drop the entire cache}, else { for every item j in the MU cache { if there is a pair [j,t j ] in U i { if t jc < t j { throw j out of the cache} else {t jc = T i } } } } for every item j Q i { if j is in the cache { use the cache s value to answer the query } else {go uplink with the query}} T l := T i } 34 17

18 The Dual-Report Cache Invalidation (DRCI) The server broadcasts every L time unit a pair of invalidation reports an object invalidation report (OIR): [T wl, T] Contents of OIR the current timestamp T a list of (o id, t id ) pairs o id : an object identifier of an object updated during the interval [T wl, T] t id : the corresponding most recent update timestamp,and t id > (T wl) a group invalidation report (GIR): [T WL, T] where W > w > 0 a fixed size report that contains the update history of the past W broadcast interval Contents of GIR a list of (G id, T id ) pairs G id : a group identifier T id : the most recent timestamp where the group G id is valid 35 The Dual-Report Cache Invalidation (DRCI) The timestamp of the groups in GIR are determined as follows: Step 1: Based on the observation that objects that are updated during [T wl, T] would be reflected in the OIR, we ignore these objects when determining the timestamp of a group. Step 2. For each group, perform the following: Among the group's remaining objects in the update history, find the one with the largest timestamp that is less than T wl. Let this timestamp be t. If no such objects exists (i.e., no updates during [T WL, T] or the updates are already included in OIR), then t = 0. The timestamp for the group is max(t WL, t) 36 18

19 The Dual-Report Cache Invalidation (DRCI) The Dual-Report Cache Invalidation Scheme Example Let T = 34, L = 4, w=2, and W = 6 Assume G1 = {o1, o2, o3, o4}; G2 = {o5, o6, o7, o8}; G3 = {o9, o10, o11, o12}; G4 = {o13, o14, o15, o16}: At time T, the update reports are The contents of OIR are as follows: {34, (o7, 26), (o8, 32), (o12, 30), (o16, 28)} The contents of GIR for groups G1, G2,G3 andg4 at time 34 are {(G1, 24), (G2, 22), (G3, 20), (G4, 12)} Suppose a mobile client disconnect at time 23 and reconnects at time 34 request objects o1, o2, o6, o7, o9, o12, o14 37 Concurrency Control Methods for Mobile Transactions a large proportion of read-only transactions and a small proportion of update transactions Stock markets and auctions The number of stock buyers or bidders is relatively few compared to the number of speculators who read the prices frequently Updates may originate from mobile clients The updates of data must be disseminated promptly and consistently Should support a serialization of mobile transactions 38 19

20 Concurrency Control Methods for Mobile Transactions Example: X : the number of units of a stock, Y: the unit price (Y) of a stock At server: U 2 U 3 At mobile client: U 3 R 1 U 2 1 st Broadcast Cycle 1 U 2 : X = nd Broadcast Cycle U 3 : Y = 0.9 time A B X=1100 Y = $1.0 C A B X=1000 Y = $0.9 C Mobile Transaction R 1 reads X = 1100 units Read-only mobile transaction of mobile client 1 Mobile Transaction R 1 reads Y = $0.9 Inconsistent Data X and Y Reads by Mobile Client 1!!! 39 Concurrency Control Methods for Mobile Transactions Problems with Conventional Concurrency Control Methods Locking or Timestamp Mechanism require considerable bi-directional communication the time required for message passing may be intolerably long Possible overloading to server 40 20

21 Concurrency Control Methods for Mobile Transactions Proposed Solutions Use weaker correctness criterion for serializability Does not guarantee a global serialization Multiversion broadcast approach For read-only transactions Conflict-serializability method Maintain Serialization Graph for conflict checking FBOCC method Based on OCC method 41 FBOCC in Broadcast Environments Backward validation the validation process is done against committed transactions Forward validation validating of a transaction is carried out against currently running transactions It provides flexibility for conflict resolution such that either the validating transaction or the conflicting active transactions may be chosen to restart It generally detects and resolves data conflicts earlier than backward validation It wastes less resources and time 42 21

22 FBOCC in Broadcast Environments At the server, there are transactions submitted for validation by different sources including mobile clients Not desirable to restart a validating mobile transaction due to high restart cost Forward validation is a better choice for the server because of 1. The flexibility to choose those active transactions that are in conflict with the validating transaction to restart 2. Only the write set of a transaction is required for forward validation It allows the read-only transactions to be validated locally and autonomously at the mobile clients 3. The server takes the active role for the decision to commit transactions 4. Given the serialization order determined by the server, the mobile clients have to determine whether their transactions can be committed by detecting any data conflicts with the committed transactions The mobile clients have to carry out the backward validation process The partial backward validation is introduced at the beginning of every broadcast cycle (Why not validating at the end of a transaction?) Any transaction that is found to read inconsistent data is restarted immediately 43 FBOCC in Broadcast Environments Principles Transactions are allowed to execute unhindered until they reach the validation point or partial validation point for mobile transactions The validation is based on the following principle to ensure the serializability If the transaction T i is serialized before transaction T j, the following two conditions must be satisfied 1. Condition 1: No overwriting The writes of T i should not overwrite the writes of T j 2. Condition 2: No read dependency The writes of T i should not affect the read phase of T j Condition 1 is ensured by performing the write phase serially in critical sections at the server Only condition 2 will be considered in the proposed validation scheme 44 22

23 FBOCC in Broadcast Environments Backward validation at the mobile clients At the mobile clients, all transactions including read-only transactions and update transactions have to perform a partial validation at the beginning of every broadcast cycle. The partial validation is carried out by consulting the control information broadcast by the server at the beginning of every broadcast cycle. If the transaction fails the partial validation, it will be aborted and restarted. Otherwise, the transaction can proceed The partial validation process is carried out against committed transactions (at the server) in the last broadcast cycle Data conflicts are detected by comparing the read set of the validating mobile transaction and the write set of committed transactions 45 FBOCC in Broadcast Environments Backward validation at the mobile clients T pv : the partial validating mobile transaction CD(C i ): the set of data objects that have been committed (updated) in the last broadcast cycle C i at the server CRS(T pv ): the current read set of transaction T pv, which is the set of data objects that have been read by T pv from previous broadcast cycles The backward validation algorithm CD(C i ) is stored in the control information table The value of C i is recorded for the final validation 46 23

24 FBOCC in Broadcast Environments Forward validation at the server At the server, validation of a transaction is done against currently running transactions Based on the assumption that the validating transaction is ahead of every concurrently running transaction still in read phase in the serialization order The write set of the validating transaction is used for data conflict detection to ensure Condition 2 The detection of data conflicts is carried out by comparing the write set of the validating transaction and the read set of active transactions If an active transaction, T i, has read an object that has been concurrently written by the validating transaction, the value of the object used by T i is not consistent Such data conflicts are resolved by restarting the conflicting transactions in the read phase. 47 FBOCC in Broadcast Environments Forward validation at the server For an update transaction submitted by a mobile client, it has to perform a final validation before the forward validation The final validation is necessary because there may be transactions committed since the last partial validation performed at the mobile client A mobile transaction may read or write data objects after partial validation and before being sent to the server for final validation There may exist some transactions at the server that are committed and in conflict with the validating transaction, T v, since its last partial validation The broadcast cycle number (i.e., C k ) of the last partial validation performed at the mobile client has to be sent to the server along with the update transaction for final validation 48 24

25 FBOCC in Broadcast Environments Forward validation algorithm T v : the validating transaction T a (a = 1,2,,n, a v): the conflicting transactions existing at the server in their read phase C k : the broadcast cycle number received along with T v T c (c = 1,2,,m) : the transactions committed at the server since C k If the history of T c is not available at the server due to space limitation, T v has to be aborted If T v is successfully validated, the write set of T v is recorded in the control information table, which will be broadcast in the next broadcast cycle The final validation results (commit or abort) of the mobile transactions are included in the control information table for acknowledgement to the mobile clients for further action. 49 FBOCC in Broadcast Environments The roles of server and mobile clients At the mobile clients, before any read or write operations are performed on data objects broadcast during a cycle, the control information transmitted at the beginning of the cycle is consulted to perform the partial validation If the transaction fails the partial validation, the transaction is aborted. Otherwise, the read or write operation can proceed A write operation on a data object is performed on a private workspace at the mobile client 50 25

26 FBOCC in Broadcast Environments The roles of server and mobile clients At the mobile clients, a read-only transaction can commit locally if it passes all the partial backward validation in the course of its execution The read-only transaction is serialized after all transactions committed before the beginning of the current broadcast cycle and is serialized before all transactions committed after the beginning of the current broadcast cycle A read-only transaction may be serialized before a transaction that is committed earlier at the server if the commit time of the transaction is later than the start time of the current broadcast cycle. 51 FBOCC in Broadcast Environments The roles of server and mobile clients For an update transaction, the read set, write set, the updated values and the broadcast cycle number of the last partial validation performed at the mobile client have to be sent to the server for final validation It is because the position of the update transaction in the serialization order is determined on the fly (at the validation point) at the server The update transaction is serialized after all transactions committed before its validation point and is serialized before all active transactions. After the final validation, the result is sent back to the corresponding mobile client To abort a transaction at the mobile clients, all copies of the data objects written in the private workspace, if any, are discarded 52 26

27 An Example of FBOCC Method Mobile Clients: Q 2 :r(a)r(b)r(c) Q 3 :r(p)r(q) U 4 :r(x)r(y)w(y) Server : U 1 : r(a)w(a) U 5 : r(y)w(y) 1) Q 2 fails the partial backward validation because CI(C i ) RS(Q 2 ) {} Client R 2 (a) R 2 (b) R 3 (p) R 4 (x) R 4 (y) W 4 (y) R 3 (q) c 3 CI(C i ) = {a} CI(C i+1 ) = {x} Server C i-1 R 1 (a) W 1 (a)c 1 C i R 5 (x)w 5 (x) c 5 C i+1 2) Q 4 fails the final validation because WS(U 5 ) RS(U 4 ) {} 53 27

Optimistic Concurrency Control. April 18, 2018

Optimistic Concurrency Control April 18, 2018 1 Serializability Executing transactions serially wastes resources Interleaving transactions creates correctness errors Give transactions the illusion of isolation