Logical and physical data structures for very small databases (VSDB)

Transcription

1 Logical and physical data structures for very small databases () Fabio A. Schreiber Cristiana Bolchini Letizia Tanca Fabio Salice Politecnico di Milano DATABASES SYSTEMS LARGE DB VLDB SYSTEM? SMALL SMALL DATABASE LARGE 1 1

2 PRESENTATION OUTLINE MOTIVATIONS DESIGN METHODOLOGY LOGICAL PHYSICAL DATA STRUCTURES ACCESS MODES COMPLEXITY POWER CONSUMPTION ANALYSIS WORK IN PROGRESS... 2 MOTIVATIONS FROM PC DOWNWARDS PALM HHELD PC SMART CARDS CELLULAR PHONES (SIM CARDS) PROFESSIONAL EMBEDDED SYSTEMS DEVICE CONTROL TELEMETRY SYSTEMS INDUSTRIAL PROCESS CONTROL DISTRIBUTED NETWORK MANAGEMENT 3 2

3 Flash EEPROM PHYSICAL STORAGE MEDIUM TECHNOLOGY ISSUES CONSTRAINTS ON MODIFY (ERASE/WRITE) OPERATIONS WRITE = PROGRAMMING WRITE ONLY ON VIRGIN OR ERASED LOCATIONS ERASE ONLY AT BLOCK LEVEL TIMING ISSUES BLOCK ALLOCATION/MANAGEMENT ENDURANCE PERSPECTIVE FROM PERFORMANCE VIEWPOINT POWER CONSUMPTION VIEWPOINT 4 PHYSICAL STORAGE MEDIUM EEPROM Flash-EEPROM Read (nsec) 150 5V 3V Program ( µsec/byte) 157 (64 B) 625 (16 B) 5V 3V Erase (sec/block) N.A. 5V 3V Cell Size (µm 0.4µm technology) Cost per bit Medium Low Endurance (program, erase operations) 10K to 100K write cycle /byte 10K to 100K erase cycle /block 5 3

4 PHYSICAL STORAGE MEDIUM: THE SMART CARD EXAMPLE ROM RAM Flash-EEPROM Ck Time Base & Watchdog Random Number Generator CPU Mod Arithmetic Processor Control logic Interrupt Controller Port Rst Gnd Vcc 6 FLASH MEMORY ORGANIZATIONS 64 KB Main Block 16 KB Boot Block 64 KB Main Block 8 KB parameter Block 8 KB parameter Block 96 KB Main Block 64 KB Main Block 128 KB Main Block 128 KB Main Block 64 KB Main Block 128 KB Main Block Symmetrically blocked component Asymmetrically blocked component 7 4

5 DESIGN METHODOLOGY IDENTIFY IDENTIFY INFORMATION INFORMATION AREAS AREAS DEFINE DEFINE VIEWS VIEWS INTEGRATE VIEWS INTO THE CONCEPTUAL SCHEMA INTEGRATE VIEWS INTO THE CONCEPTUAL SCHEMA TRANSLATE THEM INTO THE LOGICAL DATA MODEL TRANSLATE THEM INTO THE LOGICAL DATA MODEL DEFINE DEFINE LOGICAL LOGICAL VIEWS VIEWS ALLOCATE ALLOCATE THEM THEM INTO INFORMATION AREAS INTO INFORMATION AREAS ESTIMATE LOGICAL VIEW SIZE ESTIMATE LOGICAL VIEW SIZE DEFINE DEFINE PRIVACY PRIVACY PROFILE PROFILE REFINE REFINE VIEWS VIEWS ALLOCATE ALLOCATE EACH EACH LOGICAL LOGICAL VIEW VIEW ON ON CARD CARD OR OR TO TO AN AN OUTER OUTER SYSTEM SYSTEM FOR EACH FRAGMENT ALLOCATED ON CARD ESTIMATE ACCESS TYPE ESTIMATE ACCESS TYPE DATA DATA VOLATILITY VOLATILITY DESIGN DESIGN MEMORY MEMORY PROTECTION PROTECTION MECHANISM MECHANISM CHOOSE ENCRYPTION CHOOSE ENCRYPTION ALGORITHMS ALGORITHMS SELECT SELECT THE THE LOGICAL LOGICAL DATA DATA STRUCTURE STRUCTURE SELECT SELECT THE THE PHYSICAL PHYSICAL DATA DATA STRUCTURE STRUCTURE 8 CASE STUDY: PORTABLE INTERNET ACCESS DATABASE url description web_site t,o id web_login 1:n has 1:1 web_access history category bookmark login password amount 1:n mvmnt visited_on last_visited description 1:n time id web_purse balance 9 5

6 RELATIONAL DB MODEL 10 LOGICAL/PHYSICAL DATA STRUCTURES Heap RELATION LIMITED CARDINALITY (<= 10 records) Sorted RELATION MEDIUM CARDINALITY (~100 - ~1000 records) SORTED W.R.T. A FIELD Circular list RELATION MEDIUM CARDINALITY STORED MANAGED AS A CIRCULAR LIST TYPICALLY SORTED BY /time Generic RELATION MULTI-INDEX STRUCTURE 11 6

7 DB ANNOTATION 12 PHYSICAL DESIGN: DATA ORGANIZATION SORTED CIRCULAR LIST DATA STRUCTURES ARE ASSIGNED ON THE BASIS OF THE ANNOTATIONS ON THE RELATIONS HEAP HEAP SORTED CIRCULAR LIST 13 7

8 ACCESS MODES: QUERY SCAN: FETCH ALL RECORDS IN THE TABLE SEARCH WITH EQUALITY SELECTION FETCH ALL RECORDS THAT SATISFY AN EQUALITY SELECTION "find the VisitedURL for day = #05/01/2001# SEARCH WITH RANGE SELECTION FETCH ALL RECORDS THAT SATISFY A RANGE SELECTION "find the movements with amount between 10 and 100" 14 ACCESS MODES: UPDATE INSERT: REQUIRES RECORD SHIFTING WHEN SORTED FETCH BLOCK INCLUDE THE NEW RECORD WRITE THE BLOCK BACK DELETE: REQUIRES RECORD SHIFTING WHEN SORTED FETCH BLOCK DELETE RECORD WRITE THE BLOCK BACK MODIFY: FETCH BLOCK MODIFY RECORD WRITE THE BLOCK BACK 15 8

9 PHYSICAL DESIGN: BLOCK ALLOCATION GOAL: REDUCE THE NUMBER OF ERASE/WRITE OPERATIONS ACHIEVE GOOD PERFORMANCE ACHIEVE LOW POWER CONSUMPTION PRESERVE THE LIFE OF THE FLASH DEVICE SOLUTIONS: ADD A validity_bit PER RECORD ADD A deleted_bit PER RECORD INSERT DUMMY RECORDS IN BLOCKS PERFORM GARBAGE COLLECTION DURING ERASE/PROGRAM CYCLES 16 PHYSICAL DESIGN: BLOCK ALLOCATION CIRCULAR LIST Sliding window ERASE Validity Bit Deleted Bit (a) (b) (c) (d) 17 9

10 PHYSICAL DESIGN: BLOCK ALLOCATION SORTED DATA STRUCTURE HIGH PROBABILITY OF CASCADED BLOCK ERASURE / REWRITE LEAVE SOME EMPTY (DUMMY) RECORDS IN EACH BLOCK ADJACENT (EFFICIENT SEARCH) DISTRIBUTED (EFFICIENT UPDATE) 18 PHYSICAL DESIGN: BLOCK ALLOCATION SORTED DATA STRUCTURE CONCENTRATED LOWER SEARCH TIME DISTRIBUTED FEWER INSERTION CONFLICTS Block 1 n used records dummy d empty records Block 2 n dummy d new barber barber barber new host new mom mom new 19 10

11 COMPLEXITY POWER CONSUMPTION ANALYSIS DIFFERENT MEMORY ALLOCATION SOLUTIONS Adj. Dummy / No Dummy Distributed Dummy Adj. Dummy / No Dummy Distributed Dummy 20 EXPERIMENTAL RESULTS Data Block Erasures Transmitted Bits on Bus structure Strategy 10%-30% 40%-60% 70%-90% 10%-30% 40%-60% 70%-90% Heap Simple Delete bit Simple Sorted Delete bit Dummy conc Dummy dist Circular Simple List Delete bit occurred block erasures and transmitted bytes on the system bus w.r.t. the naive, no deleted bit, no dummy records solution 21 11

12 WORK IN PROGRESS... ACCURATE LOW POWER ANALYSIS USE OF ADAPTIVE DATA STRUCTURES DETAILED DEFINITION OF THE METHODOLOGY DESIGN OF TOOLS SUPPORTING THE AUTOMATIC THE SEMIAUTOMATIC STEPS OF THE METHODOLOGY 22 BIBLIOGRAPHY C. Bobineau, L. Bouganim, P. Pucheral, P. Valduriez, "PicoDBMS: Scaling down Database Techniques for Smart card," Proc. 26 th Int. Conf. on Very Large Databases (VLDB), 2000, pp C. Bolchini, F. Salice, F. Schreiber, L. Tanca, "Logical and Physical Design issues for Smart Card Databases", Technical Report no , Politecnico di Milano, to be published in ACM-TOIS. C. Bolchini, F. Salice, F. Schreiber, L. Tanca, "Logical and Physical Design for Very Small Databases", Proc. SEBD 02, Portoferraio, june 2002 W. Rankl, W. Ewffing, Smart Card Handbook Second Ed., Wiley,

13 Personal Portable Folder Information System owns CIVIL AVIATION AUTHORITY TRAFFIC CONTROL AUTHORITY DRIVING LICENSE number MUNICIPAL REGISTRY birth fam. name place birth SSN first name death PERSON citizenship domicile death place residence degree height hairs possesses profession eyes colour colour ELECTORAL CERTIFICATE code holds civil rights HEALTH AUTHORITY code of choice agency vaccinated VACCINATION type fam. name first name FAMILY DOCTOR issues PRESCRIPTION state code manages MEDICAL RECORD ups outcome DRUG EXAM HOSPITALIZATION state has SSN fam. name first name son birth citizenship place OWNER birth (PERSON) residence has has signaturecivil photo record place spouse status state IDENTIFICATION MEDICAL time min. Max. visit has BLOOD code PRESSURE VACCINATION MEDICAL RECORD chronic rate type state (LOCAL) diseases blood group allergies organ donation takes out company POLICY number premium suffering from LIFE FAMILY PUBLIC LIABILITY INSURANCE COMPANY (1 N) owns start end has license CAR PUBLIC LIABILITY number limitations place of issue type LICENSE of issue fam. name SSN first name JUDICIAL REGISTER authority action JUDICIAL AUTHORITY agency RENEWAL dependent on undergoes SUSPENSION start action end DRIVING/FLYING LICENSES subjected to takes out POLICY number name dose type result facility cause on smart-card on external media virtual relationship INSURANCE COMPANY 1 INSURANCE COMPANY N 24 13