Discovery-based Edit Assistance for Spreadsheets

Transcription

1 Discovery-based Edit Assistance for Spreadsheets Jácome Cunha Departamento de Informática Universidade do Minho Portugal Join work with João Saraiva (DI/UM) and Joost Visser (SIG) VL-HCC Corvallis, OR, USA, September 23, 2009

2 1.1 Motivation Example: Property Renting System Each row represents a renting transaction It includes information about properties, owners and clients It also contains dates and prices (formulas) Row 3 says that john has rented a property owned by tony, with address 5 Novar Dr., price per day 70, between dates 9/1/01 and 9/1/02 and paid for it.

3 1.2 Motivation Redundancy This unstructured model is valid and serves its purpose But, it contains data redundancy For example, the rent per day is repeated several times

4 1.3 Motivation Updating Problems As a result, updates can cause data inconsistence For example, updating the renting value of property pg4 must be performed in several places

5 1.4 Motivation Deleting Problems Deleting rows can be problematic, too For example, deleting the row 5 will remove all the information about property pg36

6 1.5 Motivation Modern Programming Environments Naive text editors are now replaced by powerful programming language environments (ex. Eclipse) They are specialized for the programming language under consideration and help the user throughout the editing process Knowing the language under consideration, they can detect features of the programs being edited that, for example, violate the properties of the underlying language They guide users in writing correct programs

7 1.6 Motivation This Talk In this talk, I will show how we can use well-known database techniques and programming language techniques to force users to correctly update and delete spreadsheet data

8 2.0 Overview of our Approach Infer functional dependencies (Fun Algorithm) Normalize (3NF) those functional dependencies Create a Relational Database (RDB) schema (3NF) Embed that schema in the spreadsheet Spreadsheet dependency mining and schema synthesis Relational Database Schema User Embed schema as formulas and visual objects

9 3.1 Functional Dependencies Inference Functional Dependencies A Functional Dependency (FD) denoted A B means that an element of A is uniquely associated with an element of B For example, the following spreadsheet data defines the functional dependency clientnr cname

10 3.2 Functional Dependencies Inference The Fun Algorithm The Fun algorithm (Novelli et al.) computes FDs from data It computes all the FDs defined in the data, even the ones that are not so intuitive For our example, it returns: ownerno oname totaldays clientno, cname propertyno paddress, rentperday, ownerno, oname...

11 3.3 Functional Dependencies Inference Normalize Functional Dependencies Synthesize algorithm (Maier) calculates a 3NF set of FDs It returns a set of attributes with a set of candidate keys It is necessary to choose one candidate key to be the primary key We consider some particularities from spreadsheets: Formulas: PKs can not be formulas Single value columns: too many FDs columns with only one value appear in Semantics of labels: id, number, nr, code maybe good pointers to primary keys Column arrangement: we assume that the PK is before the rest of the columns

12 3.3 Functional Dependencies Inference Normalize Functional Dependencies For our example, the functional dependencies/schema generated is: owners ownerno oname clients clientno cname properties propertyno paddress, rentperday, oname

13 4.0 Spreadsheet Programming Environments The spreadsheet programming environments have several features: Auto-completion of column values Non-editable columns Safe deletion of rows Standard editing

14 4.1 Spreadsheet Programming Environments Auto-completion of Column Values The spreadsheet environment will not allow the user to introduce two properties with the same number. Instead, it offers a list of possible values to choose from: We use the notion of FD and primary key to determine the value of some columns in the spreadsheet.

15 4.1 Spreadsheet Programming Environments Auto-completion of Column Values For example, the value of the property number (propno) determines the values of the address (paddress), rent per day (rentperday ), and owner name (oname). Consequently, the spreadsheet environment is able to automatically fill in the values the corresponding columns.

16 4.1 Spreadsheet Programming Environments Auto-completion of Column Values From the relational model inferred in the original spreadsheet we generate a set of spreadsheet formulas. Consider the FD ownerno oname. In our spreadsheet, ownerno is in column J and oname in column K. We introduce in column K the following formula: S (K, r) = if (isna (vlookup (Jr, J2 : K (r 1), 2, 0)), "", vlookup (Jr, J2 : K (r 1), 2, 0)) This formula tests if there is a value introduced in column J. In the case of a value is chosen, it searches the corresponding value in the column K and shows it.

17 4.2 Spreadsheet Programming Environments Non-editable Columns Non-primary key columns become non-editable It prevents the end-user to introduce potential incorrect data and, thus, producing update anomalies

18 4.3 Spreadsheet Programming Environments Safe Deletion of Rows To correctly delete rows in the spreadsheet, a button per row is added When the user is removing important information, this button warns him of such action giving opportunity to continue or stop the action

19 4.4 Spreadsheet Programming Environments Standard Editing The spreadsheet programming environment provides a mechanism to enable/disable the advanced features. When the user disables those features he his able to introduce data that violates the (previously) inferred relational model. When the user enables the features, the system infers a new relational model that has to be obeyed in future advanced interactions.

20 5.0 Conclusions We used data mining and database techniques to analyze and create spreadsheet programming environments We have defined Spreadsheet Programming Environments and showed how such environments can be automatically derived from the spreadsheet data The spreadsheet environment guides the user in introducing correct data We would like to test this techniques with real users

21 5.0 Conclusions We have derived a relational model and embed it as formulas and visual objects

22 5.0 Conclusions

23 6.0 Metrics from the EUSES Corpus The EUSES corpus was conceived as a shared resource to support research on technologies for improving the dependability of spreadsheet programming. It contains more than 4500 spreadsheets gathered from different sources and developed for different domains. These spreadsheets are assigned to eleven different categories. Among the spreadsheets in the corpus, about 4.4% contain macros, about 2.3% contain charts, and about 56% do not have formulas being only used to store data.

24 6.0 Metrics from the EUSES Corpus In our preliminary experiment we have selected the first ten spreadsheets from each of the eleven categories of the corpus. The number of spreadsheets per category present in the EUSES corpus, selected and processed in the evaluation. category corpus selected processed cs database filby financial forms grades homework inventory jackson modeling personal total