UNIVERSITY OF CALIFORNIA. Los Angeles. Development of. Statistical Online Computational Resources. and Teaching Tools

Transcription

1 UNIVERSITY OF CALIFORNIA Los Angeles Development of Statstcal Onlne Computatonal Resources and Teachng Tools A thess submtted n partal satsfacton of the requrements for the degree Master of Scence n Statstcs by Dushyanth Krshnamurthy 2005

2 The thess of Dushyanth Krshnamurthy s approved. Ivo Dnov Rob Gould Yngnan Wu Jan de Leeuw, Commttee Char Unversty of Calforna, Los Angeles 2005

3 TABLE OF CONTENTS 1. INTRODUCTION Statstcs Onlne Computatonal Resource (SOCR) Desgn & Implementaton Organzaton of the thess PROBABILITY DISTRIBUTION MODELING Desgn of SOCR Modeler Implemented Modelers Goodness of Ft Data Generaton STATISTICAL ANALYSIS Software Desgn of SOCR Analyss Implemented Analyss Tools GAMES Introducton Wavelet Transforms Compresson Usng Wavelets APPLICATIONS Overvew of LONI vsualzaton envronment Image Delneaton Algorthm Implementaton FUTURE DIRECTION...40 References...41

4 ACKNOWLEDGEMENTS I would lke to thank my advsor, Dr. Ivo Dnov, for havng gven me the opportunty to work wth hm. Hs advce and gudance has helped durng my study here. I am also grateful toward the unversty and my advsor for havng supported me fnancally throughout my stay at UCLA. I owe my commttee a specal thanks for ther contrbuton to ths thess. Fnally I would lke to thank my famly for ther help and support throughout my stay at UCLA. v

5 ABSTRACT OF THE THESIS Development Of Statstcal Onlne Computatonal Resources And Teachng Tools By Dushyanth Krshnamurthy Master of Scence n Statstcs Unversty of Calforna, Los Angeles, 2005 Professor Jan de Leeuw, Char The advent of technology has facltated computer based technques for statstcal analyss of data. In addton, the Graphcal User Interface (GUI) has enabled the development of hghly nteractve programs. Statstcal Onlne Computatonal Resources (SOCR) s a herarchy of portable onlne nteractve software that can be accessed through the nternet. It combnes statstcal data analyss tools wth nteractve demonstratons, smulatons, and GUIs desgned to supplement methodologcal tranng n statstcs classes. In ths thess we present three categores of tools that form part of the SOCR onlne repostory. For fttng data to probablty dstrbuton models, the SOCR Modeler s employed. Features are provded for loadng data, fttng varous models, and checkng the goodness of ft usng vsual and quanttatve results. An example s v

6 provded to show the applcaton of the Mxed Modeler tool to a medcal magng applcaton. Next, the SOCR Analyss tool provdes an nterface for performng statstcal analyss of data. We present a general framework for the SOCR Analyss tool and study the mplementaton of Aanalyss of Varance (ANOVA) as a specfc type of analyss tool. To mprove the understandng of statstcal concepts, SOCR Games provdes a set of nteractve software wth llustratons. We present a demonstraton of Fourer and wavelet transforms as part of the SOCR Games package. The effect of data compresson can be explored vsually for varous wavelet transforms. v

7 1. INTRODUCTION 1.1 Statstcs Onlne Computatonal Resources The Statstcs Onlne Computatonal Resource (SOCR) seeks to comple a herarchy of portable onlne nteractve ads for motvatng, modernzng and mprovng the teachng format n college-level probablty and statstcs courses. Ths ncludes a repostory of software tools, nstructonal materals and onlne tutorals. The SOCR resources allow nstructors to supplement methodologcal course materal wth hands-on demonstratons, smulatons, nteractve graphcal dsplays llustratng n a problem-drven manner the presented theoretcal and data-analytc concepts. In ths thess we desgned a number of applets, user nterfaces and demonstratons, whch are fully accessble over the nternet as part of the SOCR project. 1.2 Desgn & Implementaton The prmary contrbuton of ths thess s the desgn of the software modules that would allow flexblty n addton of statstcal experments and the desgn of ntutve GUI s to help understand statstcal concepts. SOCR s desgned as a set of stand alone applets, each demonstratng a partcular topc n statstcs. Each applet s composed of a top level component that s responsble for the dsplay and manpulaton of data. In addton each applet contans a number of sub components that are responsble for a partcular experment or demonstraton. Ths desgn allowed flexblty n addng new demos and 1

8 experments whle savng tme by groupng the common functonaltes nto the hgher level component. 1.3 Organzaton of the thess The remander of the thess s organzed as follows. We examne modelng of probablty dstrbuton functons usng datasets n secton 2. The desgn for the software package s dscussed followed by examples of actual probablty dstrbuton modelers and ther mplementaton. In secton 3 we examne the mplementaton of the Analyss package. We present Analyss of Varance (ANOVA) and Regresson Analyss at two examples of the Analyss tools package. In secton 4 we dscuss the mplementaton of wavelet transforms and Fourer analyss as part of the games package. Fnally we dscuss some applcatons of SOCR and suggest some future developments. 2

9 2. PROBABILITY DISTRIBUTION MODELING In statstcs, we are often gven a sample dataset that s known or assumed to be generated from some probablty dstrbuton. We are then requred to form statstcal nferences regardng the nature of the probablty dstrbuton that generated the samples. In some cases the type of dstrbuton wll be known and the parameters of the dstrbuton wll be unknown. In other cases both the type of dstrbuton and the parameters wll be unknown. In both cases there wll be parameters assocated wth the dstrbuton functon that need to be estmated. For many dstrbutons the parameter of nterest wll belong to some known set Λ. Gven a set of observatons from the dstrbuton we wll be able to estmate the parameter from the observatons usng statstcal technques that ether mnmze some crteron or have certan propertes such as producng unbased parameter estmates. Estmaton technques could be closed form solutons or teratve procedures. Detals of parameter estmaton technques can be found n [1,2]. If the dstrbuton s known a pror the problem s smply one of estmatng the parameters assocated wth the dstrbuton, assumng that the parameters are estmable. In other cases where the data comes from a completely unknown dstrbuton we could try fttng a number of dstrbutons to the data and use statstcal tests and vsualzaton to test the goodness of the model ft. The SOCR Modeler s a useful resource for unvarate dstrbuton modelng problems and provdes the followng features: 3

10 An easly accessble onlne system for fttng varous probablty dstrbuton models to data. Data n the from of a hstogram or as raw data. A Graphcal User Interface (GUI) for vsualzaton of the model ft. Data generaton technques for generatng random samples from common dstrbutons. Goodness of ft test for checkng qualty of ft. In the followng sectons we dscuss the desgn of the software module followed by some examples of actual probablty dstrbuton modelers. 2.1 Desgn of SOCR Modeler The SOCR Modeler was desgned wth a some key features n mnd such as nteractve GUIs, easy data manpulaton and the ablty to add new modelng plug-ns as they became avalable. The SOCR modeler desgn s shown n Fgure 1. It conssts of a core module that s responsble for the common functonalty of the modelers, and plug-ns that mplement functonalty specfc to each modeler such as parameter estmaton and result generaton. Core Components The bulk of the code s contaned n the core. There are four functonal modules wthn the core ncludng the GUI components, data management module, modeler nterface and 4

11 CORE PLUG-INS GUI COMPONENTS DATA MANAGEMENT NORMAL FIT MODELER BETA FIT MODELER MODELER INTERFACE DYNAMIC CLASS LOADER MIXED FIT MODELER Fgure 1. Block dagram of SOCR Modeler showng functonal components. the dynamc class loader. Among these the frst two are hghly nterrelated and work together closely to synchronze the vsualzaton of the data. Modeler GUI: The GUI conssts of three regons as shown n Fgure 2. The left pane Fgure 2. Screen shot of SOCR Modeler GUI. 5

12 dsplays the avalable modelers and the parameters assocated wth each modeler. The top porton contans a common toolbar and the remanng porton conssts of tabbed panes that dsplay the nputs and outputs. The frst tab (Data) dsplays a table for enterng data ether as hstogram data or as raw data. The second tab (Graphs) s shown n Fgure 3 and dsplays the graph used for nputtng and dsplayng the hstogram of the data. The model ft s dsplayed as a red lne supermposed over the hstogram. The probablty model can be plotted n ts orgnal scale or scaled up proportonal to the hstogram. The thrd tab (results) s shown n Fgure 4. and dsplays the results of the model ft. The fourth tab Fgure 3. Illustraton of SOCR Modeler graph showng normal dstrbuton model ft. 6

13 (About) dsplays help nformaton about the modeler and s currently under development. The fnal tab (Data Generaton) allows data generaton and s dscussed later n ths secton. Data Management: Data management conssts of the Input/Output methods for the data and the correspondng data objects. Data nput can take place through the keyboard, mouse, fle or from the system clpboard. The GUI objects used for representng and manpulatng data are the table and the graph tabs (Fgure 2 and 3). Data s entered drectly nto the keyboard ether as raw data or as a hstogram. In addton, tab separated text fles can be loaded usng the FILE OPEN button. Data can be Fgure 4. Example showng SOCR Modeler results panel. 7

14 coped from applcatons such as excel and pasted nto the table usng the PASTE button. Also, clckng the mouse on the graph panel changes the levels of the hstogram. There are three data objects for storng the data nternally. One for storng the raw data values, a second for storng the hstogram values and a thrd for storng the values estmated from the model ft. When a change s made to the data usng one of the nput methods dscussed above, a synchronzaton mechansm s trggered that converts data from the raw format to the hstogram format or vce versa dependng on the nput mechansm. Also, changes to the data recalculates the model ft usng the new data. Modeler Interface: The modeler nterface s a java class that behaves as an ntermedary between the core and the mplementatons of the modeler plug-ns. The core components send ther request for data or calculatons to ths nterface. The nterface then passes on ths request to one of the modelers based on the current selecton. For example, f the core component requres the ftted model values, t requests ths from the nterface. If the currently selected modeler s the normal ft modeler, the nterface wll send ths request to the Normal Ft Modeler whch then returns the parameter estmates to the core va the nterface. In addton, the nterface specfes certan standards that must be followed by all the modeler plug-ns. Dynamc Class Loader: When a user selects a specfc modeler through the GUI, t must be loaded at run tme and an assocaton formed between the nterface and the selected modeler plug-n. Ths functonalty s taken care of by the dynamc class loader. 8

15 The class loader along wth the nterface serve to separate the specfcatons from the functonalty of each plug-n. Ths allows easy addton of new probablty modeler plugns as long as they conform to certan standards specfed by the Modeler Interface. Ths also allows the addton of new modelers wthout recomplaton of core components. SOCR Plug-ns: The SOCR plug-ns perform parameter estmaton and provde model values. The SOCR plug-ns need to mplement the specfcatons set by the Modeler Interface for data nput and output. They are also gven access to the left pane of the GUI (Fgure 2 - left) for gettng user nput. All probablty modelers must have functons that takes raw data as nput, estmates ts parameters and returns the ftted model values. 2.2 Implemented Modelers Each model s provded wth unvarate nput data and returns a probablty dstrbuton functon (pdf) model after performng an estmate of the parameters assocated wth the model. In ths secton we dscuss some examples of modelers that have been mplemented as part of the SOCR Modeler package. Normal Modeler The normal dstrbuton s one of the most commonly used dstrbutons that arses n a number of applcatons. Detals can be found n [3,4]. The dstrbuton s completely specfed by ts mean µ and standard devaton σ. The pdf of a normal random varable s gven by 9

16 1 2 ( x µ ) 2 2σ f ( x; µ, σ ) = e (1) 2 2πσ Gven a data set Y, K 1,Y N we obtan pont estmates of the parameters from the data usng the sample mean and standard devaton usng the formula The standard devaton can be estmated as N 1 N Y = 1 ˆµ = (2) N ˆ 1 ( Y ˆ ) 2 µ = σ = (3) N 1 Equaton (3) gves an unbased estmate of the true standard devaton. We provde the user wth the opton of settng the mean and varance of the ft manually or allowng the system to estmate the parameters for the models usng equatons (2) and (3). The resultng ft s super mposed over the hstogram of the data. Fgure 3 shows an example of a normal ft where the parameters have been estmated. Fgure 5 shows the same data wth a user defned mean and varance. 10

17 Fgure 5. Example of normal dstrbuton ft wth user specfed nput parameters. Exponental Modeler The exponental dstrbuton has only one parameter called the rate parameter λ. The pdf s gven by λe f ( x; λ) = 0 λx x 0 otherwse (4) where λ s greater than zero. It s often used n applcatons to model the successve rate of occurrences of events such as arrvals of customers at a faclty. 11

18 Gven a data set Y,,Y N we can obtan pont estmates of the parameter from the data 1 K usng the closed form equaton ˆ 1 λ = (5) ˆ µ where µˆ s the sample mean. Fgure 6 shows an example of an exponental model ft. Fgure 6. Example of exponental dstrbuton model ftted usng estmated parameters. Mxed Modeler In certan applcatons there are measurements consstng of data from multple sources, each source havng a dfferent dstrbuton, dfferent parameters or both. In these applcatons there s an addtonal latent varable assocated wth each data pont 12

19 ndcatng the source t belongs to. A common example s measurements of heghts from a populaton of males and females. Ths can be modeled as a mxture of two normal varables. In ths case the latent varable assocated wth each measurement s the sex of the speces. In addton, there s also a varable that measures the proporton of males and females n the populaton. The latent varable s not always measurable and n these cases they need to be estmated along wth the other parameters. In ths secton we descrbe the mplementaton of a Mxed Modeler that fts a n normal mxture model to a gven data set. The algorthm s mplemented usng the expectaton maxmzaton (EM) algorthm that performs an teratve search to fnd an optmal soluton for the unknown parameters. We start by descrbng the normal mxture model, derve expressons for the Maxmum Lkelhood estmate of the unknown parameters and fnally dscuss the mplementaton of the EM algorthm for estmatng these parameters. Normal Mxture Model: A normal mxture model X s a contnuous random varable that conssts of n normal random varables. The pdf of the normal mxture model s gven by n 1 = PX ( x) λ j X j= 0 j (6) where 1 2 ( x µ j ) 2 2σ j X j = e (7) 2πσ 2 j 13

20 and λ j =1. Hence the normal mxture can be descrbed as a sum of weghted normal random varables where the λ j ' s denote the weght of each normal component. We can denote the set of parameters assocated wth the normal mxture by λ0 µ 0 σ 0 Θ = M λn 1 µ n 1 σ n 1 (8) Data Set: Gven a set of N measurements Y from a normal mxture dstrbuton, we can defne a set of N latent varables Z, = 1, K, N assocated wth each Y takng values 0, L, n 1 representng the normal component of the mxture that data Y belongs to. Gven latent varables Z, we can defne the pdf of Y by P Y ( y ; Z j, µ, σ ) 2 j 1 2 ( x µ j ) 2 2σ j = j j = e (9) 2πσ ML Estmates: In ths secton we derve ML estmates for the parameters n the vector Θ 0,, l n 1. Let us defne a set of varables l K where l s the number of latent varables equal to. We can form a vector from the observed values denoted as Y form a vector of latent varables denoted by Z. The ML estmate of the parameter vector Θ s then gven by 14

21 (10) ) ; ( arg max ln ˆ Θ = Θ Θ Y P ml The values of Θ that maxmzes the above expresson can be calculated by dfferentatng the functon wth respect to Θ and equatng the result to zero as shown n equaton (). 0 ) ; ( ln = Θ Θ Y P (11) The lkelhood functon n ths case s gven by = = = 1} : { 2 ) ( 2 1 0} : { 2 ) ( ) ( n Y Z Y Y Z Y l n l n n n n e e L σ µ πσ σ µ πσ λ λ θ L K (12) Takng log of the lkelhood functon, dfferentatng the result wth respect to the parameters n, and equatng to zero yelds the followng ML estmates for the weghts of the normal components as Θ N l = λ (13) Smlarly we can also derve expressons for the estmates of the mean as k k Z k l Y = = } : { ˆµ (14) and for the varance as k k Z k k l Y = = } : { 2 2 ) ˆ ( ˆ µ σ (15) 15

22 E-M Algorthm: When the latent varables s known, the estmaton of parameters Z assocated wth each normal component s straghtforward and can be estmated usng equatons (13) to (15) above. In other cases when Z s unknown or mssng, equatons (the ones just above) cannot be drectly used to obtan estmates of the parameters. In these cases the Z ' s need to be estmated along wth the parameter vector Θ. We can rewrte equaton (11) as ln P( Y Θ Θ) = E { ln P( Y, Z / )} Z ~ P( Z / Y, Θ) Θ Θ (16) and equate to zero. The estmaton of Z n the above equaton s called the E-step of the algorthm and s an expectaton operaton. Fndng parameter estmates that maxmze P ( Y, Z; Θ) s called the M-step. In the M-step of the algorthm we dfferentate the term nsde the expectaton by equate t to zero. The EM algorthm works n followng way. 1. An ntal estmate of the parameter vector calledθ 0 s provded to the algorthm. 2. The estmate of the parameter Θt at the t th teraton s used n calculatng the expected value of each Z (E-step). (t) 3. The estmates of Z and Θ are used to obtan the values of Θ t t+1 usng equatons (13) to (15) (M-step). 16

23 The EM algorthm converges to a soluton by alternatng between the E-step and the M- step. The ntal estmate Θ needs to be close to the true parameter values to ensure the 0 algorthm converges to the global maxmum. Fgure 7 shows the result after forty teratons when the algorthm has converged to a local maxmum. Fgure 8 shows convergence of the same data set wth dfferent ntal values. We see that the soluton has converged to the global maxmum. Detals of EM algorthm can be found n [5] and [6]. Fgure 7. Example of mxed model ft wth two components showng convergence to local maxmum. 17

24 2.3 Goodness of Ft Once the model parameters have been estmated we need a method for testng how well the model fts the data. Such a measure can be obtaned by usng the ch square Goodness-of-Ft test. An example of the ch-square dstrbuton can be found n [7] and detals of the test can be found n [3,8]. Ths testng method s sutable only for categorcal data. Hence we splt the data nto categores based on the hstogram of the data. Gven n observatons, and k categores, we can denote the number of observatons n each bn by where = 1, K, k. We denote the probablty of a data pont occurrng n n the th bn by p. Ths can be calculated from the pdf of the dstrbuton. The expected number of data ponts n the th bn s then gven by np. The ch-square statstc measures the devaton of the expected number of data ponts from the observed number of data ponts. The ch-square test statstc s then gven by k = 1 ( n np ) 2 χ = (17) np The ch-square statstc s compared wth a ch-square dstrbuton wth 2 ( k 1) degrees of 18

25 Fgure 8. Example of mxed model ft wth two components showng convergence to global maxmum. 2 freedom and a p-value s calculated based on the area to the rght of χ. Ths test does not depend on the dstrbuton and can be mplemented easly for varous dstrbuton modelers but the test results depends on the number of bns n the hstogram and may vary accordngly. 2.4 Data Generaton The SOCR Modeler has the provson to generate sample data from varous dstrbutons. Ths s helpful as a teachng ad and can be used to demonstrate the fttng of the models usng generated samples. Fgure 9 shows the data generaton tab. The panel allows one to 19

26 Fgure 9. Screen shot of data generaton panel of SOCR Modeler. select a dstrbuton, set the parameters and select the number of samples requred. The data s then generated as a pseudo random sequence from the dstrbuton and placed n the table. 20

27 3. STATISTICAL ANALYSIS Statstcal analyss deals wth the analyss of sample datasets from a populaton to arrve at conclusons or nferences about the underlyng populaton. There are a number of statstcal analyses methods such as hypothess testng, regresson analyss, analyss of varance (ANOVA), analyss of categorcal data and so on. These tests are qute often performed on data measurements obtaned from expermental setups. There are a number of factors n decdng on an approprate method of analyss dependng on factors such as number of populatons beng analyzed, whether the datasets are numercal, categorcal or a mx of both and more mportantly on the expermental setup. The SOCR Statstcal Analyss package conssts of applets for analyzng quanttatve datasets, performng tests on the data and dervng statstcal nferences about the data. The SOCR Analyss tool provdes a graphcal user nterface for analyzng datasets as well as examples for students to gan a better understandng of the applcaton of statstcal analyss. 3.1 Software Desgn of SOCR Analyss The desgn of the SOCR Analyss module s smlar to the SOCR Modeler. Fgure 10 shows a block dagram of the SOCR Analyss module. There s a core component that handles the data and GUI functonalty but unlke the SOCR Modeler the SOCR Analyss 21

28 GUI COMPONENTS CORE DATA MANAGEMENT ANALYSIS PLUG-INS ANOVA ANALYSIS INTERFACE DYNAMIC CLASS LOADER REGRESSION PRE-PROC INTERFACE T-TEST PRE PROCESSOR PLUG-INS DEFAULT BRAIN IMAGE INTENSITY IMAGE BASED VOLUM ES Fgure 10. Block dagram of SOCR Analyss showng functonal components. has two sets of plug-ns and nterfaces. One set of plug-ns called the Preprocessor plugns, handles the pre processng of the data whle the other set of plug-ns called Analyss plug-ns, handles the statstcal analyss of the data. Whle the focus of the desgn n the SOCR Modeler s on the data handlng by the GUI components, the focus of the desgn n the SOCR Analyss s on data formattng and processng. Analyss GUI: The Analyss GUI s shown n Fgure 11. The GUI desgn s relatvely 22

29 Fgure 11. Screen shot of SOCR Analyss graphcal user nterface. smple when compared wth the SOCR modeler GUI. The toolbar at the bottom contans most of the common functonalty. The tabbed panels conssts of a table, graph panel, results and mappng tabs. The table s the only nput mechansm for the module and can be done ether through the keyboard or from a text fle. Results are dsplayed as text n the results panel. The graph panel s avalable for the analyss plug-ns to dsplay results when avalable. The mappng panel s shown n Fgure 12. The mappng panel can be splt nto three regons. The top porton s the preprocessng panel. It s responsble for dsplayng the data formattng and pre-processng optons for the data. The center porton 23

30 called the analyss panel dsplays the optons for the Analyss plug-ns. The bottom porton dsplays the flters that can be appled to the data. Fgure 12. Screen shot showng compute panel of SOCR Analyss. Data Processng Overvew: Data processng and handlng takes place n multple stages both n the core as well as the plug-ns. An overvew of the data processng s shown n Fgure

31 External resources START Table data Preprocessor plug -n Processed data END Result Analyss Plug - n Column mappng Fgure 13.Block dagram of SOCR Analyss data flow process. The processng of data can be summarzed n the followng steps. 1.Data s ether entered nto the table or loaded drectly from a fle. 2.Once a pre-processor has been selected from the pre-processor panel, the data s passed to the pre-processor plug-n. 3.Dependng on the type of preprocessor plug-n selected, the data s processed to yeld a new set of columnar data. The pre-processor output s n the form of columnar data. The column headngs are dsplayed n the selecton box as avalable nputs. 4.The user selects some or all the columns for processng by the analyss plug-ns. The number of columns that can be selected depends on the type of analyss plug- 25

32 n selected. For example, f ANOVA s selected, the current ANOVA plug-n allows one dependant varable and up to 3 ndependent varables to be selected. 5. Once the Compute button s clcked the selected columns are passed nto the selected analyss plug-n where the statstcal analyss s performed and the results sent back to the Core for dsplay. Pre-processor Plug-ns: The data entered nto the GUI table need not be data values. They could be any type of alpha-numerc nformaton that can be understood by the preprocessor plug-n. In addton, the preprocessor plug-n can request user nput through the pre-processor panel, and has access to the fle system. Ths gves great flexblty n the desgn of a pre-processor. For example, one of the table columns could hold mage locatons and a preprocessor could be desgned that reads these mage fles and extracts nformaton from the mages for processng by the analyss plug-ns. The only requrement of a preprocessor s that they have a standard nput/output data format. Analyss Plug-ns: Once the data s converted to the standard form of a number of columns, these columns are dsplayed as a selecton opton as shown n Fgure 12. The GUI then allows the user to select the approprate processed data columns dependng on the Analyss plug-n settngs. Each analyss plug-n performs a standard task such as ANOVA, hypothess testng etc. The analyss plug-n has access to the results panel and the graph panel of the GUI for dsplayng the results. 26

33 The separaton of the statstcal analyss from the pre-processng steps s very useful for a number of applcatons and a good example of software reuse. Ths allows us to wrte statstcal analyss routnes such as ANOVA once and use t many tmes for dfferent applcatons by changng the pre-processng steps. Another advantage s the specfcaton of a standard data nput/output format that allows seamless ntegraton of the preprocessng and statstcal analyss steps. Currently, only a default pre-processor exsts that passes the data to the analyss plug-n wthout any changes. An magng preprocessor s currently under development. 3.2 Implemented Analyss Tools In ths sub-secton we dscuss an analyss plug-ns that was mplemented as part of the analyss package. Analyss of Varance It s a commonly used tool for analyzng categorcal expermental data. Detals can be found n statstcs text books and research papers [3,9]. To perform an ANOVA data consstng of a dependant varable whch s numerc and one or more ndependent varables. For example the dependant varable could be a response or output of a system whle the ndependent varables could be the settngs of the system. The results of the ANOVA calculaton s dsplayed n a tabular format (Fgure 14). 27

34 Fgure 14. Example of analyss of varance results. 28

35 4. GAMES 4.1 Introducton The SOCR Games package has a number of nteractve demos and games for explanng statstcal concepts. In ths secton we dscuss the mplementaton of a Fourer game and a Wavelet game as part of the SOCR Games package. The Fourer Game s a modfed verson of open source software that has been modfed to ft the SOCR Games framework. The wavelet Game bulds on the Fourer framework and s capable of demonstratng wavelet transforms. We start ths secton wth an ntroducton to bass functons, followed by a dscusson of Fourer seres and a dscusson of the mplementaton and functonalty of the Wavelet Game. Bass Functons The uses of bass functons arses from the need to represent sgnals (ether real valued or complex valued) as a lnear combnaton of some known sgnals wth certan propertes. From a lnear algebra perspectve ths mght be accomplshed by consderng the sgnal as belongng to a normed lnear vector space. If such a space can be found, we mght be able to represent the sgnal as a weghted combnaton of some bass functons of the vector space. An ntroducton to vector spaces and bass functons can be found n [10]. 29

36 Fourer Seres John Baptst Fourer developed the theory of Fourer seres as a way of representng perodc sgnals as a sum of trgonometrc seres. A sgnal x(t) s sad to be perodc, f for some postve value of T, x ( t) = x( t + T ) for all t. (18) The fundamental frequency for such a sgnal s gven by ω0 = 2π T. Under certan condtons a perodc sgnal can be represented by a sum of exponental bass functons that have been shfted and scaled. The exponental bass functons are of the form jkω t φ ( t) = e o, k = 0, ± 1, ± 2,K k (19) The sgnal x(t) can then be represented by k= jkωot x( t) = a e (20) k where a k are the coeffcents or the weghts of the bass functons. Ths type of representaton s called a Fourer seres expanson and s useful n analyzng the spectral content of sgnals. In addton to beng perodc, the functon x(t) must satsfy a number of condtons for the Fourer seres representaton to exst. These are called the Drchlet condtons and can be found for example n [11]. Assumng that a the representaton n equaton (20) exsts, the coeffcents are gven by a k 1 T jnω t = x( t) e o dt (21) 30

37 The Fourer coeffcents are complex valued scalars and are gven by the nner product of the sgnal wth the bass functons. Snce the coeffcents are complex we can splt them nto a phase part and magntude part. Fgure 15 shows a screen shot of the Fourer Game. Fgure 15. Example of Fourer transform of square wave. The top porton of the GUI dsplays the perodc sgnal. In ths case t s a square wave. The mddle porton shows the magntude of the Fourer coeffcents and the bottom shows the correspondng phases. The user can adjust the magntude and phases to see the changes n the sgnal or redraw the sgnal usng the mouse to see how t affects the Fourer coeffcents. 4.2 Wavelet Transforms Wavelets are scaled and shfted versons of a mother wavelet functon.. Wavelet transforms decompose a sgnal down nto ts basc consttuent components but nstead of 31

38 representng the sgnal as a seres of sne waves of dfferent frequences the wavelet transform represents sgnals as lnear combnatons of wavelet bases. The wavelet bass representaton s rregular n shape and uses compactly supported functons. Detals of wavelet transforms and ther propertes can be found n [12]. Implementaton The Wavelet Game applet contans all the code for the GUI components and ther functonalty (Fgure 16). Whenever the game needs to perform Wavelet transforms or nverse wavelet transforms, t passes the data to the wavelet transform nterface. Dependng on the plug-n that s currently loaded the approprate transform or nverse wavelet transform s performed. The plug-ns requre only two functons. WAVELET GAME GUI COMPONENTS PLUG-INS Daubeches 2 Daubeches 4 TRANSFORM INTERFACE DYNAMIC CLASS LOADER Haar Fgure 16. Block dagram of wavelet transform showng functonal components. 32

39 A transform functon that accepts a vector representng a sgnal and would return a transformed set of coeffcents. An nverse transform functon that takes a set of transformed coeffcents and returns the orgnal sgnal. Ths makes addton of wavelet transforms easy. 4.3 Compresson Usng Wavelets An applcaton of wavelet transform s data compresson. The applet demonstrates the way n whch wavelet transform compress data and how certan wavelet transforms work better on certan sgnals. Compresson of a sgnal can be acheved by takng the wavelet transform of a sgnal, and dscardng a certan percentage of the coeffcents by settng ther value to zero. Usually the smallest coeffcents are dscarded. By takng the nverse transform of the coeffcents we can study to what degree there s sgnal dstorton. Fgure 17 and 18 show wavelet transforms of a snusodal wave usng Symlett-8 wavelets and Haar wavelets respectvely. We see that when 85% of the smallest coeffcents have been removed, the nverse transform shows very lttle dstorton n the Symmlet-8 transform whereas vsble degeneraton n the Haar transform. In contrast when we use a square wave as our sgnal, we can see that the effects are reversed and the Haar wavelet gves less sgnal degradaton for the same percentage of compresson (Fgure 19 and 20). 33

40 Fgure 17. Reconstructed sne wave usng Symlet-8 wavelets wth 85% compresson. Fgure 18. Reconstructed sne wave usng Haar wavelets wth 85% compresson. 34

41 Fgure 19. Reconstructed square wave usng Symlet-8 wavelets wth 85% compresson. Fgure 20. Reconstructed square wave usng Haar wavelets wth 85% compresson. 35

42 5. APPLICATIONS The use of an object orented desgn for the packages enhances the applcablty of the SOCR modules by allowng easy ntegraton of these tools wth other java packages. We dscuss an applcaton of the Mxed Modeler ft to an Image analyss applcaton called LONI Vsualzaton Envronment (LOVE) that s currently beng developed n the Laboratory of Neuro Imagng (LONI). 5.1 Overvew of LONI vsualzaton envronment The Laboratory of Neuro Imagng's 3-D vewer, LOVE, s a versatle tool for volumetrc data dsplay and manpulaton of bran mages. LOVE s a completely portable Javabased software, whch only requres a Java nterpreter and 64 MB of RAM memory to run on any computer archtecture. It allows the user to nteractvely overlay and browse through several data volumes, zoom n and out n the axal, sagttal and coronal vews, and reports the ntenstes and the stereo-tactc voxel and world coordnates of the data. It also allows the expert users to delneate structures of nterest, e.g., sulcal curves, on the 3 cardnal projectons of the data. These curves then may be used to reconstruct surfaces representng the topologcal boundares of cortcal and sub-cortcal regons of nterest. 5.2 Image Delneaton The bran tssue s composed of whte matter, gray matter and Cerebral Spnal Flud (CSF). It s sometmes of nterest to delneate the bran mage based on the tssue type. LOVE ntegrates ths functonalty nto t s GUI (Fgure 21). The user can select crcular 36

43 regons from the bran for whch the algorthm delneates the mage based on tssue type. The pxel ntenstes for a selected regon can be modeled as samples from a weghted mxture of three normal densty functons where each pxels from each tssue type belong to one of the three normal denstes. Snce we do not know the parameters of the normal model or the tssue type each pxel represents, we can use the EM algorthm to teratvely estmate the mxed modeler parameters. Once the parameters have been estmated, we can calculate the ntersecton ponts of the normal components whch serves as hard threshold ponts to delneate the tssue types based on pxel ntenstes. 5.3 Algorthm Implementaton The left porton of the LOVE GUI (Fgure 21.) shows the selected regon of the bran n Fgure 21. LONI vsualzaton envronment showng hstogram of selected area of bran. 37

44 all three vews. The man porton of the GUI shows the hstogram of the ntenstes of the pxels from the selected regon. The user can select two ponts as a startng guess for the ntersecton ponts. The algorthm estmates the means from ths ntal guess and nputs ths nformaton along wth a startng guess for the varance and weghts to the EM algorthm. The EM algorthm runs through a number of teratons and returns the fnal estmates of the ntersecton ponts. The result of the EM algorthm s shown n Fgure 22. The estmated normal mxture model s overlad on the hstogram. There s an opton n the GUI to vsualze the resultng mxture model and the ntersecton ponts wthout the Fgure 22.. LONI vsualzaton envronment showng mxed model ft supermposed on hstogram. 38

45 hstogram. Ths s shown n Fgure 23. In addton to pckng a start pont based on user nput we randomze some of the ntal parameters a number of tmes and pck the ntal startng pont that maxmzes the log-lkelhood. Ths helps to avod convergence to a local maxmum most of the tme. Fgure 23.. LONI vsualzaton envronment showng model ft wth ntersecton ponts. 39

46 6. FUTURE DIRECTION The development of SOCR s an ongong process and s constantly beng upgraded and mproved upon. There are a number of features that are planned to be added n the near future. The categorcal ndependent varables need to be numercally specfed n ANOVA. The ablty to use alpha-numerc categores would enhance the usefulness of the applcaton. Currently, ANOVA s capable of handlng only balanced data wthout any mssng data. Regresson Analyss currently dsplays results n a tabular format. There s currently a provson for addng plots to the graph panel n the Analyss module. GUI s could be developed for dsplayng resdual plots as part of the regresson analyss results. SOCR Modeler s currently a tool exclusvely for modelng probablty dstrbuton functons. The next stage n the development of ths module would be to ntegrate bass modelers such as polynomal fts, splne fts, wavelet and Fourer transforms. The framework for wavelet transforms has been completed n the Games secton. Currently four wavelets have been mplemented. More types of wavelets wll be added to the package n the future. Fnally, the SOCR tool would beneft from a study to test ts performance. Ths could be done by testng t s speed, or by comparng t to smlar web-based tools or to some userbased benchmarks such as survey results. These addtons would help mprove the usablty of SOCR. 40

47 REFERENCES 1. M.H. DeGroot and M.J. Schervsh, Probablty and Statstcs, 3 rd ed., Addson Wesley, 2001, chap H.V. Poor, An Introducton to Sgnal Detecton and Estmaton, 2 nd ed., Sprnger- Verlag, 1994, chap. 4, pg J.L. Devore, Probablty and Statstcs for Engneerng and the Scences, 5th ed., Duxbury Press, Pacfc Grove, CA, 1999, chap. 4, chap. 10, pg , pg H. Stark and J.W. Woods, Probablty and Random Processes wth Applcatons to Sgnal Processng, 3 rd ed., Prentce Hall, 2001, chap. 2, pg J. Blmes, A Gentle Tutoral of the EM Algorthm and ts Applcaton to Parameter Estmaton for Gaussan Mxture and Hdden Markov Models, Internatonal Computer Scence Insttute and Department of Electrcal Engneerng and Computer Scence U.C. Berkeley, Aprl 1998, ICSI-TR T.K. Moon, The Expectaton Maxmzaton Algorthm., IEEE Sgnal Processng Magazne, November 1996, volume 13, ssue 6, pg AT110A_notes.dr/PPCh07.pdf, pg. 1-3,5. 9. J. Neter, M.H. Kutner, C.J. Nachtshem and W. Wasserman, Appled Lnear Statstcal Models, 4 th ed., McGraw-Hll, 1996, chap. 4, 5, pg S.H. Fredberg, A.J. Insel, L.E. Spence, Lnear Algebra, 3 rd ed., Prentce Hall,1997, chap. 6, pg A.V. Oppenhem, A.S. Wlsky and S.H. Nawab, Sgnals & Systems, 2 nd ed., 1997, chap. 3, pg I.D. Dnov, J.W. Boscardn, M.S. Mega, E.L. Sowell,A.W. Toga, A Wavelet- Based Statstcal Analyss of fmri data: I. Motvaton and Data Dstrbuton Modelng, n press, NeuroInformatcs, Humana Press,