Efficiency and design optimization

Transcription

1 Efficiency and design optimization Tuesday, Lecture 3 Jeanette Mumford University of Wisconsin - Madison Thanks to Tom Liu for letting me use some of his slides!

2 What is the best way to increase your power?

3 Your N is fixed what do you Give up? do? Let s look at the variance! Lowering the variance will increase our power

4 This lecture s focus X Improvements during study design Y GLM T

5 Efficient designs will increase power Lowering variance increases power

6 Efficient designs will increase power Lowering variance increases power 2 MFX = c(x 0 X) 1 c 0 2 w + 2 b

7 Efficient designs will increase power Lowering variance increases power 2 MFX = c(x 0 X) 1 c 0 2 w + 2 b Between-subject variance (probably unknown)

8 Efficient designs will increase power Lowering variance increases power 2 MFX = c(x 0 X) 1 c 0 2 w + 2 b Within-subject variance (probably unknown)

9 Efficient designs will increase power Lowering variance increases power 2 MFX = c(x 0 X) 1 c 0 2 w + 2 b Design matrix and contrasts (We have total control over this!!)

10 Efficiency E ciency = 1 c(x 0 X) 1 c 0

11 Interpreting efficiency Relative measure Rank designs and pick the one with highest efficiency (lowest variance)

12 Motivating illustration Let s assume we re working with convolved regressors More on this later! Two stimuli (positive and negative images) Stimulus duration = 1s ISI = 2s (fixed) Contrast of interest: positive negative How much can I increase efficiency by searching over random trial orders?

13 Motivating illustration Within- and between-subject variance estimates from real data Typically you won t be able to do this!! within-subject variance = 3036 between-subject variance = 810 N=30 What is the power as a function of effect size? alpha = 0.05

14 Motivating illustration Power for N=30 Power Most efficient Least efficient Effect Size

15 Motivating illustration Power for N=30 Power Most efficient 81% v 63% power Least efficient Effect Size

16 Things that might increase efficiency Random trial order Jittering your ITI Increasing your ITI Lengthen stimuli Increase number of stimuli

17 Which is the best design? It depends on the experimental question. E Slide courtesy of Tom Liu

18 What s to come What design is best? What is Estimation Efficiency vs Detection Power? You need to know to make educated decisions when choosing software What about efficiency for multiple contrasts? Is there a way to speed up the process?

19 Considerations Statistical efficiency Psychological considerations How you plan on modeling the data

20 Estimation vs Detection Pile of sand example

21 Estimation vs Detection Pile of sand example You can detect that something is there but what??

22 Estimation vs Detection Pile of sand example Zoom in and you can see a little better, but you can t be sure about what exactly is in here

23 Estimation vs Detection Pile of sand example You can sort of see something is there..but is is harder to detect

24 Estimation vs Detection Pile of sand example Zoom in and we can clearly estimate all of the shapes that are present!

25 Estimation vs Detection Detection Is there a signal present? What is the magnitude of the signal Estimation What is the shape of the fmri signal?

26 Image-based Example Slide courtesy of Tom Liu

27 Estimation If you are interested in the shape of the HRF, is a blocked or ER design better?

28 Estimation If you are interested in the shape of the HRF, is a blocked or ER design better? Separated ER design Use the FIR model to capture shape Jitter trial onsets to fit in the most trials within a given run GLM can separate out overlap in signal

29 Detection Which design is best for detecting signal?

30 Detection Which design is best for detecting signal? Blocked

31 Estimation? Models?

32 Estimation? Models?

33 Detection? Models?

34 Models? Detection? Y=Xβ+ε

35 The math behind efficiency To get the highest efficiency we strive for the lowest variance for a contrast

36 The math behind efficiency To get the highest efficiency we strive for the lowest variance for a contrast c must be a vector

37 The math behind efficiency What if we have multiple contrasts? 1. Calculate c(x X) -1 c for each contrast of interest 2. Average together (c 1 (X X)-1c 1 + c 2 (X X)-1c 2 + c 3 (X X)-1c 3 )/3 3. Efficiency is inverse 3/(c 1 (X X)-1c 1 + c 2 (X X)-1c 2 + c 3 (X X)-1c 3 ) This is known as A-optimality, but there are other options

38 Maximizing efficiency What kind of efficiency? Could try to maximize the efficiency to estimate the HRF shape Could maximize the efficiency to detect if a signal is present Could maximize the efficiency to detect a contrast between two trial types

39 Maximizing efficiency Depends on your model and your contrasts Can t have the best of both worlds Maximize detection and maximize estimation As detection increase estimation decreases (and vice versa)

40 Fundamental Trade-off Estimation Efficiency Periodic Semi-Random h 1 Detection Power Slide courtesy of Tom Liu

41 Maximizing efficiency What if you have multiple stimulus types? How do you choose the order? Need some type of search algorithm

42 Search algorithms Randomly try many design setups and choose one with best efficiency Would take a long time Not a very efficient efficiency search! Must also consider psychological factors

43 Permuted block design Start with stimuli blocked and randomly permute

44 Permuted block design Start with stimuli blocked and randomly permute

45 Permuted block design Start with stimuli blocked and randomly permute

46 Permuted block design Start with stimuli blocked and randomly permute

47 Permuted block design Start with stimuli blocked and randomly permute

48 Genetic Algorithm Uses the idea of genetics Two good parents should produce an even better offspring Mutations can lead to improvements

49 Genetic Algoritm

50 Genetic Algorithm

51 Improved genetic algorithm Kao et al (2008), NeuroImage Account for estimation, detection and psychological factors

52 Improved genetic algorithm

53 Options OptimizeX You ll use this in lab today! Based on convolved regressors (detection? estimation?

54 OptimizeX

55 Other options Software (Optseq and GA approaches) are limited in design types Eg. Say you have two types of stimuli (S a and S b ) that are always followed by response cues (R a and R b ) this can t be specified Based in FIR model! detection? estimation?

56 Other options Write your own code! design/start/ optseq surfer.nmr.mgh.harvard.edu/ optseq/

57 Collinearity Efficiency is a relative measure Collinearity relates to correlation between regressors High collinearity -> Low power Use the variance inflation factor (VIF) Aim for <5, but <10 in a pinch

58 What I typically do Set up the parameters for the design Write my code for generating designs Generate a few designs and check the VIF If VIF is good, I run a full efficiency search

59 Questions?