Practical Flatness. Equivalence Approach for Multi-Factor Robustness Evaluation with Application in Vaccines Development

Transcription

1 Practical Flatness Equivalence Approach for Multi-Factor Robustness Evaluation with Application in Vaccines Development Bernard G Francq Waldemar Miller, D Lin, R Rousseau, W Hoyer Contact waldemar.miller@ovgu.de bernard.x.francq@gsk.com

2 Flatness as a Concept Quality-by-Design: Risk based, data driven decisions are key! Classification of process parameters into critical and non-critical To understand relationship between Critical Process Parameters (CPP) and Critical Quality Attributes (CQA) and then estabish Design Space that have been demonstraded to provide assurance of quality. Robustness of a process is its property to stay within the specification limits (target Δ) How far can we change our experimental parameters to stay within our target margin? Flatness describes this desired relationship between dependent and independent variables 2

3 Motivating Example We want to evaluate robustness of a manufacturing process Continuous response Two factors, Duration and Temperature, refactored to a domain of 1, 1 DoE: Central composite design, 6 reps at center point, else 2 reps Apply linear regression (e.g. response surface, etc.) Significant Flatness Do the mean responses differ? e.g. -Tests on differences Practical Flatness Are mean responses equivalent? e.g. TOST Reference point 3

4 From Significance to Equivalence 4

5 Practical Flatness TOST Procedure Let,,, and,,, independent. Are these two samples equivalent? I.e.: is the difference of the means within a specified equivalence margin,. Schuirmann s TOST (Two one-sided tests): : :! vs. : vs. : " Joint hypothesis (of or ): :,, vs. :, Usually checked via 12' -CIs reject if the CI of the difference, if e.g. is within equivalence margin. ),)*+, ),,-. ),)*+, ),, 5

6 Flatness in a DoE Setting Back to our motivating example: How do we assess flatness among 9 design points? Constant Approach: Reference Approach: Many-to-many Approach: Compare 9 design points to a given textbook value Compare 8 design points to a reference (design) point Pairwise comparison of all design points 6

7 Difference CI s and TOST Procedure Problem Works in two-samples-settings. Solution Compare several differences, and accept global flatness iff individual comparisons are considered flat. Unadjusted 90% CIs (contrasts) Significant flatness decision rule: Assume flatness when all -/0-CI s of the difference cover 0! Practical flatness decision rule: Assume flatness when all - 2/0-TOST CI s lie within the equivalence margin! Difference equivalent inconclusive not equivalent (-1,1) (0,1) (1,1) (-1,0) (1,0) (-1,-1) (0,-1) (1,-1) (-1,-1)-(0,0) (-1,0)-(0,0) (-1,1)-(0,0) (0,-1)-(0,0) (0,1)-(0,0) Compared to (0,0) (1,-1)-(0,0) (1,0)-(0,0) (1,1)-(0,0) 7

8 Simulation & Multiplicity Issue in contrasts for Reference Approach Coverage Simulation 10 3 simulated DoE s 456, one per design point Number of Parameters 756 Problem 8 Comparisons: Multiple Comparisons Solution Use adjustments, e.g. Bonferroni, Šidák, Tukey, Dunnett, etc. Unadjusted-t (-1,1) (0,1) (1,1) (-1,0)(1,0)(-1,-1) (0,-1)(1,-1) Compared to (0,0) Overall 8

9 Simulation & Multiplicity Issue in contrasts for Reference Approach Coverage Simulation 10 3 simulated DoE s 456, one per design point Number of Parameters 756 Unadjusted-t (-1,1) (0,1) (1,1) (-1,0)(1,0)(-1,-1) (0,-1)(1,-1) Overall Compared to (0,0) 9

10 Simulation & Multiplicity Issue in contrasts for Reference Approach Coverage Simulation 10 3 simulated DoE s 456, one per design point Number of Parameters 756 Why the actual coverage of unadjusted or Šidák are not equal to 95% (or 66.34% for unadjusted)? Unadjusted-t Sidak (-1,1) (0,1) (1,1) (-1,0)(1,0)(-1,-1) (0,-1)(1,-1) Compared to (0,0) Overall 10

11 Spatial Correlation Issue: correlation to the center point 11

12 Spatial Correlation Issue Problem Comparisons are correlated! Solution Multivariate -distribution Distribution of the comparison vector: 8-98:, 98 9 ; 9 ) 98 0 and =>? )A. Hence the standardized comparisons vector follows a central multivariate -distribution: 898: + )A -0 B,98 9 ; 9 ) 98 0 Now: Calculate CI s using MV and Corr. (in R e.g. mvtnorm::qmvt) 12

13 Reference Approach Mean response at selected points at C 5- C,, C B 0 and corresponding matrix 9 5 D,,D B ; We obtain E comparisons to reference point C F : B :5 C C F C B C F Assume E comparisons of PIO points to reference point C F, then with PQR. E C I C F 5 D I D F ; : Var- C I C F 05Var C I.Var C F 2Cov- C I, C F 0 Calculate Comparisons CI s of ST U for Difference or Equivalence Settings 13

14 Simulation & Multiplicity Issue in contrasts for Reference Approach Coverage Simulation 10 3 simulated DoE s 456, one per design point Number of Parameters 756 Solution for overall contrasts: Multi-t is your friend Unadjusted-t Sidak (-1,1) (0,1) (1,1) (-1,0)(1,0)(-1,-1) (0,-1)(1,-1) Compared to (0,0) Overall 14

15 Simulation & Quantiles for Reference Approach Motivating Example Quantiles 4522,756, VW516, '50.05 Quantile Values* by Grid Size Quantile Value MVt-quantile standard t-quantile Sidac Adjustment 2 x 2 3 x 3 5 x 5 7 x 7 9 x 9 11 x x x 15 * quantiles calculated for both-sided 95%-CI's and 16 degrees of freedom 21 x x 31 15

16 Constant versus Reference approach: quantile Šidák Quantiles Multi-t Unadjusted t-student Reference Constant Grid Size of Contrasts (# Levels per factor) There is not so much difference in contrasting to a reference a point, or in comparing predictions to a constant value But what about the CIs? 16

17 Constant versus Reference approach: CIs 17

18 Application / Case Study Run experiment* for our motivating example Cell count Duration Y Temperature Z 5:.: Z.: Y.: [ YZ.: \ Y.: 3 Z.] Check practical flatness when comparing to the center point as a reference (contrast)! * Simulated data 18

19 Application / Case Study 19

20 Application / Case Study 20

21 Application / Case Study 21

22 Application / Case Study 22

23 Application / Case Study 23

24 Application / Case Study 24

25 Application / Case Study 25

26 Application / Case Study 26

27 Application with a given constant approach Smooth it We want a cell count = 350 or equivalently inside 350 ± 55 Design Space = Area where 90% CIs (with multi-t) lie between 295 and % CI lower 90% CI upper 27

28 Application with Constant approach Pick some points 28

29 Last Question: what if intercept model? 29

30 Last Question: what if intercept model? CIs for reference approach (contrast) collapse! CIs for constant approach converge to the classical univariate CI for a mean 30

31 Conclusion: Flatness Concept Summary Significantly flat Difference Approach : I ^ 50for all PQ_ : at least one true difference is not 0. Practically flat Equivalence Approach : I ^ a, b : all true differences within a, b Assume strict flatness if all CI s cover 0 Assume practical flatness, if the equivalence margin contains all CI s With 4, will detect any difference! Heavily depends on the width of equivalence margin There is a saturated / max. sample size for detecting flatness Both depend on the underlying regression model! 31

32 Acknowledgement & Conflict of Interest Acknowledgement To GSK Technical R&D statisticians, especially Hervé Gressard, Gaël de Lannoy, Paul Smyth Mathieu Vasselle Sylvie Scolas Conflict of Interest This work was sponsored by GlaxoSmithKline Biologicals SA. B.G. Francq, D. Lin, R. Rousseau and W. Hoyer are employees of the GSK group of companies. W. Miller is a student at Otto-von-Guericke University and is performing a traineeship at GSK. 32