Using Multicomplex Variables for Automatic Computation of High-Order Derivatives

Transcription

1 Usng Multcomple Varables for Automatc Computaton of Hg-Order Dervatves Gregory Lantone Msson Desgn Engneer JPL (prevously Scool of Aerospace Engneerng, Georga Tec) Ryan P. Russell Assstant Professor, Te Unversty of Teas at Austn (prevously Scool of Aerospace Engneerng, Georga Tec) Terry Dargent * Researc & development drectorate, Tales Alena Space rd European Worksop on Automatc Dfferentaton Monday 0t and Tuesday t June 0 INRIA Sopa-Antpols, France

2 Motvatons Workng on nd order optmal control metods (HDDP- varant of dfferental dynamc programmng) Frustrated wt epense of dervatve calculatons Analytc tedous to code or not always possble Fnte dfferencng easy but naccurate, slow Automatc Dfferentaton not stragtforward, slow New comple step metod (Squre & Trapp 998, Martens 00) accurate for frst order dervs only OBJECTIVE: Etend comple metod to ger order dervatves

3 Motvatons Senstvty Analyss Partal Dervatves of outputs w.r.t. nputs n m m n X y y y y f n X y m y Y f m k n j j k XX y f......, Computatons of Senstvtes gly desrable n many felds: Desgn Optmzaton: gradent-based on satellte trajectory optmzaton, and optmal control Inverse Problem (Data assmlaton) Curve fttng Parameter dentfcaton Nonlnear PDEs Gradent mn matr Hessan mnn tensor

4 Requrements of Senstvty Computatons n order of mportance (to us):. Accurate Improvement of algortm convergence Compute adjont state dynamc wt te same précson as te state dynamc. Fast (enoug). Easy-to-mplement Low setup tme for one problem Generalzed for dfferent problems 4

5 Estng metods Analytcal (Manual) dfferentaton Can be accurate and effcent (depends on te programmer) Knowledge of computer language needed for model mplementaton Development tme s long Error prone Mantanng dervatves an addtonal burden Dffcult on large numercal model Symbolc dfferentaton Imples usng software for symbolc manpulaton suc as Maple or Matematca Reduces model development tme Reduces errors assocated wt matematcal manpulatons Stll requres uman efforts for furter model mplementaton Mgt lead to non-effcent epressons 5

6 Estng metods Fnte Dfferencng F Central second-order accurate dfference: F ( 0 e ) F ( 0 0 Error ntroduced e Step sze ) f Step sze? Large : Averagng ~dfd dfd Small : round off error n subtracton Easest metod to mplement: only orgnal computer program s requred Development tme s mnmal Accuracy s step dependent Computatonally ntensve: N dervatves wll requre N+ functon evaluatons Often naccurate or neffcent 6

7 Estng metods Automatc Dfferentaton Analytc dfferentaton of elementary functons Repettve applcaton of te can rule Can be mplemented n two ways: Source transformaton: Produce new code to calculate dervatve based on orgnal code of a functon (e: ADIFOR, TAPENADE, ) Operator overloadng: Eac elementary operaton s replaced by a new one, workng on pars of value and ts dervatve (doublet) (e: AD0 from te Harwell Subroutne Lbrary) f sn( ( ) Dervatves of any order Accurate to macne precson: No round-off errors Complete Partal Dervatves wt sngle eecuton Computatonally more effcent tan FD Not stragtforward metod to mplement 4 ) Elementary functons Elementary operatons 7

8 Estng metods Comple-step metod (Squre & Trapp 998, Martens 00) Use comple varables nstead of real varables Found from Taylor Seres epanson: f ( ) f ( ) df O( d ) Avod te subtracton nvolved n fnte dfference Accurate to workng precson for VERY small < 0-8 Very easy mplementaton Separate smulatons for eac gradent requred (lke fnte dfferencng) Increased computatonal tme due to comple artmetc Lmted to frst-order dervatves (La, Crassds: gve tunng metods to fnd optmal step sze for nd order appromatons, stll not macne accurate ) f Im[ f ( )] Lm 0 No subtracton! 8

9 MultComple Numbers MultComple numbers (C n ) are etensons of comple numbers n ger dmensons z 0... nn... n nnn n Same formal representaton as comple numbers can be used: z z z n cross magnary terms were z Є Cn and z, z Є C n-... n 9

10 0 MultComple Numbers Eample: bcomple numbers 0 z were,,, 4 Є R, =-, =-, = c z z z were z, z Є C, =- Eample: trcomple numbers 0 z z z z were z, z Є C, =- magnary terms can be represented as matr operator eample bcomple :

11 Eample nd Dervatve From Taylor Seres f f f f H. O. T. (gnore...) f f f NOTE: f f f n =- f f f f take now te coeffcent of say e 00 and tere s no subtracton error must be representable wt double precson Im f f O we coose etremely small, from bot sdes, dvde by

12 MultComple-Step Dfferentaton Fundamental formula: f ( n) ( ) Im... n f ( n... n ) O( ) Use multcomple varables nstead of real varables Found from Taylor Seres epanson Dervatves up to any order n Same oter advantages as comple metod See paper for matematcal formaltes: Usng Multcomple Varables for Automatc Computaton of Hg-Order Dervatves, Gregory Lantone, Ryan P. Russell & Terry Dargent, ACM Transactons on Matematcal Software (TOMS) Volume 8 Issue, Aprl 0, Artcle No. 6 Detals: Step wt n eac of te magnary drectons Evaluate mult-comple functon Resultng dervatves retreved from te coeffcents of te mult-comple functon result

13 Eample Trd Dervatve Calculaton Multple varables: (,y,z) f yz Im f, y, z f yy Im f, y, z f yyy Im f, y, z

14 MultComple-Step Dfferentaton smple eample: f ( ) sn( ) e cos( ) st Dervatve Normalzed error ACCURACY: MCX and CX: ~6 dgts large range FD: ~0 dgts, requres tunng ~e step sze () 4

15 MultComple-Step Dfferentaton smple eample: f ( ) sn( ) e cos( ) nd Dervatve Normalzed error ACCURACY: MCX: ~6 dgts large range CX and FD: ~7 dgts, requres tunng step sze () 5

16 MultComple-Step Dfferentaton smple eample: f ( ) sn( ) e cos( ) rd Dervatve Normalzed error ACCURACY: MCX: ~6 dgts large range CX and FD: ~5-7 dgts, requres tunng step sze () 6

17 Implementaton Declare all te varables as comple type Or only te ndependent varables and te assocated ntermedate varables 7

18 Implementaton Declare all te varables as comple type Redefne all te functons wt comple defntons Or only te ndependent varables and te assocated ntermedate varables Easy mplementaton usng te same form of operator overloadng functons as comple numbers 8

19 Implementaton Declare all te varables as comple type Redefne all te functons wt comple defntons Or only te ndependent varables and te assocated ntermedate varables Easy mplementaton usng te same form of operator overloadng functons as comple numbers Add comple step () to te desred varable... n 9

20 Implementaton Declare all te varables as comple type Redefne all te functons wt comple defntons Or only te ndependent varables and te assocated ntermedate varables Easy mplementaton usng te same form of operator overloadng functons as comple numbers Add comple step () to te desred varable... n Partal Dervatve Calculatons f ( n) Im... n f (... ( ) n n ) 0

21 Implementaton Declare all te varables as comple type Redefne all te functons wt comple defntons Or only te ndependent varables and te assocated ntermedate varables Easy mplementaton usng te same form of operator overloadng functons as comple numbers Add comple step () to te desred varable... n Partal Dervatve Calculatons f ( n) Im... n f (... ( ) n Repeat as necessary for more ndependent varables n )

22 Implementaton Declare all te varables as comple type Redefne all te functons wt comple defntons Or only te ndependent varables and te assocated ntermedate varables Easy mplementaton usng te same form of operator overloadng functons as comple numbers Add comple step () to te desred varable... n Partal Dervatve Calculatons f ( n) Im... n f (... ( ) n Repeat as necessary for more ndependent varables Currently ave workng modules n Matlab and Fortran n )

23 Overloadng Sample Code & DEMO Overloadng eample n FORTRAN for + operator

24 Step-sze Lmts for Hg order Dervatves ) Snce error s O( ), ten >=e-6 terefore: > ~e-8 ) Because n appears n dervatve appromaton: n >ε=e-08, ) terefore: >0-08/n From above: e-8>0-08/n n<~8 for double precson (ts s upperbound, n practce sould be smaller due to dynamc range of varables, margn on error estmates.) Note tat a comple number wt n = 5 s represented wt 5 > 0 0 real numbers!!! Mn step sze and error vs. Dervatve order.00e-0.00e-08.00e-4.00e-0.00e-6.00e-.00e-8.00e-44.00e-50.00e-56.00e-6.00e-68.00e-74.00e n=order of dervatve Elementary computaton cost comparson to compute a product and ts dervatve MCX: Z *Z = ( + )*( + )= AD: { ; d( * )= d + d } Over cost of MCX versus AD: te Product ~e-6 8 ~e-08 mn (0^-08/n) error (^) 4

25 Eample: cos(),,d n (cos )/d n Relatve st toerrors 6t Cosnus on st dervatve -6 t dervatve relatveof errors cos() relatve errors relatve errors 0-5 t dervatve nd dervatve rd dervatve 4t dervatve 5t dervatve 6t dervatve Cosnus dervatve to 6 relatve errors 5 & 6 under-flow lmtaton s tep for double precson, (+)= f < e step Large coce for step 5

26 Test Case: Trajectory Trajectory State Transton Matr Satellte subject to gravtatonal force and constant nertal trust Segment propagated for 6 days st and nd -order State Transton Matrces useful n trajectory optmzaton (99 terms) Metod Sample rd -order STM Accuracy Total Relatve Compute Tme Analytcal NA.0* MultComple TAPENADE AD Fnte Dfferences *analytc computaton of STMs do not take advantage of symmetry, tme could lkely be reduced n alf, but effort s nontrval 6

27 Test Case: Gravty feld Gravty feld dervatves 00 Lunar Gravty Feld Up to trd-order Useful for satellte geodesy and trajectory optmzaton Metod Sample rd -order Senstvty Mamum Relatve Dfference wt Analytc (across all rd order terms) Total Relatve Computatonal Tme Analytcal NA.0 MultComple-Step TAPENADE AD

28 New Senstvty Landscape FD Analytcal MCX AD* Compute Speed Slow Fast Medum Medum() / slow () Ease of mplementaton Easest Hardest Medum Medum ()/Hard() Accuracy Poor Near Eact** Near Eact ** Near Eact ** Specal requrements None: functon call CAN BE a lbrary or black bo Functon call and dervatves must be consstent. CAN BE a lbrary or black bo MCX module +functon source AD module +functon source *only consdered TAPENADE () & AD0 () of te many AD toolboes avalable ** subject to round off, order of operatons, dynamc varable range errors, but not subtracton error 8

29 Concluson MultComple-Step metod was developed for computaton of partal dervatves up to any order: etenson of comple step metod to any order MultComple-Step dfferentaton combnes te best of fnte dfference, comple, and automatc dfferentaton Furter ncreasng n tool fleblty s possble troug: Prototype Modules for Fortran and Matlab Developng a scrpt to automatcally process source codes Matr and array operatons n Matlab 9

30 Tank you! We want your feedback!! 0