Custodial Integrator Automation Guide

Transcription

1 Custodial Itegrator Automatio Guide

2 Compay Cofidetial Custodial Itegrator Product Versio: V3.8 Documet Versio: 14 Documet Issue Date: April 21, 2017 Techical Support: (866) Telephoe: (781) Fax: (781) Web: byallaccouts.morigstar.com

3 Table of Cotets Cotets USING THIS GUIDE...2 AUDIENCE...2 RELATED DOCUMENTATION...2 INTRODUCTION TO CI AND CI AUTOMATION...2 CI AUTOMATION IS IT APPROPRIATE FOR YOU?...2 CRITERIA FOR RUNNING CI AUTOMATION...2 SPECIAL CONSIDERATIONS...3 COMPARISON OF CI AND CI AUTOMATION...3 SIMILARITIES AND DIFFERENCES...3 OVERALL PROCESSING FLOW FOR CI AND CI AUTOMATION...3 EXPORT OF SECONDARY FILES WITH CI AND CI AUTOMATION...6 WHEN SECONDARY FILES ARE AVAILABLE IN CI AUTOMATION...6 SWITCHING BETWEEN CI AND CI AUTOMATION...7 SET UP AND RUN CI AUTOMATION...7 CONFIGURATION OPTIONS FOR CI AUTOMATION...7 MANUAL AND AUTOMATED CONFIGURATIONS FOR CI AUTOMATION...8 OVERVIEW OF STEPS TO SET UP CI AUTOMATION...8 SETTING CI PARAMETERS...9 USING A RUNCIAUTO.BAT FILE...9 IF YOU RUN CI AUTOMATION AND NEED TO RUN CI MANUALLY...10 MONITOR CI AUTOMATION ON AN ONGOING BASIS...10 HOW TO VERIFY RESULTS...10 DETAILS ABOUT CI AUTOMATION BEHAVIOR...10 PRIMARY OUTPUT FILES...11 EXAMINE THE PRIMARY OUTPUT FILES...11 IF PRIMARY OUTPUT FILES ARE MISSING...11 SECONDARY OUTPUT FILES...12 EXAMINE SECONDARY OUTPUT FILES...12 LOG FILES...12 EXAMINE THE AUTORUNHISTORY.LOG FILE...12 FATAL ERRORS THAT STOP EXECUTION...13 NON-FATAL ERRORS THAT DO NOT STOP EXECUTION...13 APPENDIX A: CI AUTOMATION PARAMETERS...14 i

4 Table of Cotets APPENDIX B: EXAMPLES OF SECONDARY OUTPUT FILES...22 ACCOUNT TRANSLATIONS FILE...22 MISSING PRICES FILE...22 SECURITY TRANSLATIONS FILE...23 STALE ACCOUNTS FILE...23 UNTRANSLATED ACCOUNTS FILE...23 UNTRANSLATED SECURITIES FILES...23 APPENDIX C: AUTORUNHISTORY.LOG FILE EXAMPLES...25 EXAMPLE EXAMPLE EXAMPLE EXAMPLE APPENDIX D: HANDLING UNTRANSLATED SECURITIES...27 HOW CI VARIANTS HANDLE UNTRANSLATED SECURITIES...27 APPENDIX E: HANDLING STALE ACCOUNTS...28 APPENDIX F: MULTIPLE CI INSTANCES AND DATABASES...29 SETTING THE WORKING AND OUTPUT FOLDERS FOR EACH INSTANCE...29 USING MULTIPLE VERSIONS OF THE.BAT FILE...29 USING CUSTOM FILTER AND TRANSLATION FILES...29 APPENDIX G: EXAMPLE OF SECONDARY FILE OUTPUT...31 ii

5 USING THIS GUIDE Audiece This guide describes the capabilities of Custodial Itegrator Automatio (CI Automatio), which eables Custodial Itegrator (CI) to be ru from the commad lie istead of from the user iterface. The maual is writte for the IT Admiistrator who is resposible for cofigurig ad maitaiig CI. This documet describes how CI Automatio works, how it compares to CI, ad what optios are available to cofigure automatio behavior. It is helpful if the reader is familiar with CI ad how it is implemeted i the local eviromet. Related Documetatio This documet describes automatio optios. Use it i cojuctio with your other CI documetatio. Refer to: Custodial Itegrator User Guide to fully uderstad the optios available through the user iterface. Custodial Itegrator Istallatio Guide for iformatio about rutime parameters used to cofigure CI behavior. INTRODUCTION TO CI AND CI AUTOMATION Custodial Itegrator (CI) is a applicatio that trasfers data from the ByAllAccouts repository to your local eviromet ad formats that data ito files specific to your target system. CI is typically ru as a user iterface (UI) applicatio that presets to the user the various steps for the data trasfer ad data checkig process, allowig that user to iteract with it ad resolve exceptios. As a alterative to beig ru as a user iterface applicatio, CI ca be ru from the commad lie usig CI Automatio. Usig CI Automatio, CI ca be scripted to ru automatically at various times of the day to perform its ormal processig ad produce its output files. Because the user does ot iteract with the UI, careful setup ad moitorig is required to esure that CI Automatio produces the expected output. CI AUTOMATION IS IT APPROPRIATE FOR YOU? Criteria for Ruig CI Automatio CI Automatio is recommeded oly whe all of the followig items are true for you: There is a large volume of accouts to be processed. You wat to trasfer your accout data to your local eviromet as early as possible i the day so that data is ready for your staff whe they arrive at work rather tha requirig them to perform activities that cause the accouts to be trasferred. There is a low icidece of processig exceptios or you have the ability to programmatically hadle exceptios usig iformatio that CI outputs to primary or secodary files. A processig exceptio is ay data situatio that requires additioal iformatio to be provided i order to properly prepare the data for your target system. A example of a processig exceptio occurs whe a security has o uique idetifier such as a Ticker, CUSIP, SEDOL, or ISIN. This guide describes the optios CI Automatio provides for you to hadle exceptios; however, it is likely that you will still eed to hadle some types of exceptios i your target system (or usig a itermediate processor) i order for CI Automatio to work well for you. 2

6 IT staff is available to cofigure parameter files, batch scripts, ad scheduled tasks. This same staff will also eed to respod to ad troubleshoot ay CI Automatio errors that might occur whe there are chages i your local eviromet that affect CI, such as Java upgrades or other system chages. If all of these items do ot describe your situatio the CI Automatio may ot be suitable for you. Special Cosideratios There are several factors to cosider if you decide to automate CI, some of which may ot apply to your eviromet. If you pla to ru CI Automatio multiple times per day the you will eed to determie the best times to ru CI Automatio i relatio to the time that data is available from your fiacial istitutios. ByAllAccouts ca work with you to determie the typical time each day whe a majority of your accouts will have bee updated with prior day close data. This time establishes whe you wat your first ru of CI Automatio that day. You will likely eed to ru CI Automatio at least oe more time to collect data for accouts at istitutios that report data later i the day (such as 8:30 AM Easter or later). If you have a high volume of accouts you might wat to itroduce a third ru i betwee the first ad secod rus of CI Automatio. Agai, the timig of this additioal ru should be aliged with the time your fiacial istitutios report data. If you use CI with a target system such as PortfolioCeter or Morigstar Office that is expectig to receive oly oe set of files (positio, trasactio, etc.) per day ad you wat to ru CI Automatio to produce multiple file sets daily (to accelerate delivery) the you must determie how you will hadle multiple file sets i your target system. If you istall ad ru multiple istaces of CI each with a separate CI database, the you must review Appedix F: Multiple CI Istaces ad Databases for how to maage them usig CI Automatio. COMPARISON OF CI AND CI AUTOMATION Similarities ad Differeces Whe ruig CI maually, a user sees iformatio ad makes decisios about whether or ot to move forward with a ru. The user sees messages ad ca itervee or proceed. Usig CI Automatio, the behavior of CI is cotrolled by parameter settigs istead of through direct user iteractio. These parameter settigs pre-determie the choices for how to hadle commo exceptios, ad eable CI Automatio to cotiue processig through them. For example, parameter settigs ca cotrol whether CI Automatio will update stale accouts ad whether it will traslate o-validated accouts. CI ca be ru more tha oe time per day (takig delivery of multiple CI file sets) i order to obtai data as it becomes ready i the ByAllAccouts repository rather tha waitig for all data to be available before ruig CI. Likewise, CI Automatio ca be scripted to ru automatically at various times of the day to perform its ormal processig ad produce output files. For either method, exceptios detected are writte to log files. CI Automatio geerates a additioal log file that provides a history of the actios performed by each ru of CI Automatio. Overall Processig Flow for CI ad CI Automatio Note: CI ad CI Automatio have the same geeral processig flow, but there are some key differeces i how they are achieved. You eed to uderstad the differeces so you ca cofigure CI Automatio to get the behavior you wat. 3

7 The CI user iterface shows four major steps for movig data through CI to its output files. The first step, Setup, has settigs that are stored persistetly i the CI database icludig import/export defaults. Before usig CI Automatio for the first time, the Setup optios must be set through the CI user iterface. After that, each CI Automatio ru reads those settigs from the database. After it has bee cofigured, CI Automatio ca follow the same steps as CI, but without requirig ay user itervetio to move from oe step to the ext. The optios that a user would set i the user iterface durig a idividual ru of CI are maaged by parameters for CI Automatio. The autoru parameter, which ivokes CI Automatio, ca be set to ru up to ad icludig the Import, Export, ad Accept steps. The followig items compare how CI ad CI Automatio behave through all of the steps. 1. Setup Use the Setup step i the user iterface for both CI ad CI Automatio. Settigs such as CI logi ad password, output folder, ad import/export defaults, as well as accout traslatios ad security traslatios are always set through the user iterface ad stored i the CI database. Note: This maual setup is required before ruig CI Automatio for the first time. Ay chages made to Setup i the user iterface affect both CI ad CI Automatio. 2. Import data Durig the Import step, both CI ad CI Automatio import fiacial data for the accouts that have bee traslated. There are two requiremets for Import to take place i both CI ad CI Automatio: There is at least oe type of fiacial data (securities, positios, trasactios, etc.) marked for import i the Setup step. At least oe valid accout traslatio has bee defied. Whe either of the requiremets is ot met, the Import butto i the CI mai dialog is disabled. I CI Automatio, the applicatio will stop before the Import step ad a error message will be writte to the AutoRuHistory.log file. Traslated ad Utraslated accouts: Both CI ad CI Automatio will import fiacial data oly for accouts that have bee traslated; accouts that have ot bee traslated are skipped durig the Import process. For both CI ad CI Automatio, the user ca maually create accout traslatios through the Accout Traslatio dialog durig the iitial Setup step. CI Automatio however offers a additioal capability to automatically create accout traslatios. A parameter ca be set to automatically traslate utraslated accouts before importig data; as described o page 19 whe the traslateutraslatedaccouts parameter is set, CI Automatio will map each utraslated accout to the default accout idetifier for your variat of CI. By usig this parameter you ca esure that every day ew accouts will be automatically traslated ad their data imported without the eed to first ope the CI user iterface ad maually create traslatios for the ew accouts. Refer to pages 18 ad 19 for the parameters that ca be set to tue the automatic accout traslatio to your eeds. Stale accouts: A accout is cosidered stale i the BAA system whe it does ot reflect the latest data from the custodia or the accout data it got from the custodia accout was ot for the prior busiess day. I CI, stale accouts are marked i bold i the Accout Traslatio dialog; the user ca see details of these accouts i the Accout Update Status dialog. BAA will cotiue to try to get the data from the custodia util a set time of day. If the user believes the custodia has up-to-date data that BAA did ot retrieve, the user ca 4

8 ru a maual accout update for some or all stale accouts (accout update o demad). The goal is to have the stale accouts updated before startig the CI import, so CI will receive the latest data for these accouts ad export it to the output files. Whe ruig CI, the user ca request to update oe or more accouts i the Accout Update Status dialog. I CI Automatio, accout update is requested through the updatestaleaccouts parameter (page 21). Whe this parameter is set, CI Automatio will update all the stale accouts before ruig Import. Note that accout update may take a cosiderable legth of time, depedig o the umber ad size of the stale accouts. Refer to Appedix E: Hadlig Stale Accouts for strategies tha ca be used to efficietly maage update of stale accouts whe usig CI Automatio. Ivalid accout traslatios: Whe ruig CI maually, if oe or more accout traslatios is ivalid (for example the accout was deleted sice the last import), the Import butto is disabled ad o data ca be imported util the user resolves all ivalid accout traslatios. CI Automatio is differet. I CI Automatio ivalid accout traslatios are skipped durig import, but data is still imported for accouts with a valid traslatio. The AutoRuHistory.log file will cotai a message iformig that ivalid accout traslatios were foud ad have bee disabled for import. Ivalid accout traslatios are show i the CI user iterface i bold blue i the Accout Traslatio dialog ad ca oly be deleted maually through the user iterface. 3. Export to files After data has bee successfully imported to CI, the CI user ca click the Export butto to geerate files to the output folder. I CI Automatio, the Export optio ca be preselected usig the autoru:export parameter. Note that Export works oly o traslated accouts. Maagig utraslated (udefied) securities: Whe importig fiacial data CI (ad CI Automatio) verifies that each positio, trasactio, ad lot refereces a valid security. If the security caot be idetified because the symbol is missig or ivalid, the security is marked as udefied or utraslated. The presece of oe or more udefied securities i the imported data may affect the Export step depedig o the CI variat you are usig. Essetially, for some variats, both CI ad CI Automatio will export data eve if there are utraslated securities by isertig a dummy security symbol. For other CI variats, CI ad CI Automatio caot export ay data if there are utraslated securities. The variatspecific rules are the same i CI ad CI Automatio. For details about how each variat treats utraslated securities, refer to How CI Variats Hadle Utraslated Securities, page 27. If your CI variat does ot allow Export whe there are udefied securities, the Export butto i the mai CI scree stays disabled while a warig message is displayed with istructios to maually traslate the udefied securities. I CI Automatio the applicatio will stop ad exit after the Import step. A message i the AutoRuHistory.log file will idicate that udefied securities have bee foud ad eed to be maually traslated before Export ca be ru. Ulike accout traslatios, security traslatios caot be automatically geerated by CI Automatio. The user has to maually traslate the udefied securities i the CI Security Traslatio dialog ad the ru CI Automatio agai for Export to take place. Stale accouts ad positios export: If you choose ot to update stale accouts before import, kow that by default CI ad CI Automatio will ot export positios from stale accouts. You ca override this default i CI Setup by ucheckig the Exclude positios for stale accouts advaced Setup optio, thereby icludig them. Makig this chage i Setup affects both CI ad CI Automatio. I additio, for both CI ad CI Automatio the outputstaleaccouts parameter (page 17) ca be set to output a list of stale accouts that you ca the maually update. 5

9 4. Accept exported data Note: Because the Accept step writes state iformatio about the most recet dowload to the database, it is always recommeded that the user first verify that CI properly produced the output files before acceptig the exported data, whether ruig CI or CI Automatio. Whe ruig CI maually, the user clicks Accept so state iformatio about the most recet dowload is writte to the database. I CI Automatio, the Accept optio ca be preselected usig the autoru:accept parameter. Whe it is, CI Automatio performs the whole cycle of Import, Export, ad Accept. Note that to allow time to verify CI Automatio output before Accept, set parameters acceptfile ad acceptfiletimeout i your ruciauto.bat file. Refer to Appedix A: CI Automatio Parameters. Export of Secodary Files with CI ad CI Automatio CI ad CI Automatio export the primary files that are cofigured for export i Setup i the mai CI Cofiguratio scree. These files are output durig the Export step. I additio, CI ad CI Automatio ca optioally export secodary files. Secodary files cotai cofiguratio ad other auxiliary data that is ot fed ito the target system but that ca be helpful to the CI user. Secodary files are ot exported by default. For both CI ad CI Automatio, export of each secodary file is cotrolled by its associated output parameter as described i Appedix A: CI Automatio Parameters. The followig secodary files ca be exported: Accout Traslatios Missig Prices Security Traslatios Stale Accouts Utraslated Accouts Utraslated Securities Whe Secodary Files are Available i CI Automatio CI exports all requested files, primary ad secodary, durig the Export step oly ad will ot automatically export the requested secodary files if you exit CI before ruig Export. CI Automatio, o the other had, behaves differetly. Import-idepedet secodary files are available without ruig Import: With CI Automatio, you ca optioally geerate import-idepedet secodary files without ruig the sometimes time-cosumig Import step. By settig parameters to request output of Utraslated Accouts, Accout Traslatios, Security Traslatios, the ruig CI Automato i a pre-import mode, those files will be exported before CI Automatio exits. For a example of whe the capability of exportig secodary files without performig Export or Import may come i hady i CI Automato, refer to Appedix G: Example of Secodary File Output. Most secodary files ca be produced by oly ruig through Import: If you set parameters to request output of secodary files such as Utraslated Accouts, Accout Traslatios, Security Traslatios, Utraslated Securities, ad Stale Accouts, you ca ru CI Automatio through the Import step ad those requested secodary files will be exported after the Import step completes ad before CI Automatio exits. 6

10 Export of all secodary files: If you set parameters to request output of secodary files (Utraslated Accouts, Accout Traslatios, Security Traslatios, Utraslated Securities, Stale Accouts, Missig Prices) ad Automatio is set to ru up to the Export or Accept step, all requested secodary files will be exported durig the Export step after the primary files are exported. I this case the behavior is the same i CI ad CI Automatio. Switchig Betwee CI ad CI Automatio Although you may choose to ru CI i a automated way o most days, you will likely eed to ru CI maually (through the user iterface) periodically to chage settigs or check o some iformatio. Note that: You must ot ru both CI ad CI Automatio at the same time agaist the same database. There are certai CI parameters that should be cofigured idetically for maual mode ad automated mode, such as the accout idetifier scheme. It is recommeded that those parameters be set i the CI.ii file. Refer to Set Up ad Ru CI Automatio, page 7 for a explaatio of where to set parameters. For additioal details ad caveats, refer to If You Ru CI Automatio ad Need to Ru CI, page 10. SET UP AND RUN CI AUTOMATION The CI iterface is ivoked usig the ruci.bat file. To ru CI Automatio, it is recommeded that you create a customized versio of the ruci.bat file ad call it ruciauto.bat. The basis of this recommedatio lies i how rutime parameters are specified for CI ad CI Automatio. Cofiguratio Optios for CI Automatio CI has a umber of cofiguratio optios ad the way you set up CI largely depeds o your ow implemetatio ad what iformatio you wat CI to process. Those optios are described i your Custodial Itegrator User Guide. This documet describes optios for CI Automatio. Please use this guide i cojuctio with your Custodial Itegrator User Guide to fully uderstad the optios available through the CI user iterface. Refer to your Custodial Itegrator Istallatio Guide for iformatio about additioal CI rutime parameters. As described i Appedix A: CI Automatio Parameters, parameters specific to automatio are available to istruct CI Automatio to perform actios that would ormally be performed by a user such as establishig a ew accout traslatio (mappig). For example, parameters ca be set to cotrol whether CI Automatio will: Traslate utraslated accouts Update stale accouts Traslate o-validated accouts alog with validated accouts Import tax lots Export oly those accouts that are ready ad ot yet exported today if you ru CI more tha oce per day Produce secodary output files i additio (or idepedetly of) primary output files. Coordiate acceptace of files delivered from CI 7

11 Notes: If you support multiple CI istaces/databases, refer to Appedix F: Multiple CI Istaces ad Databases. If you use custom traslatios or filters for positios ad trasactios refer to Usig Custom Filter ad Traslatio Files o page 29. Maual ad Automated Cofiguratios for CI Automatio The Setup cofiguratios must be set maually i the CI user iterface, ad they are stored persistetly i the CI database. Other cofiguratios ca be cotrolled usig parameters i the CI.ii file or i the ruci.bat (or a variat of it). Ru the CI user iterface to set items i the CI database. Every time you ru CI Automatio, it will import ad export data accordig to the Setup cofiguratios set i the CI user iterface. You ca ru CI maually to chage the settigs i the Setup step. The optios available deped o the variat of CI you are ruig. They iclude: o o import/export defaults accout traslatios (mappigs) ad security traslatios Use the CI.ii file ad ruci.bat file (or a variat of it) to set parameters. As described i Appedix A: CI Automatio Parameters, you ca cotrol may cofiguratio optios whe automatig CI. For example, parameters ca be set to have CI Automatio create traslatios for utraslated accouts or produce additioal output files to fully documet complex exceptios such as utraslated securities. For iformatio settig the parameters, refer to Set Up ad Ru CI Automatio, page 7. Overview of Steps to Set Up CI Automatio 1. Ru the CI user iterface ad adjust the Setup optios which are the saved i the database. Cofiguratios set i the Setup sectio of the user iterface are persistet i the database ad used every time CI or CI Automatio is ru. The optios available deped o the variat of CI you are ruig. They iclude: CI access icludig logi, password, ad workig folder Output cofiguratio icludig file output folder Data to import ad export, such as trasactio ad positios, to cotrol which primary output files are geerated Accout traslatios (mappigs) ad security traslatios Refer to your Custodial Itegrator User Guide for iformatio about the other optios available for your variat of CI. 2. Set CI parameters. a) I the CI.ii file, put the stadard parameter settigs that you wat to use every time CI rus, whether as CI or as CI Automatio. Some CI parameters should be cofigured idetically for maual mode ad automated mode, such as the accout idetifier scheme (defaultaccoutidetifier). Set those parameters i the CI.ii file. The parameters that work for both modes are described i the Custodial Itegrator Istallatio Guide for your variat of CI. b) Copy the ruci.bat file ad ame it ruciauto.bat. Put parameters specific to automatio i the ruciauto.bat file. Those parameters are described i this guide i Appedix A: CI Automatio Parameters. 8

12 For more iformatio about the parameter files, refer to Settig CI Parameters, page Ru CI maually through the Export step, ad verify that CI has write permissios to the output folders ad that you get the output you expect. (You do ot have to complete the Accept step.) 4. Ru CI Automatio ad verify the output. Refer to Moitor CI Automatio o a Ogoig Basis, page After you get the basic CI Automatio workig, review Special Cosideratios, page 3 to determie if there are additioal factors to cosider implemetig such as timig of automatio rus, whether you eed to maage sigle file sets, ad if you eed to implemet multiple istaces of CI. Settig CI Parameters By default there are two files i which you ca adjust parameter settigs to customize how Custodial Itegrator (CI) fuctios: ruci.bat the file that ivokes CI with the user iterface. CI.ii the iitializatio file for CI. Note: Parameter values i the ruci.bat file take precedece over those i the CI.ii file. Although parameters that customize how CI rus ca be placed i either the ruci.bat file (java commad lie) or i the CI.ii file, this guide recommeds that you put the geeral parameters i the CI.ii file ad create a third file called ruciauto.bat ad put the parameters specific to automatio i that file. ruciauto.bat the file you create to ivoke CI Automatio, so that CI ca ru uatteded. I this file iclude parameters related to automatio, icludig the autoru parameter. For iformatio about automatio parameters, refer to Appedix A: CI Automatio Parameters. For more iformatio about the ruciauto.bat file, refer to the ext sectio, Usig a ruciauto.bat File. Usig a ruciauto.bat File Eve after you set up CI to ru i a scripted mode, you will still eed to access the user iterface occasioally to verify or chage a settig. Therefore it is recommeded that you have separate.bat files for ruig CI ad ruig CI Automatio. Although you wat CI to behave the same either way you ru it, there are some parameters that apply oly to the automated mode ad those should be set i the ruciauto.bat file. As explaied i Settig CI Parameters o page 9, it is recommeded that you put ito the CI.ii file all of the stadard parameter settigs that you wat to use wheever you ru CI, the specify all automatio parameters i your custom ruciauto.bat file. Usig this method, you ca the use the same CI.ii for ruig CI ad CI Automatio, ad have the parameters that apply to oly CI Automatio i ruciauto.bat. Although the.bat file you start with may be slightly differet, this example shows oe that is modified to ru CI Automatio. Autoru specific parameters are show i bold: java -cp CI.jar;extlib\sqljdbc.jar;extlib\gu-regexp.jar com.baa.aci.cache tool:pc autoru:accept traslateutraslatedaccouts:y updatestaleaccouts:y outputsecuritytraslatios:y Note: Be certai to costruct the java commad startig with the commad defied i your ruci.bat (first lie i the example). Your java commad may be differet tha the oe used i this example. 9

13 I this example, CI Automatio will traslate all utraslated accouts ad update all stale accouts before importig. It will also geerate a secodary output file that lists the maual security traslatios. Note: The parameter outputsecurittraslatios is ot specific to automatio. However it is likely to be used more i CI Automatio because there is o other way to look at the security traslatios. The same is true for the other output parameters that cotrol export of secodary files. If You Ru CI Automatio ad Need to Ru CI Maually If you have bee ruig CI Automatio ad wat to ru CI maually (via the user iterface), follow these steps. Note: These istructios assume that there is a separate.bat file for automatio as recommeded, ad it is called ruciauto.bat. 1. Compare your ruciauto.bat file ad your ruci.bat file. Esure that ay of the CI parameters described i your Custodial Itegrator User Guide are cofigured idetically for CI (maual mode) ad CI Automatio (automated mode). Although it is recommeded that those CI parameters be set i the CI.ii file, they may have bee set i the automatio ruciauto.bat file. If they are, make sure they are set idetically i the ruci.bat file. Examples of CI parameters are sqlhost, dbame, ad defaultaccoutidetifier. Note: Do ot set CI Automatio parameters (which are described i this guide) to ru CI. Refer to Set Up ad Ru CI Automatio, page 7 for a explaatio of where to set parameters. 2. Esure that CI Automatio is ot ruig ad will ot start while you are ruig CI maually. WARNING: You must ot ru CI ad CI Automatio at the same time agaist the same database. Whe CI rus (i either mode) it accesses the database ad puts state iformatio ito it. If you wat to switch from CI Automatio to ruig CI maually, stop the automated ru first, or allow it to fiish before ruig CI maually. 3. Start CI usig ruci.bat. MONITOR CI AUTOMATION ON AN ONGOING BASIS How to Verify Results This sectio describes methods you ca use to moitor CI Automatio ad to determie whether it is producig the output you expect ad whether there are exceptios that you should maage. I geeral: Examie the primary output files. Examie the secodary output files. Examie the CI log file ad CI Automatio history log file for exceptio hadlig. Details about CI Automatio Behavior The followig list describes the behavior of CI Automatio. Output files are always overwritte. There is o optio to prevet output files from beig overwritte. 10

14 Ivalid accout traslatios are disabled. This behavior allows the automatio ru to cotiue as if o ivalid traslatios existed; the output files will cotai the data for oly the valid accout traslatios. Fatal errors stop the automated executio. No-fatal errors do ot stop automated executio. Primary Output Files As part of its ormal processig, CI outputs primary data files that are iteded to trasfer data to aother system. The primary output files that ca be produced deped o the variat of CI that is beig ru, as well as the cofiguratios set i CI. The primary output files typically iclude trasactios ad positios. Depedig o the variat ad settigs, they could iclude: Accouts Securities Prices Iitial positios Positio Lots Examie the Primary Output Files Look i the output folder idetified i Setup i the CI user iterface ad esure that CI Automatio has produced the files you expect, ad that they have the cotet you expect. I most cases, you would expect to have a positio file. If so, check that it has oe or more positios ad that the data is as expected. Likewise, if you have a trasactio file, verify that it has oe or more trasactios ad that the data is as expected. If Primary Output Files are Missig If CI Automatio does ot produce ay primary output files, check the Setup settigs i CI. Those settigs cotrol which primary output files are produced by both CI ad CI Automatio. WARNING: You must ot ru CI ad CI Automatio at the same time agaist the same database. Whe CI rus (i either mode) it accesses the database ad puts state iformatio ito it. If you wat to switch from CI Automatio to ruig CI maually, stop the automated ru first, or allow it to fiish before ruig CI maually. 1. Ru CI ad check the Setup optios. 2. I the Cofiguratio tab, verify the path for the output folder. 3. Esure that Import/Export Defaults are set. Those settigs cotrol which primary output files are produced by CI or CI Automatio. The optios available are differet for each variat of CI. 4. Verify that CI has write permissios to the output folder set o the Cofiguratio tab. 5. Esure that the autoru optio i your automatio script (ruciauto.bat) is set to either Export or Accept. 6. Check the CI logs for errors or iformatio messages that may explai why ay primary output file did ot get geerated. 7. If o primary data was exported, check for the presece of the Udefied Securities file. If the file exists ad is ot empty, that may explai why Export was ot performed. 11

15 8. Ru CI Automatio agai. Secodary Output Files CI Automatio ca be cofigured usig parameters to geerate secodary output files, which ca provide additioal iformatio about the data produced. They iclude: Accout Traslatios Missig Prices Security Traslatios Stale Accouts Utraslated Accouts Utraslated Securities For detailed descriptios of Secodary Output files refer to Appedix B: Examples of Secodary Output Files. For the parameters used to output these files, refer to Appedix A: CI Automatio Parameters. Examie Secodary Output Files Examiig secodary output files ca be a importat part of moitorig the behavior of CI Automatio. CI Automatio geerates secodary output files, such as security traslatios, if parameters are set to geerate them. Look i the output folder idetified i Setup i the CI user iterface ad examie the cotet of the secodary files. Determie how to address ay issues. For example, if there are utraslated securities you ca set up the traslatios i CI maually the re-ru CI Automatio. I geeral, look for uexpected values i output fields ad for empty fields that are expected to cotai values. Also check the log files to see if there was ay error. Log Files Look for error iformatio ad istructios i the log ad history log files. CI ad CI Automatio write exceptios to log files. Whe CI Automatio rus, exceptios are writte to two differet types of log files. By default the log files are foud i the log subfolder of your CI workig folder. baaci<xxxxxx>.log files These are the ormal log files produced whe CI rus. AutoRuHistory.log file This is the automatio log file. CI Automatio does ot produce status messages that are visible to a user. Istead, it writes messages to this log file. Exceptio messages are appeded to this file for each ru of CI Automatio. If you delete or reame the file, the ext ru of CI Automatio will create a ew oe. Examie the AutoRuHistory.log file The AutoRuHistory.log cotais a ruig history of each CI startup, Accout Update, Import step, Export step, Accept step, ad CI exit. (Import, Export, ad Accept are ru if they are set i the autoru parameter.) Some examples of successful rus that show i the AutoRuHistory log are show i Appedix C: AutoRuHistory.log File Examples. 12

16 I the AutoRuHistory.log file, look for fatal errors that stop executio, as well as o-fatal errors that do ot stop executio. Fatal Errors that Stop Executio I the AutoRuHistory.log file, look for fatal errors that stop executio. Fatal errors are logged with the Requested steps icomplete message. For example: :25:47.194: mai: Custodial Itegrator V started with database 'BaaWpAci' o SQL host 'localhost' :25:47.210: mai: Error: A error occurred durig auto executio. Ifo: No import optios selected :25:47.210: mai: Requested steps icomplete :25:47.210: mai: CI exitig. Fatal errors that stop executio iclude: CI is uable to read the CI.INI file CI is uable to process commad lie parameters CI is uable to access the SQL Server database No import optio is selected The specified workig folder caot be foud The specified output folder caot be foud There are utraslated securities (CI for PC, Axys, or APX) Ay error whe tryig to access the iteret or the URL via the iteret CI fails to import AccoutView data CI fails to create or write to output files The custom traslatio file, CiTxTrasCommoCust.xml, is ot correct The trasactio filter file, CiTxFilter.xml, is ot correct The positio filter file, CiPosFilter.xml, is ot correct No-Fatal Errors that Do Not Stop Executio For o-fatal errors, the error is logged, the curret step that is beig executed is termiated, ad the ext step is attempted. No-fatal errors iclude: Ay error durig the Accout Update process. Import errors, such as loadig ay target system data (such as Axys security ad price iformatio) or validatig security data (utraslated securities ad duplicated data). Utraslated securities for CI Uiversal, TLM, ad TLM ODemad. Ivalid accout traslatios. Ivalid accout traslatios are disabled i CI Automatio, allowig automatio ru to cotiue as if o ivalid traslatios existed; the output files will cotai the data for oly the valid accout traslatios. The etry i the AutoRuHistory.log file looks like this: Error: Foud ivalid accout traslatios. Ifo: Ivalid traslatios have bee disabled. 13

17 Appedix A: CI AUTOMATION PARAMETERS Most of the parameters described i this sectio are specific to CI Automatio ad should be set i the ruciauto.bat file. However, the output parameters apply to CI ad CI Automatio. You may eed additioal parameters for your variat of CI beyod those show here, ad typically those as well as the output parameters should be set i the CI.ii file so they are used for both CI ad CI Automatio. The additioal parameters are described i your Custodial Itegrator Istallatio Guide alog with details about how to set them. Before settig ay parameters, refer to Set Up ad Ru CI Automatio, page 7 for details. Note: These CI Automatio parameters listed here are i alphabetical order, ot i order of importace. Parameter label autoru Parameter value(s) SETUP IMPORT EXPORT ACCEPT Default Descriptio Commo? oe The autoru parameter tells CI to ru automatically ad ot display ay user iterface. Iclude oe of the followig values to desigate the step up through which to ru CI Automatio. NOTE: This value is urelated to the maual Setup step i CI ad does ot replace the eed to make settigs i the user iterface as described o page 4. o o o o SETUP is a pre-import settig that causes CI Automatio to oly perform some iitial operatios ad the exit before the Import step. Certai secodary files ca be exported by ruig the SETUP step oly ad settig the associated output parameters. See Appedix G: Example of Secodary File Output for a example of a case where settig autoru=setup ca be helpful. IMPORT meas do Import step oly. EXPORT meas do Import ad Export steps. ACCEPT meas do Import, Export, ad Accept steps. Note: It is recommeded that while you are settig up ad testig CI Automatio that you do ot use the ACCEPT value. If you set parameters to request Yes 14

18 Parameter label acceptfile acceptfiletimeout Parameter value(s) (file ame) defaultnumpricedays icludelots Default Descriptio Commo? secodary files, refer to Export of Secodary Files with CI ad CI Automatio, page 6. Used with acceptfiletimeout, provides cotrol over whe the Accept step is executed durig a automated ru. If the autoru parameter is set to ACCEPT ad the acceptfile parameter is set to a fileame, the the Accept step will ot occur util the specified file exists or util the timeout occurs. See the acceptfiletimeout parameter for timeout cotrol. (umber) 10 Value is the umber of miutes to wait for acceptfile to appear. See acceptfile parameter descriptio for more details. (umber) y (The umber of days sice the last CI export, with a maximum of 6 days.) Whe this parameter is set i CI or CI Automatio it specifies the umber of days of historical price data that CI should import ad export to the Price ad Security files. The valid rage of values is 1-6 where 1 idicates oly the previous busiess day. CI will use either the umber of days from the parameter (if specified), or the umber of days sice last CI export, whichever is greater. Because positio lots (tax lots) should be imported ifrequetly, they are cotrolled with a parameter istead of with a default settig. Whe this parameter is set to y, CI Automatio imports positio lots i additio to other data (securities, positios, etc.) cofigured to be imported. Cosider creatig a additioal.bat file that is for automated tax lot gatherig ad use it sparigly. Note: This parameter is oly effective for CI Automatio, ad oly whe Rare Rare Yes Rare 15

19 Parameter label Parameter value(s) outputaccouttraslatios Default Descriptio Commo? your firm is licesed for positio lot data. Refer to the Positio Lots Guide. y Whe this parameter is set to y (yes), CI or CI Automatio geerates the accout traslatios file ACCTTRANSLATIONS_<date>.csv. Yes Whe this parameter is set i cojuctio with the autoru=setup or autoru=import, the file is exported to the output folder (as defied i the Cofiguratio tab) as soo as CI Automatio exits. See Export of Secodary Files with CI ad CI Automatio page 6 for additioal iformatio about output of secodary files i CI versus CI Automatio. For details about the file cotet ad a example of the file refer to Appedix B: Examples of Secodary Output Files. outputallsecodaryfiles y This is a catch-all parameter for secodary output files. Whe set to y, this parameter eables all of the output parameters (Missig prices file is oly exported if Prices are also exported.) Yes outputmissigpricefile y Whe this parameter is set to y (yes), CI or CI Automatio lists missig prices i the MISSINGPRICES_<date>.csv file. Yes Ulike the other secodary files that ca be output i CI Automatio without ruig the Export step, output of the MISSINGPRICES file requires ruig the Export step ad havig the Iclude prices optio selected i CI Cofiguratio scree. For details about the file cotet ad a example of the file refer to Appedix B: Examples of Secodary Output Files. 16

20 Parameter label Parameter value(s) outputsecuritytraslatios y Default Descriptio Commo? Whe this parameter is set to y (yes), CI or CI Automatio geerates the file listig maual security traslatios (SECTRANSLATIONS_<date>.csv). Whe this parameter is set i cojuctio with the autoru=setup or autoru=import, the file is exported to the output folder (as defied i the Cofiguratio tab) as soo as CI Automatio exits. See Export of Secodary Files with CI ad CI Automatio page 6 for additioal iformatio about output of secodary files i CI versus CI Automatio. Yes For iformatio about the cotet ad format of the Security Traslatios.csv file, refer to Appedix B: Examples of Secodary Output Files ad to your Custodial Itegrator User Guide. outputstaleaccouts y Whe this parameter is set to y (yes), CI or CI Automatio geerates the stale accouts file (STALEACCT _<date>.csv). Yes Whe this parameter is set i cojuctio with the autoru=import the file is exported to the output folder (as defied i the Cofiguratio tab) as soo as CI Automatio exits. See Export of Secodary Files with CI ad CI Automatio page 6 for additioal iformatio about output of secodary files i CI versus CI Automatio. For details about the file cotet ad a example of the file refer to Appedix B: Examples of Secodary Output Files. Note: The file may iclude a Export Positios colum cotaiig yes or o values to idicate whether or ot the positios were exported. Settigs i the CI user iterface cotrol whether this colum appears. To 17

21 Parameter label Parameter value(s) outpututraslatedaccouts y outpututraslatedsecurities y Default Descriptio Commo? iclude it, either or both Advaced settigs for the Iclude recociliatio positios for export optio must be selected. That optio appears o the Cofiguratio tab. Whe this parameter is set to y (yes), CI or CI Automatio geerates the utraslated accouts file, which is UNTRANSLATEDACCT_<date>.csv. Whe this parameter is set i cojuctio with the autoru=setup or autoru=import the file is exported to the output folder (as defied i the Cofiguratio tab) as soo as CI Automatio exits. See Export of Secodary Files with CI ad CI Automatio page 6 for additioal iformatio about output of secodary files i CI versus CI Automatio. For details about the file cotet ad a example of the file refer to Appedix B: Examples of Secodary Output Files. Whe this parameter is set to y (yes), CI or CI Automatio will geerate two comma-separated values (.CSV) files of iformatio about Utraslated Securities (exceptios). o o The UNDEFSEC_<date>.csv file cotais basic iformatio i a row for each utraslated security, icludig security ame ad the ame of the fiacial istitutio i which it was refereced. The UNDEFSECDET_<date>.csv file provides more detailed iformatio about each holdig ad/or trasactio that refereces the securities listed i the first file. It lists the holdigs followed by the trasactios. Whe this parameter is set i cojuctio with the autoru=import the file is exported to the output folder Yes Yes 18

22 Parameter label Parameter value(s) traslateaccoutvalidatiostatus all validatedoly traslateearliesttxdate Valid date i format: YYYYmmDD traslateutraslatedaccouts y Default Descriptio Commo? validatedoly (Today s date) (as defied i the Cofiguratio tab) as soo as CI Automatio exits. See Export of Secodary Files with CI ad CI Automatio page 6 for additioal iformatio about output of secodary files i CI versus CI Automatio. For examples of these files refer to Appedix B: Examples of Secodary Output Files. How your variat of CI behaves whe there are utraslated securities is described i How CI Variats Hadle Utraslated Securities, page 27. This parameter is used oly whe traslateutraslatedaccouts=y This parameter specifies whether CI Automatio will traslate both validated ad o-validated accouts, or oly validated accouts. o o all meas traslate both validated ad o validated accouts validatedoly meas traslate oly validated accouts This parameter is used oly whe traslateutraslatedaccouts=y Specifies the earliest trasactio trade date that will be used to filter trasactios o the first dowload. Oly trasactios with trade date o or after this date will be delivered i the first dowload. Whe this parameter is ot set or is set to (o), CI Automatio will skip utraslated accouts ad will ot import or export them. If set to y (yes), CI Automatio will automatically traslate (map) each utraslated accout to a default accout idetifier. For this parameter to work, there Yes Yes Yes 19

23 Parameter label Parameter value(s) Default Descriptio Commo? must be a default accout idetifier. The field used as the default accout idetifier depeds o the variat of CI. o o o For CI Uiversal, CI for PC, ad CI for UPC, WP Accout Number is used as the accout idetifier uless a differet field is set usig the defaultaccoutidetifier parameter. If the defaultaccoutidetifier parameter is set to WebPortfolio accout umber or WebPortfolio accout ame the whe automatio is traslatig utraslated securities it automatically appeds the iteral accout ID to ew duplicate accout idetifiers to make them uique. Note that if the total legth exceeds the maximum accout idetifier legth, the the WebPortfolio accout umber or WebPortfolio accout ame (depedig o the defaultaccoutidetifier settig) will be trucated to make it fit ito the accout idetifier legth. For CI for Axys ad CI for APX, there is o default uless it is set usig the defaultaccoutidetifier parameter. For CI for TLM ad CI for TOD, there is o cocept of "Accout Idetifier" ad those variats do ot recogize the defaultaccoutidetifier parameter so the accout is always idetified by its WP Accout Number. Whe mappig a accout, the user does ot specify a "accout idetifier" as i the other CI variats. I TLM/TOD the user ca istead specify the accout currecy (set by default to the value specified by the defaultaccoutcurrecy parameter or to USD if this parameter is ot set). Whe traslateutraslatedaccouts is set the CI Auto for TLM ad TOD 20

24 Parameter label Parameter value(s) updatestaleaccouts y Default Descriptio Commo? automatically sets each accout currecy to the default value.if defaultaccoutidetifier is available for the variat of CI you are usig, it is listed ad described i your Custodial Itegrator Istallatio Guide. Whe updatestaleaccouts is set, stale accouts are updated after utraslated accouts are traslated ad prior to the Import step. If set to y (yes) ad the autoru parameter is specified, the CI Automatio will perform a Demad Update ad force the gatherig of data from the custodia o stale accouts prior to the Import step. CI Automatio cosiders a accout to be stale if the last update date of the accout is ot today. CI performs the Demad Update oly oce ad accouts may still be stale after this update completes. CI Automatio will ot request a update o demad for accouts that show a 1007 error (bad user logi or password). You should review these accouts, update the logi ad/or password i AccoutView, ad the request a maual update. Repeated attempts to Demad Update a accout with a bad logi or password may result i the accout beig locked out by the Fiacial Istitutio ad will the require cotactig the istitutio to have access re-eabled. Yes 21

25 Appedix B: EXAMPLES OF SECONDARY OUTPUT FILES CI Automatio ca geerate secodary output files, depedig o parameters described i Appedix A: CI Automatio Parameters. Note that the ames of files output by CI Uiversal are prefixed with BAA. Accout Traslatios file Missig Prices file Security Traslatios file Stale Accouts file Utraslated Accouts file Utraslated Securities file ad Detailed Udefied Securities file Accout Traslatios file The outputaccouttraslatios parameter causes CI Automato to geerate the ACCTTRANSLATIONS _<date>.csv file. The followig is a example of a Accout Traslatios file: ACCO UNT ID WP ACCOUN T Charles Schwab Istl Merrill Lych Direct WP ACCOU NT # INSTIT UTION Charles Schwab, Ic. - Istitut ioal (Ivest met) Merrill Lych Direct (Ivest met) ACCOU NT UPDAT E STATU S INFO Update success ful o Mar 7, :13: 47 AM Update success ful o Mar 7, :17: 05 AM LAST UPDAT ED LAS T EXP ORT ED TX DATE VALIDA TOR MM SYM BOL SWEE P SYMB OL EXPO RT POSIT IONS IF STALE IMPORT STATUS TRANS LATION STATE Yes Eabled Valid Yes Eabled Valid Missig Prices file The outputmissigpricefile parameter causes CI Automato to geerate the MISSINGPRICES _<date>.csv file. The followig is a example of a Missig Prices file: SYMBOL TYPE NAME WP ACCOUNT INSTITUTION MUTUALFUND FRANKLIN AGE HIGH INCOME-C Merrill Lych Direct G0070K103 STOCK ACE LTD Charles Schwab Istl STOCK AUTOMATIC DATA PROCESSING Merrill Lych Direct STOCK AFLAC INC Charles Schwab Istl STOCK AMGEN INC Charles Schwab Istl STOCK ANADARKO PETROLEUM CP Charles Schwab Istl STOCK America Express Charles Schwab Istl

26 Security Traslatios File The outputsecuritytraslatios parameter causes CI Automatio to geerate the SECTRANSLATIONS_<date>.csv file. The followig is a example of a Security Traslatios file. See your Custodial Itegrator User Guide for more iformatio about this file. WP Name WP Ticker WP Cusip Istitutio WP Accout Symbol Symbol Type Security Type Created Last Modified D PEPPER GROUP UBS AG REG Charles Schwab, Ic Istitutioal (Ivestmet) Merrill Lych Direct (Ivestmet) DPS Ticker STOCK CX3 Ticker MUTUAL FUND Stale Accouts File The outputstaleaccouts parameter causes CI Automato to geerate the STALEACCT _<date>.csv file. The followig is a example of a Stale Accouts file: WP Accout Accout # Istitutio Stale Positios Charles Schwab Ist Modified Utraslated Accouts File The outpututraslatedaccouts parameter causes CI Automato to geerate the UNTRANSLATEDACCT_<date>.csv file. The followig is a example of a Utraslated Accouts file: WP Accout Accout # Istitutio Last Updated Validatio Charles Schwab Ist Charles Schwab, Ic Istitutioal (Ivestmet) Merrill Lych Direct Merrill Lych Direct (Ivestmet) Utraslated Securities Files The outpututraslatedsecurities parameter causes CI Automatio to geerate two output files: the utraslated securities file UNDEFSEC_<date>.csv. the detailed utraslated securities file UNDEFSECDET_<date>.csv. This file icludes the same detailed iformatio as i the CI user iterface, i two separate tables. The followig is a example of a utraslated securities file (UNDEFSEC_<date>.csv): NAME AMERIPRISE FINANCIAL INC DUKE REALTY CORP HESS CORPORATION SCH ADV CASH RESRV PREM TRAVELERS COMPANIES INC FINANCIAL_INSTITUTION Charles Schwab Ic. - Istitutioal (Ivestmet) Merrill Lych Direct (Ivestmet) Charles Schwab Ic. - Istitutioal (Ivestmet) Charles Schwab Ic. - Istitutioal (Ivestmet) Charles Schwab Ic. - Istitutioal (Ivestmet) 23