Integrator Command-Line Client

Transcription

1 Integrator Command-Line Client In addition to the web-based Integrator component of Jedox Web, Integrator projects can also be run from the command line by using the CLI tool EtlClient. This tool is located in the directory \tomcat\client. To open it, use the command etlclient.bat (Windows) or etlclient.sh (Linux). Entering -h will call up a list of the valid parameters, which are described in Command-Line-Client-Parameters. The following tasks CLI tasks are described in this article: Setting up authentication Creating server profiles Executing Integrator projects Generating project documentation Authentication Authentication with a Jedox In-Memory DB Server user is required for communication with the Jedox Integrator Server. Valid login credentials must be provided for each call of the CLI; the user can be set with option -u and password with option -pw. The IP address of Jedox Integrator is set with option -s (default value for the server is

2 ). For example, to start the Jedox Integrator project example samplebiker on a locally installed Jedox Integrator, enter the following at the command-line level: To add the project to the server: etlclient a \samples\samplerelational -u admin -pw admin To run the standard job: etlclient -p samplerelational -u admin -pw admin Server Profiles Instead of providing login credentials with each CLI call, you can user server profiles. Server profiles are created with option -spc. To use all options for a defined profile, you can use option -sp. Examples: etlclient.bat -spc MyProfile -s localhost -u admin - pw admin etlclient.bat -sp MyProfile -a samples\samplebiker etlclient.bat -sp MyProfile -p samplebiker All profiles are stored in the local file.\client\config\profiles.xml. If no profile is given, the profile with name default is used. In this case, you can omit the option -sp.

3 etlclient.bat -spc default -s localhost -u admin -pw admin etlclient.bat -p samplebiker Note: the Integrator component of Jedox Web and the command-line client work with the same Jedox Integrator. This means that jobs that were created with the Jedox Integrator web client can also be run from the command-line client. Executing a Jedox Integrator Project To execute an Jedox Integrator project, the Jedox Integrator project definition must be entered using option -p. A load and job defined in the Jedox Integrator project can then be executed. Two loads or jobs can be executed simultaneously, but only if a new job containing the desired jobs or loads has been created. If no job is determined, the job named default is executed. Examples: etlclient p samplebiker etlclient -p samplebiker -j Masterdata etlclient -p samplebiker -l Years Note: in all examples, it is assumed that login credentials are provided in the default server profile.

4 Test run Before executing loads or jobs, it is advisable to display on the screen the data sources taken as a basis. Any data source, regardless of whether it is an extract or a transformation, can be shown. For treebased data sources, the display is shown in the full hierarchy format, without attributes and levels. The number of display lines can be determined. Examples: etlclient -p samplebiker -d Products etlclient p samplebiker -d Products -n 20 The test data can also be written to a file with the > shell option. etlclient -p samples\samplebiker -d OrdersData_Trans> testfile.txt Variables The job execution can be defined more accurately using context variables. Even if a job has already been assigned variables, the variables shown at the command line level are used. Spaces must be put in quotation marks ( ) in order to be used as variable values.

5 Notation: -c <Variable1>=<Value1> <Variable2>=Value2>. or -c <Variable1>= <Value1> <Variable2>= Value2>. Examples: etlclient -p samplevariables -j SalesMonth -c Year=2006 Month=Dec etlclient -p samplevariables -j SalesMonth -c Year= 2006 Month= Dec Log If a job or load is executed from the command line level, the execution protocol is displayed directly on the command line. It contains information, warnings, and error messages that appear during the Jedox Integrator process. The level of detail during logging can be entered with the log level on the command line using option ll. Possible values are OFF, FATAL, ERROR, WARN, INFO, DEBUG, and ALL. The following final messages exist for an Jedox Integrator process: Completed successfully Completed with Warnings Completed with Errors Failed

6 The log display can be written to a file with the > shell option. etlclient -p samples\samplebiker j Initdata > InitdataLog.txt Special characters: For Windows, the following special characters must be double quoted if they are used as values for any command line option: & \ < > ^ <space>. For some characters, an additional prefix is required:! : ^! and : etlclient -p "My Project &" -j "Go^!" Exit codes Executing commands with the Jedox Integrator command line tool will return an Exit code that can be used later within the batch script: 10 Execution ended with Status Successful 20 Execution ended with Status Warnings 30 Execution ended with Status Errors 40 Execution ended with Status Failed

7 50 Execution ended with Status Stopped 60 Execution ended with Status Aborted 0 No error occurred (other commands than Execution) -1 Jedox Integrator was not reachable -2 Error on Client or Server side (e.g. Project/Job not found, Project not valid) Note: executions start with Jobs (-j) or Loads (-l), Data preview (-d), and Test (-t). call etlclient.bat -s localhost: p samplebiker -j default rem print the status code of the execution (in this case it will be 10 which means successful) echo Exit Code is %ERRORLEVEL% Execution states The following states are possible after an Integrator execution. Numeric Code String Code Description 0 Queued The Execution is present in the {@link IExecutor Executor} but not yet running. 5 Running The Execution is currently being executed and still running. 6 Paused The Execution is currently being executed and still running. 10 Completed successfully The Execution finished without encountering any errors or warnings. Everything is perfect. This is how things should be! The Execution is finished and has encountered one or more 20 Completed with Warnings warnings indicating that there are some problem within the data causing some data not to be processed successfully. No structural problems (errors) occurred.

8 The Execution is finished and has encountered one or more errors causing parts of it to fail. Nevertheless the execution finished 30 Completed with Errors trying to execute all of its parts. Note: This status can only occur if failonerror (set in config.xml) is false. The Execution has encountered an unrecoverable error and thus was terminated premature for internal fault reasons. Some parts 40 Failed of it may not have been executed. Note: If failonerror (set in config.xml) is true (default), this is the status returned, when any error occurs within the execution. 49 Stopping The Execution was tried to be terminated but it is still running. 50 Stopped The Execution was terminated premature by user intervention. Some parts where executed, some parts where not. 54 Interrupting The Execution was interrupted by user intervention 55 Interrupted The Execution was interrupted by user intervention The Execution was canceled by IExecutor#stop(Long) user 60 Aborted intervention} before it reached status running. No action was done by it. An aborted execution can NOT be reactivated and started again. The Execution is not valid and thus will not be executed. This 70 Invalid indicates that the configuration of at least one of the involved components is faulty, so that this component cannot be created and initialized. Generating project documentation Notation etlclient -pd <projectname> -j <jobname1> <jobname2>. -o <directory> 1. Create project documentation for project samplebiker without flow graphs in directory C:\EtlDoku1: etlclient -pd samplebiker -o C:\EtlDoku1 2. Create project documentation for project samplebiker with flow graphs of job Masterdata and Cubedata:

9 etlclient -pd samplebiker -j Masterdata Cubedata -o C:\EtlDoku2 3. Create documentation for job Cubedata in project samplebiker. It only contains the referenced components of this job and the flow graph of job Cubedata: etlclient -pd samplebiker.jobs.cubedata -o C:\EtlDoku3 The documentation can be displayed by opening file etldocumentation.html with a browser. Note that Internet Explorer 9 doesn t display the graphical flow graph due to insufficient support for the SVG format. The following files are generated in the output directory: Contains all project- and component-related information, etldoc.xml SVG flow graphs, references, and all component fields (references etldoc.xslt). The XSTL transformation of etldoc.xml. By adapting this etldoc.xslt file, the layout and content of the documentation can be modified (references etldoc.css). etldoc.css A style sheet (references etldoc_bg.png). etldoc_bg.png A background image. The documentation file to be displayed in the browser. etldocumentation.html Technically, it is etldoc.xslt transformation applied to etldoc.xml (requires only files etldoc.css and etldoc_bg.png).

10 Progress logs It is possible to generate additional info messages indicating the progress of a job or a data preview. This is helpful for the analysis of long-running Jedox Integrator jobs with high data volume. For one or several components of the Jedox Integrator job (extract, transform, or load), a log block size can be indicated as a command line parameter. Each time the component has processed a block, an info message will appear. Notation: -c #<componentname>.logblocksize=<n> -p samples\samplebiker -j Cubedata-c #samplebiker.extracts.orderlines.logblocksize=500 #samplebiker.loads.orderscube.logblocksize=2000 This will lead to these additional log messages: INFO : 1 rows processed from extract Orderlines INFO : 500 rows processed from extract Orderlines INFO : 1000 rows processed from extract Orderlines INFO : 2000 rows processed from load OrdersCube