Understanding the internal execution order of flows and steps

Transcription

1 HP OO 10 OnBoarding Kit Community Assistance Team Understanding the internal execution order of flows and steps What happens when a flow is run? What happens when it reaches a step? What gets calculated first? Most authors ask themselves these questions sooner or later. This document describes the execution of flows and flow steps by HP OO s execution engine. It also gives a rich set of examples, each centered on specific cases and highlighted by application-specific screenshots. The document was designed for intermediate and advanced HP OO Studio authors. It can be read at once from start to finish or simply by identifying a specific case and focusing onto it. Feel free to choose the option that better fits your needs. What we offer Overview... 2 Flow step execution... 2 Entire flow execution... 2 Flows containing parallel or multi-instance steps... 3 How can knowing HP OO s general execution assist you?... 4 Example of mixed usage of a scriptlet in the result filter, a scriptlet in the operations and in the step... 4 Example of how multi-instance step results can be merged together... 6 Example of how parallel step results can be merged together... 9 Example of enhanced variable manipulation Example of encapsulating steps to sub-flows and make use of a flow s primary output Flow level inputs in Studio and Central Example of Single Value vs List of Values as Prompts Example of Assign From with Otherwise as Constant vs <not assigned> with Otherwise as Constant Example of Assign From when the variable set exists vs when the variable set does not exist Example of Required flow level inputs HP Operations Orchestration HP Operations Orchestration (HP OO) is a next generation IT Process Automation solution that is designed from the ground up to increase automation adoption whether in a traditional data center or hybrid cloud environment. OO Community This guide is created for the OO community. To get started with HP OO 10 check out our community onboarding kit that contains four persona-based learning tracks for administrators, flow authors, end users, and integrators. Each track provides knowledge assets of the following types: Presentations Videos Tutorials Guides Best Practices HP Operations Orchestration, June 2015

2 Overview Each HP OO flow consists of steps and connections between them. When we reach a step during execution, a set of micro-steps are performed at the step level. The next section describes these steps and the order in which they are performed. Flow step execution Each step is executed in the following order: 1. The inputs are read top-down. (This includes variables that are called inside constant values, using the ${var} syntax.) 2. The worker group is assigned. 3. The action is performed. 4. The action s outputs are read top-down. (In the case of an error, the step will end at this point.) 5. The step s response is chosen. (Response names and rules are checked here.) 6. If operation level scriptlets exist, they are executed. 7. The step results are calculated. (Result filters are calculated here.) 8. If step level scriptlets exist, they are executed now. 9. The step ends. 10. According to the response, a transition is made to the next step. Entire flow execution After looking at the details of each flow step, let s look at the entire flow. Flows are executed in the following order: 1. The flow starts. 2. Flow inputs are read. 3. Flow steps are run. 4. Flow outputs are read. 5. If flow level scriptlets exist, they are executed. HP Operations Orchestration July 2015 Page 2 of 27

3 6. The flow s primary output is calculated. 7. If an Acquire Lock step was used within the flow and a Release Lock step was not, the locks are cleared at this stage. 8. A check is done to ensure all the flow branches have ended. 9. The flow ends. How locks are cleared The locks clearing step (step #7) clears all the HP OO system locks that have been acquired during the flow execution (except for locks acquired inside branches). Locks are also cleared if an exception occurs, or if an execution stops. Note that the locks mentioned are based on out-of-the-box operations and flows that are part of the HP Solutions content pack. 1. /Library/Integrations/Hewlett-Packard/Operations Orchestration/Acquire Lock 2. /Library/Integrations/Hewlett-Packard/Operations Orchestration/Release Lock 3. /Library/Integrations/Hewlett-Packard/Operations Orchestration/Wait For Lock A community guide regarding the usage of locks, is available here. Subflow execution Since a subflow is a step that encapsulates an entire flow internally, the mechanism is identical to the one described above. Flows containing parallel or multi-instance steps A parallel split step is a set of step sequences that are carried out simultaneously. Parallel split steps are best used for doing dissimilar things simultaneously and independently of each other. A multi-instance step is a step that executes simultaneously on multiple targets. Multi-instance steps are best used for doing the same thing simultaneously with different data on different targets. When the flow reaches a parallel/multi-instance split step, the following execution order is performed: 1. The inputs are read. 2. The parallel split is performed. 3. A logical step that starts the branch (lane) is run. 4. The lane s steps are run in the same way as described in Flow step execution above. 5. A logical step that ends the branch is run. 6. The system checks that all the branches are done. 7. The results from the separate lanes are merged. 8. If the step contains a scriptlet, it is run now. HP Operations Orchestration July 2015 Page 3 of 27

4 9. The flow continues and ends as a simple flow, as described in Entire flow execution above. How can knowing HP OO s general execution assist you? We have created some examples that demonstrate how understanding the order of execution can make the difference between efficiency and endless trial and error. Example of mixed usage of a scriptlet in the result filter, a scriptlet in the operations and in the step This is a short example showing how you can make use of operation level scriptlets and step scriptlets, as well as the benefits of having filtered results. This is what the example will showcase: Inputs are read Operation level scriptlets are executed Step results are calculated and filters are applied Step level scriptlets are executed Let s take the out-of-the-box Set Flow Variable operation and make use of it within a simple flow. The operation itself has an operation-defined scriptlet as shown below: We will place the operation in our flow and we will use it to: a. Define a flow variable called parentfolder that is used within the flow. b. Obtain a temporary folder called timestampfolder that is a result of this particular step (this folder can be used within the current flow as a uniquely identified folder). c. Create a global variable called childfolder that will be used outside of the flow (this folder can be used within a parent flow as a uniquely identified folder). We called our step Set Folder Structure and we placed the inputs as shown below. HP Operations Orchestration July 2015 Page 4 of 27

5 We defined the results as shown below: With a scriptlet as a result filter: To make it more fun, we added a step level scriptlet that creates the desired global variable: For example, at runtime we can end up with something like below. HP Operations Orchestration July 2015 Page 5 of 27

6 So what happens? In one single step: 1. A local variable is created: parentfodler = C:\ 2. A local variable is created, based on the filtered results of the step: timestampfolder = C:\<timestampvalue> (imagine that you can use this folder as a backup folder uniquely identified by the timestamp of the HP OO flow run) 3. A global variable is created with the help of the step level scriptlet: childfolder = C:\<timestampvalue> (you can have the same uniquely identified folder as global in order to be able to control if from outside its origin of creation). Thus the childfolder can be accessed at upper levels accordingly while the others can be used within the current flow as expected. Example of how multi-instance step results can be merged together This is a short example of how you can make use of understanding how multi-instance steps are executed in order to merge the results together and get real benefits. This is what the example will showcase: Lane split is performed Lane Steps are executed System checks for all lanes to be done The results from the separate lanes are merged We will place a multi-instance step within our flow and we will use it to: a. Trigger and execute (acceptance) tests in parallel for multiple versions of an application under test. b. Merge the results of the tests together for further usage. In order to achieve this, we created a multi-instance step with several steps inside: HP Operations Orchestration July 2015 Page 6 of 27

7 On the multi-instance (MI) step, we defined two inputs, both as List of Values (for the ease of example, we defined the inputs as constant and already assigned default comma-separated values). a. applicationtotest = our application under test b. testsuite = our tests to be executed for each application We then pass the above-mentioned inputs to a step from within the MI step, the Run Tests step. See below. HP Operations Orchestration July 2015 Page 7 of 27

8 This step triggers the actual tests and computes the results on the Results tab that we defined: Finally, it assigns the testresult to the Flow Variable. The interesting part is that we get back to the multi-instance (MI) step and define a result on it too in order to gather all the parallel results. As you can see we APPEND values from the testresult (which was defined inside the MI step) and additionally format it as desired and end up with a testresultlist, which is actually the merged results of the MI execution. For example, at runtime we can end up with something like below. HP Operations Orchestration July 2015 Page 8 of 27

9 So what happens? In the MI step defined above: a. The inputs are read. b. The lanes are split in three. (We have three comma-separated lists of values.) c. Lane steps are executed in parallel (in our case, testsuite is executed for the relevant applicationtotest). For example: app is tested using the test suite. d. The system waits for all lanes to complete. e. The results are merged together. (In our case, we end up with a comma-separated list of applicationtotest:testresult.) Example of how parallel step results can be merged together This is a short example of how you can make use of understanding how parallel steps are executed in order to merge the results together and get real benefits. This is what the example will showcase: Lane split is performed Lane Steps are executed System checks for all lanes to be done The results from the separate lanes are merged Parallel Step scriptlet is executed Note: The example is similar to the multi-instance step example. We have added the scriptlet execution to demonstrate how knowing when things occur can simplify your authoring experience. We will place a parallel step within our flow and we will use it to: a. Get different details in parallel from a targeted VM. b. Compute a list of VM details. In order to achieve this, we created the following parallel step: HP Operations Orchestration July 2015 Page 9 of 27

10 We added an input to control the targeted VM of all parallel lanes: We configured steps within the lanes with the relevant inputs and results. For example, the Get VM ID step has inputs and results as shown below. HP Operations Orchestration July 2015 Page 10 of 27

11 Similarly, the Get VM Disk Size step has a result assigned to Flow Variable and named VMDiskSize. The interesting part is that we get to the parallel step itself and create a simple step scriptlet to merge the parallel results together. Thus, we take the results of the different lanes and merge them into a single result. At runtime, we end up with what is shown below. HP Operations Orchestration July 2015 Page 11 of 27

12 So what happens? In the parallel step defined above: a. The inputs are read. b. The lanes are split in two. (We manually defined two distinct lanes.) c. Lane steps are executed in parallel. (In our case, one lane gets the VM ID while another one gets VM Disk Size.) d. The system waits for all lanes to complete. e. The results are merged together. (In our case, the parallel step context contains both the VMID and the VMDiskSize.) f. The parallel step scriptlet is executed. (In our case, the VMResult is created in both local and global context.) Example of enhanced variable manipulation This is a short example of how you can make use of understanding the order of the inputs, outputs, results and result filters computation. This is what the example will showcase: The inputs are read The outputs are read The results are computed (result filters are applied) HP Operations Orchestration July 2015 Page 12 of 27

13 As an actual example, we will use a flow with several simple steps that will: a. Fetch raw data from a server in JSON format. b. Iterate through the JSON elements and format them according to our needs. c. Add the formatted data to our final report. Let s say that for the ease of example, the raw data from our server looks something like below. {"executionid":" ","status":"completed","owner":"admin","ownerdomain":"internal","executionname":"demo01scheduled"}, {"executionid":" ","status":"paused","owner":"qa","ownerdomain":"internal","executionname":"demo02"}, {"executionid":" ","status":"running","owner":"dev","ownerdomain":"ldap","executionname":"[debug]demo03"} Let s say that what we want to achieve within our flow is to: a. Put ownerdomain and owner together in a single variable. b. Get rid of executionname. c. Get rid of all the tags and of all the characters, except the executionid tag. In other words, the result will be something like what is shown below. {executionid: ,completed,internal\admin}, {executionid: ,paused,internal\qa}, {executionid: ,running,ldap\dev}, Of course, there are various ways to do this, but we will show you one way to achieve it by simply knowing the way step and flow execution occurs. Below are the flow steps we mentioned. HP Operations Orchestration July 2015 Page 13 of 27

14 In the Fetch Data step, we get the actual raw data as shown below. We pass rawdata to the Iterator step in order to get each element one-by-one and pass it to the Filter and Format Data step. Thus, we get the raw data and extract elements one-by-one in a so-called rowdata variable as shown below. HP Operations Orchestration July 2015 Page 14 of 27

15 The interesting part is on the Results tab for Filter and Format Data As you can see, we filter field1 (which gets rowdata) to extract values of interest in our case, owner, ownerdomain and executionname. Afterwards, we create a finaldetails variable that gets data from the very same field1 and we filter this one with the help of the other results that were already defined on the same tab. Note that it is important to place this variable at the end of the list. Thus, the filters on the finaldetails variable look something like: We put owner and ownerdomain together as shown above. HP Operations Orchestration July 2015 Page 15 of 27

16 We replace the ownerdomain completely as shown above. Eventually, the flow achieves all the other things we mentioned in the beginning. Thus, at runtime, we end up with what is shown below. So what happens? On the Filter and Format Data step: a. The inputs are read. (field1 gets the value from the rowdata variable.) b. The outputs are read. (In our case, field1 is an output too and we can use it in the Results tab to pass its value to the relevant intermediary results.) c. The results are computed so that: a. The value from the origin is assigned. b. The value is filtered according to the defined filters. c. Any result that is already computed is available to be used on a subsequent result filter. (The finaldetails filter uses ${} variables defined previously on the same Results tab). HP Operations Orchestration July 2015 Page 16 of 27

17 Example of encapsulating steps to sub-flows and make use of a flow s primary output This is a short example of how you can make use of understanding how flow outputs work in a general flow and subflow usage. This is what the example will showcase: Flow inputs are read Sub-flow inputs are read Sub-flow primary output is being computed Sub-flow results are computed We will take the default flow in the previous example and replace one of its steps with a new flow that we create from scratch. Remember we had a step called Filter and Format Data with a proper logic inside that allowed us to take one raw input and transform it into a formatted result. Of course, that was good as such, but not enough. Now we want to let other authors benefit of the logic we defined while not being exposed to it. For this, we will create an additional flow as shown below. We will define a single flow level input, because this is what we will eventually need. The flow level input will be passed to the formatted step. HP Operations Orchestration July 2015 Page 17 of 27

18 The results of the Filter and Format Data step will remain the same with a single important modification. We will change the finaldetails result and assign it to a flow output field as below Next, we will go back to the Format Data Flow step and: a. Define a flow output. (In our case, formatteddata gets value from the previously defined output field.) b. Extract the primary output from our desired data. (In our case, the finaldetails output field.) Now we are ready to use the new flow in the default (parent) flow (see the previous example for more details). HP Operations Orchestration July 2015 Page 18 of 27

19 Note the changed Filter and Format Data Flow step. This step now has a different input and a different result from what it used to have, meaning rowdata from the Otherwise section is the flow level input we defined earlier that we want to get it passed to the step input. formatteddata is the flow level output we defined. As you can see, instead of having five results on this step, we now have one and all the logic occurs in the sub-flow. Thus, we are no longer exposed to it. HP Operations Orchestration July 2015 Page 19 of 27

20 Additionally, you can make use of the primary output and in the next step, (in our case, Add Data to Report) simply collect data using the Use Previous Step Result option. At runtime we end up with what is shown below. Step Filter and Format Data Flow Step Add Data to Report So what happens? When the run reaches the Filter and Format Data Flow step: 1. The inputs are read. (rowdata gets the relevant value). 2. The steps of the flow are run: a. The finaldetails output field is computed on the first step. b. The flow outputs are computed; thus, formatteddata gets the value from finaldetails. c. The primary output is computed too; thus, getting the value from finaldetails. HP Operations Orchestration July 2015 Page 20 of 27

21 3. The Filter and Format Data Flow ends; thus, we end up with formatteddata available now for the default (parent) flow. 4. The formatteddata result is computed and considering that we have no filter on it, the value is attributed as such. Flow level inputs in Studio and Central In this section, we look at how flow level inputs are treated at runtime or trigger time both in Studio and in Central. First, we will provide an overview of the input, in particular around the ways you can configure them at authoring time. There are a few things that are worth mentioning from the beginning. Inputs can be any of the following: Required or No Required Single Value or List of Values Assigned From and/or Assigned To Computed from: Constant Prompt User Previous Step Result System Accounts Logged-in User Credentials Let s see by example. HP Operations Orchestration July 2015 Page 21 of 27

22 Example of Single Value vs List of Values as Prompts In this example, we create a flow and define two flow inputs as shown below. Because Assign From is kept as <not assigned>, both inputs will take their values from the Otherwise section. In our case, this is set to prompt the user from a selection list that is available in HP OO Studio as part of the HP out-of-the-box Base content pack. In Studio, at runtime, you end up with two prompted inputs. One will allow the selection of a single value while the other one will allow selection of multiple values. In Central, at trigger time, you end up with a similar situation. Thus, the inputs can be controlled in a similar manner. Note that by using the CTRL or SHIFT keys, you can select multiple values for the List Of Values input. HP Operations Orchestration July 2015 Page 22 of 27

23 Example of Assign From with Otherwise as Constant vs <not assigned> with Otherwise as Constant In this example, we create a flow and define two flow level inputs as shown below. Note that one can get its value either from Assign From or the constant while the other one gets the value directly from the constant. In Studio, at runtime, you are not prompted for any inputs and you end up with both inputs being initialized to their constant values. In Central, at trigger time, (either go to Content Management > Flow Library and trigger a run for the flow or go to Run Management > Flow Launcher and select the same flow), you will end up with one input being prompted and its value already filled in with the constant one defined in Studio. This is the design and the expected behavior. In Central, at runtime, if you run the flow as such, both inputs will resolve to their constant values. Of course, if you alter the above input, the altered value will be assigned to it. HP Operations Orchestration July 2015 Page 23 of 27

24 Example of Assign From when the variable set exists vs when the variable set does not exist In this example, we create a flow and define two flow level inputs as shown below. We have two inputs taking values from var1 and var2 or otherwise going into a prompt for the user. We will make sure that at runtime, var1 will exist while var2 will not. Because we are discussing flow level inputs, the easiest way to achieve that is by actually creating a system property called var1 as shown below. In Studio, at runtime, you will be prompted for one input only. In Central, at trigger time, you will be prompted for no input. HP Operations Orchestration July 2015 Page 24 of 27

25 In Central, at runtime, you will be prompted for the input with the Assign From value not being initialized. In both Studio and Central, if you run the flow as such, the variables will be resolved accordingly. Note below that var1 was passed to the flow input while var2 was not (as it did not exist). Example of Required flow level inputs In this example, we will extend the previous example by marking the flow level inputs as required. We will additionally add a new input as shown below. In Studio, at runtime we end up with two inputs being prompted. Remember, this happens because var1 was also defined as a system property with the value systemproperty01. Note also that the inputs are now marked as required with an *. In Central, at trigger-time, one input is prompted. Note the * marking and the fact that you cannot actually trigger the run until the required input is filled in. HP Operations Orchestration July 2015 Page 25 of 27

26 In Central, at runtime, you will be prompted for the input with the Assign From value not being initialized. Note that this input is marked as required too. Note that the Resume button is disabled and can only get enabled once the required input is filled with a value. You ll fill in all inputs in Central: at trigger-time as well as at runtime, and the end result will be something like what is shown below. So what happens? In Central: 1. Required assignfromandvariableexists gets its value from existing var1. 2. Required assignfromandvariablenotexist does not get its value from var2 because it does not exist and it is prompted as required at runtime. 3. notassignedandrequiredprompt gets prompted as required at trigger-time. HP Operations Orchestration July 2015 Page 26 of 27

27 In Studio: 1. Required assignfromandvariableexists gets its value from existing var1. 2. Both assignfromandvariablenotexist and the notassignedandrequiredprompt get prompted as required at runtime. You have learned what happens with required inputs at trigger and runtime in both Studio or in Central. That s the end of guide. If you have any questions, you can post them on the HP OO community forums: Check the HP OO community front page for additional resources such as guides, tutorials, videos, content and more: HP Operations Orchestration July 2015 Page 27 of 27