Best practices for OO 10 content structuring

Transcription

1 Best practices for OO 10 content structuring With HP Operations Orchestration 10 two new concepts were introduced: Projects and Content Packs. Both contain flows, operations, and configuration items. Organizations typically create several projects based on requirements such as development teams, departments, and internal projects. This best practice guide describes the different principles, parameters, and conditions that should be considered in order to get an efficient and maintainable structure of the OO content repository. The best practices are flexible to organizations of different sizes and needs. HP Operations Orchestration HP Operations Orchestration (HP OO) is a next generation IT Process Automation solution that is designed from the ground up to increase automation adoption whether in a traditional data center or hybrid cloud environment. OO Community This best practice guide has been created for the OO Community. The OO Community provides knowledge assets of the following types: Presentations Videos Tutorials Quick guides Best practices Sample content Additionally, this guide describes a methodology for OO 9 users who are upgrading and need to divide the complete OO 9 repository into one or more OO 10 projects. Overview There are several questions that we need to consider when looking at a good model for structuring content in OO 10: How do we work effectively in multiple, possible independent, teams? How many projects should we have? How do we enforce governance in particular security? What do we do with shared content [between teams]? How should we structure the Central content library in terms of folders? And what about configuration items (system props etc.)? And for OO 9 users: How do we divide the big project created by the Content Upgrade Utility into smaller projects? In this guide we start by explaining some of the basic concepts related to the OO content and content structuring. Then we will look at the principles and best practices.

2 Index Overview... 1 Index... 2 Concepts... 3 Projects and content packs... 3 Dependencies... 3 Content integrity model... 3 Overview of OO 10 content promotion... 4 Folder structure... 7 Best practices for content structuring... 4 Axes of division... 5 Axis: Security... 5 Axis: Roles... 5 Axis: Content pack levels... 6 Axis: Use case specific... 6 Summary of the best practices... 9 Upgrading from OO

3 Concepts Before we dive into the discussion of dividing content into several projects, and therefore also content packs, this section presents the concepts that are required to understand some of the motivations and reasoning of how to effectively divide content into multiple projects. If you are already experienced in OO 10, you can skip this section. Projects and content packs The official Concepts Guide for HP OO explains projects and content packs (CPs) in the following way: A content pack is a file containing operations, flows, actions (Java-based or.net based), localization data, and configuration items. Content packs are deployed to the Central server and stored in the database. A content pack can be created in Studio by an author, or it can be provided by HP or a third party. Studio lets you break down your set of authored content into granular projects. Each project groups together contents of a similar nature, for example, according to functionality, development owner, geographic location, and release time lines. Studio lets you package each project to a content pack. The content pack is the read-only run time artifact for the project sources. The content pack is the artifact that is deployed to Central We highly recommend that you have read the official Concepts Guide before continuing with this best practice guide. Dependencies With the introduction of CPs in OO 10 it is important to realize that a CP, seen as an object, is not dependent on anything. However, a CP contains flows and operations that often depend on other flows and operations from other CPs. And it is also important to realize that dependencies change over time. The Base CP from HP is a good example this CP will most likely be updated over time, and every CP had flows that are depending on content from the Base CP. To learn more about handling dependencies please refer to the guide Best Practices for Content Promotion which can be found online on the OO Community. Additionally, the dependencies for the HP Standard CPs is listed and described on the Community, e.g. IT Operations. Content integrity model The content model is a model enforced by Central to ensure that the repository is always in a consistent state. For content objects (flows, operations or configuration items) and content packs we note the following properties: A content item belongs to one and only one content pack A content item is defined uniquely by: o UUID o Path Content items from one CP can depend on other content items that belong to different CPs, and this leads to a dependency of the deployment order of different CPs. We require that at any given time the Central repository is in a consistent state based on the CPs that has been deployed. This requirement leads to the following properties: There are no missing dependencies There are no duplicate UUIDs since a flow (any content item) belongs to one and only one content pack There are no duplicate paths Only one version of a content pack can be deployed at any time

4 If a flow from a CP is dependent on a flow from another CP, then either you have to deploy the latter CP first, or you can deploy both at the same deployment. A deployment will fail in case it would create an inconsistent state. Overview of OO 10 content promotion In the guide Best Practices for Content Promotion which can be found online on the OO Community, we discuss how to promote and manage CPs across several environments. In this section we give you a very brief overview of the process: First, the author submits the CP to the Artifact Repository and then the Promoter promotes the CP to the different environments according to the normal change management process (e.g. following ITIL or similar frameworks). We note that a difference between OO 10 and OO 9 is that in OO 10, the content is always deployed from the Artifact Repository, and not from one environment to another as in OO 9. In this process we capture different types of metadata of both organizational and technical nature such as CP Owner and OO Version. The data we capture helps debugging and troubleshooting, and overall minimizes the risk of changing the systems (promoting CPs is a change). Best practices for content structuring When looking at content structuring we are in fact considering how to divide our OO content into projects and also trying to understand how many projects we really need. There are no right answers to these questions since all organizations are different and have different needs. Instead, the best practices will focus on characterize the different aspects that influence our division of content. Dividing content is an ongoing effort that should follow the organizational needs. It is not required that you start by having a perfect plan for the projects that you need. It is very possible to have just one big project initially and over time create new projects or move content to new projects. Having only one project means that you will not benefit from the advantages the content pack model provides such as easier handling of content for administrators and promoters and an easy way to share common content as content packs. Depending on the internal release process testing also gets easier or at least more manageable if projects are kept smaller. Naturally dependency mapping can get more complex, but having a few shared utility CPs shared between all projects and then having a number of projects with a narrow purpose such as monitoring app stack xy really limits the impact of changes. Our expectation is that most organizations will initially use between 3 and 15 projects in particular if upgrading from OO 9.

5 Axes of division There are several aspects to separation of content and the best practices divide content along four different axes. Each axis represents a guiding line that can be used to divide content: 1. Security 2. Roles (user groups) 3. CP levels 4. Use case specific The list is prioritized in terms of importance, and we suggest using this list as basis discussions around dividing and structuring content. We examine each axis in detail in the following subsections and keep in mind that you do not need to divide content according to all four axes, but only to those that apply to your organization and usage of OO. Axis: Security As with most if not all systems we need to address security aspects first. If there is certain content that only some users should access, the security model should list this as a dedicated role. The next axis is indeed roles, but we have chosen to create an axis for security related division of content because this is the first important step when implementing an RBAC model. Examples of sensitive content could be flows that manipulate domain and LDAP servers, and security systems such as firewalls. Enforcing security is done on the SCM system (SVN server) for the projects access by the authors using Studio, and using folder and flow permissions in Central for end users running flows. With the sensitive content moved to one or several separate projects the rest of the content can be divided along the way according to the other axes of division. Please note that it is possible that some of the other axes of division ensure that content is divided in a way so that sensitive content is placed with the role that needs it, which is the next axis. However, even if the security aspects can be covered by following the other axes, we chose to have a separate axis for security so that we are certain that this aspect not overlooked which is important configuring permissions has changed in OO 10 due to the inclusion of Source Control Management (such as SVN). Axis: Roles A benefit of having projects for each team instead of one big shared project is that each team can independently of the other teams release new versions of their content packs without affecting the other teams. Roles or user groups should in general match the different development teams. Often we see that roles are based on domains such as a Windows team and a Unix team. Larger organizations might be even further division of groups. Additional grouping should be done with regards to the governance model you have in place. The best practice is to have at least one project per independent team. Some content is shared between multiple teams, and it can be difficult to decide if the content should be one or another project the next axis will look to this aspect.

6 Axis: Content pack levels In order to better visualize and understand what content that is reusable, we propose to categorize your projects and CPs into two levels; Advanced and Foundation as illustrated below: The Advanced level consists of projects that contain use case flows. And all top-level flows (flows that are executed either headlessly or interactively) should belong at this level. Some subflows and operations are also found here and all are dependent on Foundation level CPs. In the example above we see two CPs; Windows and Linux Remediation which we have separated into two to illustrate a division in an organization that has different OO teams for these two groups. The Foundation level contains the CPs that are reusable in nature. These may come from various sources and in the illustration above we have marked the standard CPs from HP in purple boxes, while the user created CPs are green boxes. We propose that all Foundation CPs are separated from Advanced CPs in order to make reuse across multiple teams easier i.e. only share the reusable components. Typically shared content are in the Foundation level. This is aligned with software industry standards with regards to modularization. Foundation level CPs typically contains sets of operations that provide integration to one or more systems. In the illustration above this could correspond to an integration to SUN LDAP, which is not part of the standard HP content, and thus it is marked as a user CP. A variant of integration operations is what we call Wrappers. These types of CPs contain flows that encapsulate other integration operations by fixing many of the inputs to predefined values (hostnames, ports, etc.), exposing only those that change with each use case. Wrapper flows is something we want to address in detail in a future best practice guide related to authoring. We recommend in most cases that a CPs should not span several levels. The illustration shows four user CPs, however depending on your organizational needs this could just as well be only two projects, if you decide to have all foundation content in a single CP and all advanced content in a single CP. The advanced CPs should be divided along the other axes. Axis: Use case specific The final axis that can be considered to divide content is related to use case specific projects. Often OO flows are created as part of project to automate and orchestrate a set of use cases, and it can beneficial to create a CP for each of these types of projects ( project here refers not an OO project, but a general work project).

7 Having several small projects rather than one big minimizes system impact when deploying new versions and with [semi-]automated content promotion process and a well-defined content promotion model there is little to no overhead in a few extra projects. It is also possible to divide the content into project by functional areas within a certain domain, such as User Management and Patching for Windows. This is something that is often done for CPs on the Advanced level when there are too many flows for one role. In other words, the final axis is used when division along the other axes still leaves some CPs so big that they are hard to manager due to e.g. coordination between teams and colleagues. Finally, we recommend having testing flows in separate CPs. It is common to have automated unit test flows, in particular for Foundation level content. And since these flows are typically only used in Studio and on a development system and by separating them it is easy to ensure that they are not promoted to a production system. It is optional to divide content along this axis. Folder structure A project contains flows, operations, configuration items, and folders. These are all wrapped in a CP before being deployed on Central. As we saw in the Content integrity model section, flows cannot have the same path even if they come from two different projects (and CPs). However, it is possible that two projects have overlapping paths. Assuming that we have two teams each with a project, they need to ensure that they are not ending up with items that have identical paths. This can be done in several way; having both open in Studio or by communicating between the teams. In the following example we have two different teams creating flows for the Windows platform; one team is responsible for end user management, and the other team for the server infrastructure. This could lead to the following two projects in OO: We observe that the paths are overlapping to the point: /Library/Company Name/Windows/ but after this point there are different folder and flow names. The best practice is to have a naming convention for paths. We recommend using a scheme similar to the following: /Library/<Company name Publisher>/<Domain>/[Area]/<Use case Department Project>/[version]/ This folder structure consists of the following elements: Company name Publisher - Having a common top level folder is advantageous since it allows both; to clearly separate own developed content from HP s, and to set generic permissions for the entire organization. Domain Domains are typical tied to larger groups within the organization, as well as logical separation for technologies. If there is content that is common to the entire organization, this level of the path could also be Common or Shared e.g. /Library/<Company name>/shared/integrations/monitoring/ /Library/<Company name>/shared/integrations/hp Service Manager/

8 Area Optional, depending on the amount of flows and the size of your organization it can be beneficial to create an extra layer based on either a technical or functional area. The level could also be used for content that is shared within the domain. Use case Department Project This level is used several ways much depending on the type and pace of development of OO flows. version Optional, useful when you need to be able to run several versions of the same flow at the same time. Another example of a flow is shown below: /Library/My Company/Linux/User Administration/LDAP/Create LDAP User We will in the upcoming guide Best Practice for Flow Authoring look at the folder structure in even more detail. Configuration items Configuration items (CIs) such as system properties and system accounts are in general something that is reused and used by many flows. The question is should you create a separate project for Cis we recommend as a best practice that you do not have a common project to hold all CIs. It is much better to keep the CI with the content that uses them since this allows for a better tracking of where the Cis come from something that was hard to manage over time in OO 9. If we consider a Foundation level project that flows to integrate with, say, SUN LDAP then it would be obvious to have a system property to hold the hostname and perhaps a system account that is used to access the LDAP. Should you one day migrate from SUN LDAP to another LDAP system, you can archive the integration flows as well as the related Cis. This keeps the Centrals better organized and up to date. Splitting configuration items into separate folders should be done carefully. We recommend testing flows after moving either flows or configuration items. New in OO With OO it is now possible to create folders for configuration items as well. We recommend, in particular for System Properties and System Accounts, to follow a similar folder structure as mentioned above. Although, as configuration items are very reusable objects we d expect a slightly less deep path. However, moving existing system properties to folders might break your flows, so test as required. Permissions In OO 10 permissions to run and view flows have been decoupled from the content when comparing to the OO 9 content model. This means that deploying content pack is the first part of content promotion process, and setting and ensuring permissions is the second part of the process. We also note that is practical that the folder structure fits the role based access control (RBAC) model as part as your overall governance model.

9 Summary of the best practices The best practices give several options of dividing OO content into multiple CPs. There are several aspects that we call axes of division and they give many options to divide content in a sensible manner: Security o By protecting sensitive content Roles (user groups) o By security o By domain o By OO development team CP levels o By high level flows o By integrations o By wrappers Use case specific o For individual projects o Further division in functional areas o Unit test flows in separate projects We recommend at least one project per independent development team. Additionally, having several smaller projects is an advantage. But you do not have to consider all four axes for the division of content, and the model is intended to be revisited over time. In line with the guide Best Practices for Content Promotion, which can be found online on the OO Community, you should be able to define the following properties for each project and CP: Owner Support Documentation location Please refer to that guide for a detailed explanation. Dividing content is an ongoing effort that should follow the organizational needs. It is not required that you start by having a perfect plan for the projects that you need. It is very possible to have just a few larger projects initially and over time create new projects or move content to new projects. In the end, the tradeoff between many small CPs and some larger CP is that; smaller CPs are easier to manage and require less coordination and results in less risk and impact, but there are more dependencies to track compared to larger CPs. Overall we lean towards smaller CPs but on reasonable level, and it is always possible to refactor a CP into two or more smaller CPs. We recommend spending time on creating a good folder structure that suits the division of content packs as well as the RBAC and security model so that managing permissions is easy and clear.

10 Upgrading from OO 9 Planning the structure and projects/cps can be done prior to starting the upgrade. An important thing to notice is that roles in OO 10 are not different from those you have in OO 9. That means that you can look at dividing content along the Roles axis and plan new CPs accordingly. During the upgrade you will run the Content Upgrade Utility (CUU) which creates a report of your flows and list which of the HP OO 10 Standard CPs that your flows are dependent on. Additionally, the CUU creates one big project with all your flows etc. and from this you can create new projects in Studio and move folders, flows, and operations to these. Initially, you can work just work with the big upgrade project from the CUU and then progressively over time create new projects. This could be done with requests for new content or in conjunction with change requests to existing flows as there are no hard requirements in OO 10 to have more than one project (although we do not recommend this). Below we list several tasks that can be done prior to upgrading to OO 10, in order to prepare a good content structure in OO 10. Step Task Notes 1 Identify Roles and Teams Identify flows and operations that are restricted due to security concerns. Identify commons and shared operations and flows, and note which configuration items they are dependent on. Run CUU to find potential issues such as Multi-instance bad practices. Ensure best practices for new content developed in OO 9. Corresponding axis: Roles. This is mostly like unchanged from OO 9 (unless there are things to improve, then this would be a good occasion). Corresponding axis: Security. Corresponding axis: CP level. This could result in one or more shared foundation CP. As with all divisions, you do not need to identify all content that is shared as this can be iteratively at later stages. As noted in the Upgrade guide a few OO 9 flow constructs are not allowed in OO 10, and we recommend installing OO 10 Central on a small test machine to know, in advance, which flows that cannot be upgrade to OO 10. For more details see the Upgrade Guide and the Multi-Instance Best Practices, both can be found on the OO Community. We recommend that you create new content OO 9 with these best practices for OO 10 in mind. E.g. operations and flows that are integrations (i.e. foundation level) are placed in individual folders so it is easy to move this to another project etc. After the upgrade the iterative process consists of the general steps: Step Task Notes 1 Create one or more foundations level projects for shared content. Corresponding axis: CP level 2 Identify and move shared content to the shared project(s). Corresponding axis: CP level 3 Create projects for the different roles (teams). 4 5 Iterate over the remaining content and identify and move content per team/role. Investigate if projects could/should be divided into smaller projects based on use cases or specific functional areas. Corresponding axes: Roles and Security Corresponding axes: Roles and Security Corresponding axis: Use case 6 Create specific projects and move the relevant content. Corresponding axis: Use case

11 As this is an iterative process, you do not need to look at all steps in conjunction with the upgrade, but you could start small and as you get more experience with OO 10, continue with the processed steps. Our expectation is 3-12 projects/cps initially for an organization upgrading from OO 9. Mapping tool To help mapping the existing folder structure to projects we created a simple Excel workbook that is good for capturing relevant data. Below is a screenshot of the sheet: In order use the sheet you can start from either the OO 9 repository or the upgraded OO 10 project created by the Content Upgrade Utility. The sheet contains four sections that capture different data points: Location: The first three levels of the folder structure. Using the example from the Folder structure section we would not keep /Library/Company Name/ but only lower lever subfolders. Projects and Content Packs: Capture which new projects to create o Project: the new project name o Axis: the axis used for the division o Frequent changes: Yes/No field if yes, then it is a potential candidate for an individual project SCM: Here we capture which SCM group that should have access to the project. Here you could specify e.g. Network: RW, All: R if everyone should be able to see the project but only the network team should be able to commit changes. Organization: These fields capture important metadata related to the projects. We recommend that this information is captured early and added to the project description (which you can set in Studio). The metadata is very important for the Best Practices for Content Promotion a guide that you can find on the OO Community and we recommend reading already now. The list of fields are: o Owner o Department o Roles (OO user groups) o AD Groups (that should be mapped to the roles) o Test Resp. team responsible for testing o Dev. Resp. team responsible for development o Support Resp. team responsible for supporting the flows in production

12 Steps to get data into the sheet 1. Find the project created by the Content Upgrade Utility (CUU) e.g.: 2. Note that it also creates a project for flows that are invalid, and if there are many flows in the "invalid" project, it is recommended to fix these first (or move/copy everything to the same project for the sake of this exercise) 3. In Studio, right click Library folder in the project and choose "Show in Explorer" 4. In Explorer, Shift+Right click the Library folder and choose "Open command window here" 5. In the prompt enter the command: dir /B /S /AD /ON > c:\temp\folder_list.txt 6. Go to the sheet called "Import" 7. From the Data menu choose "Import data" and then "From Text" and import the text file we just created in step In the wizard remember to specify that the data is delimited by \ (backslash) 9. Delete the first couple of columns so that the general /Library/[CompanyName]/ are no longer there 10. Select the first three columns and then sort (ensure that it uses all three columns to sort and not just the first) 11. Copy the data to the first sheet named "Folders" 12. Work according to the steps describing the iterative process above. You can find the Excel with sample data on the dedicated Upgrade page for OO 9: This page also has many videos and other guides.

13 Tips and Tricks Make sure you run the newest version of OO 10 + patches before upgrading content Read all the best practices guides found on the community Invest time to model a new or evaluate an existing folder structure Watch the videos on the dedicated Upgrade page for OO 9: You group rows in the Excel sheet by selecting a few and then from the Data menu choose Group: And that s the end of this guide. If you have any questions or feedback, then please post on the OO community forum: