BMC Configuration Management (Marimba) Best Practices and Troubleshooting. Andy Santosa Senior Technical Support Analyst

Size: px

Start display at page:

Download "BMC Configuration Management (Marimba) Best Practices and Troubleshooting. Andy Santosa Senior Technical Support Analyst"

Alisha Clark
6 years ago
Views:

1 BMC Configuration Management (Marimba) Best Practices and Troubleshooting Andy Santosa Senior Technical Support Analyst 9/3/2006

2 Agenda CM Infrastructure CM Inventory CM Subscription CM Software Distribution (App. Packager and Content Replicator) CM Deployment Manager 2

3 CM Infrastructure What s New in Tuner 7.0? What s New in Transmitter 7.0? 3

4 What s New in Tuner 7.0? Status Report Returns general information (for example: tuner version, release date, VM version, heap-size, arguments, and operating system information) and checks if the tuner is running Debug Report Returns specific information and checks for problems with the tuner (for example: tuner properties, thread dump information, license information, RPC configuration and connections) includes the option&<option>=f excludes the option Log Report Returns the tuner and channel history logs 4

5 Status Report 5

6 What s New in Tuner 7.0? (Cont d) Thread Dump You can get thread dump information remotely from both client endpoints and servers. runchannel -getthreaddump -tuner hostname:7717 -username admin -password "" -output C:\Temp\threaddump.txt If you want to generate thread dumps remotely, the Java VM must be launched with the Xdebug Option. On servers (transmitters, proxies, etc.), profiles do not include the -Xdebug option for performance reasons. Minimal Mode Minimal mode is now a separate process known as minituner.exe and is no longer part of the tuner. This makes minimal mode more robust and the tuner more reliable. 6

7 What s New in Tuner 7.0? (Cont d) Improved Robustness and Stability IPC service switched from DDE to named pipe. Comparing to DDE, named pipe provides a more efficient and generic bidirectional communication mechanism. It does not require applications to be GUI-based, and data exchanged can be in arbitrary format. Receipt Service BMC Remedy Change Management could initiate a change request/task on a Marimba Tuner endpoint through Deployment Manager and Policy Manager. Tuner is to provide a receipt service so as to leave a meaningful audit trail for the work performed on an endpoint. Receipts are stored in the tuner workspace. 7

8 What s New in Transmitter 7.0? More robust Transmitter storage Detection of ungraceful Transmitter shutdowns Tx Verify improvements New debug flags: Tuner TUNER/RECEIPT Transmitter TX/GC TX/DIFF TX/DSL TX/HTTP 8

9 Infrastructure - Recommendations Separate CMS, Transmitters and Windows Patch Source Java Launch Arguments: marimba.launch.javaargs=-xms128m -Xmx512m -XX:PermSize=32m - XX:MaxPermSize=128m Using Profile Property If the tuner profile property doesn t exist -> ONLY Kernel Upgrade If the tuner profile property exists, but Transmitter doesn t have the profile segment - > No Kernel Upgrade If the tuner profile property exists, but Transmitter has the profile segment -> Kernel upgrade + Profile applied Health Check Tools Replication Monitor TxVerify Transmitter Guardian 9

10 Troubleshooting - Scheduler Update schedule vs. start schedule Subscription/Policy Service -> update.schedule Patch Service -> update.schedule Inventory/Scanner Service -> start.schedule Infrastructure Service -> start.schedule Missed update/start schedule while the machine was off the network/turned off Example: Subscription update schedule at 2 AM Inventory start schedule at 3 AM Marimba schedule start delay=10000 Schedule dumper to help debugging scheduler issue 10

11 CM Inventory What s New in Inventory 7.0? Performance Debugging 11

12 What s New in Inventory 7.0? In 6.x, compliance will ALWAYS re-send all data (even when no data changes between scans). This results in an expensive delete of all compliance data followed by a re-insert of all data. This was fixed in 7.0. Using AMT Data As mac_id Intel AMT data is also now part of Intel inventory scans. Identify if machine is AMT-enabled and fetch persistent Tuner ID from AMT chip. New Diffing Logic Maintain a full report cache on the endpoint. Diff against the current inventory data in the DB if there is a checksum mismatch. Purge full report cache based on size and/or time interval. Double-Byte Storage Mirror Forwarding Mirrors on WAN can forward Inventory reports to LAN Mirrors/Master Reduce the chances of database blocking or deadlocks Task Receipts (CCM) Scan and report task receipts 12

13 What s New in Inventory 7.0? (Cont d) Old 6.X Diff Over-Write That Resulted in Full Scan Report [15/May/2006:22:23: ] - warning Over-writing older report for machine: wuscxdw221(win32: ) 1 Diff 1 Inventory Service Plugin DB Diff 1 Disk Queue: Diff 2 2 Diff 2 DB Diff 1 Endpoint ` 3 Diff 3 Disk Queue: Diff 2 Diff 3 Inventory Service Plugin DB Diff 1 Inventory Service Plugin 4 Endpoint sends Full Scan Report 5 Full Scan Report Diff Checksum Mismatch: Diff Scan Old Checksum is for Diff 2, Plugin Local Checksum Cache has Diff 1. Request Full Scan report from Endpoint Disk Queue: Inventory Service Plugin DB Delete All Insert All 13

14 What s New in Inventory 7.0? (Cont d) 7.0 Special Diff Scan Report (Instead of 6.x Full Report) [15/May/2006:22:23: ] - warning Over-writing older report for machine: wuscxdw221(win32: ) 1 Diff 1 Inventory Service Plugin DB Diff 1 Disk Queue: Diff 2 2 Diff 2 DB Diff 1 Endpoint ` 3 Diff 3 Disk Queue: Diff 2 Diff 3 Inventory Service Plugin DB Diff 1 Inventory Service Plugin 4 Endpoint generates new special diff scan report between the full report at Diff 1 state and the full report at the Diff 3 state and resends back to the plugin 5 Diff 3' Diff Checksum Mismatch: Diff Scan Old Checksum is for Diff 2, Plugin Local Checksum Cache has Diff 1. Request Special Diff report from Endpoint Disk Queue: Inventory Service Plugin DB Diff 3' 14

15 Performance SQL Server Report Center queries are basically unusable while inventory insertions are running in the background. Highly recommend enabling dirty reads if inventory insertions need to be run while Report Center queries take place. Reason: Inserts/Updates/Deletes issue an exclusive lock Select statements issue a shared lock A shared lock cannot be issued when an exclusive lock is in place and an exclusive lock cannot be issued when a shared lock is in place You can change the behavior of locking via a lock hint: - Nolock - Enables dirty reads where a Select does NOT issue a shared lock and Select does NOT honor exclusive locks - 6.x Only Report Center interactive queries can use dirty reads (enabled through property) Report Center interactive queries, reports, collections, and processing queries that another application requests can use dirty reads (enabled through UI) 15

16 Performance (Cont d) Inventory Plugin Tuning Set the scan schedule as far apart as possible (24 hours, if possible). Use the scheduling vary option to reduce the load all insert at the same time load (due to time sync d machines). 3 to 5 inventory plugin Oracle database connections appears to be optimal. More connections just result in more resources and waiting instead of increasing throughput. Disk I/O is low. CPU utilization is low. Memory is low. Java GC tuning is lower priority since currently GC s / heap size growth does not appear to be causing issues. 16

17 Debugging Customer Performance Issues Guide to Debugging Customer Performance Issues Get inventory plugin logs from all masters/mirrors/repeaters. Gather a few sample Endpoint Inventory Service channel workspaces. Oracle redo logs and/or archive logs (if archive log mode is enabled). If gathering archive logs, also request character sets, dictionary file, version/platform Oracle is running on. Tuner/History logs where inventory plugins running. System Architecture Scan schedule, # endpoints, # repeaters, # mirrors, load balancer, which plugins insert directly into database, etc. Gather any queued-up scan reports in the inventory plugin disk queue. Find out if the scan schedule is too aggressive and check if it is possible to reduce the scan schedules. In more complex environments, see if client IP Load Balancer persistence might help reducing checksum mismatches. 17

18 CM Subscription What s New in Subscription 7.0? 18

19 What s New in Subscription 7.0? Install Priority You can now set install priority values by typing numeric values per package on the Edit Policy page. In M6, you can only scroll arrow up and down to change the install priority. User-Based/Machine-Based Policies Policy will be assigned based on either user-based, machine-based, or both. In M6, the only option was both. Property: marimba.subscriptionplugin.resolvetype Value: user or machine, otherwise both. Directory Service Hierarchy Structure Policy Manager now uses an updated LDAP container structure. Enables you to more effectively extend the schema of the Active Directory infrastructure, and store and secure policies. 19

20 What s New in Subscription 7.0? (Cont d) Supports install order Supports staging new url before doing an autoupdatefrom Will not updatefrom if marimba.subscription.nodelete=true Will revert url if channel fails to install 20

21 Compliance Architectural Overview Inventory Plug-In Implements differencing for compliance. It only inserts compliance data that has been modified from the previous inventory scan on the endpoint. Subscription Service Invoked as a custom scanner by Inventory Service to calculate compliance on the endpoint. Query-Based Compliance Two types of queries in compliance engine: 1. Inventory-only query; query based solely on inventory scan data 2. Latest-data query; query based on inventory, LDAP-synced policy and Transmitter 21

22 Compliance (Cont d) Scheduling LDAP Sync The LDAP synchronization process is new in this release. LDAP synchronization optimizes the accuracy of compliance queries and reports by updating the configuration database with the latest available policy data. To optimize policy management workflow, schedule the following processes in sequence: Update policy service Perform inventory scans Schedule LDAP Sync Typically, you schedule the processes on a daily basis You can schedule LDAP Sync using the LDAP-to-Database Synchronization Service link on the Data Source tab in CMS System Settings Compliance Reporting In addition to viewing policy compliance information based on targets, you can now view compliance based on a specified package. 22

23 New Features: Compliance (Cont d) 23

24 CM Application Packager/Content Replicator What s New in Application Packager 7.0? What s New in Content Replicator 7.0? 24

the download of MSI package until pre-scripts are run Repair of

25 What s New in Application Packager 7.0? Support for MSI 3.0 No need for User Elevation when installing MSI packages Delaying the download of MSI package until pre-scripts are run Repair of MSI now shows progress bar Return code mapping App-friendly name DSL support 25

26 What s New in Content Replicator 7.0? Packaging and Installation of Content From Existing 1 source to 1 target folder N source to N target folders Installing Unix Packages Using XML Command file to specify packaging commands {5.1 screen shot} Self-Installing Channels New package command that will facilitate installation of self-installing channels {5.2 Screen shot with syntax} Channel Signing (Credit Suisse) 26

27 What s New in Content Replicator 7.0? (Cont d) No More Auxiliary Channels; Now Segments» (Segments also, only for n source - n target use case) 27

28 CM Deployment Manager What s New in DM 7.0? 28

29 What s New in DM 7.0? Main Features (Architecture Changes) Database-backed workspace Improved DS->DM communication Improved handling of log data Report Center integration 29

30 Previous Workspace Architecture File System-Based Object-tree is stored on disk in File System hierarchy Each object consists of a directory with several files Entire object-tree is loaded up and kept in memory at startup Problems Keeping all nodes in memory doesn t scale Start-up time grows linear to workspace size Workspace size is restricted by available memory Hard to: Query data Share data with other applications Maintain referential integrity Guarantee safety of the data 30

31 New Workspace Architecture Database-Backed Object-tree is stored in a database under DBTree Simple Similar to what DM uses today (PTree). Easy to learn and replace. Yet powerful Lazy loading: In-memory caching and batching of object-updates, etc. An object is a DBTree dir-node All common files are merged into a set of node-properties. Other data stored as child-nodes. Exception: Log data is stored in the logging schema. 31

32 DB-Backed Workspace How Does This Solve The Problems? Workspace-size scalability Tree no longer needs to be traversed at start up; start up time will be minimal Number of objects no longer limited by amount of memory Querying the data Logging data is stored in logging schema; querying will be easy Object data is stored in DBTree-schema; querying is possible but somewhat complicated Sharing data All data is in the database Referential integrity Yes, we can guarantee it. Safety of data Increases; databases are built to be safe datastores. 32

33 DS->DM Communication Problems DS communicates too frequently Protocol is inefficient and simplistic DS queue logs in memory Solutions Tighter control of DS communication Intelligent flushing of queue New, efficient protocol Binary Support for compression Support for new client commands (re-try later, etc.) Better use of HTTP Be efficient about keep-alives (partly done in 603S releases) 33

34 Handling of Log Data Problems DM does on-the-fly data processing (CPU-intensive) Incoming data is immediately processed UI becomes unusable when many clients upload Solutions Incoming data is placed in disk-based queue Limited set of processor threads read off of queue We get in control of CPU resources used for log-processing vs. UI May introduce latency in respect to reporting under high load This should be acceptable since the alternative is an unusable UI 34

35 What s New in Deployment Reports? Leverage The Fact That Data Is In The DB Use Report Center for building reports Separate schema for reporting Near real-time reports (for deployment status) After-the-fact reports Reports to be available in DM UI RC query-tree is accessed remotely Report can be selected from DM UI 35

36 Q & A 36

System Administration of PTC Windchill 11.0

System Administration of PTC Windchill 11.0 Overview Course Code Course Length TRN-4830-T 16 Hours In this course, you will gain an understanding of how to perform routine Windchill system administration