Wind River Lab Diagnostics

Transcription

1 Wind River Lab Diagnostics As the amount of code embedded in devices and systems continues to increase, the challenge of integrating, verifying, and validating it also expands. Engineering teams are routinely required to implement and test an ever-increasing number of features with set deployment dates, shrinking the time available for development and quality assurance (QA) testing. Developing run-time software and tools to comprehensively test embedded software is time-consuming and distracts the development team from implementing new features. Once a defect is discovered in the lab, development engineers must try to reproduce and repair it, then resubmit the product to the test teams, in a cycle that consumes essential resources. Current methods of testing, while beneficial, traditionally require lengthy debug, code, build, reload, and test cycles. If the development team is distributed over a broad geographical area, different time zones impact communication and further extend the QA phase. Development engineers are charged with writing, integrating, and verifying perfect applications, while test teams are charged with validation of product design. Both must also root out software faults that would be difficult and costly to repair after the device is deployed. Wind River Lab Diagnostics Wind River Lab Diagnostics links development and test teams in an intelligent, collaborative workflow. A scalable, distributed software diagnostics system, it enables development and test engineers to use s to inject faults, characterize running applications, perform fact-based analysis, and rapidly resolve issues encountered throughout the QA phase. Lab Diagnostics enables development teams to design testability into devices by embedding a minimally intrusive agent providing insight into running applications through s, core files, and exception logs. Lab Diagnostics enterprise-wide infrastructure allows distributed development and test teams to collect, aggregate, exchange, and analyze data from multiple devices under test at multiple locations streamlining the QA phase. The business benefits of Lab Diagnostics map to the challenges faced by OEMs today: Enables testability and supportability of devices Enables comprehensive testing of execution paths during software verification and product validation testing Shortens time to problem resolution Leverages one common tool set and knowledge-share infrastructure across functional teams and distributed sites Capitalizes, archives, and builds on the knowledge of experts Accelerates development by reducing test set-up and configuration Saves time and money throughout the development life cycle Brings better products to market faster, with lower risk Lab Diagnostics 2.1 New in This Release Automate Complex Test Sequences with Scripting The ability to manage a single, highly targeted can be a valuable diagnostic tool when engineers are actively isolating a problem. But the ability to script sequential deployment of s can make automated testing up to a thousand times faster and more efficient. Now users can pack many actions (grab, download, enable, collect data, disable) into a single, executable file for manual and long-run testing. Published application programming interfaces (APIs) on Device Management Server and Site Manager - Availability of APIs gives development and test teams increased flexibility in authoring scripts. Support for PERL scripts Table of Contents Wind River Lab Diagnostics... 1 Lab Diagnostics 2.1 New in This Release... 1 Components... 2 Productivity Workflow... 2 Cross-Development Environment... 2 Design-in Testability with the Device Management Agent... 3 Workflow... 3 Core File Workflow... 5 Laboratory Environment... 6 Post and Publish s... 6 Fault Inject, Control Execution Path, and Collect Data... 6 Post and Publish Data... 7 Analyze Data... 7 Resolve Issues... 7 Technology... 8 Specific Applications of s... 8 System Services... 9 Platform Support Matrix Enterprise Application Support Matrix Wind River Device Management Partner Ecosystem Professional Services Education Services Support... 11

2 Persistent s Now s enabled on a device can survive reboots, saving engineers time and effort when a device shuts down unexpectedly. Users can choose to develop persistent or non-persistent s. New Features and Functionality for Existing Products An upgradeable Device Management Agent enables OEMs to update devices and systems in the lab with the latest version of the Agent. Going forward, this assures backward compatibility between existing devices and new Device Management products. Instrumentation of Linux Kernel-Mode Software Now development and test teams can dynamically instrument software that executes in Linux kernel space. This complements instrumentation of applications in the Linux user space. Support for MicroTCA This hardware/software integration offers equipment manufacturers a commercial off-the-shelf (COTS) platform to expedite device development, test, and support. It supports Wind River Linux 1.4 and 1.5 boards, and VxWorks 6.x boards. OEM Development Center Developers Workbench Diagnostics Test & QA Virtual Lab Expanded VxWorks 6.x Architecture Support Now companies building applications on MIPS and ARM/XScale can use Device Management products off the shelf. Support for Wind River Linux Device Management products support Wind River Linux on Intel Architecture (IA) and PowerPC. Components Wind River Workbench Diagnostics Workbench Diagnostics is the developercentric component of Lab Diagnostics. It plugs into Wind River Workbench, an Eclipse-based development suite, and integrates with its powerful development tools. Workbench Diagnostics enables developers to design run-time software into the operating system enabling a testable platform. It facilitates development, deployment, management, log collection, and log analysis. Workbench Diagnostics enables developers to extract core files generated by VxWorks 6 and to perform core file analysis with the Workbench Debugger. When fault data from devices under test reaches the development team, Device Management Server Workbench s analytic and visualization tools expedite understanding and resolution of software defects. Device Management Server Device Management Server is an enterprise-class server application that resides in the OEM s virtual lab facility at the primary development center and connects via virtual private network (VPN) to distributed development centers. Device Management Server is the hub of Lab Diagnostics, where data is archived, aggregated, and shared by development and test engineers. With the Device Management Server, development teams can streamline collaboration and improve information sharing during development. Site Manager Site Manager is a server application to manage data flow to and from devices under test. With it, engineers can perform software verification and shorten time required to resolve issues encountered during system integration, software verification, and product validation testing. Productivity Workflow Lab Diagnostics productivity workflow shortens engineering cycles from implementation through QA testing. It offers tools and workflows to facilitate designing for testability and the collection, sharing, and analysis of device data from running applications among development and test teams. Lab Diagnostics can be used to test and diagnose running software in two settings: cross-development and laboratory environments. Cross-Development Environment Individual developers use Workbench Diagnostics and a development target to test application performance, test execution paths, and isolate run-time problems. Workbench Diagnostics provides developers with and Workbench Diagnostics Site Manager Chassis Under Test Figure 1: Lab Diagnostics Wind River Lab Diagnostics

3 core file workflows to collect device data, analyze data, and resolve problems in software modules they own. The workflow consists of: development to instrument running modules log analysis Issue resolution with corrective s workflow is useful during performance testing, execution path testing, and run-time diagnostics of intermittent issues. Core file workflow consists of: Core image extraction Core file analysis Problem resolution with corrective s Core file workflow is useful during post mortem analysis after the target crashes and reboots. In a cross-development environment, development engineers can completely observe the behavior of their running binary modules. Design-in Testability with the Device Management Agent Workbench Diagnostics incorporates run-time components that, when included in VxWorks or Wind River Linux, provide a diagnostics interface to the entire device application. The Device Management Agent enables instrumentation, core image upload, and exception log upload to supply developers with detailed diagnostics information. The run-time components are specifically designed to be minimally intrusive and have a small memory footprint. With Workbench Diagnostics, developers can easily include and configure diagnostics components using the Kernel Configurator. By including these components, developers can create VxWorks or Wind River Linux based applications designed for testability and supportability. Workflow Development During application development, engineers use Workbench Diagnostics to write, distribute, and deploy s and to collect and analyze logs. Development engineers can easily write source code and build binary modules. By referencing the application source code in Workbench Diagnostics, software developers can create on_entry, on_line, on_offset, and on_exit s to instrument entry points, lines within the body, offsets from the entry point, and exit points of functions. In the data analysis stage of the QA phase, development engineers can collect and analyze s logs generated during testing. Editor Extensions Workbench Diagnostics builds on the capabilities of the Wind River Workbench Editor to facilitate development. Workbench Diagnostics allows developers to review the application source code, determine the instrumentation point(s), and generate source code quickly. The Workbench Editor has both an application and a source code views for easy development s. Within the Figure 3: A simple example of code and instrumentation Figure 4: Project Navigator view with files selected context of application view, the editor provides a pull-down menu to facilitate auto-generation of source code templates. Within the context of the view, the editor provides highlighting of syntax. Project and Build System Extensions Workbench Diagnostics extends the Workbench Project System and Build System to organize, manage, and build binary modules. Figure 2: Kernel Configurator view with Diagnostics component selected Management In a cross-development environment, developers can use the Diagnostics Device Connect plug-in to access a Wind River Lab Diagnostics

4 Log Collection s can be written to output data logs for automatic data transfer to the host file system, or they can be written to output data directly to a console. Developers can use either method, depending on need: deep analysis or quick check. Figure 5: Device Connect view with Add Module pulldown Figure 6: Device Connect view with Deployment pulldown If s are written for unattended log collection, the Device Management Agent automatically off-loads logs to the appropriate location on the host file system. If needed, developers can poll the target from the Device Connect plug-in to transfer data immediately from the target. If s are written for immediate data output, logs can be displayed to a console, like the kernel shell. Device Connect Command Line (Automated Workflow) The Device Connect Command Line complements the GUI-based interface for engineers that prefer a command line interface. In addition, developers can automate workflows at their desk by developing scripts that access the command line interface. Figure 7: Device Connect view with Log Upload pulldown device that contains the Device Management Agent. With this set-up, developers can directly manage and control binaries on a specific target. Diagnostics Device Connect Plug-In (GUI Workflow) The Diagnostics Device Connect plugin provides a GUI-based interface to configure the Device Management Agent easily for robust communication with the target device. Once host and target communication has been established, developers can focus on rapid fault isolation with s. With binaries, developers can instrument functions methodically in the application, operating system, device drivers, interrupt-service routines, and so on. The Diagnostics Device Connect plug-in has one-click actions that enable, disable, uninstall, remove, and reset binaries. This GUI facilitates a methodical approach to determining the root cause of a fault. For each step in the sequence, developers can focus on an area of concern by temporarily placing a to collect data that will point to other areas of concern. With this tight cycle of instrument-collectremove, developers can investigate the entire application, ultimately isolating the fault to a line of code. Workbench Diagnostics allows fact-based analysis of system faults, avoiding the misdirection typical of symptom-based analysis. $ downloadsp device1 control_c.usm $ enablesp device1 control_c.usm $ disablesp device1 control_c.usm $ logupload device1 /log $ deletesp device1 control_c.usm Figure 8: Device Connect Command Line complements Device Connect Plug-In Log Analysis Workbench Diagnostics facilitates analysis of log data. Developers can easily select data sets from the host file system and review the content using Log er or Wind River System er. With Workbench Diagnostics, developers can efficiently analyze a large amount of data, quickly focus on suspicious data points, and readily associate them with the application source code. 4 Wind River Lab Diagnostics

5 Figure 9: Log er Log er Log er is a data visualization tool provided exclusively in Workbench Diagnostics. The Log er has multiple views that include tabular representation of the log data, Application Source Code view, and Source Code view to help developers easily analyze data. Wind River System er System er can graphically display logs. Workbench Diagnostics provides a conversion tool that imports logs in XML format and generates data files in System er format. Problem Resolution With Workbench Diagnostics, software developers can incorporate corrective code in s to determine the best resolutions to software issues. Using the workflow, developers can develop, manage, and analyze the efficacy of corrective s. Core File Workflow Workbench Diagnostics provides a process and the corresponding tools to generate core images, extract core images from a target, archive core files, and analyze core files. Core images record the state of the running application at the point of exception. Core files give developers the entire system image for postmortem analysis. VxWorks 6 Core Image Generation Workbench Diagnostics includes VxWorks 6 kernel components that enable core image generation. Once configured, VxWorks 6 generates core images when a fatal system exception, kernel panic, or kernel task-level exception occurs. Core File Storage and Off-Load Developers can configure VxWorks 6 for onboard core file storage or network off-load. The first method, core images are stored in persistent memory on the target and are transferred to the host during target reboot. The second method eliminates local storage by streaming the core image to the host machine or server application as the target shutdowns. Figure 10: data displayed with the System er 5 Wind River Lab Diagnostics

6 Post and Publish s The Workbench Diagnostics and Device Management Server are designed to exchange data between developers host computers and development teams server located in the lab. Development engineers can post s (source code and binary modules) with Workbench Diagnostics. The Server Connect IDE plug-in is a GUI-based interface to connect to the Device Management Server in the engineering lab. Development engineers can embody their expertise into module-specific s to be used by team members at every phase of the device life cycle. Figure 11: Workbench Debugging perspective with Target Manager attached to a core file Figure 12: Upload of s with Server Connect Plug-in Figure 13: on Device Management Server Core File Analysis Workbench Diagnostics extends the capability of Wind River Debugger to analyze core files. With Workbench Diagnostics, developers can analyze core files in readonly mode at various levels: assembly or C/C++, memory content, variable/value pairs, etc. Target Manager, Debug view, Variable view, Kernel Objects view, Stack Trace view, Memory view, Register view, and Symbol Browser fully support VxWorks 6 core file analysis. Laboratory Environment Lab Diagnostics can streamline system integration, software verification, and product validation testing in a lab environment. Device Management Server and Site Manager server applications enable teams to share knowledge, collect device data from multiple devices under test, disseminate data, analyze data, and resolve problems. The Device Management Server, a J2EE enterprise-class application, enables development and test teams to manage devices under test, source code files, binary modules, core files, exception logs, and logs. It provides a browser-based console and scripting facility for a manual or automated workflow. The Device Management Server is the repository for the development team. The view permits engineers to manage modules. Engineers can group s into device-, application-, test-, and problem-specific groupings. With the s published on the Device Management Server, development engineers can easily share their expertise. Fault Inject, Control Execution Path, and Collect Data Site Manager is the deployment and data collection application for the lab environment. Engineers can download test-specific s that have been published on the Device Management Server to run tests on devices in the lab. With the GUI interface or scripting framework, engineers can inject faults, force execution paths, and collect data from multiple devices under test. If there are multiple development sites, multiple Site Managers can be installed at every development site. With the Device, engineers can download to and enable modules on devices. s give development teams the ability to access 6 Wind River Lab Diagnostics

7 core file, exception logs, and s logs. With the Device, engineers inspect the listing of devicespecific data. If a cross-functional team needs to analyze a data set, users of the Site Manager application can upload data sets to the Device Management Server. Figure 14: Device on Site Manager Figure 15: Automation through a PERL script and control running applications by changing the argument and return values of functions. By dynamically manipulating values, engineers can test any running function, and with dynamic instrumentation, engineers can collect device data from running on devices. Test teams can advance software testing by deploying and removing a sequence of s until test coverage of device applications is satisfactory. Support for scripting enables engineers to manage multiple s in automated, complex, long-run sequences that do not require constant human supervision, meaning that more code can be tested for longer periods, with no risk of data loss if errors occur. With Lab Diagnostics, engineering teams have complete access to running applications with a minimally intrusive method. Again, there is no need to modify the application source code, rebuild the application image, reload firmware, or reboot the device. Post and Publish Data The Device Management Server and Site Manager applications are designed to aggregate and transfer device data collected from devices under test. The Site Manager collects data from devices in the lab. With the faultdocumenting devices, fault data captured during the QA process are automatically collected onto the Site Manager. With the Log, engineers can manage Figure 16: Log on Site Manager The Device Management Server is the central repository of device data collected in one or more labs. Users of the Device Management Server can easily manage many logs from multiple devices through the Log. Once users post data on to the Device Management Server, multiple developers can download logs onto the host computers for analysis. Analyze Data With the Server Connect IDE plug-in and Workbench, development engineers can download device data from the Device Management Server. Once the data is located on the host computer, development engineers can use analysis tools available in the IDE to determine how to fine-tune the application for performance, increase resiliency, and improve stability of the application. Resolve Issues Lab Diagnostics is an enterprise-wide system to distribute corrective code embodied in s. Engineers can develop, distribute, and enable s that contain possible fixes. The development team can streamline Wind River Lab Diagnostics

8 Figure 17: Analyze data captured at the time of failure the resolution process by eliminating time-consuming applications builds, reloading of application binaries, and rebooting of the system. Once the ideal fix is found, developers can easily move corrective code contained within the source code into the application source code. Technology Essential to Workbench Diagnostics is the ability to create s to gain visibility into running embedded applications. A is software used to instrument live applications dynamically without modifying the application source code, rebuilding the application, reflashing boards, or rebooting the device. s enable engineers to dynamically patch functions on running devices; access local and global variables within the scope of a function; and remotely monitor, configure, and control the execution of an application. s are minimally intrusive on device performance. They can be disabled and enabled remotely. s can be created by development engineers, leveraged by test teams, and used by support engineers. Specific Applications of s Development and Test Intelligently Exercise Subsystems and Functions Inject specific values in variables Force conditional expressions to execute code paths Inject faults for routines Alter the system s environment (e.g., change values read from sensors, set CPU and peripheral controllers register values, modify streaming data) Expedite Problem Reproduction and Resolution Force the state of the system to reproduce the error by altering the system s environment Capture a trace of global and local variables, function calls, and data during the reproduction of a problem Timestamp any point in the system to determine timing defects and performance Dynamically Debug Running Software Insert debug code (including logs and printfs) to isolate defects Insert code to test fixes Peek and poke variables, registers, and sensors to validate fixes Modify a running system without having to reboot and set up tests for every code change Force the System to Extreme Conditions Inject faults at any time during a test Force the environment of the device to a specific state (e.g., high temperature reading on a sensor) Log timestamps that quantify the performance of any part of the device Set the device to a specific state that may only be reached after hours of continuous operation (e.g., buffers full, queues empty) Eliminate the use of sophisticated test equipment (e.g., forcing the hop count to a large value in a data packet) Development Core File Analysis Log Analysis Device Log User Accounts Device Log User Accounts Device Connect Mechanism Device Connect Plug-In Server Connect Plug-In Compiler Log Conversion Tool Server Connect Mechanism Workbench Data Exchange Mechanism Control and Configuration Device Log Management and Aggregation Management and Control Database Multiple Site Management Data Exchange Mechanism Control and Configuration Device Log Management and Aggregation Management and Control Database Multiple Device Manager Workbench Diagnostics Device Management Server Site Manager Figure 18: Lab Diagnostics System Services Wind River Lab Diagnostics

9 System Services Workbench Diagnostics Services User-Specific development - For development engineers to write, build, and test s compiler - Generates binaries Auto-generation - Tool that reads the application binary and auto-generate s Core file analysis - Wind River Debugger to understand the system state log analysis - Log er and System er to analyze data logs from devices Device connect plug-in - To deploy, enable, disable, and remove s from a device directly connected to Workbench Diagnostics Server connect plug-in - To log into the server from Workbench Diagnostics; to upload s and download logs Machine-to-Machine Device connect mechanism Server connect mechanism Site Manager Application Services Authentication and authorization - Back-end service authenticates users and enforces access control to Site Manager services s for administrators, development, and test engineers - Administration view User management: Add/remove users, assign roles, edit user information, and reset passwords view for development and test engineers - management: Manage groupings, add/remove source and binary code, associate/disassociate s to devices Log view for development and test engineers - Log receipt: Review comprehensive or filtered lists of device logs - Log transfer: Push all or selected logs to the Device Management Server Device view for development and test engineers - Device management: Add/remove and initialize devices, edit device information - assignment: Add/remove -to-device association - control: Download s to devices, enable/ disable s, and remove from devices - Log transfers: Transfer logs from devices to server, transfer logs from server to host Multi-device management - Back-end service for multiple device communication: facilities initiate communication; receive core files, data, exception logs; and deploy s Control and Configuration - Configuration facilities for user accounts and device information Device log aggregation and management - Back-end service stores and categorizes device logs management and control - Management and control service for s: provides facilities to store and maintain test-specific s; and facilities to send binaries to devices, enable/disable s, and remove binaries from devices Database - Relational Database Management Service (RDBMS) for commercial databases Server-to-server data exchange - Communication service that enables secure exchange of data among server applications: includes mechanisms to initiate communication, control transfer of core files and device logs, and accept new s Device Management Server Services Authentication and authorization - Back-end service to authenticate users and enforce access control to server services s for administrators, development, and test engineers - Administration view User management: Add/remove users, assign roles, edit user information, and reset passwords Data management: Manage lists of customers, device models, model categories, processor types, and operating systems - view for development and test engineers management: Manage groupings, add/remove source and binary, associate/disassociate s to devices - Log view for development and test engineers Log receipt: Review comprehensive or filtered list of device logs transferred from multiple Site Manager applications Log transfer: Push all or selected logs to other Device Management Servers - Device view for development and test engineers Device management: Add/remove and initialize devices, edit device information assignment: Add/ remove to device association transfer: Transfer s to remote sites; initiated from Site Manager and accepted by test engineers Log transfers: Transfer logs from devices to server, transfer logs from server to host Multi-site management - Back-end service for communication with multiple Site Manager applications: offers facilities to initiate communication; receive core files, data, exception logs, and vendor-specific logs; and send s Application administration - Configuration facilities for user accounts and device information on the server Device log aggregation and management - Back-end service that stores and categorizes device logs received from multiple Site Manager applications management and control - Management and control service for s: provides facilities to store and maintain test-specific s provided by development engineering; this service has facilities to send binaries to site applications Database - Relational Database Management Service (RDBMS) for commercial databases Workbench-server data exchange - Communication service that enables exchange of data between Workbench Diagnostics and the server application: includes mechanisms to initiate communication, control transfer of core files and device logs to Workbench Diagnostics, and accept new s from development engineers Wind River Lab Diagnostics

10 Platform Support Matrix Workbench 2.6.1/ VxWorks Workbench 2.6.1/ VxWorks Workbench 2.6.1/ Linux Host Core File 1 Host Core File Host Core File Workbench Diagnostics 2.1 Windows 2000/XP SUSE Linux Rhat Linux Solaris ARM IA-32 MIPS PowerPC XScale ARM IA-32 MIPS PowerPC XScale Windows 2000/XP PowerPC NA Windows 2000/XP SUSE Linux Rhat Linux Solaris IA-32 PowerPC Standard Feature of Workbench/ Wind River Linux 1 Core file not supported on VxWorks 6.1 or 6.2 Enterprise Application Support Matrix Lab Diagnostics 2.1 Server Applications Server Platforms Application Servers Databases Browsers Windows 2000 WebLogic Oracle Internet Explorer Linux JBoss MySQL Firefox Solaris Wind River Device Management Wind River Device Management consists of two interoperable applications that create a powerful, enterprise-wide workflow and information flow: Lab Diagnostics and Field Diagnostics. Designed for use with Wind River Workbench, these applications enable companies to perform fact-based diagnostics and software fault correction across the entire device life cycle, from development to QA to field support. They are available for VxWorks 5.5, VxWorks 6.x, and Wind River Linux 1.4 and 1.5. Wind River Field Diagnostics Wind River Field Diagnostics is a scalable, remote diagnostics system that enables support engineers to securely collect and manage deployed device data to diagnose and correct faults. Field Diagnostics is an enterprise-class application that consists of a site configuration for onsite device data aggregation and diagnostics, and an enterprise configuration for device data aggregation from worldwide deployments and remote diagnostics. The application links device manufacturers with device users through a secure device data exchange infrastructure. With Field Diagnostics, device manufacturers can improve device uptime, streamline service operation, reduce service cost, and increase service revenue. This standalone product is tightly integrated with Wind River Workbench and sold as an add-on to Wind River platforms. For more information, see the Field Diagnostics product note. Partner Ecosystem Wind River s world-class partner ecosystem assures tight integration with solutions from a wide range of premier software providers. Our partners extend the capabilities of Wind River Device Management by offering out-of-the-box integration and support for key technologies in a number of fast-moving industries. Our customer support team is trained to troubleshoot partner technologies in use with Wind River products, making ours the most comprehensive and best-supported partner ecosystem in the Device Software Optimization (DSO) industry. Professional Services Wind River Professional Services, a CMMI Level 3 certified organization, enables you to reduce risk and focus on development activities that add value and differentiate your design. Our team delivers device software expertise within structured engagements that directly address key development challenges and contribute to the success of our clients. Our track record of timely delivery and indepth understanding of market and technology dynamics makes Wind River a valuable implementation partner for clients worldwide. Based on our commercialgrade project methodology, service offerings include device design, BSP and driver optimization, software system and middleware integration, and legacy application and infrastructure migration. Wind River Professional Services can configure Wind River Lab Diagnostics to meet your needs by extending processor support, extending target operating system support, validating Lab Diagnostics on Linux host environments, and integrating agents. 10 Wind River Lab Diagnostics

11 Education Services Education is fundamentally connected not only to individual performance, but also to the success of a project or entire company. Lack of product knowledge can translate into longer development schedules, poor quality, and higher costs. The ability to learn and to convert that learning into improved performance creates extraordinary value for individuals, teams, and organizations. To help your team achieve that result, Wind River offers flexible approaches to delivering product education that best fits your time, budget, and skills development requirements. Support Wind River provides full technical support for VxWorks 6.x, Wind River Workbench, Wind River Device Management, and Wind River Linux platforms. Our global support organization is staffed with engineers who have extensive experience with Wind River products and device software development. At major support centers worldwide, our local experts can help diagnose problems, provide guidance, or answer How do I? questions. Support is also available 24 hours a day at our Online Support website ( or by at support@windriver.com. If you cannot find the information you need through Online Support, please contact our global support team: North America, South America, and Asia/Pacific support@windriver.com Toll-free tel.: (800-USA-4WRS) Tel.: Fax: Hours: 6:00 a.m. 5:00 p.m. (Pacific time) Japan support-jp@windriver.com Tel.: +(00) Fax: +(00) Hours: 10:00 a.m. 5:00 p.m. (local time) Europe, the Middle East, and Africa support-ec@windriver.com Toll-free tel.: +(00)(800) France tel.: +33(0) France fax: +33(0) Germany tel.: +49(0) Germany fax: +49(0) Israel tel.: +972(0) Israel fax: +972(0) UK tel.: +44(0) UK fax: +44(0) Hours: 9:00 a.m. 6:00 p.m. (local time) Visit Wind River Online Support for fast access to product manuals, downloadable software, and other problem-solving resources for Wind River Workbench. Additional features, including patches and technical tips for common problems, are available for all customers on subscription. Online support visitors can also access a community of developers to discuss their issues and experiences. Wind River is the global leader in Device Software Optimization (DSO). We enable companies to develop, run, and manage device software faster, better, at lower cost, and more reliably Wind River Systems, Inc. The Wind River logo is a trademark of Wind River Systems, Inc., and Wind River and VxWorks are registered trademarks of Wind River Systems, Inc. Other marks used herein are the property of their respective owners. For more information, see Rev. 07/2007