The HR Avatar Testing Platform

Size: px

Start display at page:

Download "The HR Avatar Testing Platform"

Brendan Greer
5 years ago
Views:

1 The HR Avatar Testing Platform Purpose This document is intended to provide a high level overview of the HR Avatar testing platform what makes it different from other, legacy testing platforms. Overview The HR Avatar testing platform (TP) is the first testing technology platform designed from the ground up to support the requirements of mobile computing simulation based testing. At a high level, the system supports the following functions: System Functions Simulation Based Test Design Construction Testing Administration Test Results Computation, Management, Distribution Test Item Level Data Analysis Maintenance These functions demonstrate that the TP covers all aspects of testing, from test creation through test analysis maintenance. Additionally, the TP is designed to achieve superior performance against the following criteria: Design Criteria Support Simulation Driven Requirements Mobile Device Compatibility 99.9% System Instant Scalability Appropriate System Test Content ( Results) Auditability Ease of Use Seamless Integration including Direct Test Embedding Achievement of these criteria starts by designing a solid computing infrastructure. After careful analysis, we selected a cloud based solution provided by Amazon Web Services. The Amazon AWS Cloud All operational components of the TP utilize cloud computing services provided by Amazon Web Services (AWS), a division of Amazon.com. AWS is one of the largest cloud computing services providers in the industry. In 2014, Gartner placed AWS in the highest possible quadrant the Service Magic Quadrant in a comparison to other major cloud service providers in terms of

2 ability to execute completeness of vision ( 1UKQQA6&ct=140528&st=sb). The AWS services which HR Avatar uses includes: Elastic Computing Cloud (EC2) Cloud based computing resources Elastic Load Balancing Traffic distribution across EC2 Instances Simple Storage Service (S3) Fully redundant data storage Relational Database Service (RDS) A fully scalable, managed database management system Glacier Long term data storage archiving Simple Service (SES) for scalable, reliable ing Simple Notification Service (SNS) for automated message distribution response across applications Some noteworthy features of AWS include the following: On Dem (elastic) Computing Resources Computing instances backed by permanent block storage Scalability Provides ability to obtain configure additional computing, storage, network capacity easily quickly, to respond to spikes in workload or steady growth. Machine images are pre positioned ready at all times. Allows each computing instance to use storage that is retained if the instance is terminated % Availability All infrastructure is professionally monitored managed 24x7 by full time dedicated staff. Virtual Private Cloud (VPC) Customized Groups Database OS Software management Automated database replication backup Scalability Restricts access to computing resources exclusively to other instances within the same VPC specifically listed external IP addresses. Provides consistently managed, tight control of access to from all instances through one group of settings. Patches upgrades are managed by AWS staff resulting in up to date security functional capability. Offloads compute intensive database tasks to a replicated database, such as reporting results searches, leaving the primary database resources available for test administration tasks. Additionally,

3 Multi Availability Zone database deployments Reliable, secure storage using Amazon Elastic Block Storage (EBS) Scalability all backups are conducted automatically at regular intervals. Introduces redundancy into the database management system across geographic data center locations. Each Amazon EBS volume is automatically replicated within its Availability Zone to protect you from component failure. Amazon EBS encryption provides seamless support for data at rest security data in motion security between computing instances EBS volumes. Integration with the ClicFlic Authoring Tool For simulation based test authoring, the TP incorporates the SimBuilder layer, which is built as an additional programming layer that works directly with the ClicFlic intelligent media object authoring system. This layer adds additional capabilities to allow the creation of market ready simulation based assessments. For example, each simulation can be developed as a series of ClicFlic modules that are combined to create finished assessments. This allows each module to be managed in much the same way that an item would be managed on a legacy test development system. This modular architecture allows for reuse of modules greatly simplifies test maintenance enhancement. SimBuilder also allows for numerous other features, including competencies to be defined managed separately from modules, creation of common media libraries, format templates at the simulation individual item level, customized reports, many others. All of these features were incorporated to enhance productivity during test development to support automated scoring during test administration. A key feature of the SimBuilder layer, is the ability to tightly control module versions maintain an audit trail between each completed test the exact versions of each module that it used. This is a stard feature of legacy item bank systems today, the TP uses SimBuilder to implement the same capability for simulations. Simulation Driven Formatting, Scoring, State Persistence Simulation based tests add a new level of complexity to an assessment system, in both item format during scoring. A virtually unlimited number of item formats is possible. For example, a test taker may be required to evaluate a series of expense reports, write a memo, proofread a letter, evaluate a series of s, respond to co worker questions all in one series just like in real life. Each specific test taker action may evaluate a different competency each competency may use a different scoring technique.

4 The simulation authoring tool was designed to afford the ability to format test items in virtually any way desired, while associating all types of responses with specific competencies attaching relevant scoring parameters during author time. At the end of the authoring cycle, two products are produced: (1) the simulation itself, (2) the scoring documentation, in a stard machine readable (XML) format, which can read by the scoring engine to specify exactly how the test should be scored. This approach allows for a stard scoring engine to be constructed that reads this XML descriptor document to score the test. Using this design, all HR Avatar tests currently use the same scoring engine, even though the tests themselves use widely varying item types scoring schemes. However, to provide more flexibility should it be needed in the future, the TP was designed to allow for different scoring engines, so that truly unique or incompatible scoring requirements can be modeled to the precise requirements of any given test. Another challenge caused by the simulation format is the ability to restart a partially complete simulation at the place where the test was previously left off. Unlike sequential item based tests, simulations require both item responses other state indicators values in order to accurately pick up where the test left off. To accommodate this capability, the TP uses an XML based state document that includes all item responses state variables as the format for reporting responses back to the server. When a partially completed simulation is restarted, the test object simply reloads the most recently produced XML state document to restore state to precisely the right spot. Shifting the Intelligence to the Client enabling Direct Test Embedding Like their video game cousins, simulations rely on the web client (the test taker s web browser) to modify their output as they adapt to inputs from the test taker. This is in contrast to legacy testing systems, where most, if not all, formatting branching decisions were made by the server then sent to the browser as static XML. This shift to the web client results in some interesting design considerations, as well as a new exciting integration option we call Direct Test Embedding. Direct Test Embedding simply means you can embed a test inside any webpage on any website, still have all the control security you would get on a dedicated testing system. This means that partners clients can control the test taking experience more seamlessly when integrate HR Avatar assessments with their hiring processes. The test taker never needs to leave the client s system! For additional information about Direct Test Embedding, please review the HR Avatar Test Integration Guide, at 0.pdf.

5 Mobile Compatibility a Must Have for the Future There are now more mobile devices in use today than desktop computers, the trend is pushing this ratio ever higher. Many job cidates only have access to the Internet via their mobile device. In order to stay relevant, any testing system must support mobile devices. Supporting mobile devices implies that the testing system must have the ability to: 1. Support multiple browser based technologies 2. Support widely varying screen sizes 3. Support different rules for playing media on mobile devices 4. Support different video formats for different web browsers 5. Support widely differing network connection speeds Traditionally, most simulation based assessments were based on Adobe Flash technology. This was a good solution for desktop computers. However, many mobile devices, including all Apple devices most modern Android devices do not natively support Adobe Flash. Instead, these devices require the used of a set of web technologies known as HTML5. Unfortunately, several web browsers still in use do not support HTML5, such as Internet Explorer 6,7, 8. Since some browsers support only Flash (IE 6,7,8) others support only HTML5 (many mobile devices), the only viable simulation solution is to support both Flash HTML5. The TP automatically determines which technology is best for a given web browser uses that technology. Next, when testing on a mobile platform, one must contend with the fact that screen sizes vary wildly across devices. In some cases, the item format must be changed to accommodate different screen sizes. For instance, on a large screen, you can ask a series of Likert scale items on one page. However, on a small screen, a single item per page is more appropriate. The TP software is configured to detect screen size vary item format when needed. This feature is employed in many HR Avatar tests. Additionally, most mobile devices enforce rules that do not allow video to play on the device without a positive action by the user, while others play video in video players outside the web browser (Safari iphone, Opera). These restrictions require simulation software that recognizes each condition adapts to it in a way that is user friendly easy for test takers to navigate. Unfortunately, there is still not one single video format that all web browsers can play. While it has gotten significantly better, multiple formats of every video must be maintained in order to be compatible with all major browsers formats.

6 Even if there was one single video format for all web browsers, different sized screens the need to make the most of network bwidth dems different versions of each video for different resolutions. As a result of the need to have different formats for different browsers, the need to adapt to screen size differences maximize the use of connection bwidth, all HR Avatar assessments use videos that are encoded into four different versions H.264 MP4, WEBM, H.264 MP4 baseline encoded (for tablets), H.264 baseline encoded small screen resolution (for phones). The software must be smart enough to recognize which version to use for each situation. Using the Platform for Non HR Avatar Tests Thanks to it s modern, modular infrastructure object oriented software design, the TP can easily accommodate 3rd party tests test/item formats. Legacy as well as simulation based formats can be implemented reasonably quickly at a reasonable cost. In fact, the system already hosts, delivers, scores, distributes reports for several other clients who wish to take advantage of its unique features. Please contact us at sales@hravatar.com if you would like to discuss feasibility or options.

Advanced Architectures for Oracle Database on Amazon EC2

Advanced Architectures for Oracle Database on Amazon EC2 Abdul Sathar Sait Jinyoung Jung Amazon Web Services November 2014 Last update: April 2016 Contents Abstract 2 Introduction 3 Oracle Database Editions