Implementation Validation Guide For Preliminary Screener Solution

Transcription

1 Community Support Services Common Assessment Project interrai Preliminary Screener for Primary Care and Community Care Settings Implementation Validation Guide For Preliminary Screener Solution For Vendors Document Version 2.1 November 2016

2 Implementation Validation Guide for Vendors ver 2.1 November

3 Contents 1. Document Purpose Target Audience Validation Responsibilities Vendor LHINs and/or HSPs CCIM Project Team Part 1: Software Implementation Validation Purpose Scope How long should validation testing with CCIM take? Prerequisites or how to prepare for Validation Testing Software Implementation Validation Steps for Vendors Test Scenarios Scenario 1 Create a New Screener Assessment Scenario 2 Create a New Screener Assessment Scenario 3 Create a New Screener Assessment Scenario 4 Create a New Screener Assessment Scenario 5 Create a New Screener Assessment Scenario 6 Create a New Screener Assessment Scenario 7 Create a New Screener Assessment Scenario 8 Create a New Screener Assessment Part 2: IAR Test Submission Purpose Scope What tests need to be performed? Appendix A Preliminary Screener Features Checklist How to prepare for UAT? Who should conduct UAT? How to conduct UAT? How long should UAT take? Screener Assessment Creation Screener Assessment Cancellation Screener Assessment Completion Screener Assessment Management Software Offline Version Organization and Assessor Reports Extract and Submit the XML Files Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 3

4 7. Appendix B Implementation Validation Steps Appendix C IAR Submission Guidelines IAR Home Screen Web Service Configuration Uploading Assessments Manually Retrieving the Transaction Record Interpreting Results and Event Status Searching for a Client Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 4

5 Revision History Version Comments Author Date 1.0 Initial release of Implementation Validation Guide Umar Muhammad July 18, Revised for clarity and to simplify process CSS CAP Project Team April Revised to reflect minor IAR 7.0 upgrade changes IAR Upgrade Project Team November Document Purpose The purpose of this document is to guide vendors through the implementation validation process to ensure their system properly conforms to interrai and CCIM requirements and can upload assessments to the Integrated Assessment Record (IAR). Validation ensures minimal validation of core system functionality including the outcomes for the interrai specifications, additional CCIM software requirements and the IAR upload requirements. This document includes eight high-level test scenarios and an optional features checklist that acts as a User Accepted Testing (UAT) guide, in the Appendix. Together, these ensure minimal validation of core system functionalities such as correct calculation of the Assessment Urgency Algorithm provided by interrai and the additional Software Specifications Requirements document provided by CCIM. This implementation validation guide, and test scenarios contained within, does not replace the need for vendors to conduct functional unit, system and integration testing. The guide and scenarios are exclusive of these testing activities. Vendors must complete a more comprehensive vendor validation prior to health service providers (HSPs) completing their validation. This guide is made available on the CCIM website to help provide guidance to Local Health Integration Networks (LHINs) and/or HSPs through the course of the validation and test submission activities to IAR. The document is comprised of two parts pertaining to the validation process itself and one part on the next 1. Part 1 - Software Implementation Validation 2. Part 2 IAR Test Submission 3. Next Steps Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 5

6 2. Target Audience The intended audiences of this guide are: Vendors implementing an automated solution for the interrai Preliminary Screener in support of HSPs implementing the Preliminary Screener and taking part in the CSS CAP. This document, as well as supporting documents and communications, will be shared with LHINs and HSPs involved in the automation of the interrai Preliminary Screener tool. Implementation Validation Guide for Vendors ver 2.1 November

7 3. Validation Responsibilities 3.1. Vendor Development testing (unit / system and integration testing) is the responsibility of the implementing vendor. The vendor must ensure that the implemented system meets all of the software and technical requirements communicated as part of the interrai Preliminary Screener software requirements specifications release package. Vendors must complete CCIM validation prior to providing the software to HSPs. The vendors release package is located in the interrai CHA vendor area of the CCIM website at LHINs and/or HSPs To ensure a successful interrai Preliminary Screener implementation, it is recommended that a LHIN/HSP conduct User Acceptance Testing (UAT). UAT should provide assurance to LHIN/HSPs that all key requirements have been implemented accurately by the vendor. If there are any concerns or errors in the system, the LHIN/HSPs must notify the vendor(s). The Features Checklist contained in Appendix A, is a guide that LHINs and HSPs can use to assist them with this activity. HSPs should complete this prior to starting CCIM validation activities. LHINS/HSPs are required to complete two parts for CCIM validation: Entering the two test assessments into a test version of their software Submitting those test assessments to the IAR validation environment (also known as data submission) Both parts of HSP validations are required to meet the Software Validated and Implemented milestone CCIM Project Team The CCIM Project Team will work with the vendors to closely to ensure that their test submissions are correct and that they are able to submit their test data to CCIM s test version of the IAR. Please note that the CCIM Project Team is not responsible for fully testing the implemented solution delivered to the LHIN/HSPs, as that is the responsibility of the vendor. However, the CCIM project team will clarify any questions surrounding the documentation or about the requirements specification. Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 7

8 4. Part 1: Software Implementation Validation 4.1. Purpose The validation tests are meant to ensure that by coding the predetermined scenarios, the software produces correct results Scope To successfully pass the first part of the validation, the vendors need to enter the eight short scenarios listed below How long should validation testing with CCIM take? Typically validation takes a couple of days but will vary depending on whether scenarios are coded correctly Prerequisites or how to prepare for Validation Testing Vendor must have a valid interrai license. Vendor must complete unit, system and integration testing throughout the implementation process based on the requirements documentation provided as well as the communications and clarifications from weekly vendor teleconferences. Vendors may choose to leverage interrai Preliminary Screener Features Checklist Software Implementation Validation Steps for Vendors 1. Enter the eight assessments specified in Test Scenarios section below. 2. Submit the eight assessments to the IAR Validation Environment. All assessments must have consent granted. (See Part 2 IAR Test Submission below for details). 3. CCIM will review the submissions and notify the vendor when validation is complete (i.e. the HSP has passed). a. Please note that this can take more than one try to complete Test Scenarios Please note that all vendor and CCIM validations must be done using mock data. Live data is not permitted in the IAR Validation Environment. The following scenarios will be used by vendors for the business and systems testing of the automated interrai Preliminary Screener solution. Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 8

9 Scenario 1 Create a New Screener Assessment Pre-condition: Ensure all identification information has been entered within the system for the test client within Section A. **Important**: To complete the test scenarios please DO NOT enter real client information. When generating names for mock clients it is important to use names unique enough to search easily. The following site is one of many that can generate random names for testing: Please avoid using common names such as Test Client, Scenario 1, Consumer A, John Smith, Donald Duck or Brad Pitt as you may not be able to find your client in the test version of the IAR when duplicate client names are submitted. Please also ensure that you record the client names you used and Assessment IDs generated by your source test software as this information would be required for Part 2 IAR Submission. Mandatory elements and Coding Responses required have been provided in the scenarios below, you may enter appropriate responses for all other elements not listed within the scenarios Item Description Coding Legend A1-A10 Complete all mandatory fields Coding Response B1 Cognitive skills for Daily Decision Making 0 independent 0 B2a Activities of Daily Living (ADL) Bathing 0 independent or set-up help only 0 B2b Activities of Daily Living (ADL) Personal hygiene 0 independent or set-up help only 0 B2c Activities of Daily Living (ADL) Dressing Lower body 0 independent or set-up help only 0 B2d Activities of Daily Living (ADL) disability - Locomotion 0 independent or set-up help only 0 B3 Dyspnea (shortness of breath) 1 Absent at rest, but present when performed moderate activities B4 Self-Reported Health 1 Good 1 B5a B5b B6 B7a B7b C2 Instability of Conditions Conditions/diseases make cognitive, ADL, mood or behaviour patterns unstable (fluctuating, precarious, or deteriorating) Instability of Conditions Experiencing an acute episode, or a flare-up of a recurrent chronic problem Self-Reported Mood- Ask in the last 3 days, have you felt sad, depressed or hopeless Informal Helper Status Primary informal helper expresses feelings of distress, anger or depression Informal Helper Status Family or close friends report feeling overwhelmed by person s illness interrai Assessment Required 8 Could not (would not respond) 8 This is based on the Assessment Urgency Outcome score and entered by the user (see below) 1 0 Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 9

10 Assessment Urgency Algorithm (AUA) Score Score (Section C1) interrai CHA Assessment Required (Section C2) Description Description Legend Code Score generated by system for section C1 interrai CHA Assessment required 0 No 1 Yes Based on AUA Section C1 Note an AUA >= 3 indicates a CHA may be required Scenario 2 Create a New Screener Assessment Pre-condition: Ensure all identification information has been entered within the system for the test client within Section A. **Important**: To complete the test scenarios please DO NOT enter real client information. When generating names for mock clients it is important to use names unique enough to search easily. The following site is one of many that can generate random names for testing: Please avoid using common names such as Test Client, Scenario 1, Consumer A, John Smith, Donald Duck or Brad Pitt as you may not be able to find your client in the test version of the IAR when duplicate client names are submitted. Please also ensure that you record the Assessment IDs generated by your source test software as this information would be required for Part 2 IAR Submission. Mandatory elements and Coding Responses required have been provided in the scenarios below, you may enter appropriate responses for all other elements not listed within the scenarios. Item Description Coding Legend A1-A10 Complete all mandatory fields Coding Response B1 Cognitive skills for Daily Decision Making 0 independent 0 B2a Activities of Daily Living (ADL) Bathing 0 independent or set-up help only 0 B2b Activities of Daily Living (ADL) Personal hygiene 0 independent or set-up help only 0 B2c Activities of Daily Living (ADL) Dressing Lower body 0 independent or set-up help only 0 B2d Activities of Daily Living (ADL) disability - Locomotion 0 independent or set-up help only 0 B3 Dyspnea (shortness of breath) 0 Absence of symptom 0 B4 Self-Reported Health 3 Poor 3 B5a B5b Instability of Conditions Conditions/diseases make cognitive, ADL, mood or behaviour patterns unstable (fluctuating, precarious, or deteriorating) Instability of Conditions Experiencing an acute episode, or a flare-up of a recurrent chronic problem Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 10

11 Item Description Coding Legend B6 B7a B7b C2 Self-Reported Mood- Ask in the last 3 days, have you felt sad, depressed or hopeless Informal Helper Status Primary informal helper expresses feelings of distress, anger or depression Informal Helper Status Family or close friends report feeling overwhelmed by person s illness interrai Assessment Required Assessment Urgency Algorithm (AUA) Score Coding Response 8 Could not (would not) respond 8 This is based on the Assessment Urgency Outcome score and entered by the user (see below) Score (Section C1) interrai CHA Assessment Required (Section C2) Description Description Legend Code Score generated by system for section C1 interrai CHA Assessment required 0 No 1 Yes Based on AUA Section C1 Note an AUA >= 3 indicates a CHA may be required Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 11

12 Scenario 3 Create a New Screener Assessment Pre-condition: Ensure all identification information has been entered within the system for the test client within Section A. **Important**: To complete the test scenarios please DO NOT enter real client information. Mandatory elements and Coding Responses required have been provided in the scenarios below, you may enter appropriate responses for all other elements not listed within the scenarios. Item Description Coding Legend A1-A10 Complete all mandatory fields Coding Response B1 Cognitive skills for Daily Decision Making 0 independent 0 B2a Activities of Daily Living (ADL) Bathing 0 independent or set-up help only 0 B2b Activities of Daily Living (ADL) Personal hygiene 0 independent or set-up help only 0 B2c Activities of Daily Living (ADL) Dressing Lower body 0 independent or set-up help only 0 B2d Activities of Daily Living (ADL) disability - Locomotion 0 independent or set-up help only 0 B3 Dyspnea (shortness of breath) 0 Absence of symptom 0 B4 Self-Reported Health 0 Excellent 0 B5a B5b B6 B7a B7b C2 Instability of Conditions Conditions/diseases make cognitive, ADL, mood or behaviour patterns unstable (fluctuating, precarious, or deteriorating) Instability of Conditions Experiencing an acute episode, or a flare-up of a recurrent chronic problem Self-Reported Mood- Ask in the last 3 days, have you felt sad, depressed or hopeless Informal Helper Status Primary informal helper expresses feelings of distress, anger or depression Informal Helper Status Family or close friends report feeling overwhelmed by person s illness interrai Assessment Required Assessment Urgency Algorithm (AUA) Score 1 Yes 1 This is based on the Assessment Urgency Outcome score and entered by the user (see below) Score (Section C1) interrai CHA Assessment Required (Section C2) Description Description Legend Code Score generated by system for section C1 interrai CHA Assessment required 0 No 1 Yes Based on AUA Section C1 Note an AUA >= 3 indicates a CHA may be required Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 12

13 Scenario 4 Create a New Screener Assessment Pre-condition: Ensure all identification information has been entered within the system for the test client within Section A. **Important**: To complete the test scenarios please DO NOT enter real client information. Mandatory elements and Coding Responses required have been provided in the scenarios below, you may enter appropriate responses for all other elements not listed within the scenarios. Item Description Coding Legend A1-A10 Complete all mandatory fields Coding Response B1 Cognitive skills for Daily Decision Making 0 independent 0 B2a Activities of Daily Living (ADL) Bathing 0 independent or set-up help only 0 B2b Activities of Daily Living (ADL) Personal hygiene 0 independent or set-up help only 0 B2c Activities of Daily Living (ADL) Dressing Lower body 0 independent or set-up help only 0 B2d Activities of Daily Living (ADL) disability - Locomotion 0 independent or set-up help only 0 B3 Dyspnea (shortness of breath) 2 Absent at rest, but present when performing normal day to day activities B4 Self-Reported Health 3 Poor 3 B5a B5b B6 B7a B7b C2 Instability of Conditions Conditions/diseases make cognitive, ADL, mood or behaviour patterns unstable (fluctuating, precarious, or deteriorating) Instability of Conditions Experiencing an acute episode, or a flare-up of a recurrent chronic problem Self-Reported Mood- Ask in the last 3 days, have you felt sad, depressed or hopeless Informal Helper Status Primary informal helper expresses feelings of distress, anger or depression Informal Helper Status Family or close friends report feeling overwhelmed by person s illness interrai Assessment Required Assessment Urgency Algorithm (AUA) Score This is based on the Assessment Urgency Outcome score and entered by the user (see below) Score (Section C1) interrai CHA Assessment Required (Section C2) Description Description Legend Code 2 Score generated by system for section C1 interrai CHA Assessment required 0 No 1 Yes Based on AUA Section C1 Note an AUA >= 3 indicates a CHA may be required Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 13

14 Scenario 5 Create a New Screener Assessment Pre-condition: Ensure all identification information has been entered within the system for the test client within Section A. **Important**: To complete the test scenarios please DO NOT enter real client information. Mandatory elements and Coding Responses required have been provided in the scenarios below, you may enter appropriate responses for all other elements not listed within the scenarios. Item Description Coding Legend A1-A10 Complete all mandatory fields Coding Response B1 Cognitive skills for Daily Decision Making 0 independent 0 B2a Activities of Daily Living (ADL) Bathing 1 Supervision or any physical assistance B2b Activities of Daily Living (ADL) Personal hygiene 0 independent or set-up help only 0 B2c Activities of Daily Living (ADL) Dressing Lower body 0 independent or set-up help only 0 B2d Activities of Daily Living (ADL) disability - Locomotion 0 independent or set-up help only 0 B3 Dyspnea (shortness of breath) 2 Absent at rest, but present when performing normal day to day activities B4 Self-Reported Health 1 Good 1 B5a B5b B6 B7a B7b C2 Instability of Conditions Conditions/diseases make cognitive, ADL, mood or behaviour patterns unstable (fluctuating, precarious, or deteriorating) Instability of Conditions Experiencing an acute episode, or a flare-up of a recurrent chronic problem Self-Reported Mood- Ask in the last 3 days, have you felt sad, depressed or hopeless Informal Helper Status Primary informal helper expresses feelings of distress, anger or depression Informal Helper Status Family or close friends report feeling overwhelmed by person s illness interrai Assessment Required Assessment Urgency Algorithm (AUA) Score 1 Yes 1 1 Yes 1 This is based on the Assessment Urgency Outcome score and entered by the user (see below) Score (Section C1) interrai CHA Assessment Required (Section C2) Description Description Legend Code 1 2 Score generated by system for section C1 interrai CHA Assessment required 0 No 1 Yes Based on AUA Section C1 Note an AUA >= 3 indicates a CHA may be required Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 14

15 Scenario 6 Create a New Screener Assessment Pre-condition: Ensure all identification information has been entered within the system for the test client within Section A. **Important**: To complete the test scenarios please DO NOT enter real client information. Mandatory elements and Coding Responses required have been provided in the scenarios below, you may enter appropriate responses for all other elements not listed within the scenarios. Item Description Coding Legend A1-A10 Complete all mandatory fields Coding Response B1 Cognitive skills for Daily Decision Making 0 independent 0 B2a Activities of Daily Living (ADL) Bathing 0 independent or set-up help only 0 B2b Activities of Daily Living (ADL) Personal hygiene 1 Supervision or any physical assistance B2c Activities of Daily Living (ADL) Dressing Lower body 0 independent or set-up help only 0 B2d Activities of Daily Living (ADL) disability - Locomotion 0 independent or set-up help only 0 B3 Dyspnea (shortness of breath) 0 Absence of symptoms 0 B4 Self-Reported Health 1 Good 1 B5a B5b B6 B7a B7b C2 Instability of Conditions Conditions/diseases make cognitive, ADL, mood or behaviour patterns unstable (fluctuating, precarious, or deteriorating) Instability of Conditions Experiencing an acute episode, or a flare-up of a recurrent chronic problem Self-Reported Mood- Ask in the last 3 days, have you felt sad, depressed or hopeless Informal Helper Status Primary informal helper expresses feelings of distress, anger or depression Informal Helper Status Family or close friends report feeling overwhelmed by person s illness interrai Assessment Required Assessment Urgency Algorithm (AUA) Score This is based on the Assessment Urgency Outcome score and entered by the user (see below) Score (Section C1) interrai CHA Assessment Required (Section C2) Description Description Legend Code 1 Score generated by system for section C1 interrai CHA Assessment required 0 No 1 Yes Based on AUA Section C1 Note an AUA >= 3 indicates a CHA may be required Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 15

16 Scenario 7 Create a New Screener Assessment Pre-condition: Ensure all identification information has been entered within the system for the test client within Section A. **Important**: To complete the test scenarios please DO NOT enter real client information. Mandatory elements and Coding Responses required have been provided in the scenarios below, you may enter appropriate responses for all other elements not listed within the scenarios. Item Description Coding Legend A1-A10 B1 B2a Complete all mandatory fields Cognitive skills for Daily Decision Making Activities of Daily Living (ADL) Bathing 1 Modified independent or any impairment 1 Supervision or any physical assistance Coding Response B2b Activities of Daily Living (ADL) Personal hygiene 0 independent or set-up help only 0 B2c Activities of Daily Living (ADL) Dressing Lower body 0 independent or set-up help only 0 B2d Activities of Daily Living (ADL) disability - Locomotion 0 independent or set-up help only 0 B3 Dyspnea (shortness of breath) 0 Absence of symptoms 0 B4 Self-Reported Health 8 Could not (would not) respond 8 B5a B5b B6 B7a B7b C2 Instability of Conditions Conditions/diseases make cognitive, ADL, mood or behaviour patterns unstable (fluctuating, precarious, or deteriorating) Instability of Conditions Experiencing an acute episode, or a flare-up of a recurrent chronic problem Self-Reported Mood- Ask in the last 3 days, have you felt sad, depressed or hopeless Informal Helper Status Primary informal helper expresses feelings of distress, anger or depression Informal Helper Status Family or close friends report feeling overwhelmed by person s illness interrai Assessment Required Assessment Urgency Algorithm (AUA) Score 1 Yes 1 1 Yes 1 1 Yes 1 This is based on the Assessment Urgency Outcome score and entered by the user (see below) Score (Section C1) interrai CHA Assessment Required (Section C2) Description Description Legend Code 1 1 Score generated by system for section C1 interrai CHA Assessment required 2 No 3 Yes Based on AUA Section C1 Note an AUA >= 3 indicates a CHA may be required Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 16

17 Scenario 8 Create a New Screener Assessment Pre-condition: Ensure all identification information has been entered within the system for the test client within Section A. **Important**: To complete the test scenarios please DO NOT enter real client information. Mandatory elements and Coding Responses required have been provided in the scenarios below, you may enter appropriate responses for all other elements not listed within the scenarios. Item Description Coding Legend A1-A10 B1 Complete all mandatory fields Cognitive skills for Daily Decision Making 1 Modified independent or any impairment Coding Response B2a Activities of Daily Living (ADL) Bathing 0 independent or set-up help only 0 B2b Activities of Daily Living (ADL) Personal hygiene 1 Supervision or any physical assistance B2c Activities of Daily Living (ADL) Dressing Lower body 0 independent or set-up help only 0 B2d Activities of Daily Living (ADL) disability - Locomotion 0 independent or set-up help only 0 B3 Dyspnea (shortness of breath) 2 Absent at rest, but present when performed normal day-to-day activities B4 Self-Reported Health 3 Poor 3 B5a B5b B6 B7a B7b C2 Instability of Conditions Conditions/diseases make cognitive, ADL, mood or behaviour patterns unstable (fluctuating, precarious, or deteriorating) Instability of Conditions Experiencing an acute episode, or a flare-up of a recurrent chronic problem Self-Reported Mood- Ask in the last 3 days, have you felt sad, depressed or hopeless Informal Helper Status Primary informal helper expresses feelings of distress, anger or depression Informal Helper Status Family or close friends report feeling overwhelmed by person s illness interrai Assessment Required Assessment Urgency Algorithm (AUA) Score 1 Yes 1 This is based on the Assessment Urgency Outcome score and entered by the user (see below) Score (Section C1) interrai CHA Assessment Required (Section C2) Description Description Legend Code Score generated by system for section C1 interrai CHA Assessment required 0 No 1 Yes Based on AUA Section C1 Note an AUA >= 3 indicates a CHA may be required Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 17

18 5. Part 2: IAR Test Submission ***Please note that all validations must be done using mock data. Live data is not permitted in the IAR Validation environment. The IAR Validation environment is provided by CCIM for testing and validation purposes only. The IAR Validation environment is a developing environment that does not and cannot provide necessary controls to safeguard PHI. In a case of an accidental upload of PHI to the IAR Validation environment please immediately notify the IAR Support Centre at Option 8, or IAR@ccim.on.ca to have the uploaded PHI removed to minimize the risk of unauthorized PHI disclosure. How to obtain access to the IAR Test Environment Prior to any test submission, a specific vendor test account must be setup within the IAR Test Environment. CCIM will issue unique Organization identifier to be used by the vendor for all test purposes. If the vendor doesn t already have an account it will be made available by an request to iar@ccim.on.ca with a subject line of IAR Technical Request Vendor Test system access Purpose The compliance tests are meant to ensure transmissions to the IAR: Meet IAR Technical Specifications as outlined within the CCIM Data Submission and Technical Specifications V1.1 Meet IAR Technical Specifications as outlined in the IAR Release 4 Supplement Submission and Consent Specifications V1.1 Can handle simple activity sequences regarding assessment In order to successfully pass the validation, the submission process should be completed both manually and via the Web Service Scope There are four (4) submission tests that need to be completed both via the manual file uploads through the IAR interface and the IAR Web Service using the Submit Assessment operation. In order to successfully pass a test, the submission must pass through schema validation and import into the IAR without error. The tests themselves will confirm that the following criteria are met: 1. The IAR Submission Envelope is formatted correctly. 2. The IAR SOAP Envelope is formatted correctly (Web Service only). 3. All necessary Submission Header values properly configured. 4. Assessment data and IAR PHI data match correctly. 5. Assessment data passes schema validation. 6. All allowed actions (adding, updating, deletion) work as expected. 7. Consent directive flag (grant/deny of sharing assessment) works as expected 8. Error conditions are handled by vendor software. Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 18

19 For data submission to IAR please refer to the Data submission and Technical specification document on the Integrated Assessment Record section of the Vendor portal at Please refer to Appendix C IAR Submission Guidelines for additional information about the Web Service and Manual submission process steps as well as how to perform Person Search What tests need to be performed? The tests comprise of two main categories, Web Service submissions and Manual File upload. In order for a system to be deemed compliant using either method, the test steps listed for that category must be completed. The same set of assessment data can be used for both test categories. For interrai-ps submissions, vendors are requested to submit assessments that meet the Test Scenarios content requirements outlined within the interrai Preliminary Screener Implementation Validation Guide Part 1. Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 19

20 Test 1 Web Service Assessment Uploads through IAR Test 2 Manual Assessment Uploads through IAR Please refer to the IAR Validation Environment Guide (link below) for additional information: The two (2) tests involve submitting data following the five steps listed below. Both tests can contain the same assessment data. Test 1 involves submissions strictly via the IAR Vendor Test Web Service; Test 2 involves the creation of submission files and importing these files into the IAR via its Browser interface. Step 1 Single assessment using same Assessment ID Submission Method: Submission Type Web Service Manual Direction Add New Pass / Fail Pass / Fail Update (correction) Pass / Fail Pass / Fail Deletion Pass / Fail Pass / Fail Submit mock assessment; take a screenshot of that assessment in client record in IAR. Make a minor change to the client s date of birth in the previously submitted assessment and resubmit; take a screenshot of change in the IAR. Submit a deletion of the same submitted assessment. Once deleted, ensure that the assessment is no longer viewable in the IAR. Note: The steps outlined above must be performed as three separate submissions with different timestamps. Step 2 Three or more assessment records in one submission Submission Type interrai-ps Direction Add New Pass / Fail Submit 3 or more assessments in one submission file Note: The assessments may be assessment records created based on the test scenarios outlined in the Software Implementation Validation Guide Part 1 that contains test scenarios for different types of assessment urgency / priority. Multiple assessments will be contained in one file submission with one submission timestamp for the file. Step 3 Scenario coverage Submission Type interrai-ps Direction Add New Pass / Fail Submit the scenarios Note: This should be based on scenarios outlined in the software Implementation Validation Guide, Part 1 (p. 6). Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 20

21 Step 4 Assessments with Grant/Deny consent directive Submit 1 assessment with consent directive Grant ; take a screenshot of that assessment in client record in IAR. Re-submit the same assessment with the consent directive changed to Deny. Submission Type Add New Update Submission Method: Web Service Pass / Fail Pass / Fail Direction Submit 1 assessment with consent directive as Grant ; take a screenshot of that assessment in client record in IAR. Change the consent directive for that assessment to Deny in source system and re-submit that assessment with the modified consent directive Provide a screenshot of the consent directive option as it appears in vendor system UI (User Interface). Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 21

22 6. Appendix A Preliminary Screener Features Checklist This section is provided to vendors to assist them with User Acceptance Testing activities to complete on their vendor software prior to starting CCIM validation How to prepare for UAT? Review Implementation Validation Guides for the assessment software you are validating focus on Features Checklist Review any additional HSP specific requirements e.g., you may ask your vendor to provide additional reporting or features Prepare test cases to ensure that the software correctly handles everything in the Features Checklist (see Appendix A) Who should conduct UAT? One or more people in your HSP who: o o o Understand how to use the assessment tool Have experience with assessments on the front line Have completed training on the tool 6.3. How to conduct UAT? Refer to the Features Checklist in Appendix A to track your test results Complete each of your test case scenarios and keep track of your results If you have any scenarios that do not pass, check that they are keyed in correctly: o o o Coding accuracy Dates are correct Scenarios are complete If you have checked all the above and there are outstanding issues, take these to your vendor for resolution. Examples include: o o o A misunderstanding about how to code something Possible software bug that needs to be fixed by the vendor Issue with settings in your environment Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 22

23 6.4. How long should UAT take? Generally UAT should take no more than 1-2 days depending on the assessment and whether it is coded according to the scenarios The process will be expedited when user ID and passwords are set up (either by technical lead or vendor) prior to testing and an experienced assessor is conducting the actual testing 6.5. Screener Assessment Creation Reminder: Actual results answers should always be YES. Alert your vendor if any answer is NO. Step Description Expected Result Actual Results Yes No 1 Start a new screener assessment Assessor is able to create a new screener assessment with a unique ID Can you create a new screener assessment? Is the screener assessment ID displayed unique? 2 Check the screener assessment status displayed The screener assessment is assigned an In Progress status Is the screener assessment status displayed as In Progress? 3 Enter few screener assessment responses in any order within sections Screener assessment responses can be entered randomly in any order or sequence Can you provide responses randomly? 4 Enter notes in one or more sections of the screener assessment Notes are appended to each section within the screener assessment Are notes appended to the end of each section? 5 Save screener assessment User is able to save screener assessment at any time while still in progress Can you save a screener assessment in this status at any time? 6 Perform a search for the specific client or screener assessment record The status of the screener assessment record for that client should remain as In Progress and prior responses provided must be displayed Is the screener assessment record displayed with an In Progress status? Does the system display previous saved responses? 7 Screener assessment completion continuation User is able to continue entering responses within the same screener assessment record Does the system allow user to continue from where last response was entered? 8 Skip entry for Section B2 and Section B3 and enter responses for all required questions for the screener assessment, and mark as complete An error message is displayed preventing user from marking the screener assessment record as complete Is error message displayed preventing the user from marking the assessment as complete? 9 Complete all required mandatory fields for the screener assessment and mark as complete Marked complete successfully and no error message is displayed Can you mark it as complete? 10 View Assessment Urgency Algorithm (AUA) AUA score is automatically calculated and displayed once all mandatory fields are complete Can user view the screener Assessment Urgency Algorithm? Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 23

24 Step Description Expected Result Actual Results Yes No 11 Perform a search for the specific client or screener assessment record The user can view the screener assessment record and the Outcome Score (AUA), with the status also displayed as Complete Is the entire screener assessment and Outcome Score (AUA) displayed? Is the status of the screener assessment Complete? 6.6. Screener Assessment Cancellation Step Description Expected Result Actual Results Yes No 1 Create a new screener assessment, enter responses to some fields, and choose to cancel the screener assessment record with a cancellation reason Screener assessment status is changed to Cancelled Is the screener assessment status changed to Cancelled? 2 Search for the Cancelled screener assessment record to verify responses entered prior to cancellation and cancellation reason entered is stored The record displays responses provided prior to cancellation, cancellation reason is entered, and status is Cancelled Is a Cancelled status displayed for the screener assessment record? Is the cancellation reason displayed? Are values of response provided are displayed? 6.7. Screener Assessment Completion Step Description Expected Result Actual Results Yes No 1 Enter current date in the Client Date of Birth field An error message is displayed if client birth date is current date Is error message displayed preventing the user from entering current date as DOB? 2 Enter a birth date less than current date The date is accepted as a valid birth date Is this date accepted as a valid birth date? 3 Enter a date for Date screener signed as complete that is less than the Assessment Reference Date An error message is displayed indicating the Date screener signed as compete cannot be less than the Assessment Reference Date Is error message displayed? 4 Enter a Date screener signed as complete that is greater than the Assessment Reference Date The Date screener signed as complete is accepted as valid Is the date entered accepted as a valid entry date? 5 Enter no value for Health Card number (A4a Screener) and no value for the field Reason for No Health Card number entered (Administrative Face sheet); enter all other mandatory values and mark the screener assessment as complete An error message is displayed preventing user from marking the screener assessment record as complete indicating that Either specify a Health Card number in section (4a) or select a Reason for no Health Card number Is the error message displayed? Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 24

25 6.8. Screener Assessment Management Step Description Expected Result Actual Results Yes No 1 Search and view client screener assessment(s) (based on defined search criteria client ID, status, etc.) An historical list of client screener assessment record is displayed Are client screener assessment records displayed based on search criteria? 2 Print In Progress and Completed screener assessments (should include outcome score (AUA) and header information) Screener assessments can be printed Is screener assessment, outcome score (AUA), and header information printed properly? 3 Print screener assessment without Notes section Screener assessment can be printed without the notes Is user able to print screener assessment without notes? 4 Print screener assessment with Notes section Screener assessment can be printed with notes Is user able to print screener assessment with notes? 5 Export one or multiple screener assessment records to CSV or Excel file formats Screener assessment records are exportable to Excel and/or CSV Able to export one or more screener assessment records to CSV or Excel file format? 6 Print blank screener assessment Blank screener assessment should have the same look and feel as paper version of the interrai Preliminary Screener Is the blank screener assessment printout version the same as a paper version? 6.9. Software Offline Version Step Description Expected Result Actual Results Yes No 1 In the offline mode repeat the steps in Screener Features Checklist Section 1, Screener Assessment Creation Refer to expected results Refer to actual results for Screener Assessment Creation 2 Ensure that once you are back in the office and connected to your server you are able to upload the screener assessment created The screener assessment records created offline are uploaded successfully Can you upload the screener assessment? 3 Search for screener assessment records created in offline version of the software You are able to search and view the screener assessment records online and all the information displayed matches the offline version Are the records created offline searchable in the online version and do the number of records and information match? 4 In the offline mode repeat the steps in Screener Features Checklist Section 2, Screener Assessment Cancellation Refer to expected results Refer to actual results for Screener Assessment Cancellation 5 In the offline mode repeat the steps in Screener Features Checklist Section 3, Screener Assessment Completion Refer to expected results Refer to actual results for Screener Assessment Completion 6 In the offline mode repeat the steps in Screener Features Checklist Section 4, Screener Assessment Management Refer to expected results Refer to actual results for Screener Assessment Management Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 25

26 6.10. Organization and Assessor Reports Step Description Expected Result Actual Results Yes No 1 Ensure your vendor has developed the following Assessor Reports: 1. Client s CAPs and Outcomes Report 2. Client Progression Report 3. Client Assessment Summary Report The date generated in the report should be accurate when checked against the actual assessment record Report features such as filters, format of display, sort are working properly Is the date generated accurate? Are all the features working properly? Extract and Submit the XML Files Step Description Expected Result Actual Results Yes No File name should conform to the following format: [AssessmentType]<YYYYYMMDD><HHMM><OOOO><ZZ>.<999>.xml [RAI CHA]< ><08:30><1234><IA>.<002>.xml 1 Extract and submit assessment data in XML format Where: [Assessment Type], RAI CHA, YYYYYMMDD the date when the export process started; HHMM - time when the export process started (note 24 hour clock is used); OOOO organization ID, or parent organization id used in assessment application in multi-site environments (in case of OCAN); Is the generated file in the required format? ZZ - target system suffix, IA for Integrated Assessment; File sequence number, a sequence number of extracted file if export is broken into multiple parts, default value Extract and submit files that include CHA, FS, AL, DB and MH Only completed assessments files will be submitted in XML file, the status must = Complete. Supplements based on the conditions outlined will be submitted as part of assessment Triggered and Not Completed Triggered and Completed Not Triggered and Completed Do all assessment records have a Completed status? Are there assessment records with any one of the conditions outlined? Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 26

27 7. Appendix B Implementation Validation Steps This diagram is a high level summary of the validation steps within the full implementation cycle. Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 27

28 8. Appendix C IAR Submission Guidelines 8.1. IAR Home Screen HSPs will be set up with two roles for testing and validation: Uploader and Viewer The Viewer role will have access to view assessments ONLY The Uploader role will have access to submit assessments and review upload log reports ONLY 8.2. Web Service Configuration Your source test software environment will need to be configured to submit automatically to the IAR Validation environment. If you require assistance with your source test software environment configuration, please contact your vendor. Please also ensure that your vendor provides instructions for completing a web service submission. Configuration Details: Configure your source test software with the following URL: Configure your source test software with the user ID, the password, and test Organization ID provided by CCIM upon request o NOTE: The Test Organization ID needs to be configured into the source software system in order to test both the manual and web service submissions Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 28

29 8.3. Uploading Assessments Manually Step 1 The Menu bar is found on the left side of the IAR window The Menu bar is the primary navigation method Each menu contains one or more options, which are links to IAR functions Menu options are dependent on access privileges granted to the user Click on Submissions to expand the menu Then click Upload Step 2 Click Browse to pick the submission file you would like to upload In the dialog box that appears, locate the save file and click it: Click Open Click Upload Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 29

30 Step 3 Once you have uploaded the file, IAR will return the message below Record the TX number (also referred to as the Transaction ID) 8.4. Retrieving the Transaction Record Step 1 Click on Review History Step 2 If you noted the Transaction ID, enter it in the Transaction ID field Step 3 If you didn t record the Transaction ID, click on Search without adding any search criteria (the search results will return all transactions within the last 15 days) 8.5. Interpreting Results and Event Status Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 30

31 The results, as shown above, display a minimum of four (4) audit records (events) for each successful upload submission: Received Submission Process Submission Store Consent Store Assessment If all four events are successful, the assessment document will be displayed in the appropriate client s record. Open any of the audit events by clicking on Event Status of Success or Fail this will provide further information Searching for a Client Select the Person Search option on the Menu Bar Add search criteria re person to search In order to find a client, follow the steps outlined below Easy Steps for Searching: Step A1 Ontario Health Card Number OR Step B1 - Client Last Name Step B2 Client First Name Step B3 Date of Birth IMPORTANT: Unlike source software that provides a short list with minimal search criteria added, the IAR requires a minimum of the First and Last Names correctly spelled, as well as the Date of Birth. Once search criteria are entered, click Search to retrieve the client s record. If the uploaded assessment was successfully stored and the search results come back with no matching client, add more criteria (e.g., health card number) Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 31

32 Once the client s name is displayed, click on it to view associated assessments Implementation Validation Guide for Vendors ver 2.1 November 2016 csscap@ccim.on.ca 32