[Maria Jackson Hittle] Thanks, Michael. Since the RSR was implemented in 2009, HAB has been slowing shifting its focus from data submission to data

[Michael Costa] Welcome to today s Webcast. Thank you so much for joining us today! My name is Michael Costa. I m a member of the DART Team, one of several groups engaged by HAB to provide training and technical assistance to Ryan White grantees during the implementation of the RSR. Today s Webcast is presented by Maria Jackson Hittle of the WRMA/CSR Data Support Team. Maria will give you an in depth look at the different data elements collected in the RSR client level data file and the common data validation warnings and errors. She ll also tell you how to review submitted data to verify that the data you submit are both complete and correct. At any time during the presentation, you ll be able to send us questions using the Question function on your control panel on the right hand side of the screen. You ll also be able to ask questions directly live at the end of the presentation. You can do by clicking the raise hand button (on your control panel) and my colleague, Titi, will conference you in. You can also click the telephone button and you ll see a dial in number and code. We hope you consider asking questions live, because we really like hearing voices other than our own. Now I ll turn this over to our presenter. Maria? 1

[Maria Jackson Hittle] Thanks, Michael. Since the RSR was implemented in 2009, HAB has been slowing shifting its focus from data submission to data quality. It s no longer just about submitting your data on time. Now, it s about submitting complete and accurate data on time. HAB is challenging you to improve your data quality. And, in support of that effort, we have scheduled three Webcasts focusing on RSR Data Quality. The first Webcast, presented by Ellie Coombs with the DART Team, was held last week. She explained the concept of data quality and its importance. She told you how you can review your data before upload and the strides RSR System Vendors are making to ensure data quality at the point of data entry. Ellie also told you about the tools being developed by RSR System Vendors to help you review and clean your data before you create your client level data XML file. If you missed last week s Webcast, I strongly recommend that you view it when it s posted to the TARGET Center Website. Today s session is going to focus on client level data validation. Your 2012 RSR will go through a more rigorous review than ever before. More, and more detailed, validation checks have been added to the RSR System. So, my goal with today s session is to help you prepare by telling you about the new validation checks you ll encounter when you submit your 2012 RSR, and to warn you about the validation checks that may impede submitting your report. This year will be the first year your RSR may be rejected because of poor data quality in your client level data file. Next week, you will hear from Elisa Peet, who will tell you all about the tools that have been added to the RSR System, and how you can use those tools to verify that the data submitted to HAB match the data in your system. Be sure to sign up for that event on the TARGET Center Website. 2

Let me warn those of you who are new to the RSR: Today s session is not for beginners. It builds on information presented in previous trainings and Webcasts. But for the benefit of our newbies, I m going to quickly run through the basics of client level data reporting and data validation. I may say some things you ve never heard before. Don t panic. As Michael mentioned, you can ask questions at the end of the presentation. After the rapid fire primer on client level data and data validation, I m going to discuss a few selected RSR client level data elements and their associated system validations. Unfortunately, there s no time to go over each and every data element and its validations today. Instead, my goal is to give you a foundation to build on as you pursue better and better data quality. After that, we ll wrap up with a few final thoughts about data validation.

Introduction to the RSR All providers that indicate in their Provider Report that they deliver direct client services are required to upload client level data. The data are uploaded into the RSR Provider Report as an XML file. If you don t have the first notion about what XML could be, there are resources on the TARGET Center Website that will explain XML. 4

2011 Annual Ryan White HIV/AIDS Program 2/5/2013 Regional Data Training The client level data XML file should include one record for each client who received a RWHAP funded service during the reporting period. And each client record must include the client s encrypted Unique Client Identifier (euci). The client s record may also contain up to 66 data elements, grouped into 3 sections: The client s demographic information (Items 1 15); The RWHAP funded core medical and support services the client received (Items 16 45); and The client s clinical information (Items 46 66), if applicable. Providers can determine which data elements are required for a particular client by looking at Appendix A of the RSR Instruction Manual. Appendix A shows what we call the dot chart. 5

Introduction to the RSR 10/25/2012 This slide shows a portion of the chart in Appendix A. As you can see, the services are listed in the column headings and the client level data elements are listed in the row headings. When a dot appears at the intersection of a service and data element, that data element has to be reported for the recipient of that service. Remember, when it comes to the RSR client level data file, it s about the service. The service the client receives dictates the data elements the provider is required to report on for that client. 6

After you create the client level data file, it is uploaded into the Provider Report. The Provider Report and client level data are validated together. Data validation is a process that compares your data with the data requirements outlined by HAB. Each client record in the XML file is compared with the HAB s data requirements. This process helps ensure the data you provide for your clients are complete and make sense. There s an illustration of the concept of data validation on the right. HAB s data requirements are represented by a blue box. Notice how the first, second, and fourth client records are blue boxes, just like the data requirements? Since the client records match the data requirements, these data will not trigger a system validation notice. Now look at the third client record, which is represented by a red box. The data in that client s record don t match HAB s data requirements. The client data represented by the red box will trigger a system validation notice. 7

The validation checks will return an error, warning, or alert notification when the rules behind the validation check are violated. Each one identifies a potential data issue that may keep your data from properly representing your program. Think of it as a traffic light. Errors: Receiving an error message is like hitting a red light. You have to fix your data before you can submit it to HAB. Warnings: Receiving a warning is a lot like hitting a yellow light. You can proceed; but you should do so with caution. In other words, you can submit your data with a warning; but you should do so cautiously. You should review all of the warnings in your validation report before you submit your RSR. If the data are correct, submit your report with a meaningful comment that explains your data. If the data are incorrect, fix your data. Or, if you do not have time to fix the data, submit the data with a comment telling HAB how you re going to fix the issue before you submit the 2013 RSR. Alerts: Finally, there are alerts. Data alerts are informative; they are there to help you identify potential issues with your data. And, as with a green light, you can proceed with data submission at will. You do not need to try to fix the data that caused an alert before you submit your RSR, and you do not need to write a comment to explain an alert. But you should always review the data that triggered the validation notice. Remember, the automated RSR system validation checks are just one aspect of data validation. Equally (if not more) important are the manual validation checks that you will perform. 8

So please be sure to take the necessary time to review your data before and after you upload it into the RSR system. Why? Because even with the automated RSR system data validation checks, incorrect data can still be submitted to HAB if the data are properly formatted and meet HAB reporting requirements. For example, the system checks cannot tell if the total number of clients or service visits reported is too high or too low. But you can, if you compare the reports generated in your local systems with the reports (such as the upload confirmation report) in the RSR system. Start reviewing your data right now. Don t wait! You are a critical part of the data validation process! And, without your active participation in this process, there is no way HAB can use the data with confidence. 9

Now that we have covered client level data reporting and the concept of data validation at a very high level, let s get down into the weeds. 10

In the interest of time, we won t discuss all of the client level data elements or the validation checks that are applied to those data elements. As I mentioned before, my goals are to give you a solid foundation on which to build your expertise and to alert you to pitfalls that impede the successful submission of your 2012 RSR. And though I am not going to go over everything today, I encourage you to learn about each of the data elements by reviewing the RSR Instruction Manual, and to download and review the complete list of the validation checks in the RSR Web system by following the steps shown here on this slide. If you have questions while you are reading those materials, give us a call at Data Support. We ll be happy to answer them. 11

The first section of the client level data report collects demographic data elements on each client reported. Here is a list of all of the data elements that may be collected in the clientlevel data file. As I mentioned, the demographic data elements reported for each client are determined by the services the client received. For example, providers will report all of the demographic data elements for clients who receive OAMC, MCM, and NMCM services. Meanwhile, providers will report only basic demographic data elements for recipients of other core medical and support services. 12

Again, HAB is raising the bar on data quality. One important aspect of data quality is completeness. Beginning this year, you will get an alert if any of the demographic data elements shown on this slide are missing. You should be reporting something for each of these data elements, even if that something is unknown. 13

But even reporting Unknown for the Poverty Level, Housing Status, or Medical Insurance demographic data elements is not good enough. These demographic data elements have the lowest completion rates nationally. So, for the 2012 RSR, you will also receive a validation alert if you report Unknown for any of these elements. Providers should be collecting information on each client s poverty level and health insurance to verify the client s eligibility for the Ryan White HIV/AIDS Program. 14

In addition to validation checks for missing and unknown data, the RSR system includes validation checks that verify the data being reported are logical and internally consistent. Let s look at the data element First Service Visit Date as an example. The client s first service visit date is the date of the client s first service visit, EVER, at the agency. So this visit may have occurred before the start of the reporting period. Also, it may or may not be the client s first HIV related visit or RWHAP funded visit. And because it is the first visit date, the date should never change in subsequent reports. Even if the client drops out of care for a time, the client s first service visit date at your agency does not change. 15

Given those reporting rules, HAB has developed a series of validation checks verifying that the data being reported for a client s first service visit date make sense. For example, your data will trigger a warning notification if a client s first service visit occurs before the client s birth year, after the client s date of death, or after any of the client s other service visits. Notice that you will receive an error if the client s first service visit occurs after the end of the reporting period. HAB only wants data for clients who received a service during the reporting period. If the first service visit occurred after the reporting period end date, the client s record should not be included in the client level data file. And this is not the only validation check in the demographic information section that will trigger an error. 16

You will also receive a validation error if you do any of the following: Report that a client s date of death occurred after the reporting period end date; Report a transgender status for a client who is not transgendered; Do not report a transgender status for a client who is transgendered; or Report that a client s AIDS diagnosis year occurs after the reporting end date. Remember, you cannot submit your RSR with errors. So, if you encounter any of these data issues, you will be required to revise your client level data in your source system, and then generate and upload a new (revised) XML file. 17

Ryan White HIV/AIDS Program Funded Services The next section of the client level data, Items 16 45, collects information on the Ryan White funded Core Medical and Support Services the client received. The criterion for including a client s record in the client level data XML file is that the client received at least one RWHAP funded core medical or support service during the reporting period. Each client record must have at least one RWHAP funded service reported in this section. HAB has also added new checks that help ensure providers are reporting services according to HAB s guidelines. You should not report more than one service visit in each category per day. For the 2012 RSR, you ll receive a validation alert if you report that the client received more service visits in a category than there are days in the applicable quarter. For example, there are 92 days in the fourth quarter of the 2012 reporting period, October 1 through December 31. You cannot report more than 92 service visits for a single client in a single service category for the fourth quarter of 2012. Remember: For each day, only one service visit in each category may be reported for the RSR even if the client receives more than one service in a particular category during the day. You shouldn t report that a client has more RWHAP funded Outpatient/ambulatory medical care visits in Item 16 for the reporting period than the total number of HIV Outpatient/Ambulatory medical care dates in Item 48. In other words, the number of Outpatient/ambulatory visits reported in Item 16 must be equal to or less than the total number of HIV Outpatient/Ambulatory Service Visit dates reported in Item 48. You shouldn t report that a client received a service if you did not have an active contract for that service at the time your client received it. You ll get a validation alert if you do. The bottom line is you should not provide services with your RWHAP funds that you are not funded to deliver. And this is actually an excellent place to expand our discussion beyond client level data for a moment. 18

There are an important series of validation checks that are performed for every service reported as funded in the Grantee Report, delivered in the Provider Report, and uploaded in the clientlevel data file. The first two checks compare the services reported as funded in the grantee report with the services reported as delivered in the Provider Report: If a provider delivered a service that was not funded by the grantee, the provider will receive a validation error. For the 2012 RSR, providers cannot report that they delivered a service that a grantee did not fund! If a grantee specifies a service as funded that the provider did not deliver, the provider will receive a validation warning. For example, let s say a Part D provider is funded for legal services and permanency planning services. The provider delivers legal services, but none of its clients need permanency planning services. The provider will receive a validation warning because permanency planning services are funded but the provider did not deliver this service. In this situation, it is appropriate for the provider to submit its report with a warning. The next two checks compare the services reported as delivered in the Provider Report with the services that were uploaded in the client level data file: If a provider reports that it delivered a service that none of its clients received, this means the service was not uploaded in the client level data XML file, and the provider will receive a validation warning. If a provider reports that a client received a service that it did not deliver, the provider will receive a validation warning. 19

Introduction to the RSR The final group of data elements collected in the client level data upload are the clinical information data elements. They were developed to help HAB evaluate if clients are receiving a consistent level of care across all service providers. This section is slightly different from the RWHAP funded services section. This is because if the client received even one Ryan White supported outpatient/ambulatory medical care service, the provider will report all of the client s clinical information, regardless of who paid for or delivered those clinical services. 20

The clinical information data elements required of all HIV positive clients who receive a RWHAP funded OAMC service are shown on this slide. For the 2012 RSR, you will receive a validation alert if any of these items are missing for these clients. 21

For the clients who were not screened (No, Unknown, Not Medically Indicated) for Tuberculosis (TB), Hepatitis B (Hep B), or Hepatitis C (Hep C) during in the reporting period, you will also report if the client was screened for TB, Hep B, or Hep C since his or her HIV diagnosis. You will only report CLD Items 63 64 for your female HIV positive clients and Items 65 66 for your pregnant HIV positive clients. You ll receive an alert if you don t report these data for the applicable clients. 22

Again, the clinical information data elements should only be reported for HIV positive clients who received a RWHAP funded outpatient/ambulatory medical (OAMC) service during the reporting period. You will receive a validation alert if these data are missing for these clients. You ll also receive a validation alert if you report clinical information for HIV positive clients who DID NOT receive a RWHAP funded outpatient/ambulatory medical care service or for clients who are HIV negative, indeterminate, or have unknown HIV status. You will not have to fix your data or write a comment to submit your RSR if you receive these alerts. But please don t ignore them. Print them out, and take some time to think about how to fix them in the coming year. 23

As with the demographic data elements, HAB has added a series of checks to ensure the data reported make sense, are internally consistent, and meet HAB s reporting requirements. Let s look at the four clinical information data elements with validation checks that will return an error notification for the 2012 RSR: Items 47, 48, 49, and 50. Item 47 collects the client s first HIV Outpatient/ambulatory medical care visit with this provider. Where the first service visit date could be for any service (for example, case management services), the first OAMC visit should meet the RWHAP definition of an outpatient/ambulatory medical care visit. But, like the first service visit date, the first OAMC visit may have occurred before the start of the reporting period, may or may not be a RWHAP funded visit, and does not change in subsequent reports. One other important point about the first OAMC visit date: It may or may not be same date as the date of the client s first service visit with this provider (CLD Item 1). Item 48 collects all dates of the client s outpatient/ambulatory care visits in the provider s HIV care setting with a clinical care provider during the reporting period, regardless of the payer. Item 49 collects all of the client s CD4 test dates and results that occurred during the reporting period. Item 50 collects all of the Viral load test dates and results that occurred during the reporting period. 24

For the 2012 RSR, you will receive an error if you report that a client s first HIV OAMC visit (CLD Item 47) occurred after any of the client s other HIV OAMC visit dates (CLD Item 48). Remember, Item 47 is the client s FIRST OAMC visit. And notice how I stressed the words during the reporting period when I described Items 48, 49, and 50. HAB doesn t want data from outside the reporting period for these clinical information data elements. So, if you report OAMC visit dates, CD4 test dates, or Viral load test dates that occurred BEFORE the reporting period, you will receive an error notification. 25

You will also receive an error notification if you report a client with any of the following: A first HIV Outpatient/Ambulatory medical care visit after the reporting period end date; An HIV Outpatient/Ambulatory medical care visit (as reported for Item 48) after the reporting period end date; CD4 test dates after the reporting period end date; or Viral load test dates after the reporting period end date. These are the errors you may encounter when you validate your client level data. 26

You can now receive errors for every section of your client level data. Remember: You will not be allowed to submit your data if you have any of the errors we ve just discussed. You must fix your data before you can submit your RSR. You must find and fix the erroneous client data in your local system, generate a new client level data file, and then upload that file into your Provider Report. Sometimes you have to submit your report with warnings. Keep in mind that alerts and warnings are not bad. Triggering a validation warning (or alert) doesn t immediately indicate a problem with the data. It just means that HAB wants you to take another look at the data and proceed cautiously with your submission. If the data are correct, submit with a warning comment that explains the data. And if the data are incorrect, fix you data. Or, if you can t fix you data before the deadline, submit your RSR with a warning comment that tells us how you plan to fix the issue, so you do not encounter the same warning or alert for the 2013 RSR. Don t submit valid but inaccurate data. Review your data both before and after you upload it into the RSR system. Again, you cannot rely on the RSR system validations alone. Though your data may pass the system validations, the data may still be incorrect. There are a number of data errors that may occur which the system cannot detect. Use your local system tools to check the data before you upload them into the RSR system. You know your program best. It s up to you to ensure the data reported to HAB are complete and accurate. Note: Your manual checks should not be performed by hand with a record by record review of your client level data. Use your data system s reporting features to help you review your data. Or, use the X ERT application to review the data in your client level data XML file before you upload it. Then verify the information in the Web system is correct by comparing the data in your local system with the data in the RSR system using the tools in the RSR system. 27

Be sure to note which data issues cannot be fixed for 2012, and then use the 2012 RSR submission to improve your data collection, management, and reporting practices for 2013. Resolve to review your provider s data on a regular schedule throughout the year. By focusing on your data regularly in 2013, you will ease your data submission process for the 2013 RSR. Also, be sure to talk with your providers. Let them know what you expect for their data. As grantee, you are obligated to monitor each provider s performance. And monitoring their data is one way you can meet that obligation. 28

Educate yourself and your providers about data quality using the RSR technical assistance resources available on the HAB and TARGET Center Websites. 29

If you have questions or need help, please call someone. There s no wrong door! Any of the contacts shown on this slide are ready to assist you with data validation and submission. We ll make certain that your call is triaged and directed to the group that will be the most helpful to you. Thank you for joining us today, and good luck with your 2012 RSR submission! [Michael Costa] We will now take your questions. You can send us questions using the Question function on your control panel on the right hand side of the screen. You can also ask questions directly live. You can do this by clicking the raise hand button (on your control panel). If you are using a headset with a microphone, my colleague, Titi, will conference you in; or, you can click the telephone button and you will see a dial in number and code. We hope you consider asking questions live, because we really like hearing voices other than our own. As you exit this Webcast today, we d like your feedback on this presentation. On your screen you will see three evaluation questions appear. Please take a moment to respond to each. We appreciate your feedback very much, and we ll use this information to plan future Webcasts. Once again, feel free to contact any of the TA contractors for all of your questions about data and reporting. There are no questions you can t ask when looking for answers for the HAB data support contractors. 30

Thanks again for joining us today! 30