11.0 Random Assignment

Transcription

1 11.0 Random Assignment Random assignment is the procedure by which enrolled youth will be assigned to either the Usual Services or ASPIRE Services groups. Random assignment is performed in a computer system, Random Assignment System or RAS created by Mathematica Policy Research (MPR). The RAS will be used to (1) verify eligibility for ASPIRE, (2) gather contact information for the youth and their parent/guardian, (3) check that the youth has not been previously assigned, (4) conduct random assignment (or assignment to a previously enrolled sibling s group), (5) automatically create letters notifying enrollees of their random assignment, and (6) produce reports on recruitment and enrollment progress. All ASPIRE Staff must receive Social Security Suitability Clearance prior to access being granted to the Random Assignment System Access and Use of the RAS Access to the RAS will be based on staff s role on the project. Permissions levels are administrator and counselor. Administrators have access to reports & account creation; whereas counselors have data entry privileges only. Procedures for all staff include: Random assignment must occur within two business days of the enrollment meeting. If there is an illness or emergency and random assignment cannot be completed within two business days, the Site Coordinator will decide how to proceed, as soon as possible. Random assignment must be completed prior to leaving on vacations or other outoffice events Log in to the RAS Create an Account. Before the ASPIRE staff member can access the RAS, the program administrator will need to create an account for him/her. Once the account is set up, the staff member will be sent an account creation to invite him/her to use the system. The staff member should click on the link in the to validate the address. This address will serve as the username and is not case sensitive. The staff member will then be prompted to create a secure, case-sensitive password that is at least eight characters long and contains at least one letter, one number, and one special character (such #, $, or %). Next the staff member will be prompted to pick a method for the multifactor authentication process. He/she can either enter a cell 11.0 Random Assignment 1 of 15

2 phone number to receive a text or a phone number to receive a call. Once a password is created, the link in the original account creation will no longer be valid. SAVE WEBPAGE. The RAS login page (Screenshot 11.1) will appear. Enter the username ( address) and password and then click the Log in button. The staff member will then begin the multifactor authentication process. Depending on the method chosen, the staff member will either receive a text message with a six digit numeric code that he/she will have to text back to the sender or a phone call that will prompt him/her to press the # key. The authentication process can be changed after login. Screenshot 11.1 RAS Log in Page If the ASPIRE staff member forgets his/her password, he/she may click the Forgot your password? link. The staff member will receive an with a link to change the password. The staff member must complete the password change and log in within 15 minutes or the link will become inactive and the account will be locked. If an incorrect password is entered three times, the account will be locked. After each unsuccessful login attempt, the staff member will be reminded of how many attempts remain before he/she is locked out of the system. A supervisor must then be contacted to reset the password. Once the password is reset, the staff member will receive an with a link to create a new password. A new password and login must be created within 15 minutes or the link will become inactive. After the staff member can successfully log in, the RAS will show the welcome page (Screenshot 11.2). To prevent access to the RAS by unauthorized users, the system will automatically log out after 20 minutes of inactivity. Work must be saved regularly by 11.0 Random Assignment 2 of 15

3 clicking the Save Draft button located at the bottom of the RAS data entry page (Screenshot 11.7), which will prevent any unsaved changes from being lost and inform RAS of continued activity. Additionally, the system will auto-save any work on the Enroll Youth page every five minutes to avoid the loss of data entered by ASPIRE staff. If a staff member is logged into RAS for two hours, the system will automatically logout and the staff member will need to log in again. RAS will display a countdown clock in red text at the top of the page 15 minutes prior to the automatic two-hour log out. Screenshot 11.2 RAS Welcome Page Users will have access to the system on a need-to-know basis. Depending on user level, the staff member may be restricted from viewing certain items on the top menu bar and/or data from the Find Youth menu. For example, Case Managers and Site Coordinators in one state will only be able to view detailed information for youth enrolled in their state. Site Coordinators and the Project Director will be able to run reports and manage user accounts Enter Information from AIMS Random Assignment Form Instrument Click on the Enroll Youth link on the menu at the top of the screen (Screenshot 11.3). This menu will appear on nearly all pages and can be used to navigate around the website Random Assignment 3 of 15

4 Screenshot 11.3 Enroll Youth Participation Link This will take the staff member to the eligibility validation screen (Screenshot 11.4). The staff member will be prompted to enter the youth s SSN and DOB (date of birth) first to confirm that he or she is eligible for ASPIRE. Screenshot 11.4 Eligibility Validation Screen The youth s SSN and DOB will be compared with those of all eligible youth. If eligibility cannot be confirmed based on the youth s SSN and DOB, the staff member should verify that the information entered is correct. Once the information is verified with the youth and the parent/guardian, if eligibility is still not confirmed you should inform them that they are not eligible for the ASPIRE project. The youth s SSN and DOB will also be compared with those of all previously enrolled youth. If the youth is found to be already enrolled in ASPIRE, an error message will reveal that he or she has already been randomly assigned to a study group (Screenshot 11.5). If the individual is eligible and not already enrolled in ASPIRE, the RAS data entry page will come to the screen Random Assignment 4 of 15

5 Screenshot 11.5 Error Message for Already Enrolled Youth When eligibility is confirmed, the system will display the RAS date entry page (Screenshot 11.6). The two initial fields (SSN and DOB) and the youth s name will be pre-populated. Additionally, the pre-determined unique study identification number will be displayed on the screen. Screenshot 11.6 RAS Data Entry Page 11.0 Random Assignment 5 of 15

6 Enter the relevant data fields the AIMS Random Assignment Form into the RAS. The Random Assignment Form has been designed to reflect of the order of the RAS screen. If both the RAS screen and the AIMS Random Assignment Form instrument are displayed at the same time, data entry will be easier. Table 11.1 contains a list of required and optional fields. Though some fields are not required, we recommend that ASPIRE staff try to obtain as much information as possible from the youth and their parent/guardian. Random assignment can still be completed without entering the optional data. Table 11.1 Required and Optional Fields for Random Assignment Required Fields for Random Assignment Youth Data Fields Youth first name Youth last name Optional Fields (do not need to complete optional fields to do random assignment) Youth middle initial Youth address Youth signature Youth address line 2 Date of youth signature At least one telephone number for youth a Youth Social Security number Youth date of birth Youth gender Youth address line 1 Youth city/state/zip Region b Parent/Guardian Data Fields Parent/guardian first name Parent/guardian last name Parent/guardian signature Parent/guardian middle initial Parent/guardian address Parent/guardian Social Security number Date of parent/guardian signature Parent/guardian address line 2 At least one telephone number for parent/guardian c Parent/guardian date of birth Parent/guardian gender Parent/guardian address line 1 Parent/guardian city/state/zip 11.0 Random Assignment 6 of 15

7 If any implausible or unlikely information is entered, the system will display an alert message describing the issue in red text at the top of the screen and/or next to the field in question. Screenshot 11.7 provides an example of an error in which an invalid address was entered. The system will also display an alert for any missing data fields if the staff member attempts to conduct random assignment without entering data into all of the required fields. If the parent does not have a phone number--which is a required field--the parent can use the phone number of a close contact who can easily pass on a message for the parent to return the case worker s call. Examples may include a close relative, friend, or neighbor. If optional fields are left blank, random assignment can proceed. Screenshot 11.7 Error Message: Invalid Address and Missing Fields Based on the information provided on the consent form, the RAS will match the SSN and DOB of a youth s sibling (if one is noted on the form) against the SSNs of all previously enrolled youth. If no match is found, the RAS will ask for confirmation that the sibling information is accurate before proceeding to the next step. If a youth s sibling data match those of a previously enrolled youth, the applicant will be assigned to the same study group as the sibling. If no sibling is noted on the consent form, the SSN and DOB of the youth s parent/guardian will be compared to those of the parents/guardians of all previously enrolled youth. If a match is found, indicating that a sibling previously enrolled, the youth will be assigned to the same study group at the sibling. If a sibling was not identified by the RAS and was inadvertently enrolled into a different group than the first enrolled sibling, they will be assigned to the same program group. If the not-identified sibling was enrolled in ASPIRE services, the other sibling will also be 11.0 Random Assignment 7 of 15

8 eligible for those services. If this situation arises, the Site Coordinator will immediately report this to the RAS Help Desk and notify the ASPIRE Project Director. They will manually change the program group in both RAS and APL will manually change the assignment in AIMS. It is important to remember that for purposes of national evaluation, the sibling with the group assignment change will be treated as a non-research case. Siblings will be defined as eligible youth living in the same household. When youth live in different households, that sibling relationship will rely primarily on self-reporting. If it has been noted that a sibling had previously applied they will be treated as such. If it wasn t noted, they will not be treated as siblings. If a parent asks if a youth living in another household should be recorded as a sibling follow up questions to determine the level of interaction between the two youth living in separate households should be asked. Such questions could be as simply as do you have frequent interactions with this youth? Random Assignment of Youth to the ASPIRE Services or Usual Services Groups The RAS will automatically assign the youth to either the ASPIRE services group or the Usual Services group. After information collected during the informed consent process has been entered, the ASPIRE Staff click the Continue to Random Assignment button located at the bottom of the page. The RAS will then save the case and check that all required data have been entered without errors. The ASPIRE staff may also click the Save Draft button at any time if random assignment cannot be completed in one sitting. Save data frequently so they are not lost upon automatic log off after two hours. Saving data frequently will also signal to the RAS that the page is still active. If all required data have been entered and there are no identifiable data errors, the RAS will assign the youth to one of the two study groups: (1) the ASPIRE services group or (2) the Usual Services group (Screenshot 11.8) Random Assignment 8 of 15

9 Screenshot 11.8 Results Screen If a staff needs to put an applicant through random assignment and the RAS system is unavailable, contact the RAS Help Desk, as follows: By at PROMISEHelpDesk@mathematica-mpr.com By phone at (855) from 8:00 a.m. to 8:00 p.m., Eastern Time. Please leave a voice message with name and number and the call will be returned as soon as possible Please do not include any personal information about a specific youth (e.g., names, DOBSs, SSNs) in or voice messages Updating Youth Contact Information If a youth s address and/or phone number have changed, this information should be updated in the AIMS Recruitment, Enrollment & Case Management, Address and Phone Number instrument. It is not required to update information in the RAS. MPR may occasionally request the updated contact information for use in the national evaluation. Youth, parent/guardian, and sibling DOBs and SSNs cannot be revised by ASPIRE staff in the RAS, as they are key fields for validation checks. If this information needs to be revised during the enrollment period, contact the RAS Help Desk for assistance Running Reports in RAS The ASPIRE RAS will allow Site Coordinators and the ASPIRE Project Director to run a series of reports for their respective programs. Those with permissions will have a 11.0 Random Assignment 9 of 15

10 Reports link on the top menu bar. Click Reports to view the reports landing page. The reports landing page will have links to the following reports: Enrollment by Month, Enrollment by Region, and Enrollment by Study Group. Enrollment by Month The top section of this report will display basic information about ASPIRE such as: start date, end date, percentage of enrollment period elapsed, target enrollment, total number of enrollees to date, and percentage of target enrollment achieved. The bottom section will include a detailed table on actual enrollment by month to each of the study groups and in total. The current month will be displayed; however, the counts will not be final until the last business day of the month. Each month will be broken down into number of enrollees in the ASPIRE services group, number of enrollees in the usual services group, and total number of enrollees. Enrollment by Region The top section of this report will display basic information about ASPIRE such as: start date, end date, percentage of enrollment period elapsed, target enrollment, total number of enrollees to date, and percentage of target enrollment achieved. The bottom section will include a detailed table on actual enrollment by region to each of the study groups and in total. Each region will be broken down into number of enrollees in the ASPIRE services group, number of enrollees in the usual services group, and total enrollees in that region. Enrollment by Study Group The top section of this report will display basic information about ASPIRE such as: start date, end date, percentage of enrollment period elapsed, target enrollment, total number of enrollees to date, and percentage of target enrollment achieved. The bottom section will include a detailed table on actual enrollment by study group assignment, the ASPIRE services group and the usual services group. The report also provides the Study ID number and the associated study group at the individual level Checking for previous enrollment and verifying group assignment Before enrolling a youth or delivering ASPIRE services to a youth whose study group status is uncertain, enter the youth s SSN and DOB in the Find Youth screen to check for a previous random assignment or to view the individual s study group assignment Random Assignment 10 of 15

11 Program staff may also check AIMS Random Assignment Form instrument to confirm that a youth is in the program group before delivering ASPIRE services to him or her Verifying group assignment integrity As part of the early assessment reports for the process study, MPR will request extracts from AIMS of the number of youth who have been recorded as ASPIRE and/or Usual Services youth. MPR will compare these extracts to the reported number of youth assigned to the program and usual services group as recorded in the RAS. This check will ensure that youth who have been randomly assigned to a study group in the RAS are being recorded as belonging to the same study group in AIMS. The APL team will conduct similar checks at frequent intervals Random Assignment System (RAS) to ASPIRE Information Management System (AIMS) Record and Confirm Group Assignment Once the youth s placement has been made in RAS, staff must return to AIMS to record the group assignment. Staff will indicate into which group the youth was placed, on the Random Assignment Form in AIMS. This instrument also asks whether that placement was based upon a sibling s group placement, or the youth s alone. Staff will be required to electronically sign the Random Assignment Form, indicating that the assignment they have marked is accurate and consistent with RAS. Site Coordinators will monitor the completion of this form, and provide additional verification. Appendix C: AIMS Data Points recruitment and enrollment, rows describe fields documenting random assignment entry and confirmation in AIMS Confirming Group Assignment and Assigning Case Managers The Site Coordinator will confirm the youth s group assignment in AIMS Recruitment, Enrollment & Case Management, Random Assignment Form by comparing the youth s group designation with that in the RAS within two business days of the random assignment. If the assignment matches, the Site Coordinator will lock the Random Assignment Form. This measure ensures that the youth s group assignment will not be 11.0 Random Assignment 11 of 15

12 changed. Site Coordinator will then inform the staff who conducted the enrollment to make the notification phone call and mail the letter within three business days. The Site Coordinator will then inform the case manager of a new client assignment within two business days of notification of random assignment. Referring to the participant by SID number rather than their name to protect personally identifiable information, the Site Coordinator will notify the Case Manager by to document the assignment. The Site Coordinator will record the assignment of the case manager in AIMS as well. Record in AIMS Recruitment, Enrollment & Case Management, Intake By CM Of Aspire Services Group instrument After confirming the youth has been notified by phone and mail, the Case Manager will make initial contact with the youth and their parents/guardians within three business days of phone notification of group assignment, to set up a face-to-face meeting. A face-to-face meeting will occur within 30 days of random assignment. Timeline: Enrollment Meeting to first contact should take one week: 1. Enrollment Meeting 2. Within 2 business days, submit youth for random assignment in RAS. 3. Within 2 business days from RAS, Site Coordinator confirms random assignment and assigns youth to a Case Manager. 4. Within 3 business days of the Site Coordinator confirming group assignment in AIMS, the staff who conducted the enrollment notifies youth and parent of group assignment. 5. Within 3 business days from phone notification, Case Manager should make first contact with youth and/or the family. Adjusted Timeline (enrollment to first contact) for telephone Enrollment Meeting: 1. Mail enrollment packet to youth (1 week prior to scheduled telephone meeting). 2. Conduct telephone enrollment meeting, requesting that youth and parent/guardian sign documents and put them in paid envelope to mail back on the same day as the meeting. (The enrollment packet will contain 2 copies of each document. The family should sign and keep one copy for their records, and send the other copy to ASPIRE.) Inform the family that a follow up call will occur in 5 days days after enrollment meeting, call family if documents have not been received. 4. When documents are received, enter information into RAS. 5. See Enrollment Meeting to first contact for timeline following RAS. Case Managers should keep in mind that youth assigned to the ASPIRE Services group will remain in that group for the duration of the study, unless they choose to withdraw 11.0 Random Assignment 12 of 15

13 from the program. This includes youth whose status has changed while in the project, such as youth on SSI who transition to a non-cash status during the project. AIMS Data Points: If a youth does withdraw, rows are used to record withdrawal information. Before the data can be entered, the Site Coordinator will have to unlock the form Notify the Youth Participant of His or Her Group Assignment The Notification Process The ASPIRE staff completing the enrollment process will inform participants of their group assignment within 3 business days of random assignment being complete. Staff will notify participants of their group assignment personally (in-person or over the phone), as well as mailing the official group assignment letter generated via RAS. 1. Print Group Assignment Notification Letter Once random assignment has been successfully completed, the RAS will produce a letter notifying the applicant of the results. The RAS will be customized to the ASPIRE states programs. The system will populate the top portion of the letter with the applicant s name and address. MPR will also provide the programs with Spanish versions of generic (non-customized) random assignment letters. 2. Notifying by Mail All participates must receive a letter in the mail confirming their assignment in writing. After random assignment has been completed, take the RAS-generated study group assignment letter and mail it to the youth. A copy of the written assignment confirmation will be kept in the participant s ASPIRE file. 3. Notifying Over the Phone Follow the Talking Points informing of Random Assignment provided on the ASPIRE Staff website and inform the youth of their study group assignments, the services they are eligible to receive, and any next steps. Do not leave a message on the first attempt. Any messages that contain personally identifiable or sensitive information. The staff member who conducted the enrollment meeting should also be prepared to make the group assignment notification phone call Random Assignment 13 of 15

14 Appendix C: AIMS Data Points recruitment and enrollment, rows are used to document notification. Tips for Making the Notification Phone Call: Introduce yourself and remind the youth/parent your role with ASPIRE. Explain that random assignment has been completed, and they will be receiving a letter in the mail describing their group assignment. Let them know the youth s group assignment. If assigned to ASPIRE Services: o Explain that they have been assigned a Case Manager. o The Case Manager will contact them within 3 business days to set up their first meeting. If assigned to Usual Services: o Explain their continuing role with ASPIRE, and that they will be contacted periodically to completed surveys. o Remind them of the valuable resources and options available on the Resource List provided during the enrollment meeting. Ask them if they have any questions. Thank them again for participating. Tips for Dealing with an Upset Youth or Parent/Guardian: It is possible that youth and parent/guardians may be unhappy with their program assignment. Emphasize that this is a study and assignment to either group is beneficial for the youth as well as for improving future service delivery. Use the Talking Points with Your or Parent/Guardian upset with Random Assignment found on the ASPIRE Staff website, to address their concerns. Offer to resend the Resource List. If the individual is still upset after the script has been read to them, they may also be referred to the Site Coordinator. After notifying the youth and family, return to AIMS Enrollment Data Collection Form and record that notification has occurred Random Assignment 14 of 15

15 11.4 Case Management Begins The Site Coordinator will assign the Case Manager by recording the assignment in AIMS and ing the Study ID number to the Case Manager. The Case Manager will contact the youth and parent within three days of the youth and parent receiving notification of their group assignment, and schedule a face to face meeting to occur within 30 days of youth s enrollment in ASPIRE Summary of Steps from Initial Recruitment Mailing to Service Provision FROM TO TIMELINE Letters mailed Return response indicating interest Assigned for recruitment follow up Interest expressed by potential enrollee Enrollment meeting Recruitment to Enrollment Follow up calls to nonrespondents = Site Coordinator assigns staff to recruitment = Call to schedule enrollment meeting = Enrollment meeting is held = Enrollment to Random Assignment Random Assignment = After 1 week 2 business days 2 business days 2 weeks 2 business days Random Assignment to Notification and Service Provision Random Assignment Site Coordinator 2 business days confirms group = Random Assignment Site Coordinator assigns 2 business days Case Manager (if ASPIRE = Site Coordinator Notify by phone of 3 business days confirms group group assignment, and = Notification of group Case Manager makes 3 business days assignment initial contact with new = Initial phone contact First face-to-face 30 days from with new client meeting = Random 11.0 Random Assignment 15 of 15