Assessment Tool Benchmark Time Frame Person/Group Analysis Assessment Method Responsible

Transcription

1 Southern Illinois University Carbondale College of Applied Sciences and Arts School of Allied Health Radiation Therapy Program Assessment Plan 14 Analysis and Actions Based on the 2012/2013 Radiation Therapy Class MISSION: The mission of the Radiation Therapy Program through Southern Illinois University Carbondale is to provide a quality program integrating education, research and service in order to meet the needs of the profession and improve health care of the people and communities we serve. GOAL #1: Prepare the student to practice as a competent entry level professional Radiation Therapist by offering a balanced curriculum and quality didactic/clinical instruction. Outcomes Assessment Tool Benchmark Time Frame Person/Group Analysis Assessment Method for Responsible Evaluation 2012/2013 # Avg 8/17 1. Graduates indicate overall satisfaction with education from the program. Graduates will be surveyed approximately 6 months following graduation. Graduate Survey Graduates will report average score of 4.0 on a scale of 1-5 ( Excellent or Very Good ) on item #12 of graduate survey. Survey Monkey question #12 # Avg 9/ /2011 # Avg # Avg # Avg 2007/2008 # Avg Action Met benchmark. No action necessary. Sending several s to try and increase response rates. Survey renumbered so question is now #12. Survey renumbered 2. Employer indicates graduate was adequately prepared to perform as an entry-level radiation therapist. Employers will be surveyed approximately 6 months following graduation of student. Employer Survey The returned employer will reflect average score of 4 on a scale of 1-5 on ( Strongly Agree or Generally Agree ) response to item #IIH. Survey Monkey question #11 Post gradation 2012/2013 # Avg 5/7 #IIH 4.25 Avg 4/7 2010/2011 #IIH 4.80 Avg #IIH 5.0 Avg #IIH 4.89 Avg 2007/2008 #IIH 4.86 Avg Benchmark met. Continue to However need to increase return rates. 3. Graduates will pass ARRT Radiation Therapy certification exam on first attempt. Graduates will sit for ARRT certification exam following graduation. ARRT Report if available/graduate Feedback/ARRT Website A five year average of 80% of the graduates will pass the credentialing exam on first attempt Fall Most Recent Five year Average = 97.8% (90/92 graduates) 16/17 = 94.1% for 2012/2013 class 16/16 = 100% for class 17/17 = 100% for 2010/2011 class 19/20 = 95% for class 22/22 = 100% for class 17/18 = 94.4% for 2007/2008 class 100% for 2006/2007 class Benchmark met. Continue to 1

2 100% for 2005/2006 class 100% for 2004/2005 class 100% for 2003/2004 class 100% for 2002/2003 class 100% for 2001/2002 class 78.5% for 2000/2001 class 2012/2013 # Avg #4 4.6 Avg # Avg # Avg 5/7 #IA 4.50 Avg #IB 4.50 Avg #IIH 4.25 Avg #IIIK 4.25 Avg 4/7 2010/2011 #IA 4.40 Avg 4. Employers indicate graduate demonstrates essential skills and knowledge necessary to work effectively with other health care practitioners. Employers will be surveyed approximately 6 months following graduation of student. Employer Survey The returned employer will reflect average score of 4 on a scale of 1-5 on ( Strongly Agree or Generally Agree ) response to items #IA, #IB, #IIH, #IIIK. Survey Monkey questions 3, 4, 11, &16 #IB 4.40 Avg #IIH 4.80 Avg #IIIK 5.0 Avg 5/9 #IA 5.0 Avg #IB 4.83 Avg #IIH 5.0 Avg #IIIK 5.0 Avg 5/20 #IA 4.89 Avg #IB 4.89 Avg #IIH Avg #IIIK 4.89 Avg 9/ /2008 #IA 4.71 Avg #IB 4.71 Avg #IIH 4.86 Avg #IIIK 4.71 Avg Benchmark met. Continue to Noticed drop in data but believe it is skewed will low response rate however we are concerned. 2

3 7/ / /17 100% 16/16 100% 5. Students will be clinically competent upon graduation Students clinical competencies. Clinical Competency Check Lists Each students will complete 97% of the suggested competencies Summer 2010/ /17 100% 20/20 100% 22/22, 100% 2007/ /18, 100% 2006/ /14, 100% Met benchmark. Continue to Increased benchmark to 97% from 95% spring GOAL #2: Provide didactic and clinical experiences that lead to research in educational, professional, or health care issues relating to radiation therapy. Outcomes Assessment Method Assessment Tool Benchmark Time Frame for Evaluation Person/Group Responsible Analysis Action 1. Students demonstrate essential skills to plan and execute research on various topics. Students will be evaluated on six research assignments. Grades on the required research projects while at clinical rotations. Students will achieve an average score of 87.5% on assignments. Clinic Coordinator 2012/2013 HW1 = 98.5% HW2 = 97.3% HW3 = 97.9% HW4 = 97.9% HW5 = 96.3% QA = 90.9% HW6 = discontinue Average = 96.5% 17/17 students Total 100% scored at or above benchmark of 85%. HW1 = 97.9% HW2 = 98.0% HW3 = 98.6% HW4 = 96.9% HW5 = 95.6% QA = 90.6% HW6 = 95.4% Average = 96.1% Total 100% scored Met benchmark. Continue to Raised Benchmark to 87.5% from 85% HW6 decreased from the 2006/2007 year due to modifications. 3

4 at or above benchmark of 85%. 2010/2011 HW1 = 98.3% HW2 = 97.2% HW3 = 98.0% HW4 = 97.0% HW5 = 95.4% QA = 91.8% HW6 = 95.1% Average = 96.1% 17/17 students Total 100% scored at or above benchmark of 85%. HW1 = 99.2% HW2 = 98.4% HW3 = 98.2% HW4 = 96.3% HW5 = 96.3% QA = 97.8% HW6 = 95.9% Average = 97.4% 20/20 students Total 100% scored at or above benchmark of 85%. 22/22 students HW1 = 98.7% HW2 = 97.5% HW3 = 97.9% HW4 = 96.9% HW5 = 96.1% QA = 94.3% HW6 = 95.8% Average = 96.7% Total 100% scored at or above benchmark of 85%. 22/22 students 4

5 GOAL #3: Provide avenues to students for professional development and growth within the profession. Outcomes Assessment Method Assessment Tool Benchmark Time Frame for Evaluation Person/Group Responsible Analysis Action 1. Graduates will become members of professional organizations. 2. Graduates will maintain their CE activities to grow with the profession Graduates will be surveyed approximately 6 months following graduation. Graduate. Survey Monkey Question 9 Graduate. Survey Monkey Question 10 50% of the graduates join the ASRT within six months post graduation. 50% of graduates will participate in the required CE activities to maintain their ARRT Certification. 2012/ %-5/8 students indicated they are members of the ASRT. 67%-6/9 students indicated they are members of the ASRT. 9/16 returned 2010/ %-2/6 students indicated they are members of the ASRT. 6/17 returned 20.0%-1/5 students indicated they are members of the ASRT. 5/20 returned 10.0% - 1/10 students indicated they are members of the ASRT. 10/22 total were returned 62.5% 2007/ / % 5/8 students indicate they have started their CE requirements 38% 3/9 students indicate they have started I believe the professionalism project is helping with this even though many are not joining. need to increase awareness of this. I believe employment is also a factor. 2007/2008 increased from 29% in 2006/2007 class. still believe this is low due to employment numbers. I believe this low due to low employment numbers. Added Professionalism project in the 2007/2008 class. This increased 5

6 Graduates will be surveyed approximately 6 months following graduation. their CE requirements 2010/ % 3/6 students indicate they have started their CE requirements 20.0% 1/5 students indicate they have started their CE requirements 20.0% 2/10 students indicate they have started their CE requirements 2007/ % 3/8 students indicate they have started their CE requirements from 17.3% in the 2006/2007 class. 6

7 GOAL #4: Provide avenues for students to develop and apply skills in effective communication necessary for successful radiation therapy practice. Outcomes Assessment Method Assessment Tool Benchmark Time Frame for Evaluation 1. Students demonstrate effective communication skills. Growth Evaluations are performed during each clinical rotation. 7 th week Growth Evaluations Summer. Students will achieve an average score of 4.0 on a scale of 0-5 ( Acceptable to Excellent ) on item #I. Summer Person/Group Responsible Analysis 2012/2013 #I Avg 4.88 #I Avg /2011 #I Avg /17 students #I Avg /20 students #I Avg /22 students 2007/2008 #I Avg 4.72 Action Met benchmark. Continue to Changed to 7 th week growth eval from summer semester for the 2011 report. 2. Graduates demonstrate effective communication skills. Employers will be surveyed approximately 6 months following graduation of student. Employer Survey The returned employer will reflect average score of 4 on a scale of 1-5 on ( Strongly Agree or Generally Agree ) response to item #III I. Survey Monkey Question 13 and 14 # avg, # avg 5/7 #III I 3.75 avg 4/7 2010/2011 #III I 4.9 avg 5/9 #III I 4.67 avg 6/8 #III I 4.89 Avg 2007/2008 #III I 4.57 Avg 9/12 Did not meet Benchmark for. Will emphasize the importance of communication. One employer had concerns about phone use. 2010/2011 Increased 0.2 Decreased almost 0.2 7

8 GOAL #5: Provide avenues for students to develop and apply skills in critical thinking and problem-solving necessary for successful radiation therapy practice. (Split the original goal 4 into goals 4 and 5 to meet accreditation requirements 2013). Outcomes Assessment Method Assessment Tool Benchmark Time Frame for Evaluation Person/Group Responsible Analysis RAD %Avg RAD 370 no data, computer crash RAD %Avg RAD 390 no data, computer crash RAD %Avg 18/18 students Action RAD %Avg RAD %Avg RAD %Avg RAD %Avg RAD %Avg Benchmark met. Continue to 1. Students demonstrate effective problem solving and critical thinking skills. For all fall semester courses a Comprehensive Final Exam must be completed to move on to the spring semester. Final Exam in RAD 360, 370, 380, 390, and 400 Students will achieve an average score of 80% out of 100%. Fall, Clinic Coordinator 2010/2011 RAD %Avg RAD %Avg RAD %Avg RAD %Avg RAD %Avg 18/18 students A couple scores were lower this year but the class did struggle with concepts some. Every score increased in comparison to the year before. RAD %Avg RAD %Avg RAD %Avg RAD %Avg RAD %Avg 20/20 students RAD %Avg RAD %Avg RAD %Avg RAD %Avg RAD %Avg 24/24 students 8

9 2. Graduates demonstrate effective problem solving and critical thinking skills. Employers will be surveyed approximately 6 months following graduation of student. Employer Survey The returned employer will reflect average score of 4 on a scale of 1-5 on ( Strongly Agree or Generally Agree ) response to item #IF. Survey Monkey Question /2013 #8 4.4 Avg 5/7 #1F 3.75 Avg 4/7 2010/2011 #1F 4.5 Avg 5/9 #1F 4.67 Avg 6/8 #IF 4.56 Avg 2007/2008 #IF 4.57 Avg 9/12 did not meet benchmark met. Discuss with advisory board. However believe results are skewed due to low return rate. 9

10 Goal #6: Provide a clinical and didactic environment which leads to the development of clinical skills and competence appropriate to an entry level radiation therapist. Outcomes Assessment Method Assessment Tool Benchmark Time Frame for Evaluation Person/Group Responsible Analysis Action 1. Students will demonstrate simulation laboratory competence Simulation Laboratory Testing at various times during fall semester Simulation Laboratory Testing Student averages will not be less than 80% for all of the labs during the semester Fall Clinical Coordinator 2012/2013 Lab down; not available for testing, but was used for learning exercises, positioning, and immobilization. (new lab installed fall 13) Lab down; not available for testing, but was used for learning exercises, QA, positioning, and immobilization. 2010/2011 Lab down; not available for testing, but was used for learning exercises, QA, etc Lab down; not available for testing, but was used for learning exercises, QA, etc Brain 94.3% Spine 0%- LAB DOWN Pituitary 95.2%, Pelvis/Breast 95.0% Total 94.8% for average of three laboratory tests 2007/2008 Total 94.1% average for the four laboratory tests Final Bid has been accepted for the new Acuity simulator. Install should start summer /2011 PO is in process to purchase a 2006 Acuity Simulator. Needing to purchase new sim. Currently getting quotes. 2008/209 Met benchmark. Continue to Each increased or remained at 100% 10

11 2. Students will be clinically competent upon graduation Students clinical competencies. Clinical Competency Evaluations Each students will complete 97% of the suggested competencies Summer 2012/ /17 students 100% 100% 2010/ /17 students 100% 20/20 students 100% 22/22 students 100% Met benchmark. Continue to Increased benchmark to 97% from 95% for cycle. 3. Students demonstrate continual improvement of skills as they progress through the program. Growth Evaluations are performed during each clinical rotation. 7 th week Summer Growth Evaluations. Students will achieve an average score of 4.0 on a scale of 0-5 ( Acceptable to Excellent ) on item # II. 2012/2013 #II Avg 4.94 #II Avg /2011 #II Avg /17 students #II Avg /20 students #II Avg /22 students Met benchmark. Continue to 4. Students demonstrate professional development and growth as a result of didactic and clinical experiences. Growth Evaluations are performed during each clinical rotation. 7 th week Summer Growth Evaluations. Students will achieve an average score of 40 on a scale of 0-50 ( Acceptable to Excellent ) on the evaluation overall. 2007/2008 #II Avg 4.53 Growth Evaluation 2012/2013 Avg Growth Evaluation Avg Growth Evaluation 2010/2011 Avg /17 students Avg /20 students Avg /22 students 2007/2008 Avg Met benchmark. Continue to 11

12 5. Employer indicates graduate was adequately prepared to perform as an entry-level radiation therapist. Employers will be surveyed approximately 6 months following graduation of student. Employer Survey The returned employer will reflect average score of 4 on a scale of 1-5 on ( Strongly Agree or Generally Agree ) response to item #IIH. Survey Monkey question /2013 #11 Avg 4.6 5/7 #IIH Avg /7 2010/2011 #IIH Avg 4.8 5/9 #IIH Avg /8 #IIH 4.89 Avg 9/12 Benchmark met. Continue to 12

13 PROGRAM EFFECTIVENESS OUTCOMES Outcomes Assessment Tool Benchmark Time Frame for Evaluation Person/Group Responsible Analysis Action Most Recent Five year Average = 97.8% (90/92 graduates) 16/17 = 94.1% for 2012/2013 class 16/16 = 100% for class 1. Graduates will pass ARRT Radiation Therapy certification exam on first attempt. 2. Graduates will have employment within the radiation therapy profession post graduation. ARRT Report if available/graduate Feedback Graduate Survey. A five year average of 80% of the graduates will pass the credentialing exam on first attempt 80% of the graduates will have employment within six months post graduation. Five year average. Fall Approximately 6 months post graduation. 17/17 = 100% for 2010/2011 class 19/20 = 95% for class 22/22 = 100% for class 17/18 = 94.4% for 2007/2008 class 100% for 2006/2007 class 100% for 2005/2006 class 100% for 2004/2005 class 100% for 2003/2004 class 100% for 2002/2003 class 100% for 2001/2002 class 78.5% for 2000/2001 class Five Year Average = 84.18% 2012/2013 9/9 = 100% 4 unknown, 4 went on to further education, 9/9 = 100% 2 unknown, 4 went on to further education, 1 working x-ray(not seeking as going to grad school) 2010/2011 9/10 = 90% 1 unknown and 7 went on to further education 10/16 = 62.5% 6 not employed and 4 went on to further education 13/19 = 68.4% 6 not employed, 3 went on to further education 2007/ /14 Students 85.7% 4 graduates went to dosimetry school Met Benchmark. Continue to Stress the importance of taking the test immediately upon graduation Summer 2012 JRCERT indicated this statistic is to take into account graduates actively seeking employment. This is why we are at 100% for the year. - Continue to monitor as employment rate is increasing. Not taking program capacity. 2010/2011 lowered enrollment by 10% for this year. Currently 17 enrolled. For kept 10% reduction. - Should consider lowering the 13

14 2 unemployed (unsure on the two as we no longer have contact info) 2006/2007 9/10 Students 90% 2005/ /18 Students 100% One student went to dosimetry school number accepted into the program. Will talk to school director about this. 3. Students who start the program will complete the program. Program Completion Rate/Student Retention Rate. 80% of the students starting the program will graduate from the program. Summer 2012/ /18 students 94.4% 16/18 students 88.8% 2010/ /18 students 94.4% 20/20 students 100% 22/24 Students 91.6% 2007/ /20 Students 90% 2006/ /16 Students 87.5% 2005/ /20 Students 90% Met benchmark. Continue to 4. Graduates will evaluate the program positively. Graduate survey. The returned graduate will reflect average score of 4 on a scale of 1-5 on ( exceptional or adequate ) response on item #11. Survey Monkey Question # /2013 # average score on 8/17 sent out # average score on 9/16 sent out 2010/2011 # average score on 6/17 sent out # average score on 5/20 sent out # average score on 10/22 sent out 2007/2008 # average score of on 8/18 sent out Met benchmark. Continue to Renumbered survey for year Renumbered survey for year 5. Employers will evaluate the program positively. Employer survey. The returned employer will reflect average score of 4 on a scale of 1-5 on ( Excellent or Very Good ) response to item #IV. Survey Monkey Question # /2013 # avg 5/7 sent out #IV 4.25 avg 4/7 sent out 2010/2011 #IV 5.00 avg 5/9 sent out #IV 5.00 avg 6/8 sent out #IV 4.78 avg 2007/2008 #IV 4.57 avg 9/12 sent out Met benchmark. Continue to Believe drop in satisfaction is skewed due to low response rate. Concerned about drop in score. 14

15 6. Employers will evaluate the program positively. Employer survey. The returned employer will have an 80% yes response for the following question: If given the opportunity would you hire a graduate of this program again? Yes or No Survey Monkey Question # / Yes 0 No 5/7 sent out 4 Yes 0 No 4/7 sent out 2010/ Yes 0 No 5/9 sent out 6 Yes 0 No 6/8 sent out 8 Yes 0 No One did not answer 2007/ Yes 0 No One did not answer 9/12 sent out Met benchmark. Continue to 15