Frame for Evaluation. Spring Semester

Transcription

1 Southern Illinois University Carbondale College of Applied Sciences and Arts School of Allied Health Medical Dosimetry Program Assessment Plan Spring 2017 Analysis and Actions MISSION: The mission of the Medical Dosimetry Program through Southern Illinois University Carbondale is to provide a quality program integrating education, research and service in order to meet the needs of the profession and improve the delivery of health care for the people and communities we serve. GOAL #1: Prepare the student to practice as a competent entry level professional Medical Dosimetrist by offering a comprehensive curriculum and quality didactic/clinical instruction. Outcomes Benchmark Analysis Action 1. Graduates indicate overall satisfaction with education from the program. 2. Employer indicates graduate was adequately prepared to perform as an entry-level dosimetrist. Assessment Method Graduates will be surveyed approximately 6 months following graduation. Employers will be surveyed approximately 6 months following graduation of student. Assessment Tool Graduate Survey Employer Survey Graduates will report average score of 4.0 on a scale of 1-5 ( Excellent or Very Good ) on item #10 of graduate survey. Q11 on Survey Monkey The returned employer surveys will reflect average score of 4 on a scale of 1-5 on ( Strongly Agree or Generally Agree ) response to item #IIH. Time Frame for Spring Spring Person/Group Responsible Program Director Program Director 2006/2007 #10 Avg /3 Surveys 2007/2008 #10 Avg 5.0 7/7 Surveys 2008/2009 #10 Avg / /2010 #10 Avg / /2011 #10 Avg 5.0 6/15 surveys 2011/2012 #11 Avg 4.5 8/ /2013 #11 Avg / /2014 #11 Avg / /2015 #11 Avg / /2016 #11 Avg / /2007 #IIH Avg 5.0 2/3 surveys 2007/2008 #IIH Avg 4.8 5/7 surveys 2008/2009 #IIH Avg 5.0 5/10 surveys 2009/2010 #IIH Avg 4.62/5 6/12 Met benchmark. No action necessary. Became #11 for Spring 13 plan as we separated two questions. Benchmark met. Continue to monitor Will continue to send multiple s to help garner more response. 1

2 Q11 on Survey Monkey 2010/2011 #IIH Avg /10 surveys 2011/2012 #IIH Avg 5.0 4/10 surveys 2012/2013 Q11 Avg /9 surveys 2013/2014 Q11 Avg 5.0 4/10 surveys 2014/2015 Q11 Avg /13 surveys 2015/2016 Q11 Avg 5.0 4/11 surveys 2001/2002 1/1 Students BJH 2002/2003 2/2 Students BJH 3. Graduates will pass MDCB certification exam on first attempt. Graduates will sit for MDCB certification exam the year following graduation. MDCB Report if available/graduate Feedback A five year average of 80% of the graduates will pass the credentialing exam on first attempt Fall Program Director 2003/2004 2/2 Students BJH 2004/2005 1/1 Students BJH 2005/ Students SIUC 2006/2007 3/3 Students SIUC 2007/2008 7/7 StudentsSIUC and the 4 th graduate from 2006/2007 took the exam and passed her first attempt. 2008/2009 9/10 Students 90%SIUC 2009/2010 9/11 Students 82%, two have not tested. 2010/ /14 93%, one did not test. Results from March 2012 exam. 2011/ /13 SIUC 2012/ / % SIUC 2011-Concerned about students waiting to take the exam. Will encourage the importance of not waiting Met Benchmark. Continue to monitor Reviewing the RAD 555 exam as there was a correlation with those who failed. 2

3 2013/ /17 SIUC, one did not test 2014/ /14 SIUC, Five did not test for personal reasons and application problems. 2015/ /20 85% SIUC. 2006/2007 #IA Avg 4.5 #IB Avg 4.5, #IIH Avg 5 #IIIK Avg 5, 2/3 surveys 2007/2008 #IA Avg 4.6 #IB Avg 4.8, #IIH Avg 4.8 #IIIK Avg 4.4, 5/7 surveys 4. Employers indicate graduate demonstrates essential skills and knowledge necessary to work effectively with other health care practitioners. Employers will be surveyed approximately 6 months following graduation of student. Employer Survey The returned employer surveys will reflect average score of 4 on a scale of 1-5 on ( Strongly Agree or Generally Agree ) response to items #IA, #IB, #IIH, #IIIK. Survey Monkey Question 3, 4, 11, and 16 Spring Program Director 2008/2009 #IA Avg 5.0 #IB Avg 5.0, #IIH Avg 5.0 #IIIK Avg 5.0, 5/10 surveys 2009/2010 #IA Avg 4.83 #IB Avg 4.83 #IIH Avg 4.67 #IIIK Avg 4.83, 6/12 surveys 2010/2011 #IA Avg 4.86 #IB Avg 4.86 #IIH Avg 4.71 #IIIK Avg 4.71, 7/10 surveys 2011/2012 #IA Avg 5.0 #IB Avg 5.0 #IIH Avg 5.0 #IIIK Avg 5.0, 4/10 surveys 2012/2013 Benchmark met. Continue to monitor. 3

4 #3 Avg 4.8 #4 Avg 4.8 #11 Avg 4.75 #16 Avg, 5 5/9 surveys 2013/2014 #3 Avg 4.75 #4 Avg 4.75 #11 Avg 5.0 #16 Avg, 5.0 4/10 surveys 2014/2015 #3 Avg 5.0 #4 Avg 5.0 #11 Avg 4.88 #16 Avg 5.0 8/13 surveys 2015/2016 #3 Avg 5.0 #4 Avg 4.75 #11 Avg 5.0 #16 Avg 5.0 4/11 surveys 5. Students will be clinically competent upon graduation Students clinical competencies. Clinical Competency Check Lists Each students will complete 97% of the suggested competencies Summer Program Director 2005/2006 5/5 students 2006/2007 3/3 students 2007/2008 7/7 students 2008/ /10 students 2009/ /12 students 2010/ /15 students 2011/ /13 students 2012/ /16 Met benchmark. Continue to monitor. Increased benchmark to 97% Requiring cohort to participate in AAMD plan challenge. Add interview comp for 2017/2018 4

5 students 2013/ /18 students 2014/ /19 students 2015/ /20 students 5

6 GOAL #2: Provide didactic and clinical experiences that lead to research in educational, professional, or health care issues relating to medical dosimetry. Outcomes 1. Students demonstrate essential skills to plan and execute a research project. Assessment Method Students will be evaluated on two research assignments. Assessment Tool Benchmark Time Frame for Grades on the two required research papers while at external clinical rotations. Students will achieve an overall average score of 87.5% on the two research papers. Summer Person/Group Responsible Analysis 2006/2007 scored at or above benchmark of 85%.3/3 students 2007/2008 scored above 85% (avg on both 97.9%) 2008/2009 scored above 85% (avg on both 98.5%) 2009/2010 scored above 85% (avg on both 99.2%) 2010/2011 scored above 85% (avg on both 97.1%) 2011/2012 scored above 85% (avg on both 95.0%) 2012/2013 scored above 85% (avg on both 98.2%) 2013/2014 (overall avg score was 96.67%) 2014/2015 (overall avg score was 97.21%.) 2015/2016 (overall avg score was 93.9%.) Action Raised benchmark to 87.5% for 2013 plan Continue to encourage students to submit to/present with the professional organizations. 6

7 GOAL #3: Provide avenues to students for professional development and growth within the profession. Outcomes Assessment Method Assessment Tool Benchmark Time Frame for Person/Group Responsible Analysis Action 1. Students will gain a strong understanding of the professional requirements within the medical dosimetry field. 2. Graduates will participate in CE activities for certification. Students will complete the professional development paper and presentation. Graduates will be surveyed approximately 6 months following graduation. Completion of paper and presentation assignments. Graduate surveys. of students will proficiently complete these assignments. 25% of the graduates will start earning CE credits. (Note, the exam is typically completed in the January/February after graduation.) Spring Spring, Approximately 6 months post graduation Added professionalism and critical thinking to these exercises in Completed again in 2011, 2012, 2013, 2014, 2015, completed this assignment. 2014/ % indicate participation in CE activities. 4/ / % indicate participation in CE activities. 2/11 Met Benchmark. Continue Encourage students to start CE activities as soon as possible. Students just tested in January/February and this survey is done in April.- considering removing this item as CE are not counted until the following year. 7

8 GOAL #4: Provide avenues for students to develop and apply skills in effective communication necessary for successful medical dosimetry practice. Outcomes 3. Students demonstrate effective communication skills. 4. Graduates demonstrate effective communication skills. Assessment Method Growth s are performed during each clinical rotation. Employers will be surveyed approximately 6 months following graduation of student. Assessment Tool Benchmark Time Frame for Final Growth Employer Survey Students will achieve an average score of 4.0 on a scale of 0-5 ( Acceptable to Excellent ) on item #I. The returned employer surveys will reflect average score of 4 on a scale of 1-5 on ( Strongly Agree or Generally Agree ) response to item #III I. Survey Monkey questions 13 (verbal) & 14 (written). Summer Spring Person/Group Responsible Analysis 2006/2007 #I Avg 5.0, 3/3 students 2007/2008 #I Avg 4.79, 7/7 students 2008/2009 #I Avg 4.7, 10/10 students 2009/2010 #I Avg 5 12/12 students 2010/2011 #I Avg /15 students 2011/2012 #I Avg /13 students 2012/2013 #1 Avg /16 students 2013/2014 #1 Avg /18 students 2014/2015 #1 Avg /19 students 2015/2016 #1 Avg /20 students 2006/2007 #III I Avg 4.5, 2/3 surveys 2007/2008 #III I Avg 4.2, 5/7 surveys 2008/2009 #III I Avg 4.6, 5/10 surveys 2009/2010 #III I Avg 4.67, 6/12 surveys 2010/2011 #III I Avg 4.57, 7/10 surveys 2011/2012 #III I Avg 4.75, 4/10 surveys 2012/2013 #13 Avg 4.75, #14 Avg /9 surveys 2013/2014 #13 Avg 5.0, #14 Avg /10 surveys Action Benchmark met. Continue 8

9 2014/2015 #13 Avg 5.0, #14 Avg /13 surveys 2015/2016 #13 Avg 5.0, #14 Avg 5.0 4/11 surveys 9

10 GOAL #5: Provide avenues for students to develop and apply skills in critical thinking and problem-solving necessary for successful medical dosimetry practice. (Split the original goal 4 into goals 4 and 5 to meet accreditation requirements Spring 2013). Outcomes 1. Students demonstrate effective problem solving and critical thinking skills. 2. Graduates demonstrate effective problem solving and critical thinking skills. Assessment Method A passing score must be achieved on the Comprehensive Final Exam in RAD 555 to successfully complete the program. Employers will be surveyed approximately 6 months following graduation of student. Assessment Tool Benchmark Time Frame for Final Exam in RAD 555 (Previously the 570B Course) Employer Survey Students will achieve an average score of 80% out of. The returned employer surveys will reflect average score of 4 on a scale of 1-5 on ( Strongly Agree or Generally Agree ) response to item #IF. Survey Monkey Question 8 Summer Spring Person/Group Responsible Analysis 2006/2007 Avg 86%, 3/3 students 2007/2008 Avg 93.8%, 7/7 students 2008/2009 Avg /10 students 2009/2010 Avg 90.8% 12/12 students 2010/2011 Avg 90.5% 15/15 students 2011/2012 Avg 91.8% 13/13 students 2012/2013 Avg 89.2% 16/16 students 2013/2014 Avg 89.8% 18/18 students 2014/2015 Avg 92.28% 19/19 students 2015/2016 Avg 87.48% 20/20 students 2006/2007 #IF Avg 5.0, 2/3 surveys 2007/2008 #IF Avg 4.8,5/7 surveys 2008/2009 #IF Avg 4.6, 5/10 surveys 2009/2010 #1F Avg 4.83,6/12 surveys 2010/2011 #1F Avg 4.71,7/10 surveys 2011/2012 #1F Avg 4.50, 4/10 surveys 2012/2013 #8 Avg 4.8, 5/9 surveys 2013/2014 #8 Avg 4.75, 4/10 surveys 2014/2015 #8 Avg 4.88, 8/13 surveys 2015/2016 #8 Avg 5.0, Action Benchmark met. Continue Benchmark met. Continue Added research paper and presentation on the topic of critical thinking. This has appeared to help Research paper requires the student to list ways to improve critical thinking. Presentation and discuss is also required. 10

11 4/11 surveys 11

12 GOAL #6: Provide a clinical and didactic environment which leads to the development of clinical skills and competence appropriate to an entry level medical dosimetrist. Outcomes Assessment Method Assessment Tool Benchmark Time Frame for Person/Group Responsible Analysis Action 2006/2007 3/3 students 2007/2008 7/7 students 2008/ /10 students 2009/ /12 students 1. Students will be clinically competent upon graduation Students clinical competencies. Clinical Competency s Each students will complete 97% of the suggested competencies Summer 2010/ /15 students 2011/ /13 students 2012/ /16 students 2013/ /18 students 2014/ /19 students 2015/ /20 students Increased benchmark to 97% for spring 2013 plan. 2. Students demonstrate continual improvement of skills as they progress through the program. Growth s are performed during each clinical rotation. Final Growth Students will achieve an average score of 4.0 on a scale of 0-5 ( Acceptable to Excellent ) on item # II. Summer 2006/2007 #II Avg 5.0 3/3 students 2007/2008 #II Avg 4.86, 7/7 students 2008/2009 #II Avg 4.7, 10/10 students 2009/2010 #II Avg 5 12/12 students 2010/2011 #II Avg /15 students 2011/2012 #II Avg /13 students 2012/2013 #II Avg /16 students 2013/2014 #II Avg /18 students 2014/2015 #II Avg /19 students 2015/2016 #II Avg /20 students Updated Growth based on CI feedback to include more specific feedback measures in 2016 for the 2016/2017 cohort. 12

13 3. Students demonstrate professional development and growth as a result of didactic and clinical experiences. 4. Employer indicates graduate was adequately prepared to perform as an entrylevel dosimetrist. Growth s are performed during each clinical rotation. Employers will be surveyed approximately 6 months following graduation of student. Final Growth Employer Survey Students will achieve an average score of 40 on a scale of 0-50 ( Acceptable to Excellent ) on the evaluation overall. The returned employer surveys will reflect average score of 4 on a scale of 1-5 on ( Strongly Agree or Generally Agree ) response to item #IIH. Survey Monkey Question 11 Summer Spring 2006/2007 Growth Avg /3 students 2007/2008 Growth Avg /7 students 2008/2009 Growth Avg 48.4, 10/10 students 2009/2010 Growth Avg 50, 12/12 students 2010/2011 Growth Avg 49.43, 15/15 students 2011/2012 Growth Avg 49.92, 13/13 students 2012/2013 Growth Avg 49.69, 16/16 students 2013/2014 Growth Avg 49.83, 18/18 students 2014/2015 Growth Avg 49.58, 19/19 students 2015/2016 Growth Avg 49.95, 20/20 students 2006/2007 #IIH Avg 5.0 2/3 surveys 2007/2008 #IIH Avg 4.8 5/7 surveys 2008/2009 #IIH Avg 5.0, 5/10 surveys 2009/2010 #IIH Avg 4.67,6/12 surveys 2010/2011 #IIH Avg 4.71,7/10 surveys 2011/2012 #IIH Avg 5.0,4/10 surveys 2012/2013 #11 Avg 4.75, 5/9 surveys 2013/2014 #11 Avg 5.0, 4/10 surveys 2014/2015 #11 Avg 4.88, Updated Growth based on CI feedback to include more specific feedback measures in 2016 for the 2016/2017 cohort. Benchmark met. Continue 13

14 8/13 surveys 2015/2016 #11 Avg 5.0, 4/11 surveys 14

15 PROGRAM EFFECTIVENESS OUTCOMES Outcomes Assessment Tool Benchmark Time Frame for Person/Group Responsible Analysis Action 2001/2002 1/1 Students BJH 2002/2003 2/2 Students BJH 2003/2004 2/2 Students BJH 2004/2005 1/1 Students BJH 2005/2006 5/5 Students SIUC 1. Graduates will pass MDCB certification exam on first attempt. MDCB Report if available/graduate Feedback A five year average of 80% of the graduates will pass the credentialing exam on first attempt Fall 2006/2007 3/3 Students SIUC 2007/2008 7/7 Students SIUC 2008/2009 9/10 Students 90%SIUC 2009/2010 9/11 Students 82% SIUC (two did not test) 2010/ /14 Students 93% SIUC (one did not test) 2011/ /13 Students 2012/ /16 Students 93.8% 2013/ /17 Students (one did not test) 2014/ /14 Students (five did not test) 2015/ /20 Students 85% Met Benchmark. Continue Concerned that the MDCB report does not break out by class. Also concerned about students not testing when eligible one did not test due to application error. Five year average of 95.0% (76/80) 2. Graduates will have employment within the medical dosimetry profession post graduation. End of program paperwork 50% of the graduates will have employment at time of graduation. Summer 2006/2007 of students had employment at time of graduation. 3/3 students 2007/2008 of students had employment at time of graduation. 7/7 students 2008/ % of students had employment at time of graduation 2009/ % of students had employment at time of graduation 2010/ /15 73% of students had employment at time of graduation 2011/2012 5/ % of students had employment at time of graduation 2012/2013 9/ % of students had 2012 and Did not meet benchmark. Will start discussing the need to look for employment earlier in program. Continue Starting to see more job advertisements but we are tracking 6 months and 12 month dates and meeting benchmarks Still have many job 15

16 Graduate Survey. 80% of the graduates will have employment within six months post graduation. Five year average. Approximately 6 months post graduation. Spring employment at time of graduation 2013/2014 7/ % of students had employment at time of graduation 2014/2015 7/ % of students had employment at time of graduation, 8 did not respond. 2015/2016 7/14 50% of students had employment at time of graduation, 6 did not respond. 2001/2002 1/1 Students 2002/2003 2/2 Students 2003/2004 2/2 Students 2004/2005 1/1 Students 2005/2006 5/5 Students 2006/2007 3/3 Students 2007/2008 7/7 Students 2008/ /10 Students 2009/ /12 Students 2010/ /15 Students 2011/ /13 Students 92.3% 2012/ /16 Students 2013/ /18 Students 2014/ /13 Students (13 responded but sure other 6 do too) 2015/ /11 Students (11 of 20 responded.) advertisements coming in weekly Modified survey question for 2015/2016 cohort. Not yet completed. 3. Students who start the program will complete the program. Graduate Survey Program Completion Rate/Student Retention Rate. 85% of the graduates will have employment within twelve months post graduation. Five year average. 80% of the students starting the program will graduate from the program. Five year average. Approximately 12 months postgraduation. Fall semester. Summer Overall 70/71 Students for Five year average of 98.5%. 2012/2013 class surveyed in /16 of students 2013/2014 class surveyed in /18 of students 2014/2015 class surveyed in /11 responded. 2015/2016 class surveyed in August of 20 responded, 7 did not respond. 2001/2002 1/1 Students 2002/2003 2/2 Students 2003/2004 2/2 Students 2004/2005 1/1 Students 2005/2006 5/5 Students 2006/2007 3/4 Students 75% 2007/2008 7/7 Students 2008/ /10 Students 2009/ /12 Students 2010/ /16 Students 93.8% 2011/ /14 Students 92.9% 2012/ /17 Students 94.1% Modified survey question for 2015/2016 cohort. Met benchmark. One student removed for poor performance in 2011/2012. One student left for personal reasons 2012/2013. One student left for personal reasons 16

17 2013/ /18 Students 2014/ /19 Students 2015/ /22 Students 90.1% 2013/ /2015- one student dropped day one and does not count as not in official 10 day count. 4. Graduates will evaluate the program positively. Graduate survey. The returned graduate surveys will reflect average score of 4 on a scale of 1-5 on ( exceptional or adequate ) response on item #10. Survey Monkey #11 for 2016 and beyond. Spring 2006/2007 #10 average score of 4.67 on 3/3 surveys sent out 2007/2008 #10 average score of 5 on 7/7 surveys sent out 2008/2009 #10 average score of 4.7/5 on 10/10 surveys sent out 2009/2010 #10 average score of 4.86/5 on 7/12 surveys sent out 2010/2011 #10 average score of 5/5 on 6/15 surveys sent out 2011/2012 #11 average score of 4.5/5 on 8/13 surveys sent out 2012/2013 #12 average score of 4.82 on 11/16 surveys sent out 2013/2014 #8 average score of 4.71 on 14/18 surveys sent out 2014/2015 #11 average score of 4.60 on 10/19 surveys sent out 2015/2016 #11 average score of 4.82 on 11/20 surveys sent out 2015/2016 -one student dropped in middle of fall semester and one did not complete the program. 5. Employers will evaluate the program positively. Employer survey. The returned employer surveys will reflect average score of 4 on a scale of 1-5 on ( Excellent or Very Good ) response to item #IV. Survey Monkey question #20 Spring 2006/2007 #IV average score of 4.5 on 2/3 surveys sent out 2007/2008 #IV average score of 4.6 on 5/7 surveys sent out 2008/2009 #IV average score of 4.6 on 5/10 surveys sent out 2009/2010 #IV average score of 4.67 on 6/12 surveys sent out 2010/2011 #IV average score of 4.57 on 7/10 surveys sent out 2011/2012 #IV average score of 5.00 on 4/10 surveys sent out 2012/2013 #20 average score of 4.75 on 17

18 5/9 surveys sent out 2013/2014 #20 average score of 5.0 on 4/10 surveys sent out 2014/2015 #20 average score of 5.0 on 8/13 surveys sent out 2015/2016 #20 average score of 4.75 on 4/11 surveys sent out 2006/2007 Yes 2 No 6. Employers will evaluate the program positively. Employer survey. The returned employer surveys will have an 80% yes response for the following question: If given the opportunity would you hire a graduate of this program again? Yes or No Survey Monkey question #25 Spring 2/3 surveys sent out 2007/2008 Yes 5 No 5/7 surveys sent out 2008/2009 Yes 5 No 5/10 surveys sent out 2009/2010 Yes 5 No 0 6/12 surveys sent out (one left blank but gave high marks) 2010/2011 Yes 6 No 0 7/10 surveys sent out (one left blank but gave high marks) 2011/2012 Yes 4 No 0 4/10 surveys sent out 2012/2013 Yes 4 No 0 5/9 surveys sent out (one skipped) 2013/2014 Yes 3 No 0 4/10 surveys sent out 2014/2015 Yes 8 No 0 8/13 surveys sent out 2015/2016 Yes 4 No 0 4/11 surveys sent out 18