Career Program Outcomes Assessment Report Prepared by the MCC Career Education Department, October 2014
Introduction The Missouri Department of Elementary and Secondary Education requires that completers of career programs take a Technical Skills Assessment, wherever such an assessment is available and appropriate. The assessment should be industry-recognized. Many, but not all, of the MCC career programs have a TSA. Beginning with the 2011-12 school year, all programs are also expected to assess at least one program outcome, report the results, and identify any needed interventions or follow-up actions. The information summarized here was collected as part of the program review process and refers to assessment activities conducted during the 2013-14 school year. Apparel and Textiles APTX is evaluated through the APTX 275 Portfolio class. In this class, a panel of judges who work in various facets of the industry interview graduating students and evaluate their portfolios. The rubric and procedure has been approved by DESE as a TSA. There was one program completer in spring 2013, and she passed. No actions are currently planned based on this result. Automotive Automotive TSA results (NA3SA exams) for MCC-Longview Automotive are in the 80th percentile nationwide with a 96% pass rate (178 passed out of 185 taken). Twenty-two of the twenty-three students who took exams passed over 65% of them. Only four failed even one of the exams. There is a suggestion to do pre and post-testing, however that would increase the cost from approximately $700 to $4,000. The program is in the process of looking at the lowest scoring area (Steering and Suspension) and plans to determine action steps to improve this area. The faculty is also looking at individual task areas to increase student learning. In addition, the AUTO program assessed the following program outcome: Students will demonstrate the knowledge of thorough application of safety rules and regulations. The written exam used to assess this outcome is an online product through CCAR, and it is an industry standard. The name of the product is SP2 (Safety and Pollution Prevention). One hundred and forty seven students took the assessment during Fall 2013 and Spring 2014 (the certificate is good for one year). One hundred percent passed the assessment. All instructors will continue to cover safety as it relates to the specific area that is being learned in the class and laboratory for that specific course section. Business The business program uses A*S*K Business Fundamentals as the TSA for Management, and National Occupational Competency Testing Institute (NOCTI) products for student TSAs in the other areas: Advanced Accounting for the Accounting emphasis, and administrative assisting for the office management emphasis. Fifteen of the seventeen (88%) taking the A*S*K exam passed. Nine of twelve
(75%) passed the accounting exam. Both students taking the administrative assisting exam passed, but the program is being discontinued for other reasons. For 2013-14, the business program participated in the general education assessment project instead of assessing a CTE program outcome. They identified discipline outcomes for the transfer program, assessed one of them, and then also assessed critical thinking using an accounting cycle project. Child Growth and Development Students exiting the program are required to take the NOCTI (National Occupation Competency Testing Institute) test for Early Childhood Education and Care. The overall scores and scores from all areas but one were below the national average for spring 2014. Eight out of the 11 students passed the exam. Since that time, a new program coordinator has been hired who will also serve as faculty. Revision of the curriculum is being planned. Computer Science - Cisco The Cisco Exam serves as the TSA, and is required for all students taking CSIS 290 as their capstone class. The program has taken steps to improve the scores on the TSAs. It is suspected that many low scores were due to the new version of the industry certification, since students were completing the curriculum geared toward the prior certification. To aid students who fell under the older curriculum we provided access to bridging material. The following steps were taken: 1) Updated curriculum and sent changes through MCC s curriculum process. 2) Provided access to bridging material for students who completed prior curriculum. 3) Required practice certifications for students in CSIS 113 and CSIS 213. In addition, the program assessed the following performance indicators in 2013-14: Uses internetworking operating system software for router and switch configuration to control the flow of information through a network. Troubleshoots and repairs problems with information flow through a network. The assessment used was the CSIS 213 Accessing the WAN practical final. 85% of the students adequately achieved the skills measured in the program outcome. This is a 6% decline from FY13. The program will implement a newly updated skills based assessment along with newly updated curriculum for FY15. Computer Science and Information Systems Web Technologies The CIW certifications are used as the Web Technologies TSA. In fall 2013, five students took the exam. One of three areas assessed reported majority of students not successfully passing (Site Development Associate). More course time and emphasis has been placed on the Site Development Associate materials. Spring 2014 had one student who took and passed this part (and both others) of the assessment.
The program outcome assessed was: Demonstrate professional oral and written communication skills. A project-based assessment (administered via SAM) and common comprehensive final were given to selected sections of CSIS students beginning Fall 2013. Nearly half of the 147 students who took the skillbased exam scored 96 or higher, and 120 of them 76 or higher. Although results were not particularly low, the program has made significant changes to introductory curriculum in CSIS 110 and CSIS 115. Dental Assisting The national board exam serves as the TSA, and it has three parts: radiology, infection control, and chair-side assisting. Fall 2013 scores were as follows: ICE (Out of 17 students in Fall, 2 failed, none retook due to financial constraints, ICE 88% Pass) and RHS Exams (Out of 17 students in Fall, 5 failed, 2 retook and Passed, RHS 82.3% Pass) In response the program has provided board prep software for students from learning express. Mock National Board Exams were also offered to our students twice in fall and spring prior to their actual board dates. Emergency/Paramedic The national registry exam serves as the TSA for this program. It is anticipated that a practical skills section will be added to the exam for 2016. The program is adding practical skills lab courses to help with student preparation. Engineering Technology This program has no TSA available. The ADDA exam is a possibility for CADD but involves considerable expense to become an ADDA testing center. The program outcome assessed is: Produce a completely dimensioned and annotated orthographic construction document from a three-dimensional representation of an object. This document should be produced to a standard scale and to an established set of CADD and drafting standards using an industry standard CADD software package. The results were good and no changes are planned this year based on the results. Graphic Design A portfolio assessment scored by local industry representatives serves as the TSA. Twenty of twenty-two students completing a portfolio (91%) were given a passing score by a panel of employers. The results will be discussed by the faculty. Health Information Technology The RHIT exam serves as the TSA for this program. To date graduates have successfully passed the national certification exam with scores above the national average. The 2014 graduating class was required to purchase a competency review text book that was bundled with the cost of the national exam. This will require the students to sit for the national certification exam within 4 months after
graduation (majority of the students will be sitting for the exam in August 2014) or $300 will be forfeited. Also, an RHIT competency course was incorporated into curriculum for spring 2014. The program analyzes scores on the RHIT credentialing exam, comparing MCC-PV scores to national scores as well as doing item and category analysis to determine if there are weak areas that need to be addressed. Additionally the curriculum is analyzed annually to determine if updates or changes should be made. No program outcomes were assessed formally this past year, because due to curriculum revision the program outcomes had to be revised. Courses were also restructured with new texts to meet the deliverables. Outcomes were reviewed at the April 29, 2014 Advisory Board meeting. Heating and Air-conditioning ICE was administered on a limited scope for the first time in the program. The results were below expectations. The textbook used in the core classes of the program will be updated to better cover required material. The program will also make the latest troubleshooting software simulations available to instructors to better prepare students for the TSA. The program outcome assessed was: The program will graduate individuals who exhibit competence with entry-level skills in the heating, ventilation, and air conditioning profession. It was assessed through the TSA. Law Enforcement The Law Enforcement (Police Academy) program uses the Peace Officers Standards and Training (POST) exam, which is the licensure required for employment in Missouri. In the past year, 37 graduates took the exam and all 37 (100%) passed. No immediate program changes are planned based on these results. Occupational Therapy Assistant The program TSA is the national board exam from NBCOT. Thirteen of the fourteen students who took it in spring 2014 passed the first time. Current course content, schedule, clinical assignments and instructional methodology appear to be successfully supporting student success. Paralegal There is no standard national test available as a TSA for the program. The capstone Internship class is the primary assessment tool. The program outcome assessed was: Perform in a professional and ethical manner as a working paralegal and a productive member of the legal team. All students have passed for the last two years, and no major changes required. Physical Therapy Assistant The program uses the national boards from NPTE as a TSA. Results have historically been excellent (nearly always 100% first-time pass rate). This continued for 2013-14 with 34 out of 34 (100%) students
passing the licensure examination. The program has continued earlier implemented changes including requiring students to enroll in an online score builder course during clinical rotations, requiring students maintain a portfolio of old quizzes and exams for study purposes, and a comprehensive internal end-ofprogram exam required for all. Practical Nursing The NCLEX national exam is used as the TSA. The Practical Nursing Program improved student performance on TSAs by increasing live clinical simulation and using more ATI practice examinations. A curriculum alignment with ATI has been completed. Data from ATI exams revealed areas within the curriculum needing improvement, and faculty worked on alignment and quality of delivery. An NCLEX review has been added to the end of the program to review content and provide test taking strategies to increase the student s. The program outcome assessed is: Applies the nursing process for basic nursing care to healthy or ill clients across the lifespan. It was assessed by the ATI Comprehensive Predictor, which evaluates the student s knowledge base and also estimates probability of passing the NCLEX exam. The Comprehensive Predictor showed 15 students with a 95% likelihood of passing, and provided recommendations for areas to review. As a result of the recommendations, 25 students passed the exam on their first attempt. Skills learned in the first semester will now be integrated throughout the entire program. Radiological Technology The program uses the national registry exam from ARRT as its TSA. All eighteen students who took the exam this year passed. Last year s pass rate was also 100%. The program will continue to make updates to curriculum to remain aligned with national standards. Registered Nursing The RN/ADN program uses the NCLEX exam as its TSA. The pass rate for Spring 2013 graduates was 90.3% with 52 students taking the exam. For the 72 December 2013 graduates who took it, 90.3% passed. Thirteen May 2014 completers have taken the exam so far, with a 92.3% pass rate. The program plans to increase simulation time in areas where clinical sites are difficult to find. The accreditors are also encouraging reduction of pre-requisites. Surgical Technology The national certification exam (CST) is used as a program TSA. 15 out of 16 students tested (94%) passed the exam this year. This success rate is a huge improvement over the previous year s pass rate of 60%, and may at least partially be attributed to changes made based on last year s assessment work. Veterinary Technology
The program uses the national exam as its TSA but does not have access to scores by student. There were eighteen students who took the licensure exam this year, and all 18 passed. The program outcome assessed was: Obtain national certification and was based on NTVE results. The program is updating curriculum in three courses to reflect current industry standards. The coordinator will also seek advisory board approval to change the program prerequisites. Students have had to finish general education courses after the technical coursework was finished, and had to wait until November rather than July to take the VTNE. The change should improve the students chances of completing their degree when the program is finished, and improve exam performance. Welding The AWS SENSE Modular exams are used as the TSA. The faculty is investigating the effectiveness of the material that augments the lectures. Modifications have been made to the materials used to augment lectures for the 2014-15 academic year. The program outcome assessed was: The program will graduate individuals that meet or exceed the American Welding Society s guidelines for entry-level employees in welding technology. It was assessed using the indicator: Demonstrate ability to apply welding, cutting, and print reading skills to layout and fabrication. The results were good, but the rubric will be modified to simplify the points being addressed, enabling better analysis of results.