USING THE NATIONALLY NORMED SME FUNDAMENTALS OF MANUFACTURING EXAMINATION AS A PROGRAM OUTCOMES INDICATOR

Transcription

1 USING THE NATIONALLY NORMED SME FUNDAMENTALS OF MANUFACTURING EXAMINATION AS A PROGRAM OUTCOMES INDICATOR Gregory Neff, P.E. 1 Abstract? This paper discusses the use of relevant, nationally normed examinations, one of the 12 assessment methods compiled by Gloria Rogers of Rose-Hulman University. The Accreditation Board of Engineering and Technology mentions this method as a typical assessment measure in both the Engineering Criteria 2000 and Technology Criteria 2000 accreditation criteria. The proposed 2004 revision to the technology criteria requires that: Each program must utilize multiple assessment measures in a process that provides documented results to demonstrate that the program objectives and outcomes are being met. INTRODUCTION The Mechanical Engineering Technology and Industrial Engineering Technology programs from the Department of Manufacturing Engineering Technologies and Supervision (METS) at Purdue University Calumet have a course requirement worth 10% on the syllabus of the one credit senior project survey course (required by the curriculum). Students must take the nationally normed Society of Manufacturing Engineers (SME) Fundamentals of Manufacturing Examination. The vast majority of students from both programs take their first jobs in the manufacturing industry due to the job market in the Northwest Indiana and Chicagoland areas. This paper will examine the benefits and costs of the SME Fundamentals of Manufacturing examination also known as the Certified Manufacturing Technologist (CMfgT) Examination in the light of the Engineering Criteria 2000 (EC2000) and the Technology Criteria 2000 (TC2K) accreditation criteria [1]. A comparison of various alternatives will be made. These include: the Engineering Intern (EI) exam often called the Fundamentals of Engineering (FE) exam or the Engineer in Training (EIT) exam from the National Council of Examiners for Engineering and Surveying (NCEES), the National Institute for Certification in Engineering Technologies (NICET) certification program, the Certified Manufacturing Engineer (CMfgE), Certified Manufacturing Integrator (CET) and Certified Engineering Manager (CEM) exams also from SME, the various (12) ASQ exams, various (9) fluid power certifications from the Fluid Power Society, American Society for Production and Inventory Control (APICS) Certified Production and Inventory Manager (CPIM) exam, Geometric Dimensioning and Tolerancing (GD&T) certification or other American Society of Mechanical Engineers (ASME) examinations, and the National Association of Industrial Technology (NAIT) Certified Industrial Technologist (CIT) exam. Nationally normed examinations for technology or engineering programs outside of the mechanical, industrial or manufacturing areas will not be considered in this paper. ASSESSMENT UNDER TC2K OR EC2000 The METS department at Purdue University Calumet has defined ten tools [2] for collecting assessment data with responsibilities defined for every staff or faculty member in the department. The tools were structured to include all the assessments required by the University as well as the various accreditation agencies. Designing the tools with this aim has allowed the METS department faculty to minimize paperwork and focus on actual continuous improvement efforts rather than simply writing multiple reports to satisfy various constituents who are essentially looking for the same information. The ten assessment tools developed by the METS faculty focus on assessing the following five general areas: 1) Faculty, 2) Student enrollment and activities, 3) Facilities and equipment, 4) Curriculum, and 5) Teaching and learning. The METS department assessment tools are as follows: Tool Assessment Data for: T1 - Summary of annual faculty data Faculty T2 - Enrollment Summary Student Professional Activities T3 - Equip./Tool/Mach./Space/Tech. Survey Facilities T4 - Summary of Dept. Curriculum Documents Curriculum T5 - Individual Course assessments T6 - Employer/Alumni Surveys T7 - Graduate Exit Surveys T8 - a through k matrix (Program outcomes assessment) T9 Nationally normed exams T10- Advisory Board Input Summary Appropriate Areas FIGURE 1. METS DEPARTMENT ASSESSMENT TOOLS 1 Purdue University Calumet, MET Program, th Street, Hammond, IN , gneff@purdue.edu 135

2 An example of a Tool can be seen below in FIGURE 2. Upon review of this Tool it becomes obvious that the Tool is nothing more than the prescription of what assessment data needs to be looked at and evaluated in order to verify the department and program goals are met. Assessment Tool # 9 SME CMfgT Exam I. Data What data is needed 1) Comparison between PUC pass rates and national student pass rate 2) Percentage of questions correct in the various MET and IET subject areas vs. time and cumulative. Where data comes from 1) Manufacturing Engineering Certification Institute of SME Examination analysis When data should be gathered and by whom Use of the data 1) SME faculty advisor. 2) May and December of each year 3) Requires typing results onto a spreadsheet 1) Individual students who pass will use the certification as a credential for job searches or promotion, and on business cards and resumes 2) Having taken one national exam might encourage additional certification or registration. 3) Department will Graph and tabulate results to support A through K outcomes assessment 4) Comparison of pass rate with national average as an indication of program strength Correlation to program and department goals II. III. 1) Demonstration data for TC2K Criterion I: a, b, f, h, k 2) Program Goals: 3) Department Goals: Evaluation and Assessment of Data Results from each administration of the exam will be graphed by % correct versus knowledge area to track how students are performing. Outcomes / changes / improvements The program coordinators will review test results summary. Changes in curriculum may prove to be needed from this analysis. If particular courses need changing the program coordinator and course lead instructor should take appropriate action. FIGURE. 2 NATIONALLY NORMED EXAMINATION ASSESSMENT TOOL EXAM REQUIREMENTS For a nationally normed examination to be useful for assessment it must:? Be relevant to the program educational objectives and outcomes? Be available for senior students to take without requiring on-the-job experience? Provide feedback to the teaching faculty on how the program s students did overall on the exam compared to other program s students and on the different subject areas covered in the exam? Be reasonably priced.? Be available each term or semester 136

3 POSSIBLE EXAMS A number of certification exams are available which are nationally normed. Some of the exams that are relevant to mechanical, industrial, or manufacturing engineering technology students are described below. EI/ FE Exam The Engineering Intern or Fundamentals of Engineering exam is one of the few nationally normed engineering examinations that are suitable for assessment. The exam is eight hours long. It is available to all engineering seniors at EAC ABET accredited programs around the country and is available to seniors or graduates from TAC ABET programs in 28 of the 50 states. The price of the exam and application is cheapest in the state of Indiana, which in 2000 charged only $20 for the application and exam. Other states charge as much as Illinois $ or $180 in Guam. The National Council of Examiners for Engineering and Surveying (NCEES) have begun cooperating with state boards of registration and engineering schools in providing programs with feedback for summative outcomes assessment. Besides information on how well students performed on individual topic areas such as dynamics or statics, information is now being provided on how well students did compared to students at similar institutions elsewhere. Lefevre [3] suggested that Engineering programs should strongly consider using the FE exam topic-level performance data as part of their program assessment. But he warned that Institutions must remember that the primary purpose of the FE is to assess minimal technical competence. Watson [4] suggested that the use of the FE information is compromised by the fact that a) the FE examination is not taken by all engineering students, b) students who are required to take the examination are typically not held accountable for the results, and hence may not be motivated to succeed, and c) the students who voluntarily take the FE are highly motivated to satisfy the preliminary requirement for the PE. Along these lines, Drnevich and Tener [5] went on record in opposition to mandatory use of the PE/FE exams as assessment tools at least for civil engineering programs. They pointed to the provided FE Reference Book that contains definitions and equations as threatening to reduce the exam to an exercise in plug and chug. They also suggested that the exam is a rather shallow instrument due to the large number of multiple-choice questions spread out over many topics requiring that each question be relatively simple. One disadvantage is that the feedback given by the NCEES is not specific enough to generate much in the way of initiatives for program improvement, which is the motivation for changing to the new accreditation criteria. The feedback for the morning consists of 12 percentages. The afternoon feedback varies with the specialty from 11 civil topics to 20 industrial engineering topics with a percentage for each. SME Fundamentals of Manufacturing Exam The METS department at Purdue University Calumet requires senior MET and IET students who are taking the capstone senior project sequence to take the Society of Manufacturing Engineers Fundamentals of Manufacturing Examination as part of their senior project survey course. The exam is three hours long and administered by the faculty during the 14 th week of the semester. It costs students who become student members of SME (for $15), an additional $95, not counting study materials. This cost is not too different from the cost of a typical textbook. Nonmembers would pay $190 to register for the exam. The percentage of Purdue University Calumet students passing the December, 2001 exam was 50%. The percentage of all college and university students passing the exam from February 6, 2001 to December 13, 2001 was 58% (410 out of 737). At this point we are collecting baseline data on how our population with its characteristic of being required to take the exam performs.? Changes/Improvements Implemented: Three faculty members have volunteered to teach an 8-hour review course for students over two Saturdays.? Future plans for improvement: More detailed feedback on what types of questions students missed has started to be provided by SME. The department plans to use the data to improve the review course and eventually improve required credit courses to help students score higher on the exam. The body of knowledge covered by the exam is shown in FIGURE 3. The 2 nd column shows the number of questions on the December, 2001 exam on that topic. An example of the feedback provided by SME is shown for the December,2001 exam in the 3 rd column. It shows the percentage of correct answers for that topic by Purdue University Calumet students who took the exam then. The 4 th column shows a comparison with all students from 42 schools taking the exam in Passing score on the exam is 60%, which is a higher percentage than the passing score of 70 on the FE exam. This passing score on the FE exam corresponds to 50% or less. SME gives more feedback to schools than NCEES does. Module 1. MATHEMATICS, APPLIED & ENGINEERING SCIENCES, & MATERIALS APPLICATION 1.1 Mathematics 12/13/01 # Questions%Correct National Algebra Trigonometry Analytical Geometry Calculus Applied and Engineering Sciences Metrication/SI System Physics Chemistry Statics Dynamics Fluid Mechanics Thermodynamics/Heat Transfer Electrical Circuits/Electronics

4 1.3. Materials Application 12/13/01 # Questions%Correct National Metals (Properties and Applications) Plastics/Polymers (Properties and Appl) Composites (Properties and Applicat.) Ceramics (Properties and Applications) Fluids (Properties and Applications) - - Percentage (%) of test on this module 22.3 Module 2. PRODUCT/PROCESS DESIGN & DEVELOPMENT 2.1 Research and Development # Questions%Correct National Product R&D Process R&D Market/Sales/Life Cycle Analysis Intellectual Property Protection Design Design Management Concurrent Engineering Design for X (Mfg/Assy/Maint/etc) Drafting/Drawing/Engineering Graphics CAD/CAM/CAE Applications Simulation/Engineering Design Analysis Tolerance Analysis/GD&T Engineering Economics/Value Analysis Product Prototype Build and Test Process Development and Test - - Percentage (%) of test on this module 10.0 Module 3. MANUFACTURING PROCESS APPLICATIONS & OPERATION 3.1 Manufacturing Process Applications & Operations# Quest.%Correct National Material Removal Processes Fabrication Processes Hot and Cold Forming Processes Casting and Molding Processes Electrical/Electronics Mfg. Processes Heat Treatment Processes Joining, Welding, &Assembly Processes Finishing Processes Bulk and Continuous Flow Processes Material Handling/Packaging Hand Tool Use/Machine Operating Percentage (%)of test on this module 10.0 Module 4. PRODUCTION SYSTEM and EQUIPMENT DESIGN/DEVELOPMENT 4.1 Production System Design & Development # Quest.%Correct National Infrastructure/Plant Location Analysis Facility Planning/Plant Layout Process Planning Capacity Planning Production/Manufacturing Syst. Design Process Documentation/Work Instruct Tool and Equipment Selection Process&Equipment Capability Analysis Cost Justification Production System Build/Test Human Factors, Ergonomics, & Safety Maintenance Systems Environmental Protection/Waste Mgmt Equipment/Tool Design and Development Cutting Tool Design Workholding Tool Design Die/Mold Design Gage Design Machine Design Power Systems (Mech/Elec/Fluid) Control Systems (Mech/Elec/Fluid) - - Percentage (%) of test on this module 25.4 Module 5. AUTOMATED SYSTEMS and CONTROL 5.1. Automated Systems and Control # Quest.%Correct National Automated Systems (Hard/Flexible) CNC/PLC/Computer Control CIM Systems Computer Systems and Networks Information Technology/Database Systems - - (MIS, etc.) Enterprise-wide Systems Integration - - (MES,ERP,etc.) Percentage (%) of test on this module 10.8 Module 6. QUALITY and CUSTOMER SERVICE 6.1. Quality and Customer Service # Quest.%Correct National Customer Focus (Research/Test/Satisfact.) Quality System and Standards (QS/ISO/CE Mark/etc) Probability and Statistics Statistical Control Methods (Sampling/Charting/etc) Problem Analysis & Solving (Fishbone/Pareto/FMEA/etc) Factor Analysis (DOE/Correlation/etc) Capability Analysis (Process/Equipment/etc) Inspection/Test/Validation Metrology Reliability Analysis System/Process/Continuous Improvement - - (BPR/kaizen,etc) Customer and Field Service - - Percentage (%) of test on this module 10.8 Module 7. MANUFACTURING MANAGEMENT 7.1. Manufacturing Management # Quest.%Correct National Strategic Planning/Global Competitiveness Organizational Design and Management Project Management Personnel Management Methods (x/y/team/matrix/etc) Human Behavior/Motivation/Leadership Labor Relations Education/Training Operations Research, Analysis, & - - Forecasting Production Organization Systems (agile/lean/mass/etc) Material & Resource Management/Logistics Accounting/Finance/Economics Business/Engineering Ethics and Social Responsibility Standards, Laws, and Regulations Percentage (%) of test on this module 6.9 Module 8. PERSONAL EFFECTIVENESS 8.1. Personal Effectiveness # Quest.%Correct National Interpersonal Skills (listening, courtesy, etc.) Negotiating & Conflict Management (persuasion, conflict resolution) Presentation Skills & Oral Communication (formal & informal) Written Communication Skills (reports, computer literacy) Innovation & Creativity (idea generation & - - acceptance) Learning & Knowledge Transfer (info research & sharing; education - - Percentage (%) of test on this module 4.6 Total 130 Questions (100%) FIGURE 3 SME CMFGT TEST BODY OF KNOWLEDGE 138

5 Other Certification Exams There are a number of other certifications available from various organizations. Would any of these lend themselves to program assessment?? NICET is the National Institute for Certification in Engineering Technologies affiliated with the National Society for Professional Engineers (NSPE). NICET has certifications in mechanical engineering technology, industrial engineering technology, civil engineering technology, and electrical/ electronics engineering technology. The technician certification tests ($150) are for individuals with appropriate work experience, college training, or both. Their usefulness for Associate degree level technology program assessment would depend on NICET s willingness to provide appropriate feedback to schools. To become a certified mechanical or industrial engineering technologist requires a TAC/ABET accredited degree, payment of a fee ($75 for the initial level), filling out a personal information form, a technologist work history form and getting a recommendation or endorsement from a professional who is familiar with the applicants work history, technical capabilities & background, but no exam.? NAIT is the National Association of Industrial Technology. Their certification is called the Certified Industrial Technologist (CIT). Cost is $30. The three-hour exam is closed book, 160 multiple-choice questions divided into four categories: Production Planning & Control, Safety, Quality, and Management & Supervision. The exam is intended to be used for program assessment.? SME CMfgE, CEI and CEM are each $195 for members or $390 nonmembers. The CMfgE is the Certified Manufacturing Engineer. This certification is not suitable for program assessment because in order to take the 3-hour exam the applicant must have 10 years of manufacturing related education and/or work experience. Few if any students would qualify to take this exam in most programs. The CEI is the Certified Enterprise Integrator, which requires 8 years of related education and/or work experience to take the 5-hour exam. The CEM is the Certified Engineering Manager. It too requires 8 years of related education and/or work experience to take the exam.? ASME Certifications include the $360 GD&T Certification that has two levels, each requiring passing a 4 or 6 hr question multiple-choice examination. The Fossil Fuel Boiler Operator (QFO) has 6 levels and is based on education, experience and written tests. This and the other three operator certifications, QHO, QMO, and QRO are highly specialized, so are not suitable for program assessment of engineering or technology programs.? ASQ is the American Society for Quality, which has 12 certifications. All require relevant work experience, except the very lowest, Certified Quality Improvement Associate (CQIA) which accepts the AS degree. CQIA cost is $155 for members, $260 for non-members to take the 3- hour 100-question exam. No school feedback is available.? APICS is the American Production and Inventory Control Society. Their Certified in Production and Inventory Control (CPIM) and Certified in Integrated Resource Management (CIRM) programs require 5 three hour exams each costing $100/$150 for members or $135/$185 for nonmembers respectively. No experience is required but these are not generally suitable for program assessment because of their narrow focus, lack of feedback to schools, and cost.? Fluid Power Society has nine certifications available. Student fees are $40 for six of the exams: mechanic or technician for any of industrial hydraulic, mobile hydraulic or pneumatic systems. No job experience is required. Specialized exams have limited use for program assessment. The 75 question exams are 3 hours long. A three hours hands-on exam is also required for certification. CONCLUSION Several nationally normed standardized exams are available from professional organizations. The exams considered in this paper were originally developed for individual use to document knowledge, skill, and ability through a certification or professional licensing process but can reflect on the quality of preparation of program graduates compared to other universities. The FE, CMfgT, and CIT exams can also be used for program assessment. For mechanical, industrial, or manufacturing engineering technologies, the SME CMfgT exam is a good choice. For engineering programs, the FE exam is the gold standard and due to its low cost in Indiana probably should be required in that state. AUTHOR GREGORY P. NEFF has served as a TAC/ABET program evaluator since He currently serves on the American Society of Mechanical Engineers (ASME) Committee on Technology Accreditation in charge of training new MET evaluators and will be starting a five-year term as an ASME commissioner on the Technology Accreditation Commission in summer, He was a program evaluator on a TC2K accreditation visit in REFERENCES [1] ABET web site with technology accreditation criteria, [2] Neff, Gregory and Scachitti, Susan, The Assessment Cookbook: Recipes for Successfully Meeting TC2K Criteria, Proceedings of the 2002ASEE Annual Conference and Exposition, June 16-19, Montreal, Quebec, CA, Session [3] Lefevre, Walter, Smith, John W., Steadman, John W., and White, Kenneth R., Using the Fundamentals of Engineering Examination to Assess Academic Programs, NCEES, Clemson, S.C., November, [4] Watson, John L., An Analysis of the Value of the FE Examination for the Assessment of Student Learning in Engineering and Science Topics, Journal of Engineering Education, July 1998, pp [5] Drnevich, Vincent P. and Tener, Robert K., Opposition to Mandatory Use of PE/FE Exams as Assessment Tools, Proceedings of the 1998 ASEE Annual Conference and Exposition, June 28-July 1, Seattle, WA, Session