Tunxis Community College eportfolio Project Evaluation Report submitted by Professor Laura M. Gambino September 30, 2011

Transcription

1 Tunxis Community College eportfolio Project Evaluation Report submitted by Professor Laura M. Gambino September 30, 2011 eportfolio Project Background and Goals Tunxis Community College, located in Farmington, Connecticut serves approximately 4,000 degree and certificate seeking students each year and offers its students a quality, yet affordable education in an accessible and supportive environment, fostering the skills necessary to succeed in an increasingly complex world. ( Mission and Vision, TXCC website). In 2009, Tunxis joined the LaGuardia Community College Making Connections Mini-Grant and Seminar Program. Out of that participation, the Tunxis eportfolio Project was designed. The eportfolio Project, first and foremost, addresses the College s primary mission of increasing student success, recognizing that eportfolio technology can and should augment best practices and pedagogy. Furthermore, the eportfolio Project is an integral part of institutional assessment, allowing faculty, staff, and administration to collaborate and share best practices in teaching, learning, and assessment, while addressing the needs of our external constituents. In terms of expanding and strengthening connections with the community, eportfolio provides program advisory boards the opportunity to participate in capstone assessment. Since portfolios emphasize critical thinking, communication and reflection, this project engenders the development of skills necessary to succeed in an increasingly complex world. The eportfolio Project work is centered on four primary goals: Develop and implement a comprehensive eportfolio system that will include the following: o Showcase student work in meeting course, program, and general education abilities o Showcase student work in meeting career objectives Create a vibrant teaching and learning community using eportfolio

2 TCC eportfolio Project Evaluation Report 2 Design a comprehensive faculty development plan centered on eportfolio Use eportfolio as the primary tool for institutional assessment For each of these goals, specific outcomes were developed related to faculty development, student support, technology, and assessment and evaluation (See Appendix A). Areas of focus for the eportfolio Project include Liberal Arts and Sciences, First Year Experience, and career programs. eportfolio Project Implementation Implementation of the eportfolio Project began during the summer months prior to the Fall 2009 semester. Tunxis successfully migrated to a new eportfolio platform, Digication, and approximately fifty faculty and staff members attended professional development workshops to learn the new software. In the fall, the eportfolio Project leaders began their first faculty professional development eportfolio Seminar Series. Ten faculty members from a range of Liberal Arts and Sciences disciplines attended the seminar. The goal was for each of them to incorporate the use of eportfolio into one or more of their courses. The seminar consisted of four, two-hour workshops and introduced faculty to the pedagogy of eportfolio, reflection, and assessment. During the fall semester, eportfolio was used in the Computer Information Systems, Business Office Technology, Dental Hygiene, and Early Childhood Education Programs. In addition, all full-time faculty members in developmental English courses and the First Year Experience course integrated eportfolio into the curriculum. Additionally that semester, as part of student support, student technology mentors were hired and trained to work in the eportfolio lab and run introductory eportfolio workshops for

3 TCC eportfolio Project Evaluation Report 3 students. The lab was open 20 hours per week. Faculty members were able to schedule visits to the lab for their classes; open training sessions and walk-in hours were scheduled as well. During the spring semester, eportfolio use continued to grow at the College. A second professional development eportfolio Seminar Series was held in conjunction with Tunxis Center for Teaching. Eighteen faculty members volunteered to attend that seminar. At the end of the Spring 2010 semester the Business and Technology Department held their first capstone eportfolio student showcase. On May 5, 2010, the Computer Information Systems (CIS) and Business Office Technology (BOT) programs held a joint advisory board meeting and eportfolio showcase. Students from the CIS Coop Ed/Work Experience course and the BOT Administrative Practicum shared their showcase and assessment eportfolios with advisory board members and members of the Tunxis management team. Two weeks later, the Dental Hygiene program held a similar event with their students. Both showcases provided the opportunity for advisory board members, faculty, and Tunxis administration to observe how students use eportfolios to demonstrate program and general education outcomes. During the academic year, use of eportfolio at Tunxis continued to grow. The Business Administration program began using eportfolio as part of its capstone course and held a capstone showcase. A third cohort of ten faculty participated in the eportfolio seminar series. A new faculty professional development workshop was developed, eportfolio: Continuing Conversations. These workshops were designed for experienced eportfolio faculty. Each semester, two, two-hour workshops were offered focusing on topics such as reflection, assessment, and eportfolio pedagogy. The eportfolio lab expanded its hours to thirty per week with three student technology mentors. All First Year Experience faculty members, full-time and adjunct, integrated eportfolio into their courses. Tunxis also began investigating the use of

4 TCC eportfolio Project Evaluation Report 4 eportfolio for advising and transfer of Liberal Arts and Sciences and General Studies students. In the spring, eportfolio capstone student showcases were held in Computer Information Systems, Business Administration, Business Office Technology, and Dental Hygiene. The Early Childhood Education program used information from student eportfolios as part of the NAEYC accreditation process. The eportfolio Project also sponsored its first college-wide student eportfolio competition and showcase featuring first year, Liberal Arts and Sciences and career program eportfolio students. The showcase was attended by approximately forty faculty and staff. eportfolio Project Evaluation Plan Several evaluation tools and data collection methods were used to evaluate the impact of eportfolio on faculty. Faculty who participated in the initial Digication summer training were surveyed regarding the software and its applicability to classroom use. Faculty in the eportfolio Seminar Series were asked to complete a reflection narrative at the conclusion of the seminar discussing their seminar experience. Data related to faculty use of eportfolio in courses and sections was collected each semester. An eportfolio student survey was created and administered each semester of the academic year. This survey focused on students initial impressions of the eportfolio. A capstone student survey was designed and administered to students enrolled in program capstone courses during the Spring 2010, Fall 2010, and Spring 2011 semesters. This survey focused on both the value of the eportfolio as a learning and showcase tool, and on the students awareness of the integration of knowledge both within courses in the major and general education courses across a program s curriculum. An advisory board survey was developed and used after each

5 TCC eportfolio Project Evaluation Report 5 program student showcase event during those same semesters to assess the value of eportfolio as a tool for demonstration of knowledge and its use for employment or transfer. Student success (C- or better) and retention (semester-to-semester) data was collected each semester in developmental English courses, comparing eportfolio and non-eportfolio sections. College-wide retention rates were examined in fall 2010, based on the number of eportfolio sections a student was enrolled in during the previous spring 2010 semester. In addition, coordinators for each program provided data on the number of capstone students using eportfolio to successfully demonstrate program and general education outcomes. Lastly, a continual increase each semester in the number of student eportfolios, faculty participation in eportfolio seminars, and an increase in the number of courses and sections using eportfolio would serve as evidence of project success. Tunxis eportfolio Project Evaluation Plan Goal Measure 1A. Create a vibrant teaching and learning community using eportfolio 1B. Design a comprehensive faculty development plan centered on eportfolio 2A. Student engagement with eportfolio 2B. Increased student understanding of eportfolio 2C. Student awareness of integration of knowledge within a degree program Faculty Training Survey Number of faculty using eportfolio in courses Number of faculty who participate in the eportfolio Seminar Series Number of faculty who participate in the eportfolio Continuing Conversations series Faculty reflection narratives eportfolio student survey ( ) Capstone student survey ( ) Number of student eportfolios created 3. Increased student success and retention Comparison of success (C- or better) rates in eportfolio/non-eportfolio sections in developmental English Retention rate comparison in developmental English College-wide retention rates based on number of eportfolio courses

6 TCC eportfolio Project Evaluation Report 6 Tunxis eportfolio Project Evaluation Plan Goal Measure 4. Effective use of eportfolio for assessment Percentage of students using eportfolio to demonstrate program outcomes Percentage of students using eportfolio to demonstrate general education outcomes eportfolio use in program accreditation and review process Advisory Board Survey eportfolio Project Evaluation Evidence and Analysis Goal 1A and 1B: Faculty and staff participated in a 4-hour introductory workshop during the summer of 2009 to learn Digication, Tunxis new eportfolio software platform. Both faculty and staff surveyed at the conclusion of the training. Twenty-one percent surveyed said they would use eportfolio in all their fall courses and fifty-one percent of those surveyed said they would use eportfolio in selected courses. Forty percent said they would use eportfolio as a way to demonstrate general education abilities and forty-six percent would use eportfolio to demonstrate program abilities. It should be noted that these numbers were most likely lowered by the inclusion of non-teaching staff in the survey. During the Fall 2009 semester, ten full-time faculty members voluntarily participated in the first eportfolio Seminar Series. This workshop series consisted of four, 2-hour sessions with an intense eportfolio pedagogical focus. Eighty-percent of these faculty members successfully integrated eportfolio into one or more of their courses during the Fall 2009 semester. A second cohort of eighteen full-time and adjunct faculty signed up to participate in a similar series during the spring 2010 semester. Eighty-three percent of this second cohort integrated eportfolio into their courses during either the spring 2010 or fall 2010 semester. A third cohort of twelve faculty participated in the fall 2010 seminar series. In this group, six of the participants were

7 TCC eportfolio Project Evaluation Report 7 newly-hired faculty who were required to attend the seminar. Seventy-percent of this cohort integrated eportfolio into their courses during the school year. Beginning in the Fall 2010 semester, the eportfolio: Continuing Conversations workshops were held twice a semester, for experienced eportfolio faculty. Attendance was voluntary; faculty could attend one session, or all four sessions as their schedules permitted. On average, fifteen faculty members attended each workshop. Faculty participants were asked to complete reflection narratives at the conclusion of the eportfolio Seminar Series. The eportfolio: Continuing Conversations attendees were asked to provide qualitative feedback at the end of each workshop. Faculty comments in these reflections and feedback suggest that the professional development workshops are valuable teaching and learning activities. For example: In January, I knew nothing about eportfolio and online assessment. Today, I feel more comfortable with the tools and how to apply them in the class. I was intimidated by the new medium, but once I sat in front of a computer and started noodling around the site I became more comfortable. I found that I need individual attention at times, so Marguerite, Sally, and Laura were all instrumental in guiding me through my learning of eportfolios. Continue to have these eportfolio Conversations workshops. We need to talk more about reflection. The time spent working with other faculty on assignments is really valuable. Faculty use of eportfolios in courses continues to grow each semester. Currently sixtyfive faculty members (full-time and adjunct) currently teach courses with an eportfolio

8 TCC eportfolio Project Evaluation Report 8 component. Sixty distinct courses and 110 sections used eportfolio during the academic year. One hundred thirty-six distinct faculty and staff eportfolios exist in Digication. Goal 2: A student survey was given to students enrolled in courses using eportfolio during the first year of the project. Survey results suggest that students knowledge of eportfolios increased and that students saw the eportfolio as helpful to them during their education. Eighty-nine percent of students indicated they will maintain their eportfolio in the upcoming semesters and fifty-nine percent said they would use an eportfolio to showcase their work. These results indicate that students see the value of eportfolio as a tool for learning and showcase. Only twenty-eight percent of students said they would use their eportfolio to demonstrate general education and/or program abilities. Additional work needs to be done to emphasize the value of using eportfolio to meet these objectives when students are first introduced to the eportfolio. Capstone student survey results also support the value of eportfolios for students. Seven capstone courses in programs that use eportfolio as both a learning and assessment tool and a showcase tool administered the survey over three semesters. Cumulative results (n=79) indicate that students have an understanding of integrative learning as related to both courses in their program and real-world experiences. Ninety-nine percent agreed or strongly agreed that they understand how classes relate to their major. Ninety-five percent agreed or strongly agreed that they understand how classes fit together. Ninety-six percent of students agreed or strongly agreed that they use accumulated knowledge in current coursework. Ninety-seven percent see meaningful connections between coursework and real-world experiences.

9 TCC eportfolio Project Evaluation Report 9 Additionally, 83% of capstone students surveyed agreed or strongly agreed that eportfolio will be a valuable tool for transfer/job opportunities and 86% agreed or strongly agreed that eportfolio is a good way to showcase achievements/growth. During the past two years 3,064 distinct students have created eportfolios in Digication. One hundred sixty-eight students created complete or capstone showcase and/or assessment eportfolios during that same time period. Goal 3: The Office of Institutional Research compiled student success and retention data in developmental English. Student success is defined as achieving a C-(70%) or better in a course. Retention is defined as re-enrollment at the College from one semester to the next. Using data from the spring and fall 2010 semesters, an increase in student success rates was seen in eportfolio sections as compared to non-eportfolio sections in all developmental English courses. Students (n=422) in ENG*065 Integrated Reading/Writing I, had a success rate of 66.85% if enrolled in an eportfolio section as compared to 63.9% in non-eportfolio sections. Students (n=582) enrolled in ENG*075 Integrated Reading/Writing II, had a success rate of 65.38% in eportfolio sections as compared to 61.27% in non-eportfolio section. Students (n=366) enrolled in ENG*093 Introduction to College Reading and Writing, had a success rate of 61.22% in eportfolio sections as compared to 58.58% in non-eportfolio sections.

10 TCC eportfolio Project Evaluation Report 10 Additionally, the cumulative retention rate from Fall 2010 to Spring 2011 for students enrolled in developmental English (ENG*065, ENG*075, ENG*093) courses was 78.1% in eportfolio sections as compared to 72.3% in non-eportfolio sections % Fall 2010-Spring 2011 Retention Rate Developmental English Students 78.00% 76.00% 74.00% Retention Rate 72.00% 70.00% eportfolio Sections non-eportfolio Sections The Office of Institutional Research also examined the retention rates of students collegewide. Retention rates from the spring 2010 to fall 2010 semesters were calculated based on the number of eportfolio courses a student was enrolled in during the spring 2010 semester.

11 TCC eportfolio Project Evaluation Report 11 Students enrolled in no eportfolio courses had a spring to fall retention rate of 52.7%. Students in one eportfolio course had a retention rate of 60.9%. Students in two eportfolio courses had a retention rate of 66.2%. Students in three eportfolio courses had a retention rate of 71.4%, and students in four eportfolio courses had a retention rate of 66.7%. Goal 4: In degree programs that use eportfolio in their courses, program coordinators were asked to track the number of students who successfully used the eportfolio to demonstrate achievement of program outcomes and general education outcomes in the capstone course in the program. Ninety-nine percent of students (n=170) successfully used the eportfolio to demonstrate achievement of program outcomes. Ninety-nine percent also used the eportfolio to demonstrate achievement of general education outcomes. During the past two years, student eportfolios were used as part of the Dental Hygiene accreditation process for the Commission on Dental Accreditation. The Early Childhood Education program used eportfolio information as part of its NAEYC accreditation process. The Business Administration program will be including information from student eportfolios as part

12 TCC eportfolio Project Evaluation Report 12 of the accreditation process for the Accreditation Council for Business Schools and Programs. This past spring, the use of eportfolio was highlighted by the Computer Information Systems Program Review Team in their evaluation report as a valuable component of the CIS program, noting the extent to which the CIS program (and Tunxis in general) uses eportfolio and Digication for the purposes of: Providing a means for students to track their own academic progress and acquire a sense of accomplishment. Providing a means for faculty to assess how well students are achieving course and program outcomes. Providing a means for students to collect their work in a format which can be showcased to employers. (CIS Program Review, 2011) In a survey given to program advisory board members (n=25) after each capstone student showcase, 96% strongly agreed or agreed that eportfolio is an effective tool to demonstrate student learning and that eportfolio is an effective tool for students to showcase their work to potential employers or four-year schools, 84% agreed or strongly agreed that eportfolio is an effective tool for programmatic assessment. Additionally, when surveyed on student assessment and outcomes achievement, 91.7% agreed or strongly agreed that students successfully demonstrated achievement of selected program abilities and 84% agreed or strongly agreed that students successfully demonstrated achievement of selected general education abilities. Conclusion As a result of the eportfolio Project, which grew out of Tunxis participation in the Making Connections Mini-Grant and Seminar Program, eportfolio pedagogy is a successful

13 TCC eportfolio Project Evaluation Report 13 teaching, learning and assessment practice at Tunxis Community College. EPortfolios are a valuable tool for students to integrate knowledge, demonstrate learning, and showcase their work for transfer or employment. Evidence gathered to date clearly shows that eportfolio has a positive impact on students and their success at Tunxis Community College. There is an increase in both student success and retention rates in developmental English. There is also an increase in retention rates for students enrolled in eportfolio courses. Student survey evidence suggests that students themselves see the value in using and maintaining their eportfolios. The current level of participation in professional development activities and sustained interest in faculty professional development work centered on eportfolio indicates the value of creating an eportfolio-based teaching and learning community. The number of faculty members using eportfolio in their teaching, and the number of faculty who have created their own eportfolio also indicates the success of these faculty development efforts. Capstone students in eportfolio-based programs are able to use eportfolio to demonstrate achievement of program and general education outcomes. They also use eportfolio to showcase their work for career and/or transfer opportunities. EPortfolio work has been used successfully in the accreditation and program review process as well. The benefit of using eportfolio as a student assessment tool and as an institutional assessment tool is evident. This evidence suggests that Tunxis needs to continue with its efforts to use eportfolio as the primary assessment tool at the College.

14 Tunxis Community College eportfolio Project Evaluation Report submitted by Professor Laura M. Gambino September 30, 2011 Appendix A eportfolio Project Timeline Goal Tasks Person(s) Responsible Faculty Development Introduce new eportfolio platform to Mini-Grant participants from LAS, Department Chairs Existing eportfolio users (full-time and adjunct) in CIS, BOT, DH, ECE, Tech/Eng migrate to new platform Coordinate eportfolio professional development for Mini-Grant participants Digication Training Digication Training Meet with faculty to discuss migration implications Coordinate and schedule 4 2-hour sessions Plan LGCC visit for targeted training Gambino Digication staff Gambino Digication staff Program Coordinators and Gambino Yawin, Terrell, Gambino, eportfolio faculty and students Yawin, Terrell, Gambino Measurable Outcome Completion of 4 hour training and pre and post training survey Completion of 4 hour training and post training survey Faculty feedback regarding migration issues Completion Date Summer 2009 Summer 2009 Summer 2009 Feedback via survey from participants Fall 2009 Feedback from participants Fall 2009

15 TCC eportfolio Project Evaluation Report 15 Create a long-term faculty development plan for eportfolio Project Meet with Center for Teaching (CFT) and Academic Assessment Planner to outline and brainstorm Identification of second LAS cohort Dwyer, Cassidy, Terrell, Yawin, Gambino Gambino, Terrell, Yawin Outline of faculty development plan December LAS faculty identified Fall 2009 Plan and conduct an eportfolio professional day for faculty and staff Rooke, Dwyer, Gambino, Terrell, Yawin, Mini-Grant Participants Completed survey Spring 2010 Introduce new cohort of Program Coordinators (DARC, HS, CJ, FA, VFA, BA) Schedule training sessions for use of eportfolio as program assessment tool Gambino, Carbone, Bencivengo Feedback Spring 2010 Student Support Formalize plan for mentoring program for eportfolio students Facilitate implementation planning for PCs Gambino, Rooke Completed implementation plan for program September 2010 Recruit lab workers MIT, Gambino Hiring of 3-4 student lab workers September 2009 Design and implement technology training for lab workers MIT, Gambino Completed training plan, feedback via various eportfolio faculty and student surveys December 2009

16 TCC eportfolio Project Evaluation Report 16 Technology Maximize technological resources to implement eportfolio project Design and implement leadership/facilitation training for lab workers Develop eportfolio website 1.0 Develop eportfolio tutorials 1.0 Assessment and Evaluation Develop a comprehensive set of assessment tools to measure eportfolio project Other Maximize financial resource opportunities to implement eportfolio project Develop a set of eportfolio entry points for students Design one survey instrument for each component of the project Add eportfolio questions to existing survey instruments (SASE, CCSSE) Gather baseline data and set benchmarks for future assessment Pursue grant opportunities Work with DC/PCs to identify appropriate entry points? Completed training plan, feedback via various eportfolio faculty and student surveys December 2009 Student lab workers, Revillini, existing eportfolio users Launch completed website May 2010 Student lab workers Launch completed tutorials May 2010 Dwyer, England Initial survey instruments completed Summer 2009 Broader- scale survey instruments Spring 2010 completed Dwyer, England Survey updated Fall 2009 Dwyer, England Completed report Fall 2009 Gambino, Rooke,? Acquisition of funding Spring 2010 Gambino Set of designated eportfolio classes Spring 2010

17 TCC eportfolio Project Evaluation Report 17 Showcase student work to internal and external stakeholders Research 3 credit/4 contact hour issue Design an event (or multiple events) to showcase student work Rooke, Gambino Gambino and other PCs Response from system office Fall 2009 regarding feasibility Stakeholder survey Spring 2010