Making e-assessment Work A L E A N M A N A G E M E N T A P P R O A C H. Rogō Sy s t em Manager

Size: px

Start display at page:

Download "Making e-assessment Work A L E A N M A N A G E M E N T A P P R O A C H. Rogō Sy s t em Manager"

James Small
5 years ago
Views:

1 Making e-assessment Work A L E A N M A N A G E M E N T A P P R O A C H Dr Simon Wilkinson Rogō Sy s t em Manager

2 Presentation History Performance Management Lean Management Open Source Exam Lifecycle

3 History formative True/False question 2005 first Summative Exam 2009 first video used 2010 first audio used 2011 Rogō becomes open source ,000 exam papers delivered online

4 Performance Management THE 4Ps OF PERFORMANCE

5 Purpose Are students fit for purpose? Are assessments fit for purpose? Is the IT fit for purpose?

6 People Academics Administrators External Examiners Students

7 Process Writing Reviewing Answering Reporting Feedback

8 Performance!

9 Lean Management REMOVING WASTE

10 What is Lean? The core idea is to maximize customer value while minimizing waste.

11 People Academics Administrators External Examiners Students

12 What is Lean? Understand customer value Example Identify questions which need to be modified

13 What is Lean? Understand customer value Value stream analysis Example Send out s Receive s Save Attachments Create Word doc Cut & Paste Read Comments

analysis Create flow Example Send out Emails

14 What is Lean? Understand customer value Value stream analysis Create flow Example Send out s Receive s Save Comment Attachments Online Create Word doc Cut & Paste Read Comments

15 What is Lean? Understand customer value Value stream analysis Create flow Customer pull Example Report compiled when needed

16 What is Lean? Understand customer value Value stream analysis Create flow Customer pull Pursue perfection Example Shortcut added to edit questions

17 Open Source DEVELOPMENT MODEL

18 Open Source Software Free license Source code is included Anyone can modify the code Modified versions can be redistributed public function compile_report($recache) { $results_cache = new ResultsCache($this if ($recache or $results_cache->should_ $this->recache = true; } else { $this->recache = false; } $moduleid = Paper_utils::get_modules($t $this->moduleid_in = implode(',', array $this->exclusions->load(); $this->load_answers(); $this->set_log_late(); $this->load_absent(); $this->find_users(); $this->load_metadata(); $this->load_overrides(); $this->load_results(); $this->adjust_marks(); $this->add_rank(); $this->convert_moduleids(); $this->flag_subpart();

19 Successful OSS Systems Linux Firefox MySQL Android Apache Moodle

20 Rogō To create, as a community, a leading e-assessment management system that is scalable, secure, useable and extensible. rogo-oss.nottingham.ac.uk

21 Exam Lifecycle MANAGEMENT IN PRACTICE

22 Exam Lifecycle Question writing Blueprinting Internal peer review External Examiner review Standards Setting

23 Exam Lifecycle Question writing Blueprinting Internal peer review External Examiner review Standards Setting Questions presented Answers recorded Crash Recovery

24 Exam Lifecycle Question writing Blueprinting Internal peer review External Examiner review Standards Setting Questions presented Answers recorded Crash Recovery Post-Exam Analysis of performance Fix incorrect answers Exclude poor questions Export marks Student feedback

25 Exam Lifecycle Question writing Blueprinting Internal peer review External Examiner review Standards Setting Questions presented Answers recorded Crash Recovery Post-Exam Analysis of performance Fix incorrect questions Exclude poor questions Export marks Student feedback Refine Items

26 Problem Authentication Post-Exam What happens when student forgets password?

27 Solution 1 Authentication Post-Exam Password lists Mr Bilbo Baggins h0bb1t Mr Bugs Bunny wh1te46 Mr Daffy Duck green26 Miss Piggy p1ggy2

28 Solution 2 Authentication Post-Exam Paper guest accounts User Password Name user1 passwd1 user2 passwd2 user3 passwd3

29 Solution 3 Authentication Post-Exam Online guest accounts

30 Problem Exam Scheduling Post-Exam How to handle exam scheduling?

31 Solution Exam Scheduling Post-Exam

32 Solution Exam Scheduling Post-Exam

33 Solution Exam Scheduling Post-Exam

34 Problem Accessibility Post-Exam How can students with special needs be accommodated?

35 Solution Accessibility Post-Exam Interface support User profiles

36 Problem Poor Questions Post-Exam How can questions be excluded quickly? Paper pass mark re-calculated? Audit trail of the exclusion?

37 Solution Poor Questions Post-Exam

38 Solution Poor Questions Post-Exam

39 Solution Poor Questions Post-Exam Student marks adjusted Paper total adjusted Pass Mark adjusted Audit trail

40 Problem Standard Setting Post-Exam How to speed up the Hofstee process?

41 Solution Standard Setting Post-Exam

42 Problem Feedback Post-Exam Did I pass? What is my score? What did I get right/wrong? How do I compare? Students

43 Problem Feedback Post-Exam Question reuse is good practice! Avoid questions in public domain! No time to rewrite paper each year! No time for personal tutorials! Staff

44 Solution Feedback low score Post-Exam

45 Solution Feedback mid score Post-Exam

46 Solution Feedback good score Post-Exam

47 Solution Performance Summary Post-Exam

48 Conclusion Purpose Process Open Source People

Polytechnic e-learning: LMS Manual - LECTURER

Polytechnic e-learning: LMS Manual - LECTURER JPP & USTM 2014 TABLE OF CONTENTS 1. ACCESSING LMS... 1 2. LOG IN... 1 3. MANAGE STUDENT SELF-ENROLMENT... 1 4. MANAGE COURSE LANDING PAGE... 1 5. MANAGE COURSE