CCMS Guidance for Producing Assessment Material Using Workflow Management for Test Constructor

Transcription

1 CCMS Guidance for Producing Assessment Material Using Workflow Management for Test Constructor If you have any questions regarding the content of this document, contact Last updated: 17/07/17 Version 4

2 Table of Contents 1. Test Constructor Role Stages of the RFT workflow to be completed Receiving and Completing Tasks in CCMS... 4 Collaboration Test Constructor... 4 Stage 5: Check typeset material... 4 Appendix Appendix Page 2 of 15

3 1. Test Constructor Role Test Constructor you will create assessment material(s) from authored items. The purpose of the Test Constructor is to: 1. Critically review authored items to ensure fitness for purpose in terms of validity, reliability, accessibility, comparability and consistency. 2. Construct complete assessment material(s) including the mark scheme/answer keys and assessment grids and check they: are in accordance with instructions provided by the Assessment Standards Senior Managerand Test Coordinator as appropriate are submitted on approved templates and follow OCR house style requirements are fit for purpose i.e. valid, reliable and error free are consistent with the requirements of the specification are consistent with the level of demand evidenced in sample assessment material are consistent with the format and style of assessment evidenced in sample assessment material effectively discriminate across the full ability range are mapped/tagged to assessment objectives/learning outcomes and any other requirements that form part of the assessment model minimise bias in line with the Equality Act protected characteristics take into account all copyright implications and requirements use language which has been made as inclusive as possible include content that is either original material or appropriately sourced, has been produced exclusively for OCR and does not appear in any third party material. 3. Formally sign-off all assessment material(s) and related mark scheme/answer keys and assessment grids that can go forward for typesetting. 4. Proofread, and correct as necessary, the typeset version of the assessment material(s). 5. Record changes to the material(s) in the Audit Log and assist with any queries throughout the production cycle. 2. Stages of the RFT workflow to be completed. Assessors will have access to the material only when their input is required. Collaboration Test Constructor Manager Access: Test Constructor, Assessment Standards Senior Responsibility: Test Constructor: Review Version 1, upload amended Version 2, update relevant forms, you can choose to return to Author or Test Coordinator. Material will continue to loop until you are happy the material can be typeset. Select options when uploading assessment material When material is fit for purpose; submit to OCR Assessment Standards Senior Manager: Review material if needed Check typeset material Responsibility: Page 3 of 15 Access: Test Constructor Test Coordinator (provided on request) Test Constructor: Review typeset version against final version confirmed for typesetting, answer any queries from the typesetter, amend typeset material if needed, upload amended material or approve material Once material is fit for purpose; submit task Test Coordinator: Oversee stage, review amended material if required. Will not make changes.

4 3. Receiving and Completing Tasks in CCMS CCMS log in link: notifications are sent to users on receipt of a new task in the system, they are sent from Documentum Administrator (dmadmin@cambridgeassessment.org.uk). Whenever a new task is received in CCMS the notification is held along with any others generated and then sent out as a collated single at two points during the day at 6am and 6pm. Do not reply to this notification, send s to ocr.ccmsteam@ocr.org.uk. Collaboration Test Constructor Review all materials received from Author. They should have updated the following documents. - Question Paper (and any other items) - Mark Scheme - Assessment Grid - Specification Over Time Grid - Copyright Information Form - Equality Act Form Make any necessary amends and upload the updated documents. Update the Audit log. If you have any comments for the Author or Test Coordinator to consider, or amends you would like them to make, add them to the Audit log. Complete and submit the task. Select one of the following options: o To Author o To Test Coordinator o Material is fit for purpose The last, is to confirm that the paper is ready for typesetting. Before you approve the paper you can send the task to either the Author or Coordinator, then they can send it back to you or to one another, until the material is finalised. As Constructor, only you will have the option to approve the material for typesetting. Therefore, if the Author or Coordinator approves the paper, they will have to submit the task to you, and then you can submit it as fit for purpose. More detailed instructions about completing this stage can be found in Appendix 1. Stage 5: Check typeset material Check the PDF of the Question Paper against the word document of the paper you approved for typesetting. Ignore the front cover, final details to be confirmed at sign off. At this point you are only checking if the content of the material is correct. If amends are needed, mark-up the PDF document using the Adobe stamps provided at your proofreading training (let OCR know if you have not completed the training). Upload the mark-up PDF to CCMS and update the Audit log. Complete and submit your task. Select the relevant option amends are needed or material is fit for purpose. Once amended you will receive a new version of the PDF for approval. More detailed instructions about completing this stage can be found in Appendix 2. Page 4 of 15

5 Appendix 1 Collaboration Test Constructor Step 1 Retrieve documents. Select the Assessment Material tab to access the Question Paper, Mark Scheme and Insert/Resource Booklet uploaded by the Author. Documents will automatically save to your computer Page 5 of 15

6 Select the Forms tab to access all other documents. To review documents choose the Download option, i.e. the Blueprint. Page 6 of 15

7 To make amends to a document choose the Download to Edit option. A key icon will appear next to the document names, indicating that you have locked them for editing, as below. Page 7 of 15

8 Step 2 upload your Second Version Select the Assessment Material tab to upload the Question Paper, Mark Scheme and Insert/Resource Booklet. To upload the second version of the Question Paper and Mark Scheme, select Yes. Page 8 of 15

9 Question Paper Mark Scheme Page 9 of 15

10 Select the Forms tab to upload the documents you have locked for editing, i.e the Audit log or the grids. Step 3 Submit your task. Before you do, check if all documents under the Forms tab have been unlocked, i.e. the key icon not showing. If you see the key you need to upload that document, even if no changes were made. When all files under the Forms tab have been uploaded you will be able to submit the task. is When you submit the task, it will disappear from your task list and progress to the assessor you have selected. Page 10 of 15

11 the Test Coordinator to notify them that you have completed your assessment material, so they are aware the paper has progressed. You have now completed your task and a notification will be sent to the assessor you have selected. Page 11 of 15

12 Appendix 2 Check typeset material You need to have completed the Proofreading training to complete this task, contact the Assessor Training team if you haven t. (Assessor.ManagementTraining@ocr.org.uk) Step 1 Retrieve the documents you need to review. Question Paper PDF typeset version of the paper, which you need to annotate. Question Paper the paper you have approved for typesetting. (if available) QP Comments typesetter s comments for you to consider. Documents will automatically save to your computer. Page 12 of 15

13 Step 2 proofread the PDF of the Question Paper, and then upload the relevant material. Your check is to confirm one of the following: no errors to the assessment material: o Update the Audit log, refer to p.10 of Appendix 1 to download and upload the document. OR amends are required o Update the Audit log, refer to p.10 of Appendix 1 to download and upload the document. o Make the amends onto the typeset PDF version of the assessment material using Adobe and the stamps provided to you. Then upload the annotated assessment material, as per the screenshot below. Page 13 of 15

14 You have now uploaded the annotated PDF. Step 3 Complete and Submit the task. Once amended, you will receive a new version of the PDF for approval. Page 14 of 15

15 You have now completed your task and an will be sent to the OCR CCMS team. Page 15 of 15