banks, test constructional tools, learning systems and assessment delivery systems 10/06/2011

Transcription

1 QTI: IMS Question & Test Interoperability Specification Laura Farinetti Dipartimento di Automatica e Informatica Politecnico di Torino laura.farinetti@polito.it 1 IMS Question & Test Interoperability (QTI) specification Describes a data model for the representation of question (assessmentitem) and test (assessmenttest) data and their corresponding results reports Enables the exchange of this item, test and results data between authoring tools, item banks, test constructional tools, learning systems and assessment delivery systems 2 1

2 IMS Question & Test Interoperability (QTI) specification Uses XML syntax Pedagogy neutral Subject neutral Able to display formatted text, pictures, sound files, video clips, interactive applications or applets 3 IMS Question & Test Interoperability (QTI) specification Supports several types of user response item selection, text input, numeric input, xy-position selection and group selection that can be combined with several different input techniques radio button, check box, text entry box, mouse xy position dragging or clicking, slider bar, 4 2

3 QTI elements Tests are called assessments An assessment has one or more questions Questions plus their associated data are called items Data needed to deliver the question, such as layout rendering information, the score if it is correct, the feedback to be given, 5 QTI specifications AssessmentItem Explains the internal structure of an individual item (i.e. a question) Composed of Assessment several sections Section Section Item Item 6 3

4 QTI specifications AssessmentTest A test is a group of assessmentitems with an associated set of rules that determine which of the items the user sees, in what order, and in what way the candidate interacts with them The rules describe the valid paths through the test, when responses are submitted for response processing and when (if at all) feedback is to be given 7 QTI specifications A test is a logical aggregation of several QTI items Moreover, a test contains instructions on how to create groups of items within a test shuffle the sequence of items presented to the user specify when the candidate may submit answers for response processing provide feedback to the candidate at different points within a test 8 4

5 QTI specifications A test is a logical aggregation of several QTI items Moreover, a test contains instructions on how to create groups of items within a test shuffle the sequence of items presented to the user specify when the candidate may submit answers for response processing provide feedback to the candidate at different points within a test 9 Question anatomy Attributes Statement Stylesheet Variable declaration Choices Logic for marking (response processing) Example

6 Question anatomy Choices SAME logic for marking (response processing) Example 2 11 Question anatomy Feedback Example

7 Question anatomy Data generated automatically according to a logic (template processing) If it takes 32 women 3040 minutes to dig a hole, how long would it take 63 women? 20 minutes. Example 4 13 QTI elements Sections are used to group a series of questions within an assessment E.g. different section for each subtopic, to know the score obtained for Assessment each section as well as over Section the assessment as a whole At the end of the assessment a result report is generated Section Item Item 14 7

8 Item types Multiple choice True false Multiple response Image hot spot Fill in the blank Select textt Slide Drag object, drag target Order objects Match items Connect points 15 AssessmentItem: structure AssessmentItem <AssessmentItem> Attributes </AssessmentItem> Item Variable Declaration Template Processing Stylesheet ItemBody Response Processing ModalFeedback 16 8

9 Attribute Characteristic of an item Composed of A name predefined by the QTI specification (e.g. Title ) A corresponding value (e.g. Test on set associative cache ) supplied by the item author Attribute values are read and written by the systems which process QTI items No additional attribute can be created 17 Attribute Set of attributes predefined by the specification IDENTIFIER - Uniquely identifies item (mandatory) TITLE - A short descriptive title for the item (mandatory) LABEL - A short label l for the item (optional) LANGUAGE - QTI supports different languages (optional) 18 9

10 Attribute ADAPTIVE - Is this item adaptive? (mandatory); adaptive means that it changes appearance or scoring (or both) in response to each of the user attempts; otherwise it is a non-adaptive item (the default) TIMEDEPENDENT - Will time taken to respond to the question affect user score? (mandatory) TOOLNAME - Name of the tool which created the item (optional) TOOLVERSION - Version of the tool which created the item (optional) 19 Attributes - example Example

11 Attributes - example <assessmentitem xmlns=" xmlns:xsi=" instance" xsi:schemalocation=" imsqti_v2p1 imsqti_v2p1.xsd" identifier="choice" title="unattended Luggage" adaptive="false" timedependent="false" > 21 Item Variable Declaration An item variable is an internal variable declared inside a question used to hold information temporarily during the run time processing of an item Three types of item variable response variables: hold answers submitted by the candidate outcome variables: hold scores attained by the candidate template variables: used to generate cloned items 22 11

12 Response Declaration Section Declares one response variable inside the item <responsedeclaration identifier="response" cardinality="single" basetype="identifier"> <correctresponse> <value>choicea</value> </correctresponse> </responsedeclaration> Name of the response variable Can hold one identifier Correct response for this variable 23 Response Declaration Section ASSIGNED during ItemBody Interactions Response Variable DECLARED in Response Declaration REFERENCED inside Response Processing 24 12

13 Outcome Declaration Section Declares one outcome variable inside the item <outcomedeclaration identifier="score" cardinality="single" basetype="integer"> <defaultvalue> <value>0</value> </defaultvalue> </outcomedeclaration> Name of the outcome variable Can hold one integer value Default value for this variable 25 Outcome Declaration Section ASSIGNED during Response Processing Outcome Variable DECLARED in Outcome Declaration REFERENCED inside Modal & Integrated Feedback 26 13

14 Template Declaration Section Declares one template variable inside the item <templatedeclaration identifier="people" cardinality="single" basetype="string" mathvariable="false" paramvariable="false" /> Name of the template variable Can hold a single string value 27 Template Declaration Section ASSIGNED during Template Processing DECLARED in Template Declaration Template Variable REFERENCED inside ItemBody REFERENCED inside Response Processing 28 14

15 Template processing Template processing generates cloned items A template processing section specifies a sequence of template rules which are evaluated by a delivery engine Template rules generate values for one or more template variables These values are used inside the ItemBody section to populate values in the item text 29 Template variables A PEOPLE MIN If it takes 3 women 40 minutes to dig a hole, how long would it take 6 women? 20 minutes. Example 4 Item 53 B PEOPLE RESPONSE (Response variable) 30 15

16 Template processing <responsedeclaration identifier="response" cardinality="single" basetype="float" /> One response <templatedeclaration identifier="people" variable cardinality="single" basetype="string" mathvariable="false" paramvariable="false" /> <templatedeclaration identifier="a" cardinality="single" basetype="integer" mathvariable="false" paramvariable="false" /> <templatedeclaration identifier="b" Four template cardinality="single" basetype="integer" mathvariable="false" paramvariable="false" /> variables <templatedeclaration identifier="min" cardinality="single" basetype="integer" mathvariable="false" paramvariable="false" /> 31 Template processing Assign template <templateprocessing> variable PEOPLE the <settemplatevalue identifier="people"> value men or <random> women or children <multiple> <basevalue basetype="string">men</basevalue> <basevalue basetype="string">women</basevalue> <basevalue basetype="string">children</basevalue> </multiple> </random> </settemplatevalue> <settemplatevalue identifier="a"> <randominteger min="2" max="4" /> </settemplatevalue> Assign template variable A the value 2 or 3 or

17 Template processing If A equals 2 then <templatecondition> Assign template variable B <templateif> the value 4 or 6 or 8 or 10 or 12. <match> <variable identifier="a" /> <basevalue basetype="integer">2</basevalue> </match> <settemplatevalue identifier="b"> <randominteger min="4" max="12" step="2" /> </settemplatevalue> </templateif> <templateelseif> <match> <variable identifier="a" /> <basevalue basetype="integer">3</basevalue> </match> Else If A equals 3 then 33 Template processing <settemplatevalue identifier="b"> <random> Assign template B the value 6 or 12 <multiple> <basevalue basetype="integer">6</basevalue> <basevalue basetype="integer">12</basevalue> </multiple> </random> </settemplatevalue> </templateelseif> <templateelse> <settemplatevalue identifier="b"> <random> <multiple> <basevalue basetype="integer">8</basevalue> <basevalue basetype="integer">12</basevalue> </multiple> </random> Else </settemplatevalue> Assign template B the value 8 or 12 </templateelse> </templatecondition> 34 17

18 Template processing Assign template variable <settemplatevalue identifier="min"> MIN the value (120 / A) <integerdivide> <basevalue basetype="integer">120</basevalue> <variable identifier="a" /> </integerdivide> </settemplatevalue> <setcorrectresponse identifier="response"> <integerdivide> <basevalue basetype="integer">120</basevalue> <variable identifier="b" /> </integerdivide> </setcorrectresponse> </templateprocessing> Assign response variable RESPONSE the value (120 / B) 35 Template processing <itembody> <p> If it takes <printedvariable identifier="a" /> <printedvariable identifier="people" /> <printedvariable identifier="min" /> minutes to dig a hole, how long would it take <printedvariable identifier="b" /> <printedvariable identifier="people" /> to dig a similar hole? </p> <p> Answer: <textentryinteraction responseidentifier="response" expectedlength="8" /> minutes. </p> </itembody> 36 18

19 Template processing The ItemBody section uses these variables to generate a single cloned item Uses the printedvariable element to insert the value of a template value at a particular point in the question text PEOPLE = women A = 3 B = 6 (note: because A=3, B can only take the value 6 or the value 12) MIN = 40 RESPONSE = 20 If it takes 3 women 40 minutes to dig a hole, how long would it take 6 women? 37 Item Body Contains the text, graphics, media objects and interactions that describe the item content and information about how it is structured Specifies The item as seen on screen The interaction(s) between the user and the item 38 19

20 Interaction Allow the candidate to interact with the item Through an interaction, the user selects or constructs a response The user responses are stored in the responsevariables Each interaction ti is associated with (at least) one response variable 39 Internal structure ItemBody blockinteraction choiceinteraction matchinteraction blockstatic <p> <div> custominteraction positionobjectstage 40 20

21 Internal structure blockinteraction blocks the most commonly used block type: action types such as multiple l choice, matching gaps and slider bars (11 types) blockstatic blocks do not support interactions, but specify visual components such as text, pictures and tables custominteraction blocks allow to extend the specification and build new interaction types positionobjectstage blocks blocks support images onto which the user can place other visual objects 41 choiceinteraction Example

22 choiceinteraction Two blockstatic <itembody> blocks <p>look at the text in the picture.</p> <p><img src="images/sign.png" alt="never LEAVE LUGGAGE UNATTENDED" /></p> <choiceinteraction responseidentifier="response" shuffle="false" maxchoices="1"> <prompt>what does it say?</prompt> <simplechoice identifier="choicea">you must stay with your luggage at all times.</simplechoice> <simplechoice identifier="choiceb">do not let someone else look after your luggage.</simplechoice> <simplechoice identifier="choicec">remember your luggage when you leave.</simplechoice> </choiceinteraction> </itembody> One choiceinteraction block If user selects the second option, the response variable RESPONSE is set to ChoiceB 43 choiceinteraction <responsedeclaration identifier="response" cardinality="single" basetype="identifier"> <correctresponse> <value>choicea</value> </correctresponse> </responsedeclaration> The correct answer of the question appears earlier in the item, where the response variable RESPONSE is declared, incorporating the correctresponse element 44 22

23 matchinteraction Example 5 45 matchinteraction Declaration of the item variables <responsedeclaration identifier="response" cardinality="multiple" basetype="directedpair"> <correctresponse> <value>c R</value> <value>d M</value> RESPONSE stores one or <value>l M</value> more pairs of variables <value>p T</value> </correctresponse> <mapping defaultvalue="0"> <mapentry mapkey="c R" mappedvalue="1" /> <mapentry mapkey="d M" mappedvalue="0.5" /> <mapentry mapkey="l M" mappedvalue="0.5" /> <mapentry mapkey="p T" mappedvalue="1" /> </mapping> </responsedeclaration> <outcomedeclaration identifier="score" cardinality="single" basetype="float" /> The mapping element assigns a value to any correct combination (e.g. C R ) entered by the user 46 23

24 matchinteraction <itembody> <matchinteraction responseidentifier="response" shuffle="true" maxassociations="4"> <prompt>match the following characters to the Shakespeare play they appeared in:</prompt> <simplematchset> <simpleassociablechoice identifier="c" matchmax="1">capulet</simpleassociablechoice> <simpleassociablechoice identifier="d" matchmax="1">demetrius</simpleassociablechoice> <simpleassociablechoice identifier="l" matchmax="1">lysander</simpleassociablechoice> <simpleassociablechoice identifier="p" matchmax="1">prospero</simpleassociablechoice> </simplematchset> 47 matchinteraction <simplematchset> <simpleassociablechoice identifier="m" matchmax="4">a Midsummer-Night's Dream</simpleAssociableChoice> <simpleassociablechoice identifier="r" matchmax="4">romeo and Juliet</simpleAssociableChoice> <simpleassociablechoice identifier="t" matchmax="4">the Tempest</simpleAssociableChoice> </simplematchset> </matchinteraction> </itembody> The matchmax attributes places an upper limit of how many associations can be specified against a particular value in a list 48 24

25 Response Processing Allows an item author to embed marking instructions inside an item These instructions are expressed in a high-level language, not tied to any particular delivery tool or scoring engine Similarities between response processing and template t processing Both execute logic written using QTI rules and update item variables 49 Response Processing section Example

26 Response Processing section Declaration of the item variables <responsedeclaration identifier="response" cardinality="ordered" basetype="identifier"> <correctresponse> <value>driverc</value> <value>drivera</value> Ordered list of <value>driverb</value> </correctresponse> identifiers </responsedeclaration> <outcomedeclaration identifier="score" cardinality="single" basetype="integer" /> 51 Response Processing section ItemBody section <itembody> <orderinteraction responseidentifier="response" shuffle="true"> <prompt>the following F1 drivers finished on the podium in the first ever Grand Prix of Bahrain. Can you rearrange them into the correct finishing order?</prompt> <simplechoice identifier="drivera">rubens Barrichello</simpleChoice> <simplechoice identifier="driverb">jenson Button</simpleChoice> <simplechoice identifier="driverc">michael Schumacher</simpleChoice> </orderinteraction> </itembody> 52 26

27 Response Processing section Processing section <responseprocessing> <responsecondition> <responseif> <match> <variable identifier="response" /> <correct identifier="response" /> </match> <setoutcomevalue identifier="score"> <basevalue basetype="integer">2</basevalue> </setoutcomevalue> </responseif> If current value of RESPONSE equals the correct value of RESPONSE then Set SCORE to 2 53 Response Processing section Processing section <responseelseif> <match> <variable identifier="response" /> <ordered> <basevalue basetype="identifier">driverc</basevalue> <basevalue basetype="identifier">driverb</basevalue> <basevalue basetype="identifier">drivera</basevalue> </ordered> </match> <setoutcomevalue identifier="score"> <basevalue basetype="integer">1</basevalue> </setoutcomevalue> </responseelseif> Else if current value of RESPONSE equals the sequence DriverC,DriverB,DriverA then Set SCORE to 1 (i.e. give partial credit for nearly-correct response) 54 27

28 Response Processing section Processing section <responseelse> <setoutcomevalue identifier="score"> <basevalue basetype="integer">0</basevalue> </setoutcomevalue> </responseelse> </responsecondition> </responseprocessing> Else Set SCORE to 0 55 Response Processing Templates Offer an alternative to hard wiring logic inside an item Templates are prewritten segments of response processing which the item author can reference inside a responseprocessing element 56 28

29 Response Processing Templates Example 1 - template 57 Response Processing Templates <responseprocessing template=" question/qti_v2p0/rptemplates/match_correct.xml" t t / t t l" /> <?xml version="1.0" encoding="utf-8"?> <responseprocessing xmlns=" xmlns:xsi=" xsi:schemalocation=" imsqti_v2p0 imsqti_v2p0.xsd"> Template 58 29

30 Response Processing Templates <responsecondition> <responseif> <match> <variable identifier="response" /> <correct identifier="response" /> </match> <setoutcomevalue identifier="score"> <basevalue basetype="integer">1</basevalue> </setoutcomevalue> </responseif> <responseelse> <setoutcomevalue t identifier="score"> <basevalue basetype="integer">0</basevalue> </setoutcomevalue> </responseelse> </responsecondition> </responseprocessing> 59 Feedback The item author can specify feedback at different points during user interaction with an item Three types of feedback Modal Feedback: presented to the user immediately after response processing 60 30

31 Feedback Integrated feedback: integrated into the item itembody users are free to update their responses while viewing integrated feedback not dependent on having submitted an answer for response processing Test Feedback: presented to the user conditionally based on the value of test Outcomes 61 Modal and integrated feedback Integrated feedback Example 3 Modal feedback 62 31

32 Modal and integrated feedback 63 Modal and integrated feedback Declaration of the item variables <responsedeclaration se ec at o identifier="response" e S cardinality="single" basetype="identifier"> <correctresponse> <value>mgh001c</value> </correctresponse> </responsedeclaration> <outcomedeclaration identifier="score" cardinality="single" basetype="float" /> <outcomedeclaration identifier="feedback" cardinality="single" basetype="identifier" /> 64 32

33 Integrated feedback ItemBody section <itembody> <choiceinteraction responseidentifier="response" shuffle="false" maxchoices="1"> <prompt>who is the President of Mexico?</prompt> <simplechoice identifier="mgh001a">george W Bush <feedbackinline outcomeidentifier="feedback" identifier="mgh001a" showhide="show">no, he is the President of the USA. </feedbackinline> </simplechoice> <simplechoice identifier="mgh001b"> Tony Blair <feedbackinline outcomeidentifier="feedback" identifier="mgh001b" showhide="show">no >No, he is the Prime Minister of England. </feedbackinline> </simplechoice> 65 Integrated feedback ItemBody section <simplechoice identifier="mgh001c"> Vicente Fox <feedbackinline outcomeidentifier="feedback" identifier="mgh001c" showhide="show">yes. </feedbackinline> </simplechoice> <simplechoice identifier="mgh001d">ariel Sharon <feedbackinline outcomeidentifier="feedback" identifier="mgh001d" showhide="show">no, he is the Prime Minister of Israel. </feedbackinline> </simplechoice> </choiceinteraction> </itembody> Compare current value of outcome variable called outcomeidentifier with value of identifier. ifi Do their values match? If showhide is set to show AND the values in step 1 match THEN show the feedback message Else hide the feedback message 66 33

34 Modal feedback Response Processing section <responseprocessing> <responsecondition> <responseif> <match> <variable identifier="response" /> <correct identifier="response" /> </match> <setoutcomevalue identifier="score"> <basevalue basetype="float">1</basevalue> </setoutcomevalue> </responseif> <responseelse> <setoutcomevalue identifier="score"> <basevalue basetype="float">0</basevalue> </setoutcomevalue> </responseelse> </responsecondition> <setoutcomevalue identifier="feedback"> <variable identifier="response" /> </setoutcomevalue> </responseprocessing> 67 Modal feedback Modal Feedback section <modalfeedback outcomeidentifier="feedback" t e identifier="mgh001c" showhide="show">yes, that is correct. </modalfeedback> <modalfeedback outcomeidentifier="feedback" identifier="mgh001c showhide="hide">no, the correct answer is Vicente Fox. </modalfeedback> 68 34

35 Stylesheet An item may use one or more external stylesheets to enhance its appearance <stylesheet href="shared/orkney.css" type="text/css" /> <itembody> <div class="rightpane"> <object data="shared/orkney.html" type="text/html" /> </div> <div class="leftpane"> <p>read the text about the Orkney Islands and then decide if the following sentence is correct or incorrect.</p> <choiceinteraction responseidentifier="response shuffle="false" false maxchoices="1"> <prompt>you can fly direct from London to Kirkwall.</prompt> <simplechoice identifier="t">correct</simplechoice> <simplechoice identifier="f">incorrect</simplechoice> </choiceinteraction> </div> </itembody> 69 AssessmentTest: structure AssessmentTest <AssessmentTest> </AssessmentTest> t> Attributes Outcome Declaration TimeLimits TestPart OutcomeProcessing TestFeedback 70 35

36 Attribute Attributes of the whole assessmenttest element IDENTIFIER - Uniquely identifies the test (mandatory) TITLE - Short title of the test (mandatory) TOOLNAME - Name of the tool which created the test t (optional) TOOLVERSION - Version of the tool which created the test (optional) 71 OutcomeDeclaration An assessmenttest element can contain zero, one or more outcome declaration elements An outcome declaration declares one outcome variable whose scope is the entire test Stores a value relevant to the candidate s performance at the test level 72 36

37 OutcomeDeclaration Other outcome variables are used at the item level Item outcome variables generally determine the value of test outcome variables E.g. in a test composed of ten items, the value of a test outcome variable may be calculated as the sum of the values of ten item outcome variables 73 Overall approach to OutcomeDeclaration Declare test outcome variables in the outcome declarations section of assessmenttest For each item response submitted by the candidate First execute the item s response processing (if declared inside assessmentitem): this will update item s outcome variables Then execute test s outcome processing (if declared inside assessmenttest): this will update test s outcome variables End for 74 37

38 OutcomeDeclaration 7 outcome variables <?xml version="1.0" encoding="utf-8"?> <assessmenttest xmlns=" g _ v2p1" xmlns:xsi=" xsi:schemalocation=" imsqti_v2p1 imsqti_v2p1.xsd" identifier="rtest-25" title="basic Statistics as Outcomes"> <outcomedeclaration identifier="ncorrecta" cardinality="single" basetype="integer" /> <outcomedeclaration identifier="ncorrectb" cardinality="single" basetype="integer" /> <outcomedeclaration identifier="nincorrect" i i cardinality="single" basetype="integer" /> <outcomedeclaration identifier="nresponsed" cardinality="single" basetype="integer" /> 75 OutcomeDeclaration <outcomedeclaration identifier="npresented" cardinality="single" basetype="integer" /> <outcomedeclaration identifier="nselected" cardinality="single" basetype="integer" /> <outcomedeclaration identifier="percent_correct" cardinality="single" basetype="float" /> <testpart navigationmode="nonlinear submissionmode="simultaneous"> <outcomeprocessing> </assessmenttest> Sections explained later 76 38

39 TestPart Top-level sections that compose an assessmenttest A testpart contains one or more internal subsections called assessmenttest assessmentsections Allow the test author to disaggregate what would otherwise be a linear set of items testpart assessmentsection assessmentitemref 77 TestPart An assessmenttest contains one or more testparts A testpart contains one or more assessmentsections An assessmentsection may reference one or more assessmentitems, via the assessmentitemref element and/or contain one or more nested assessmentsections 78 39

40 Example A QTI test does not physically contain full items It points to items by specifying their external filenames 79 Functionality associated with Item Groups Item Groups are managed via attributes and elements 80 40

41 TestPart - attributes and elements identifier (mandatory attribute ) Uniquely distinguishes an individual testpart within one assessmenttest navigationmode (mandatory attribute ) determines the sequence in which the candidate must answer items in the testpart linear : the candidate must answer each item one after the other and may not navigate back to a previously answered item nonlinear : the candidate may answer the items in any sequence 81 TestPart - attributes and elements submissionmode (mandatory attribute ) determines when the candidate is permitted to submit item responses individual : the candidate submits the response for one item before answering any other item in the testpart simultaneous : responses for all the items in this testpart are submitted for response processing in one batch at the end of the testpart 82 41

42 TestPart - attributes and elements precondition (optional element) evaluated just before a candidate enters a testpart if a precondition expression evaluates to false, then the associated testpart will not be presented to the candidate more than one precondition can be associated to a testpart the precondition element is also defined in sectionpart 83 TestPart - attributes and elements branchrule (optional element) determines where next? after a testpart has been completed by the candidate if a branchrule evaluates to true, either the candidate will be directed to another nominated testpart or the entire test will be terminated immediately more than one branchrule can be associated with a testpart 84 42

43 TestPart - attributes and elements itemsessioncontrol (optional element) specifies a list of attributes which affect each item session in the testpart attributes:maxattempts, showfeedback, allowreview, showsolution, allowcomment, allowskipping, validateresponses timelimits (optional element) specifies a minimum and/or a maximum time period for the candidate to answer all the items in the testpart 85 TestPart - attributes and elements testfeedback (optional element) specifies test (not item) level feedback, which can either be displayed during or at the end of a testpart more than one testfeedback element can be associated with a testpart Note: all the features apply to just one testpart different testparts within the same AssessmentTest can have different features 86 43

44 SectionPart - attributes and elements Contains attributes and elements which are common to both assessmentsection and assessmentitemref sectionpart stores a number of common parameters shared by both types of child element identifier (mandatory attribute) t uniquely distinguishes the section or the item reference within the overall test 87 SectionPart - attributes and elements required (optional attribute) if set to true means this child element must appear in the final selection fixed (optional attribute) relates to item shuffling within a section if set to true, this element must remain in its original position when the final shuffled sequence is determined precondition (optional element) determines if this item or section will be skipped 88 44

45 SectionPart - attributes and elements branchrule (optional element) contains rule(s) which identify, under certain circumstances, an alternative item or section itemsessioncontrol (optional element) contains parameters to control the allowable states for each item session timelimits (optional element) determines the period of time during which the candidate may answer this item or this section 89 AssessmentSection - attributes and elements title (mandatory attribute) stores a string title,which may or may not be displayed to the candidate visible (mandatory attribute) if set to true, the candidate is aware that this is a distinct section within the overall testpart if set to false, the candidate is not aware of this section s existence per se, and all items inside it will appear to belong to an outer section 90 45

46 AssessmentSection - attributes and elements keeptogether (optional attribute) applies to invisible sections only if set to false, the items in this section may be mixed with items from a parent section if set to true, the items in this section must remain en bloc selection (optional element) determines how many child elements may be chosen by a delivery engine from an assessmentsection 91 AssessmentSection - attributes and elements ordering (optional element) determines the sequence in which the chosen child elements will be presented to the candidate if the Boolean attribute shuffle is set to true, the delivery engine will randomize the sequence in which the items are presented 92 46

47 AssessmentSection - attributes and elements rubricblock (optional element) determines the introductory text displayed for this assessmentsection. the audience of this text is determined by the attribute view, which can take one or more values from a predefined set (e.g. candidate, author, tutor, ) 93 AssessmentItemRef - attributes and elements href (mandatory attribute) includes an item in the test the attribute value should be the URI of the external file containing the item category (optional attribute) associates a user-defined category (e.g. biology ) with an item these categories can be detected and filtered within outcome processing 94 47

48 AssessmentItemRef - attributes and elements variablemapping (optional element) renames, within the scope of a test, a variable declared inside an item weight (optional element) assigns a weighting for this item specifically in the context of this test templatedefault (optional element overrides the default value of a template variable declared by the item based on an expression evaluated at testlevel 95 TestPart -example <?xml version="1.0" encoding="utf-8"?> <assessmenttest xmlns=" xmlns:xsi=" xsi:schemalocation=" imsqti_v2p1 imsqti_v2p1.xsd" identifier="rtest-13" title="steve's Test"> <!-- Provide support for branching. --> <testpart navigationmode="linear" submissionmode="individual"> i di id <itemsessioncontrol showfeedback="true" maxattempts="1" showsolution="false" /> <assessmentsection identifier="sectiona" title="section A" visible="true"> 96 48

49 TestPart -example <rubricblock view="candidate"> <p>instructions for Section A</p> </rubricblock> <assessmentitemref identifier="item034" item034 href="item034.xml"> <itemsessioncontrol maxattempts="0" /> </assessmentitemref> <assessmentitemref identifier="item160" href="item160.xml"> <!-- this item only if the the previous response was identifier Y --> <precondition> <match> <variable identifier="item034.response" /> <basevalue basetype="identifier">y</basevalue> </match> </precondition> </assessmentitemref> 97 TestPart -example <assessmentitemref identifier="item063" href="item063.xml"> <!-- this item only if the the previous response was identifier N --> <precondition> <match> <variable identifier="item034.response" /> <basevalue basetype="identifier">n</basevalue> </match> </precondition> </assessmentitemref> </assessmentsection> 98 49

50 TestPart -example <assessmentsection identifier="sectionb title="section B" visible="true"> <itemsessioncontrol maxattempts="2" /> <rubricblock view="candidate"> <p>instructions for Section B</p> </rubricblock> <assessmentitemref identifier="item347" href="item347.xml"> <!-- skip item653 if response is identifier "N" --> <branchrule target="item656"> t <match> <variable identifier="item347.response" /> <basevalue basetype="identifier">n</basevalue> </match> </branchrule> 99 TestPart -example <!-- special targetitem means exit this section of the test --> <branchrule target="exit_section"> <not> <match> <variable identifier="item347.response" /> <basevalue basetype="identifier">y</basevalue> </match> </not> </branchrule> </assessmentitemref> <assessmentitemref identifier="item653" href="item653.xml" /> <assessmentitemref identifier="item656" href="item656.xml" /> </assessmentsection> </testpart> </assessmenttest>

51 TimeLimits Option to specify the time period available to the candidate for answering questions The time constraints can be applied to Asingleitem A group of items The whole test 101 TimeLimits For each, the test author may specify a timelimits element, which contains two attributes mintime (in number of seconds, integer value) maxtime (in number of seconds, integer value) Either or both of these attributes may be specified

52 TimeLimits TimeLimits element of assessmenttest The total time covers not only the sum of times taken to answer each item, but also any extra time the candidate spends in navigating the screen to move amongst questions A delivery engine can keep track of the time taken to complete a test This value is held in a predefined QTI response variable called duration This variable value can be referenced inside outcome processing logic 103 TimeLimits <?xml version="1.0" encoding="utf-8"?> <assessmenttest xmlns=" xmlns:xsi=" xsi:schemalocation=" imsqti_v2p1 imsqti_v2p1.xsd" identifier="rtest-09" title="steve's Test"> <!-- Provide support for controlling the duration of an item attempt (both maximum and minimum) in the context of a specific i test. -->

53 TimeLimits <testpart navigationmode="linear" submissionmode="individual"> <itemsessioncontrol i t showfeedback="true" " maxattempts="1" showsolution="false" /> <timelimits maxtime="3600" /> <assessmentsection identifier="sectiona" title="section A" visible="true"> <rubricblock view="candidate"> <p>instructions for Section A</p> </rubricblock> <assessmentitemref identifier="item034" href="item034 item034.xml xml"> <itemsessioncontrol maxattempts="0" /> <timelimits mintime="180" maxtime="600" /> </assessmentitemref> 105 TimeLimits <assessmentitemref identifier="item160" href="item160.xml" /> <assessmentitemref identifier="item063" href="item063.xml" /> </assessmentsection> <assessmentsection identifier="sectionb" title="section B" visible="true"> <itemsessioncontrol maxattempts="2" /> <timelimits maxtime="7200" /> <rubricblock view="candidate"> <p>instructions for Section B</p> </rubricblock> <assessmentitemref identifier="item434" href="item434.xml" /> <assessmentitemref identifier="item065" href="item065.xml" /> <assessmentitemref identifier="item365" href="item365.xml" /> </assessmentsection> </testpart> </assessmenttest>

54 OutcomeProcessing Specifies logic which assigns values to one or more test outcome variables Outcome processing takes place each time the candidate submits the responses for an item, when in individual submission mode or a group of items, when in simultaneous submission mode 107 OutcomeProcessing The values assigned to outcome variables can be used to display test-level (not item-level) feedback to the candidate branch to another section or item within the assessmenttest

55 OutcomeProcessing - example <outcomedeclaration identifier="ncorrecta" cardinality="single" basetype="integer" /> <outcomedeclaration identifier="ncorrectb" cardinality="single" basetype="integer" /> <outcomedeclaration identifier="nincorrect" cardinality="single" basetype="integer" /> <outcomedeclaration identifier="nresponsed" cardinality="single" basetype="integer" /> <outcomedeclaration identifier="npresented" cardinality="single" basetype="integer" /> <outcomedeclaration identifier="nselected" cardinality="single" basetype="integer" /> <outcomedeclaration identifier="percent_correct" cardinality="single" basetype="float" /> 109 OutcomeProcessing - example <testpart navigationmode="nonlinear" submissionmode="simultaneous"> Execute once <assessmentsection identifier="sectiona" title="section A" visible="true"> <rubricblock view="candidate"> <p>instructions for Section A</p> </rubricblock> <assessmentitemref identifier="item034 href="item034.xml" /> <assessmentitemref identifier="item160" href="item160.xml" /> <assessmentitemref identifier="item063" i i i href="item063.xml" /> </assessmentsection>

56 OutcomeProcessing - example <assessmentsection identifier="sectionb" title="section B" visible="true"> <selection select="2" /> <rubricblock view="candidate"> <p>instructions for Section B</p> </rubricblock> <assessmentitemref identifier="item434" href="item434.xml" /> <assessmentitemref identifier="item065" href="item065.xml" /> <assessmentitemref identifier="item365" href="item365.xml" /> <assessmentitemref identifier="item656" href="item656.xml" /> </assessmentsection> </testpart> 111 OutcomeProcessing - example <outcomeprocessing> <setoutcomevalue identifier="ncorrecta"> <numbercorrect sectionidentifier="sectiona" tifi /> </setoutcomevalue> <setoutcomevalue identifier="ncorrectb"> <numbercorrect sectionidentifier="sectionb" /> </setoutcomevalue> <setoutcomevalue identifier="nincorrect"> <numberincorrect /> </setoutcomevalue> <setoutcomevalue identifier="nresponsed"> <numberresponded /> </setoutcomevalue> <setoutcomevalue identifier="npresented"> <numberpresented /> </setoutcomevalue> Predefined QTI functions

57 OutcomeProcessing - example <setoutcomevalue identifier="nselected"> Predefined d QTI <numberselected /> functions </setoutcomevalue> <setoutcomevalue identifier="percent_correct"> <product> <basevalue basetype="float">100.0</basevalue> <divide> <numbercorrect /> 100 * (numbercorrect / <numberselected /> </divide> ide> numberselected) </product> </setoutcomevalue> </outcomeprocessing> 113 TestFeedBack Components at the assessmenttest level access (mandatory attribute) during : test feedback is shown immediately after response processing atend : test feedback appears only once, after submission of a response to the last item attempted t in the test t outcomeidentifier (mandatory attribute) refers to a previously declared outcome variable

58 TestFeedBack showhide (mandatory attribute) show or hide identifier (mandatory attribute) used in conjuction with the outcomeidentifier and showhide if the value of this attribute is included in the value of outcomeidentifier, and the showhide attribute is set to true, then feedback will be shown otherwise feedback will not be shown 115 TestFeedBack title (optional attribute) can be displayed to the candidate flowstatic (zero or more elements) contain the actual test feedback to be rendered to the candidate fixed text or, more likely, information based on responses submitted so far

59 More examples Multiple choice question Example

60 Association question Example Text entry question Example

61 Hot text question Example Hot spot question Example 11 Item

62 Select point question Example 12 Item