ACER Online Assessment and Reporting System (OARS) User Guide

Transcription

1 ACER Online Assessment and Reporting System (OARS) User Guide January 2015

2 Contents Quick guide... 3 Overview... 4 System requirements... 4 Account access... 4 Account set up... 5 Create student groups (tags)... 5 Create student accounts (candidates)... 6 Administering tests... 9 Assigning tests... 9 Un-assigning unstarted tests Candidate log in Test navigation Reopening tests Closing tests Generating reports Group report Report tools Sorting group reports Scale report Individual reports Viewing question content directly from a report Sharing reports Staff accounts Edit student information in bulk P age OARS System User Guide January 2015

3 Quick guide Purchase assessments Set up account Administer tests Generate reports Create staff accounts Manage account over time 3 P age OARS System User Guide January 2015

4 Overview Welcome to ACER online assessment and reporting, a time-effective alternative to paper tests and surveys. Tests are administered online and reports are available instantly once tests are completed. System requirements Minimum screen resolution of 1024x768 DSL or cable internet connection minimum 56 kbps Compatible computers and devices PC / laptop Apple Mac ipad Other tablet devices Compatible browsers Internet Explorer version 8 and above (with compatibility mode switched off) Google Chrome Mozilla Firefox Safari Account access Each school has a unique web address for online testing, which looks similar to this example - All teachers and students at the school log in at the same web link. The unique web address was ed to the account administrator upon registration, and again when the account was approved by ACER. The account administrator username and password were nominated by the person who entered the account details at the time of registration. TIPS Save the unique web address as a bookmark or shortcut. Add the link to the school s intranet page for easy student access. If the link is misplaced, find the URL by using the Find My School service at and remember to bookmark the web address once it appears on-screen. 4 P age OARS System User Guide January 2015

5 Account set up Once assessments or surveys are purchased, to prepare for testing begin by entering student information into the account. Create student groups (tags) The optional system for grouping students in ACER s online assessment and reporting system is called tagging. Tags are only necessary to identify characteristics students have in common besides year levels as year level information is recorded separately in the system. Tags can be used to report on particular groups of students after their tests are completed, and they can be created and assigned to students at any time. Students can be tagged with as many groups as required. Tag names can be entered in any format. Common tags are classes and extension groups. To create tags, go to the Tags Manage page. Enter a tag name into the new tags box then click create or press enter on the keyboard. The tag will be added to the box of tags for the school as shown above. To add more than one tag to the account at the same time, type multiple tag names with a comma (,) between each tag - e.g. Class A, Class B. 5 P age OARS System User Guide January 2015

6 Create student accounts (candidates) Students must be added to the account as candidates to sit tests. There are two ways to create candidates from the Candidates Create page, in bulk OPTION 1 or individually OPTION 2. OPTION 1 bulk candidate upload 1. Ensure any required tags (optional) have been added to the account as outlined in the section Create student groups (tags). 2. Download the *.xls Excel template. The spreadsheet template contains two examples as shown below, which should be removed from the file after observing their format. 3. Fill in each column of the template, using the guidelines below. The information can be copied and pasted from a school database file. Column A Family Name Column B Given Name Enter each students surname(s). Apostrophes, hyphens and spaces are allowed. Enter each students first name(s). Apostrophes, hyphens and spaces are allowed. 6 P age OARS System User Guide January 2015

7 Column C Username Column D Password Column E DOB Column F Gender Column G Tags Column H Unique ID Column I Year Level Enter a unique username for each student. Apostrophes and spaces are not allowed. Student ID numbers are recommended. Can be an address. Enter a password for each student that is eight characters or longer. Students can be allocated the same password as their usernames are unique. Alternatively, each password can be unique or groups of students can share a password. Leave blank to create a unique random ten-letter password for each student. Enter students date of birth. The format must be DD/MM/YYYY or DD-MM-YYYY. Enter students gender. The format can be full words (Male/Female and male/female) or single letters (M/F and m/f). Entering data into this column is optional. Tags to assign to each student. Separate multiple tags with a comma (,) e.g. Year 2, Class 2B, Extension. Tag names must be an exact match to the tags created in the account, including spaces and letter case. If not being used leave the column blank but do not delete the heading Tags at the top of the column. Entering data into this column is optional. It can be used to record another identifier for each student, such as an ID number from a different database system this information can then be included in report exports spreadsheets after tests are sat. If not being used, leave the column blank but do not delete the heading Unique ID at the top of the column. Enter one of the following: Prep, Year 1, Year 2, Year 3, Year 4, Year 5, Year 6, Year 7, Year 8, Year 9, Year 10, Year 11, Year 12, Staff. Staff should be used for any teacher/dummy logins. 4. Save the file using the save as option in Excel, and the correct file type will be set by default, which is *.xls do not change the file type. Name the file something relevant (e.g. TrialCollegeYear2.xls), and save it to a folder or location which will be accessible later. 5. At the Candidates Create page, click on browse or choose file. Locate the file saved at step 4 and click open. To finish, click to upload to import the file to the account. OPTION 2 individual candidate entry 1. Fill out the form with a student s details. All fields besides Unique ID are mandatory. Given Name Family Name Username Password Year Level School Year enrolled DOB Gender Unique ID Tags Student's first name(s). Apostrophes, hyphens and spaces are allowed. Student s surname(s). Apostrophes, hyphens and spaces are allowed. Unique combination of letters, numbers, hyphens and underscores. Can be an address or student ID number. Must not contain apostrophes (') or spaces. Eight or more of any combination of numbers, letters or symbols. Leave blank to create random ten-letter password. Password can be the same for all candidates. Enter one of the following: Prep, Year 1, Year 2, Year 3, Year 4, Year 5, Year 6, Year 7, Year 8, Year 9, Year 10, Year 11, Year 12 or Staff. Staff can be used for any staff and dummy logins. The School Year enrolled selection will lock to the current year for all new students that are created. Format must be DD-MM-YYYY or DD/MM/YYYY, the year must be four digits. Tick male or female. Unique identifier such as student ID number. This field is optional and can be left blank. Tick one or more tags to assign to the student. This is optional. 7 P age OARS System User Guide January 2015

8 2. Once all required fields are entered, click create. TIPS To assign tags to students after they have been created as candidates, go to the Tags Assign page. At step 1 tick the tag to be assigned, at step 2 tick the boxes next to the relevant candidates that require the tag then click assign. Tags can be unassigned from candidates in bulk from the Tags Unassign page, or individually from the Candidates View page - by clicking on a candidate s name and scrolling down the page to their tags. January 2015

9 Administering tests Once candidates have been created in the account, they must have a test assigned to their login. Assigning tests Tests are assigned to candidates from the Tests Assign page. If unsure which test to assign, information about which test levels are appropriate for each year level can be found under the relevant section of the Help menu. A test preview button is available on the Tests Assign page once a test level has been selected from the drop down menu (see below). 1. Select the test and test form (level) to assign from the two drop-down boxes. 2. Click the search button next to Year Level, or the search button next to Tags. If searching by year level, select the relevant year level from the dropdown list and then click show candidates the candidate list will update to only show students in the year level selected. If searching by tag, tick the box next to the relevant tag of students, and then click show candidates the candidate list will update to only show students in the tag or year level that was selected. Click select all candidates (or, tick the boxes next to the relevant students of the group selected if not assigning this test to all candidates in the year level or tag selected) 3. Click unassign. Once assigning is complete, candidates can log in and sit their tests. Select the relevant test and test level Click to access a preview of the test selected. Click to search by tag or by year level Select the candidates to assign the test to, and click assign 9 P age OARS System User Guide January 2015

10 TIP Repeat steps 1 and 3 if assigning more than one test to the same group - e.g. Maths and Comprehension. Un-assigning unstarted tests* Unstarted tests can be unassigned in bulk from the Tests Unassign page. * If started or complete tests need to be deleted, oars@acer.edu.au with the details of the request. 1. Select the test and test form (level) to unassign from the first two drop-down boxes. 2. Select not started from the status drop down box. Another status (started or complete) can be selected here to view the lists of candidates, but not to unassign the tests. 3. Select a date or date range if relevant (e.g. if a test was incorrectly assigned to some students today but correctly assigned to others yesterday, choose today s date), otherwise leave as all. 4. Click the search button next to Year Level, or the search button next to Tags. If searching by year level, select the relevant year level to unassign tests from at the dropdown list then click show candidates - the candidate list will update to only show students in the year level selected. If searching by tag, tick the box next to the relevant tag of students to unassign the tests from, and then click show candidates the candidate list will update to only show students in the tag that was selected. 5. Click select all candidates (or, tick the boxes next to the relevant students of the group selected if not unassigning this test from all candidates in the year level or tag selected). 6. click unassign. Select the relevant test and test level Select not started Select the candidates to unassign the test from, then click unassign 10 P age OARS System User Guide January 2015

11 Candidate log in Candidates log in to sit tests or surveys from the school s unique web address. Prior to testing dates, have the unique URL added to the school s intranet page or as a bookmark on computers or devices for easy student access. A list of candidate usernames and passwords can be generated from the Candidates View page by following the steps below. 1. Click the search button next to Year Level, or the search button next to Tags. If searching by year level, select the relevant year level from the dropdown list then click show candidates the candidate list will update to only show students in the year level selected. If searching by a tag, tick the box next to the relevant tag of students, and then click show candidates the candidate list will update to only show students in the tag that was selected. Click select all candidates (or, tick the boxes next to the relevant students of the group selected if not assigning this test to all candidates in the year level or tag selected) then click assign. 2. Click show login details and then export login details as a spreadsheet. Once candidates log in with their username and password, a page with assigned tests will be visible. Advise each student to check that their own name appears in the top right hand corner of the page (as students may have similar logins). Instruct students to click on the arrow next to the name of the appropriate test to be sat, and directions will appear on the next page. Read the administration directions to students, to ensure they understand the test process and have an opportunity to ask questions before commencing. When all students are ready to begin, instruct them to click the start test button. 11 P age OARS System User Guide January 2015

12 This is the directions screen for PAT-R Comprehension TIP Detailed teacher administration instructions for each test are available under the Help menu. Test navigation There are two ways candidates can navigate through their test. 1. Click Next to move to the next question. 2. Use the progress indicator bar in the top right corner of the screen as navigation. The question boxes in the progress indicator bar will change colour when the question has been answered. Questions can be skipped and returned to later. Unanswered questions will remain unshaded in the progress indicator bar. If the text is longer than the screen can display, candidates will be able to use the scroll bar on the right hand side of the screen to view remaining text. 12 P age OARS System User Guide January 2015

13 To finish the test, candidates can either click End (located in the progress indicator bar), or click Next on the final page of the test. Before the test closes completely the following screen will appear, which indicates the number of unanswered questions. Clicking finish the test will submit the candidate s answers to the database, and their results will be scored by the reporting component. Once a candidate clicks finish the test they can t access their test questions again. If for any reason a candidate needs to exit their test (fire drill or other emergencies), instruct them to close the browser/window without clicking finish. When they log in next they will be returned to the question they were working on. Reopening tests If a candidate accidentally clicks finish the test, the account administrator can reopen the test by following the steps below. 1. Navigate to the Candidates View page. Search for the candidate s name and click on it. 2. Scroll down the page to view the candidate s completed tests, and click reopen next to the relevant test sitting. A pop-up box will appear, asking for confirmation that the test should be reopened click OK. The candidate will be able to complete the test when they log in next, and their previously chosen answers will still be selected. Closing tests If a candidate does not submit their test, the results aren t able to be reported on. If a candidate forgets to click finish, administrators can close the test by following the steps below. 3. Log into the school s account and navigate to the Candidates View page. Search for the candidate s name and click on it. 4. Scroll down the page to view the candidate s started tests, then click close test next to the relevant test sitting. A pop-up box will appear, asking for confirmation that the test should be closed click OK. Staff will be able to report on the test results once this is done, and the candidate will no longer be able to log in and answer any further questions. 13 P age OARS System User Guide January 2015

14 Generating reports Once candidates submit their test answers a report can be generated. Click on Reports from the menu. If administering ewrite or Social-Emotional Wellbeing Survey (SEW), refer to the test specific report documents located under the Help menu. On Reports page select the following. 1. The test to report on (e.g. PAT-R Comprehension) 2. The test level to report on (e.g. Test 3) 3. The year level to norm the candidates against (e.g. Year 3)* 4. The tags/candidate groups to report on (e.g. Year 3). If not reporting on any particular group of candidates, do not select a tag. To report on multiple tags, change the drop-down box selection from all to one or more. 5. The reporting period (this is optional and not relevant if candidates have only sat the test level once). Click report to load the results a group report will generate first. *The year level selected determines which norm reference group candidates will be compared to. This affects stanine and percentile rank report data, but not scale scores. When choosing which year level to compare candidates to, consider the month in which the norm data was collected refer to the Help page for this information. 14 P age OARS System User Guide January 2015

15 Group report The group report contains the following information. 1 Candidates 2 Raw Score 3 Scale Score 4 Stanine 5 Percentile Rank 6 Question Difficulty 7 Question Strand 8 Percentage Correct The students chosen answers are displayed under each question number. Correct answers are represented with green ticks, incorrect answers are coloured red with the incorrect option chosen by the student displayed, and NA represents any questions students did not answer. Test question content is accessible in a pop-up window by clicking on a question number. For more detailed information about this feature, see the section Viewing question content directly from a report Report tools Click to change the test or tag to report on. Click the first icon to view a question percentage correct graph. Click the second icon to view a stanine frequency graph. Click the first icon to export the group report as an Excel spreadsheet file (*.xls). Click the second icon to download an individual report batch in PDF. Click to access a scale score report. 15 P age OARS System User Guide January 2015

16 Sorting group reports Group reports can be sorted based on the following information. Date Candidate name Gender Raw score Scale score Stanine Percentile rank Question number Question difficulty Question classification Percentage correct To sort, click on the arrows located next to the relevant information The report can be changed by clicking the test name. The norm reference group students are compared to can be changed from the drop-down box. After a new selection the change is applied instantly. 16 P age OARS System User Guide January 2015

17 Scale report The scale report shows the distribution of candidates scale scores along the relevant PAT scale (there is a different scale for each PAT area, e.g. the PATc scale is for PAT-R Comprehension). Each candidates scale score is plotted as a single dot. Hold the mouse cursor over a dot to see the student name, scale score and test sitting date it corresponds to. The table below the scatter plot shows the name of each candidate in the report. The shaded rows represent achievement levels that are 10 scale scores wide. Click on a symbol in a row (or an area in the scatter plot) to see a pop-up description of the skills students typically demonstrate at that achievement level. The PAT Resources Centre (an extra online teaching content resource) is based on these achievement levels, or bands. Note that placement in an achievement level is approximate as there is an error of 3 scale scores or more around each individual scale score. The box and whiskers plots show the norm distribution (based on the national sample) for each year level on the PAT scale. The box shows the middle 50% of students in the norm distribution. The white dash in the box is the mean score for that year level. The bottom whisker extends to the bottom 5% and the top whisker extends to the top 95% of the norm distribution. Click on a year level and the norm distribution will appear in yellow across the scale. Click to purchase PAT Resources (if logged in as an account administrator), or to access the Resource Centre if already purchased Norm distribution plots Click on an achievement level to filter the students shown in the scatter plot. All is set by default. The number in the gray circle indicates the total students in that band. Search for a candidate s name here 17 P age OARS System User Guide January 2015

18 Individual reports To view an individual candidates report, click on a candidate name from the group report. Most difficult question Easiest question Green circles represent correct responses, and red squares indicate incorrect ones. Click on each circle or square to view the question content in a pop-up box. The red dotted line shows the student s scale score, and the gray area around the scale score represents the error margin. To view the individual report grouped by classification, rather than the default (unit grouping), change the selection from the group on drop-down box. The position of the question indicates the scale difficulty level the lower the circle or box, the easier the item; and the higher the circle or box, the more difficult the item. An individual report can be converted to a PDF for printing, by clicking view this report in PDF. There is also a PDF icon on the group report page to download a PDF batch of all individual reports in the group. TIPS For more information on interpreting PAT results, refer to the document titled Interpreting ACER Test Results on the Help page. A link to a series of short videos to help answer some of the frequently asked questions about the PAT Series is available on the Help page. 18 P age OARS System User Guide January 2015

19 Viewing question content directly from a report When a question number is clicked on from a group or individual report*, a pop-up box showing the question content appears. The pop-up box also provides statistics on how candidates in the report answered this question. The first column is the number of candidates in the report that chose that option. The second column provides the same figure as a percentage. The option outlined in blue indicates the correct answer. Also listed in the pop-up box are the question difficulty, question classification and skill descriptor based on the PAT scale. Navigate through the test questions by using the navigational tool at the top right corner of the pop-up box. To exit the question pop-up and return to the group report, click anywhere in the dark green area of the screen, or click the X in the top right hand corner. *Question content is inaccessible via a shared report. 19 P age OARS System User Guide January 2015

20 Sharing reports Administrators have the option to share reports with others via a web link. The link allows another person to access the same report without logging into the school s account. Note that share report links expire one week after the report is generated, and test question content is inaccessible from all share report links. Staff that require access to question content also, or more frequent access to reports can be set up as Report Generators refer to the staff accounts section for setup instructions. On the Reports page, to generate a link to share a report with others tick the share report box before loading the results. The web link to share the report will appear at the top of the group report that generates. 20 P age OARS System User Guide January 2015

21 Staff accounts Account administrators can assign access rights to other staff, enabling them to manage candidates or view reports using a separate username and password to the account administrator. There are two ways to add staff logins at the Staff Create page, in bulk OPTION 1 or individually OPTION 2. OPTION 1 bulk staff upload 1. Download the *.xls Excel template. The spreadsheet template contains two examples as shown below, which should be removed from the file after observing their format. 2. Fill in each column of the template, using the guidelines below. The information can be copied and pasted over from another file. Column A Name Enter each staff members surname(s). Apostrophes, hyphens and spaces are allowed. 21 P age OARS System User Guide January 2015

22 Column B Column C Username Column D Password Column E Roles Column F Tags Column G Year Level Enter each staff members address. Enter a unique username for each staff member. Apostrophes ( ) and spaces are not allowed. Usernames must be unique across the account between all candidates and other staff members. Enter a password for each staff member that is eight characters or longer. Staff can be allocated the same password as their usernames are unique. Enter a role(s) to assign to each staff member. The roles to choose from are Candidate Manager, Report Generator or Candidate Manager and Report Generator. Separate multiple roles with a comma (,). Enter tags to assign to each staff member, which will determine which candidates they have access to. Separate multiple tags with a comma (,) e.g. Year 2, Class 2B, Extension. Tag names must be an exact match to the tags created in the account, including spaces and letter case. Enter Year Level to assign to each staff member, which will determine which candidates they have access to. Please follow the format provided for specifying Year Levels. 3. Save the file using the save as option in Excel, and the correct file type will be set by default, which is *.xls do not change the file type. Name the file something relevant (e.g. TrialCollegeStaff.xls), and save it to a folder or location which will be accessible later. 4. At the Staff Create page, click on browse or choose file. Locate the file saved at step 4 and click open. To finish, click to upload to import the file to the account. OPTION 2 individual entry 1. Navigate to the Staff Create page, and enter the following information for a staff member: Contact name Contact Username Password Password (again) Enter the staff member s name Enter the staff member s address. Enter a username for the staff member. It must be something unique to the account across all staff members and candidates. Enter a password for the staff member that is eight characters or longer. Enter the same password again. 22 P age OARS System User Guide January 2015

23 2. Tick a box to assign a role to the staff member. Selecting a role is mandatory. The role selected here defines the functionality the staff member has access to. Each role is described below. Candidate Manager: Report Generator: This staff member can create candidates and tags, and assign tests, but not view reports. This user has immediate access to any candidates that they create. The administrator of the school s account can allow this user access to other candidates by assigning them to the staff member. This staff member can only view reports on candidates assigned to them. The administrator of the school s account needs to assign candidates to the staff member. Candidate Manager and Report Generator: This staff member can create candidates and tags, assign tests and view reports. They have immediate access to any candidates that they create. The administrator of the school s account can allow this staff member access to other candidates by assigning them. 3. Once a role is selected, select a tag or Year Level to assign to the staff member. This sets out which candidates they have access to. Tick the box next to a tag name and the tag will move up the page. Multiple tags can be selected. If a mistake is made, untick the tag and it will move back down into the tags section. September 2014

24 Tick a tag to assign to the staff member Once ticked, the tag will move to this section 4. Once all information has been entered, click create. 5. Remember to provide the staff member with their username and password, as well as the unique web address to log in at. Repeat the process to create a login for another staff member. TIPS Account administrators have access to all candidates and all reports by default. Once a tag is assigned to a staff member, if candidates are added or removed from the tag later on, the staff member s access is automatically updated. 24 P age OARS User Guide September 2014

25 Edit student information in bulk After students have been added to the account as candidates, in time their details may need to be edited. If there are multiple students that require changes, this can be completed in bulk from the Candidates Edit page, by following the guidelines below. 1. To edit every candidate in the list, skip straight to step 2. If only editing particular groups of students, tick the boxes next to the appropriate tags for these candidates. 2. Click download file to download an Excel (*.xls) spreadsheet of the candidates current details. The exported file includes candidates Family Name, Given Name, Username, Password, Date of Birth, Gender and Tags all of which can be edited. A system id for each student appears in the A column, do not edit the information in this column (it is used by the system to ensure the correct student s details are edited). 3. Make the changes required; ensuring the candidate format guidelines below are followed. 25 P age OARS User Guide September 2014

26 Column A System ID Column B Family Name Column C Given Name Column D Username Column E Password Column F DOB Column G Gender Column H Tags Column I Unique ID Column J Enrolled Column K Year Level Column L School Year DO NOT EDIT. This id is used by the system to ensure the correct student s details are edited. Enter each students surname(s). Apostrophes, hyphens and spaces are allowed. Enter each students first name(s). Apostrophes, hyphens and spaces are allowed. Enter a unique username for each student. Apostrophes and spaces are not allowed. Student ID numbers are recommended. Can be an address. Enter a password for each student that is eight characters or longer. Students can be allocated the same password as their usernames are unique. Alternatively, each password can be unique or groups of students can share a password. Leave blank to create a unique random ten-letter password for each student. Enter students date of birth. The format must be DD/MM/YYYY or DD-MM-YYYY. Enter students gender. The format can be full words (Male/Female and male/female) or single letters (M/F and m/f). Entering data into this column is optional. If tags are being used, enter the tags to assign to each student. Separate multiple tags with a comma (,) e.g. Class 2B, Extension. Tag names must be an exact match to the tags created in the account, including spaces and letter case. Entering data into this column is optional. It can be used to record another identifier for each student, such as an ID number from a different database system this information can then be included in report exports spreadsheets after tests are sat. If not being used, leave the column blank but do not delete the heading Unique ID at the top of the column. Enter one of the following: Enrolled or Not Enrolled. (Not Enrolled can be used for students who have left or graduated) Enter one of the following: Prep, Year 1, Year 2, Year 3, Year 4, Year 5, Year 6, Year 7, Year 8, Year 9, Year 10, Year 11, Year 12 or Staff. Enter one of the following: 2015, 2014, 2013 or Save the file using the save as option in Excel, and the correct format will be set by default, which is *.xls do not change the file type. Name the file something relevant, and save it to a folder or location which will be accessible later. 4. On the Candidates Edit page, click on browse or choose file. Locate the file saved at step 3 and click open. To finish, click to upload to import the file and apply the changes. 26 P age OARS User Guide September 2014