User Group testing report

Transcription

1 Use Goup testing epot Deliveable No: D6.10 Contact No: Integated Poject No : SafetyNet Aconym: SafetyNet Title: Building the Euopean Road Safety Obsevatoy Integated Poject, Thematic Pioity 6.2 Sustainable Suface Tanspot Poject Co-odinato: Pofesso Pete Thomas Vehicle Safety Reseach Cente Egonomics and Safety Reseach Institute Loughboough Univesity Holywell Building Holywell Way Loughboough LE11 3UZ Oganisation name of lead contacto fo this deliveable: SWOV Due Date of Deliveable: 30/06/2006 Submission Date: 30/06/2006 Repot Autho(s): Katalijn Ritsema van Eck and Divea Twisk (SWOV) Poject Stat Date: 1st May 2004 Duation: 4 yeas Poject co-funded by the Euopean Commission within the Sixth Famewok Pogamme ( ) Dissemination Level RE Resticted to a goup specified by the consotium (including the Commission Sevices) sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 1

2 Executive Summay This epot pesents the esults fom two use goup testing session on the ERSO website ( in the peiod , and the implications of these esults fo the design of the website in tems of stuctue and content. The epot is a poduct of WP6 (Deliveable D6.10) and is only disseminated to the membes of the SafetyNet consotium. Its pupose is to epot on the wok in pogess, and the cuent status of the use goup testing. The fist use goup testing has led to the cuent stuctue and content of the website. The second has seved as a test whethe it is in line with needs of the use goup. We conclude that the website in its cuent fom is positively eceived by lage goups of uses. This has been achieved by involving uses in the ealy stages of its development. In 2007 a final use goup study will be caied out. We aim to involve a use goup that is boade in outlook and moe epesentative of the potential use goup we aim fo. The second half of 2007 will be used to adapt the website on the basis of the esults of this last use goup testing. sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 2

3 Contents Euopean Road Safety Obsevatoy Use Goup testing epot 1 Intoduction Backgound The impact of use goup testing Stuctue of the epot Method fist testing peiod Most impotant esults fist testing peiod Quality of web texts Best way of pesenting web texts poduced by the Editoial Goup Opening page Database Geneal sevices Changes made to the website in between Method and esults second testing peiod Objective Method Most impotant esults second testing peiod Concluding emaks and futue plans Appendix A Questionnaie use goup testing sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 3

4 1 Intoduction This epot pesents the esults fom two use goup testing session on the ERSO website ( in the peiod , and the implications of these esults fo the design of the website in tems of stuctue and content. The epot is a poduct of WP6 (Deliveable D6.10) and is only disseminated to the membes of the SafetyNet consotium. Its pupose is to epot on the wok in pogess, and the cuent status of the use goup testing. 1.1 Backgound Within SafetyNet, a website is being ceated. This website should A) fom a window to SafetyNet esults and B) fom the basis fo a Euopean Road Safety Obsevatoy (ERSO). The goal of the ERSO website is to have all elevant oad safety infomation available -easily and quickly- fo policy makes and othe oad safety pofessionals in Euope. The ERSO website has fou pats: a data pat, a knowledge pat, a sevices pat and a SafetyNet pat. 1.2 The impact of use goup testing Quality is of pime impotance fo the success of the website. Howeve, the quality of the website is not only dependent on its content, but even moe impotantly dependent on the extent to which the websites accommodates to the needs of its (potential) uses. Theefoe in the development of the website the expeiences of uses play a cucial ole. To gea the website to these uses needs, use goup testing has been done. With the testing we like to find out what the wishes of the uses ae. What navigation option do uses like? What style of witing is appealing? In this epot, the esults of seveal use goup testing ae descibed. Thee have been two peiods in which the website was tested: The fist testing peiod was in May, June and July Afte that, we impoved the website. The second testing peiod was in May Stuctue of the epot We will fist descibe the method and the esults of the fist testing peiod (chaptes 2 and 3). Chapte 4 pesents the changes that esulted fom the fist testing peiod. Then we will descibe the method and the esults of the second testing peiod (chapte 5). In chapte 6 you find an outline fo the use goup testing in sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 4

5 2 Method fist testing peiod In the fist testing peiod, thee have been thee testing moments: May, June and July In May 2005 the so-called 'RSPI National Expets' wee asked to fill in a questionnaie. The RSPI (Road Safety Pefomance Indicatos) National Expets goup is one of the two goups of national epesentatives fom the EU membe states (and Noway and Switzeland) that ae egulaly consulted by the SafetyNet team. The national expets goups ae woking goups set up by the EC and - fom the pespective of SafetyNet act as independent oad safety infomation specialists. In many cases its membes belong to the pimay taget goup of the ERSO website. 23 of the national expets esponded to the questionnaie. In June 2005, just befoe the SafetyNet plenay meeting in Mykonos, anothe questionnaie was sent out to all uses attending the meeting. Since the meeting was esticted to SafetyNet patnes, this implies that only SafetyNet patnes eceived the questionnaie at that time. 10 uses esponded. In July 2005, fo the last time a questionnaie was sent out, to the Rosebud use goup. 39 people of the Rosebud use goup esponded to the questionnaies. Evey time the questionnaie was spead via . Uses whee asked to fill in a wod document (which was attached to the ) and send it back to us. The fist time, in May, this questionnaie mainly focused on the issue how (layout-wise) to pesent web texts. The second time, in June, we impoved the old questionnaie and added some exta questions. So the questionnaie contained some exta layout questions concening the knowledgebase, and some questions on the opening page and the database. Since the two goups we sent the questionnaie to, did have elations with the SafetyNet poject, we wanted to ask a thid independent goup thei opinion on the site. The Rosebud use goup consists patly of policy makes, which ae ou main taget goup. To avoid a vey long questionnaie, we made thee vesions with diffeent questions (vesion A, B and C). The thee diffeent questionnaies sent to the Rosebud use goup wee andomly divided. We epeated the questions that wee aleady used in the questionnaies to the RSPI National expets and the SafetyNet goup (in vesion A and C), and asked some new questions (vesion B). Evey questionnaie, A, B and C, had a diffeent topic. Questionnaie A was focused on how to pesent web texts; questionnaie B on the content of web texts and questionnaie C contained some exta layout questions on the knowledgebase, questions on the opening page, the link list and the database. sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 5

6 3 Most impotant esults fist testing peiod 1. Quality of web text 2. Best way of pesenting web texts poduced by the Editoial Goup 3. Opening page 4. Database 5. Geneal sevices 3.1 Quality of web texts Objective The objective is to wite clea web texts at the level of detail that ou taget goup (mainly policy makes) wants/ needs Results This questionnaie was only sent to the Rosebud use goup. Of the Rosebud use goup, 30% esponded to ou questionnaie. The age of the espondents was between 29 and 75 and the aveage age was 49. Half of the uses esponding woked fo national o local govenment, 30% fo a eseach institute, the est fo a consultancy agency o the univesity. As you can see in table one and two, uses esponded positive to the two texts on Speeding and Road Safety Management. 12 espondents said they ageed with the quote "I like this infomation". One peson esponded neutal. The espondents wee vey positive about the infomation. When we asked if the infomation was too detailed o not detailed enough, uses esponded neutal o said that was not the case. The infomation was tustwothy and easy to undestand. Thee ae some subtle diffeences, though, between Road Safety Management and speeding. What can we lean fom those? Maybe we can conclude that o Uses pefe staight fowad infomation o Uses pefe not too detailed infomation o Uses pefe shote chaptes On the othe hand, Road Safety Management, as a subject, is vey philosophical, Speeding is moe staight fowad, so that makes it difficult to compae the scoes between subjects. Speeding QUESTIONS I don't agee Neutal I agee 1. I like this infomation This is the infomation I need sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 6

7 3. I could use this infomation fo my wok The infomation is too detailed The infomation is not detailed enough The sections ae too shot The sections ae too long I can easily find the infomation I am looking fo The infomation is tustwothy The content is easy to undestand The English used is easy to undestand Table 1: scoes of the osebud use goup on speeding. Road Safety Management QUESTIONS I don't agee Neutal I agee 12. I like this infomation This is the infomation I need I could use this infomation fo my wok The infomation is too detailed The infomation is not detailed enough The sections ae too shot The sections ae too long I can easily find the infomation I am looking fo The infomation is tustwothy The content is easy to undestand The English used is easy to undestand Table 2: scoes of the Rosebud use goup on Road Safety Management Conclusions We ae heading in the ight diection. The assumption we made that uses pefe shot chaptes is coect. Uses seem to pefe infomation that is applicable to situations at hand. We need to keep that in mind all the time. sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 7

8 3.2 Best way of pesenting web texts poduced by the Editoial Goup We tested fou ways of pesenting infomation: Tee. You navigate though the infomation via a menu on the left of the sceen. Diagam. You navigate though the infomation via a diagam, explaining the elationship between pats of the infomation via boxes and aows. Pdf. A Pdf, document, to be uploaded. Shot long. An intoduction, like a summay, on the web, with the possibility to upload a Pdf document if you want moe infomation Objective We want a layout, navigation options, and a way of pesenting the infomation that helps uses find thei way. A quick glance should aleady give insight in whee to find what. What we also stive fo in the pesentation of the infomation, is that uses not only find what they ae looking fo, but they find additional elevant infomation as well. So if, fo example, uses look fo infomation on speed cameas, they should also find infomation on othe countemeasues Results The uses said that this "finding additional infomation" was mostly the case with diagam and tee, moe than when they used Pdf o shot/ long as navigation tools. Diagam and tee scoed highest on othe functionality questions as well, like finding infomation quickly. Diagam is best if you aleady have some knowledge on the subject, if not, it can be confusing, because it is quite a lot of infomation at once. The navigation tool tee (a menu on the left) is vey intuitive, but doesn't give something exta. Pdf has a lot of pactical advantages, like the possibility to scoll, do a wod seach, to copy the text, to pint the text etc. It is impotant that uses get a good oveview of the text. If we cut the text into smalle pats, thee is the isk that uses loose the oveview. On the othe hand, it can be difficult to keep the oveview if the whole text is available at once and it is not possible to see whee you ae in the text. The most impotant is, that uses at all times know whee they ae in a lage scheme of the text. This could be achieved by using a tee, which maks the location of the pat you ae eading (see figue 1). In shot: Uses want to know whee they ae in a text. This can be achieved with a tee (see figue 1). Uses want to quickly undestand what the subject is all about. This can be achieved by offeing a diagam at the beginning of the text (see figue 2). Uses want to have easily accessible seaching and pinting options fo the whole document. This can be eached by making a Pdf document available (see figue 3). sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 8

9 Figue 1: With a tee (a menu on the left) you have the possibility to show uses whee thee ae in the text. Figue 2: a diagam gives uses a quick glance at the whole subject teated. sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 9

10 Figue 3: a Pdf document makes it possible to do a wod seach though the whole document and to pint the whole document at once. Taking all the comments seiously, this would mean that we should offe thee navigation outes to the uses: a tee, a diagam and a Pdf document. We have tested these thee navigation options togethe and the esults wee positive, the thee options complement each othe. That is why we choose to have these thee ways (moulds) of pesenting the infomation. It must be clea ight away, though, that thee ae thee 'moulds' fo the same infomation. We did eceive some othe feedback on the website to. In the text, we had text boxes with exta infomation o pactical examples. Uses eally liked these text boxes. Uses want to be able to go back to the home page all the time Conclusions With the thee ways of pesenting the infomation (Tee, Pdf, Diagam) we have eached the objective of making the infomation clea and easy to handle, fo uses with diffeent levels of knowledge. 3.3 Opening page Objective We stive to make an opening page that catches the eye, but woks on old computes as well. The opening page should make the stuctue of the web site clea within seconds Results Duing the fist testing peiod, we have tested the following opening pages fo the knowledgebase: Blog: Shot boxes of infomation pe subject (see figue 4). Diagam: A diagam, in ou test the New Zealand hieachy, giving an oveview of all oad safety infomation (see figue 5). Tee: A simple menu on the left of the page (see figue 6). sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 10

11 Figue 4: Blog. sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 11

12 Figue 5: Diagam. Figue 6: Tee. sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 12

13 Duing the fist test, blog and tee scoed highest. The only poblem was that, mainly in the blog option, uses thought it was annoying, that they had to scoll down that much to find you subject. 10 subjects should be the maximum. Duing the second test, we gave moe infomation on how the diagam would have to wok. As a esult, diagam and menu scoed highest duing the second test Conclusions A simple menu keeps scoing high points, but is not eally flashy. A blog might not be pactical, a diagam might be too complicated. We need to futhe exploe the options with the help of a gaphical designe. 3.4 Database Objective We would like to have a database on the web site with the functionalities uses need. It must be intuitive in it's use Results The uses we asked about the databases, did not use databases all that much. 66% of the uses, used a database a few times pe yea and 33% of the uses neve used a database. Although not using databases all that much, uses do pefe to have a database (80%) on the web site, but only with explanatoy text with conclusions fom the data (80%). We have tested the data system Cognos/Poweplay. The data system scoed positive. Uses liked the pesentation of data (80%) and thought it was useful to have Poweplay in ERSO (80%). Uses did pefe to have explanation on the data. 80% ageed that a couse would be useful, and only 50% undestood, without explanation, that they could dag data into the table Conclusions It is good that we plan to have a database in ERSO. Poweplay could be a good tool fo the web site. 3.5 Geneal sevices Objective To have a geneal sevices pat, whee uses can find all infomation they need apat fom data and knowledge. Like fo example: a link list, seach engine and who-is-who Results We only tested the link list. 90% of the espondents said that the navigation options fo the link list wee good, but only 30% could easily find the link they wee looking fo. These esults ae a bit confusing, so some exta testing is needed. The link list could also be a bit moe detailed. sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 13

14 3.5.3 Conclusions The link list needs to be impoved. Othe pats of the geneal sevices sections still need to be tested. sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 14

15 4 Changes made to the website in between All uses said yes to the question: "do you think that the web site, if finished, could be a valuable souce of infomation fo you?" This tells us that we ae moving in the ight diection, but of couse a lot needs to be done. In the changes we make we have to keep in mind that navigation, stuctue and pinting ae vey impotant to uses. Between the fist use goup testing peiod and the second use goup testing peiod, a lot has happened: We have changed the entie look fom 'SWOV style' to SafetyNet style. Fo this we have had the help of the media depatment at Loughboough Univesity. With this, we have changed oveall layout. The media depatment has designed an opening page fo the website, based upon an idea given to us duing the plenay meeting of SafetyNet. We have added seveal web texts. We have added an exta pilla; SafetyNet. So the website now has fou pillas: Knowledge, Data, Sevices, SafetyNet. We have managed to make one layout fo the diagams. We have added boxes with pactical examples in all web texts. We have shotened the paagaphs of Road Safety Management, and have the guideline of maximum of 500 wods pe heading. We have added a seach engine. We have filled the data section with infomation coming fom the SafetyNet deliveables. We have added a home page. sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 15

16 5 Method and esults second testing peiod 5.1 Objective The website has all elevant functionalities. We want to assess the following questions: What do uses like, what do they miss, what do they advise. 5.2 Method The second testing peiod took place duing the SafetyNet confeence in Pague duing the 10 th and 11 th of May We have invited all confeence visitos to come to ou wokshop, whee they had the possibility to look at the website on 10 computes with intenet connection. Duing that peiod, we asked them to give us feedback in two ways. 1) oally, we wee walking aound and wee available fo questions and/ o feedback. 2) via a small questionnaie on pape. The questionnaie is pesented in Annex 1. We have eceived 34 filled in questionnaies. Of that goup, 58% was a eseache, 11% woked on developing new policies and 30% was advising othe oganisations. The goup thus had a lage pecentage of eseaches. Of couse all paticipants came to Pague fo the oad safety confeence, with a main focus on data collection and analysis. This means, that we do have to keep in mind that these uses might have diffeent pioities than policymakes, ou main focus goup fo the website. 5.3 Most impotant esults second testing peiod The uses wee faily positive about the website as it is. 72% of the uses said that they will use the site egulaly. 75% of the uses said the site povided infomation and data they need fo thei wok. The uses would like to have moe infomation on data collection and data-analysis. They wee the least inteested in a detailed account of individual studies. In the questionnaie, uses could choose fom a list of subject they would like to have a web text on. Thee was a lage inteest in a web text on Fatigue. Othe web text the uses wee inteested in wee: Childen, Youngstes and The collection and analysis of oad safety data. The uses eally appeciated the possibility to compae between counties and would like to see that extended in a link to county pofiles with oveviews of ules and egulations. They also thought that a calenda of oad safety confeences should be added. sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 16

17 We also got geneal feedback on the website. Some examples of emaks made ae: Coections in woding o spelling. Some of the diagams wee too complicated. Wok with mouse-oves. Open extenal links in an extenal popup. Don't use "next" at the bottom of a page as a navigation tool. Changes to the link list. sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 17

18 6 Concluding emaks and futue plans The website in its cuent fom is positively eceived by lage goups of uses. This has been achieved by involving uses in the ealy stages of its development. Howeve the cuent use goups ae not a good epesentative sample of the potential use goup that we would like to attact to the site. As yet the uses ae in one way o an othe elated to the SafetyNet poject, o ae uses that ae vey inteested in developments like these (ealy adoptes). In the pevious stages of its development this bias was beneficial as we could count on thei enthusiasm. Howeve in the next stages, we need to test whethe the site also confoms to the expectations of a less select and much boade goup of policy makes, in paticula to policy makes in the "new" Euopean counties. Moeove, we will exploe the needs of NGOs and stakeholdes such as Licensing agencies, ca industy, and lobby goups. In the fist half of 2007 a final use goup testing will be oganized diected at a boade use goup. The second half of 2007 will be used to adapt the website on the basis of the esults of this last use goup testing. sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 18

19 Appendix A Questionnaie use goup testing 2 The ERSO website A.1 Read this fist The objective of the ERSO website is to give you the infomation you need. You expeiences with the site and you opinions will help us to impove it. Theefoe we ask you just 10 minutes of you time to complete this questionnaie. To answe the questions, please tick the appopiate boxes like this.. You can tick moe than one box pe question. 1. You oganisation? Cental Govenment Local govenment Road authoity Police Othe: 2. You wok? Geneal activities and only some on taffic safety Specialist in taffic safety 3. Main components of you wok? To do eseach To develop new policies To advise othe oganisations Othe: 4. If you seach fo info on taffic safety, what level of detail do you need most? agee neutal Disagee Mainly a shot oveview of best pactices A detailed desciption of poblem analyses and measues sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 19

20 A detailed account of individual studies Data illustating diffeences between county Refeences to scientific papes and epots Infomation on data collection and data-analysis Othe: At the end of 2006 the following special topics will be available in ERSO: Speeding Road safety management Dink diving Roads Vehicle safety and potection systems Post impact cae Pedestians and cyclists Motoised two-wheeles Novice dives Olde dives Road safety at wok 5. Which 5 othe topics should we addess in the next yea: give you pefeences. Childen Youngstes Distaction Fatigue Dug diving Wok zone safety The collection and analysis of oad safety data Othe topics o.. o.. 6. You opinion about the content: To what extent do you agee with the following statements? agee neutal Disagee 1. I will use this site egulaly 2. This site povides infomation and data I need in my wok 3. The infomation is easy to find 4. The quality is high and tustwothy sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 20

21 5. The possibility to compae between counties is useful 6. A calenda of oad safety confeences should be added 7. I like the complete infomation on the SafetyNet Deliveables 7. What exta sevices would you appeciate? agee neutal Disagee 1. An electonic newslette about updates on the website 2. County pofiles with oveviews of ules and egulations 3. A countemeasues database + eseach papes 4. Couses on data collection and analyses Othe: Thank you vey much fo you contibution! Would you like to be kept infomed about the pogess, please leave you addess hee. sn_wp6_d6p10_usegoup_testepot_300606_final.doc Page 21