Innovative solutions for data capturing strategies

Transcription

1 Innovative solutions for data capturing strategies Cecilia Colasanti 1, Manuela Murgia 1 1 Istat, cecilia.colasanti@istat.it, manuela.murgia@istat.it Abstract The Italian National Statistics Institute (Istat) is going to develop new software and data capturing methods in accordance with latest user and market requirements. In order to offer a good service for citizens, Istat is taking into consideration many factors like technological evolutions, necessity of collecting data more and more rapidly and social changes. In this article we want to describe the strategy that Istat is adopting to get this target. This strategy is based on two ways: one way is through LimeSurvey (for questionnaires with almost simple administration rules) and the other through XsiMpLe (for questionnaires with a more complex structure). Both these tools can be framed into data capturing process standardisation, keeping in high consideration data security problems (data integrity and authentication). The proposed strategy represents the starting point for a production process standardisation and it allows an efficient use of human resources with high IT skill. Keywords: data capturing, limesurvey, XsiMpLe 1. Introduction The Italian National Statistics Institute (Istat) is going to develop new software and data capturing methods in accordance with latest user and market requirements. In order to offer a good service for citizens, Istat is taking into consideration many factors like technological evolutions, necessity of collecting data more and more rapidly and social changes. As the time goes by, it is possible to observe that on one hand it is always more difficult to contact people by land telephone and find them at home or to talk directly with respondents, and, on the other hand, the large use of internet and mobile technologies encourage the use of web for data capturing. So the traditional ways to collect data are going to be used side by side with new and updated technologies. In addition, national and international laws go into direction of cost reduction for public administration, maintaining high standard quality. Besides, Istat has already experienced the positive impact on data quality resulting from the internalization of activities concerning the implementation of electronic questionnaires, since it requires specialised and skilled personnel that know which characteristics software must have in order to fulfil the methodological requirements of statistical surveys. In this article we want to describe the strategy that Istat is adopting to get this target. This strategy is based on two ways: one way is through LimeSurvey (for questionnaires with almost simple administration rules) and the other through XsiMpLe (for questionnaires with a more complex structure). Both these tools can be framed into data capturing process standardisation, keeping in high consideration data security problems (data integrity and authentication). The proposed strategy

2 represents the starting point for a production process standardisation and it allows an efficient use of human resources with high IT skill. 2. Possible solutions to be adopted in data capturing strategies 2.1 Limesurvey Limesurvey is an open source software that has a lot of features and allows to manage easy or medium complexity surveys. This is the perfect tool to implement opinion or administrative surveys. It doesn t have hard controls that are typically used for economic or households/individual surveys. We want to analyse this product referring to: technical aspects (user, administration, template management); organisational aspects (skill required, personnel, etc); how the tool has been used in Istat Limesurvey allows to manage users with different grants. For each survey, it is possible to classify them as respondents, that can only submit the application form, or as weak/power administrators. Using the panel administration it is possible to associate one or more users and groups to each survey and to choose if an administrator can read, write, execute, restart a survey or manage his own participants. This is an important feature, for its impact on organisational aspects, that guarantees the separation of roles between each responsible, the respect of privacy among surveys on the system, the possibility to early individuate potential points of failure on the system. (fig. 1 - user manager) The tool allows to choose between an anonymous and not-anonymous survey with a public registration of respondents or a selected group. In the second case, the

3 administrator can import users on the system (by using.csv files or LDAP software), and automatically send invitations, reminders and token by . Using cookies or session based surveys, participants can buffer answers to continue survey at a later time. It is possible to change the basic template for any survey using a template editor for creating your own page layout and integrate pictures and movies. It is also important to underline that Limesurvey supports more than 50 different languages in front-end and back-end. Referring to the survey, you can implement many question types and add conditions for questions depending on earlier answers (skip logic or branching). At the end, it is possible to enhance import and export functions to text, csv, pdf, spss, R, xml and MS Excel format and to elaborate basic statistical and graphical analysis. A critical point of this tool is represented by the features related to data evaluation and visualisation. For this reason, during the whole statistical process, Limesurvey is a good tool to collect data instead of analysing and representing them. It is required a basic IT knowledge to configure this tool and a PHP skill for designers that want to deeply customise it (see below the case study Regulation of Sistan law ). This kind of professional skills are common in a modern organisation, so it is quite easy to produce and put on line survey in this way. We adopted Limesurvey in Istat for the following surveys: (i) wellness of employees, (ii) Regulation of Sistan law, (iii) monitoring of teleworkers, (iv) methodological competences. (i) (ii) (iii) (iv) About 2100 Istat employees have been interviewed through a web survey with more or less 60 different questions to understand the satisfaction level at work. 285 users have been invited to participate in a survey about the Sistan law reform. They could answer with a short or long text. About 300 people are involved into monitoring of teleworkers. Each of them has to choose an answer into a system of indicators. About 500 people are asked about their methodological competences. (i) Wellness of employees Quality results as well as good products mainly depend on high quality job and good relationship among employees at social and professional level. The aim of the First survey on employees satisfaction was to facilitate the enforcement of Istat human capital and consequently the quality of delivered services. This survey is divided into 5 sections and take information on (i) individual job experience; (ii) Istat reputation; (iii) satisfaction for internal services; (iv) individual information; (v) opinion on administrative services. To make the compilation easier, convenient technical solutions were adopted: survey design and interview s length, questions wording and type to questions (radio button, list, check button, etc.).

4 (fig. 2 question of wellness survey) (ii) Regulation of Sistan law The National Statistical System (Sistan) is the set of public and private offices, configured as a large net, that gives the official statistical information to the citizens. Belong to the net: Istat, public offices of statistical information, statistical offices of central/local government administrations, self-government firms, public administrations identified through a special law released by the Prime Minister, Municipalities, local health services, Chamber of Commerce, private statistical offices that provide a public service. To reform the regulation of Sistan law, it was asked to the Sistan members their opinions about the main sections of the present law. In this case, the kind of questions is very simple (long text), and particular attention was concentrated on graphical aspects, to modify the basic style-sheet. (fig. 3 regulation of sistan law)

5 (iii) Monitoring of teleworkers Three years ago telework was introduced in Istat. About fifty activities, suitable to be teleworkable, were chosen among all the Istat processes. One of the main aspects was connected to monitoring of teleworkers job. For this reason quality and quantity indicators were associated to the tasks selected. Each teleworker s responsible, through an on line survey, had to fill in a form with convenient indicators (tipically executed work out of expected work within a fixed time period), the description of virtual or physical place where the output can be found, calendar of activities expected and other useful comments. (iv) Methodological competences In Istat the human resources with methodological competences are located in both Central Methodological Direction and Statistical Production Directions with a mixed organisational model. In this context it seems to be difficult to get, effectively and efficiently, high quality level of processes. For this reason it has been created a net structure, based on people competences, that involves researchers belonging at central and production Directions. The net s knots represent the competence s centre. People skills are designed through a survey, implemented with Limesurvey, divided in two groups of questions relating to personal skill (individual methodological competences) and social skill (the net of people with same competences). Multiple choice questions, logical jumps and conditions are the main ingredients of this survey. 2.2 XsiMpLe The idea of developing a product based on open source technologies and able to implement electronic questionnaires suitable for any kind of data collection technique (from Data Entry to CATI/CAPI and WEB) was borne in Istat in 2008 and was presented during the last NTTS 2009 conference (Capparucci et al., 2009). From this idea, we developed a prototype of product called XsiMpLe whose characteristics and functions are well described in a paper presented during the 62 th Wapor Conference in Lousanne (Degortes et al. 2009). In this present paper we will shortly describe the product suggesting the reading of the above mentioned paper for further details Main characteristics of XsiMpLe XsiMpLe is a prototype of software, developed in Istat, and based on open source technologies/tools (XML, Javascript, Dom, Firefox). It is not available to the open source community since it represents the starting point of a new project aimed at providing a new product that will go beyond the limits of XsiMpLe. This new project is anyway based on the same approach used for XsiMpLe and therefore it is necessary to describe it. We created an XML based language (a dialect proper of the statistical questionnaire world) easy to use for the implementation of electronic questionnaires. We then created an XSLT compiler transforming our XML dialect into a XHTML document.

6 A browser will take in charge the XHTML document and will be the questionnaire user interface. We called our XML dialect XsiMpLe. Why the name XsiMpLe? For two reasons: to make it clear that is based on XML standards and to make it evident that the software is very simple to use: also a non IT expert, who has at least knowledge in text editors and browsers, can use it. Apart from simplicity, its main features are represented by checks performed on client side and the capability of managing very complex questionnaires. In more details: performing checks on client side during the data collection phase means that an error message appears to the respondent/interviewer as soon as a question is answered and the cursor moves to the following question. In this way there is no need of submitting data to have they checked. This was possible by using generalised checking functions implemented with Javascript; with XsiMpLe it is possible to implement very complex questionnaires, where complexity is in terms of skipping rules and consistency checks. In fact, it has been created to be also used for CATI or CAPI surveys whose electronic questionnaires can be sometimes very complex IT tools. This is because, in both cases, the questionnaire is interviewer administered and this makes it possible to prevent data from errors during the data collection phase, enhancing data quality and speeding up the editing and imputation phase. Besides, XsiMpLe can be adapted to different data collection techniques (CATI, CAPI, WEB) since its layout can be easily changed in order to respect the peculiar characteristics of each survey mode. This allows for process standardization because, in case of mixed-mode survey, the same software or, better, the same lines of code can be used for the implementation of the questionnaire to be administered in different ways. It manages any type of questions format and performs controls on single questions as well as on more interrelated questions (consistency controls). In case of error a window appears, showing a customised error message and the list of variables involved in it. By clicking on the variable it is possible to go back to the questionnaire and correct the keyed value(s). The pictures below show some of the possible question formats as well as some of the checks that can be implemented with XsiMple. (fig. 4 - XsiMpLe functionalities) The above picture shows a radio question or single choice question and an open question. Below a drop-down list and a multi-choice question are shown. In this last

7 case if more than two answers are checked, an error message appears to the respondent/interviewer. (fig. 5 - XsiMpLe functionalities) (fig. 6 - XsiMpLe code) The above figure is an example of XsiMpLe code. A questionnaire can contain more than one section: the tags <datablock> and <title> are used to identify and to assign name to sections. Each variable is defined by the tag <field>; inside the tag <field> there are some attributes to describe variable characteristics: its name and size, its tablename, i.e. the name of the table into the RDB where it is recorded, its dbtype, i.e. its type into the RDB. Under the tag <field> other three tags can be founds: <question>, <type...> and <check>: <question> contains the question to be administered; <type...> specifies the format of the variable on the browser. For example: <typetext> specifies a text type, <typeradio> specifies a radio type if a radio

8 button question is needed. They are suitable for questions whose answer categories are mutually exclusive; <check> specifies a call to one or more generalised Javascript functions to perform controls on the keyed variables. In our example, we use the refuse check on variable companyname and a range check on variable job. All checks are performed client side: the former allows not answering the question (answers are compulsory unless specified), the latter produces a warning message if out of range values are keyed. Anyway, when the questionnaire has a very complicated interview flow together with many consistency checks among variables, XsiMpLe alone may not be the best and easy way for the implementation of questionnaires. Therefore, we have implemented another flexible and dynamic XML language to be used together with XsiMple. We have called it: XDoMsaiLing (it uses two web tools, an XML file and the DOM - Document Object Model). Its functioning is based on the principle that all questions to be administered belong to a hierarchical tree. Questions defined in the XsiMpLe questionnaire represent a straightforward tree. Then, by adding skipping rules, hierarchy among variables is created thus making the final tree corresponding to the designed interview flow. Also error conditions hard errors and warnings- can be easily managed by adding them inside the tree. <question1/> <question2/> <question3/> <question4/> <question5/> <question6/> customised navigation.xml <question7/> <question1/> <question2> <condition>question2=1 <true> <question3/> <question4/> </true> <false> <question5/> </false> </condition> </question2> (fig. 7 - XDoMSailing for complex questionnaires) The default navigation.xml contains the list of all questions (variables) of the questionnaire. Ad hoc changes are necessary to customise the tree according to the questionnaire flow. These changes are made using three types of tags: the condition tag, the error tag and the warning tag. To test XsiMpLe we implemented the electronic questionnaire of a web survey among quality mangers belonging to different NSIs aimed at evaluating an Istat product, called SIQual, according to the European Statistics Code of Practice followup and quality methodology project. To manage respondents accounts XsiMple needs a secure system to lean on. In Istat this is represented by INDATA website. We used it to host a questionnaire implemented with XsiMpLe Besides, as the web survey mentioned above did not cover all possible types of questions and checks one can meet when developing an electronic questionnaire, we

9 decided to test XsiMpLe functions by implementing with it the electronic questionnaire of an old edition of a CATI survey called Birth survey. The test showed that the product is able to satisfy also the requirements of this data collection technique (scheduling of calls not included) and therefore that XsiMpLe can be used of any data collection mode XsiMpLe new version As time goes by we realised that this prototype has already a potential shortcoming: it properly worked with open source browsers like FireFox, but not with Internet Explorer. In other words it wasn t perfectly cross-browsers. We realised that to make it totally cross-browsers it was necessary to add too many lines of code. This would have created problems for its maintenance and for future developers. Therefore we decided not use native Javascript for our generalised function but to use another ingredient : JQuery ( This is a framework that has many advantages like the fact that it manages the use of different browsers making our product able to be executed by any of them without the need of adding any line of code. Besides JQuery is based on the motto Write less, do more ( This means that its syntax is very synthetic and efficient and this is very useful for us since we had to write much more lines of codes to obtain the same results. The new version of XsiMpLe, that will probably go under a new name, will be realised next year. We are currently using it for the implementation of the Coverage Survey of the Sixth Italian Agricoltural Census starting in April This will be an interviewer administered web survey requiring some key variables to be checked during the data collection phase itself. (fig. 8 XsiMpLe new version)

10 3. Conclusions The different characteristics of the two software allow to say that if one looks for the best product he will not find it because the best product doesn t exist while it does exist the most suitable product to solve the different survey s needs. Therefore it is advisable to choose the product that satisfies at best the specific requirements of each survey. In general, we can say that Limesurvey is more suitable for simple questionnaires and it is quick to configurate, while XsiMpLe can be very useful in case of complex surveys where complexity is still in terms of skipping and checking rules. Once the system is set up, which software has to be used depends on the questionnaire complexity. As a consequence, the proposed strategy suggests the conjoint use of the two products. Considering the top management s point of view, this allows an efficient use of human resources because those with higher IT skills can be used only for really complex cases or for specific software customisation, while in the other cases it is involved only personnel with a lower IT profile. So it is possible to define and to put in practice a standard and generalised production process. Thanks to the good results in terms of efficiency and management obtained from the use of the two products in simple pilot surveys, the next steps will be their use for more complex questionnaires. References Capparucci et al. (2009) Comparative analysis among open source and commercial software for the development of electronic questionnaires for statistical surveys NTTS 2009, Bruxelles Cianchetta R., Pagliuca D. - (2005), "Soluzioni Open Source per il software generalizzato in Istat: il caso di PHPSurveyor", Collana Documenti Istat, n. 17. Degortes et al. (2009) An innovative open source strategy for the development of electronic questionnaires for statistical surveys WAPOR Conferece, Lousanne.