ESSPROS Task Force on Methodology November Qualitative data review

Transcription

1 EUROPEAN COMMISSION EUROSTAT Directorate F: Social statistics Unit F-5: Education, health and social protection Luxembourg, 24 October 2017 DOC SP-TF ESSPROS Task Force on Methodology November 2017 Qualitative data review Meeting of the ESSPROS Task Force on Methodology Luxembourg, November 2017 BECH Building (Eurostat) B2/464

2 Qualitative data review 1. INTRODUCTION The following document reviews the "qualitative data" currently collected and disseminated for ESSPROS (qualitative information, list of schemes, quality reports, footnotes, etc.) and then considers how this may be managed more effectively and efficiently in the future. In this document, "qualitative data" is taken to be primarily non-numerical information. Such data may complement the quantitative data by describing what is covered (descriptive information) or by clarifying the methods used to produce the data and any issues which affect it (technical information). It may even have no direct links with the data, simply providing context (contextual information). 1 Good qualitative data are vital to ensure effective validation of the quantitative data, assessment of its quality, and the correct use and interpretation of the data. The importance of the latter point has been heightened by the recent dissemination of detailed ESSPROS data by scheme. Further, recent work to establish links with other datasets (e.g. MISSOC, LMP) has demonstrated that establishing detailed links and comparing data relies heavily on the names of detailed benefits covered by individual data points being reported in the qualitative information in both datasets. In this context, the way this information is currently collected and disseminated in ESSPROS could be improved. With the introduction of SDMX for the quantitative data 2, and the recent work to review the qualitative information, there is an opportunity to rethink and improve on how the various items of ESSPROS qualitative data are currently collected and disseminated in order to improve their efficiency and effectiveness. This document is split into two sections. The first section provides an inventory of the qualitative data currently collected and how it is disseminated. The second section describes possible ways in which the qualitative data might be more efficiently collected, validated and disseminated, culminating in some ideas for discussion by the Task Force. 1 In statistical terminology, these "qualitative data" are sometimes referred to as: - "reference metadata" (or explanatory metadata), describing the contents and the quality of the statistical data, including explanatory texts on the context of the statistical data, methodologies for data collection and data aggregation as well as quality and dissemination characteristics; - "paradata": information related to a statistical data collection or production process. 2 In statistical terminology "structural metadata" is sometimes referred to as information used to identify, formally describe or retrieve statistical data, such as dimension names, variable names, dictionaries, datasets technical descriptions, etc. For example, structural metadata includes the titles of the variables and dimensions of statistical datasets, as well as the units employed, code lists, data formats, potential value ranges, time dimensions, value of flags, classification used, etc. 2

3 2. INFORMATION CURRENTLY COLLECTED AND DISSEMINATED This section is split into two parts, one outlining qualitative data collected as part of the datasets and one outlining data collected in quality reports Datasets Four datasets include some form of qualitative data. These are the Qualitative information (QUALI), Quantitative data for the Core system (QUANTI), the Pension beneficiaries module (BENEF) and the Net benefits (NET) module. The QUALI dataset is, as the name suggests, entirely qualitative, while the other datasets include only small pieces of information that complement the quantitative data Qualitative information (QUALI) ESSPROS Qualitative information (QUALI) is collected annually to provide detailed information on the social protection schemes and the detailed benefits they provide. The specifications of this data are set out in Appendix II of the ESSPROS manual. 1 of this appendix describes the purpose of the Qualitative information as providing the following: 1. in-depth information on social protection schemes 2. the means to evaluate the classification of schemes and benefits applied 3. a clear basis for footnotes in publications and for the ESSPROS database in New Cronos (sic) 4. a means to respond to questions from users on the data by scheme 5. the basis for the validation of ESSPROS data (Core System and modules on pension beneficiaries and net social protection benefits) Further, Table A of appendix sets out the qualitative information to be collected as the following: - General information: 1. Period covered 2. Date of updating qualitative information 3. Responsible organisation, person and contact - General description of each scheme: 1. English and original (in national language) name of the scheme 2. Scheme movement (year of introducing/abolishing/merging/splitting of the scheme) 3. classifications of the scheme 4. reference legislation and/or regulation 5. organisation(s) responsible for running the scheme 6. financing of the scheme 7. scope (protected people under scheme) 8. bibliography and web bibliography (optional) 9. history of the scheme 10. notes (other important information) 3

4 - Description of each benefit: 1. English and original name of benefit 2. Year of introducing or abolishing the benefit 3. General conditions for granting the benefit 4. The categories of beneficiaries 5. Taxation of benefit This information is collected using an Excel template in which each piece of general information and each piece of information about a scheme is reported in a separate cell while all information about a benefit is reported in a single cell according to the corresponding benefit s classification by detailed benefit type. Further, information about several benefits may be grouped together when they share the same classification and are provided by the same scheme. In addition to this, information is also collected on the time period covered by the qualitative information i.e. which reference years are covered in a separate worksheet to the rest of the data. Except for particular cases, the qualitative information covers the whole timeseries. Every year it is updated with the latest information available. The qualitative information is disseminated via Eurostat s dedicated page on social protection in the same format as it is transmitted Quantitative data for the Core System (QUANTI) ESSPROS Quantitative data for the Core System (QUANTI) is collected annually providing data on expenditure and receipts of ESSPROS schemes. Traditionally this has been transmitted to Eurostat via an Excel questionnaire but SDMX is gradually being adopted and expected to replace the Excel questionnaire in the future. As part of the collection of this dataset three sorts of qualitative information are collected: - Scheme list: This identifies the schemes for a given country. Some guidance on this is provided in the document List and classification of schemes (Scheme_List): General Instructions and Guidelines for completion of the excel questionnaire available on CIRCABC 3. For each scheme listed this identifies the name of the scheme (in English and national language), its acronym, its classification (according to the criteria set out in section of the ESSPROS Manual), the year it became active and the year it was closed. Note that at present this information is provided via an Excel template (SCHEMES) that can be used to convert the information to SDMX format (this is an intermediate solution towards the full implementation of SDMX). - Footnotes: Footnotes associated with the quantitative data are collected. Some guidance on this is provided in the document Instructions to fill in the ESSPROS quantitative data questionnaire available on CIRCABC 3. In the QUANTI questionnaire the following information is required for each footnote: T-Code (e.g ), Category (e.g. old age pension) and the Footnote text. Footnotes can be provided for any level of aggregation. 3 Available at: 4

5 Note that at present this information can be provided via the traditional Excel questionnaire (QUANTI) or, if SDMX is being used, there exists a specific Excel template for collecting this information (CSTEMP). - Classification of pension schemes: Provides information for each scheme identified as a pension scheme on the scope of the scheme in terms of the types of pension benefits it provides and to categorise the scheme according to its mode of financing and the stakeholder bearing the risk. This data is optional and specific guidelines on how to complete it are not systematically distributed with the launch of the data collection for the Core System. However, starting from the 2018 collection, they will be included amongst the documentation on CIRCABC. Note that at present this information can be provided via a specific Excel questionnaire to be sent by or, if SDMX is being used, there exists a specific Excel template for collecting this information (CSTEMP). In terms of the dissemination of this information, only part of the scheme list is systematically disseminated on CIRCABC as part of the dissemination of the data by scheme Pension beneficiaries (BENEF) module ESSPROS Pension beneficiaries (BENEF) data is collected each year. It primarily focuses on collecting the numbers of pension beneficiaries for selected categories of benefits. Traditionally this information has been transmitted to Eurostat via an Excel questionnaire but SDMX is gradually being adopted and expected to replace the Excel questionnaire in the future. As part of the collection of this dataset three sorts of qualitative information are provided: - Footnotes: Footnotes associated with the pension beneficiaries data are collected. There appear to be no guidelines on the requirements but the template for providing them is self-explanatory. It is quite different from the template used for footnotes to the quantitative data (QUANTI) of the Core System. Data providers simply need to provide the footnote text in the appropriate cell. Footnotes can be provided for any level of aggregation. Note that at present this information can be provided via an Excel template (modified version of BENEF) that can be used to convert the information to SDMX format (this is an intermediate solution towards the full implementation of SDMX) or, if SDMX is being used, there exists a specific Excel template for collecting this information (BENTEMP). - Supplementary information (only some items): The pension beneficiaries module collects some items of supplementary information to complement and put the main data context. The specifications for this are set out in appendix III of the ESSPROS manual. Among these items there are two which can, in some sense, be considered qualitative information (even if they may be numeric in nature). These are the compulsory data which has to be provided at both all schemes level and at scheme level and further broken down by gender: o The legal or standard age of retirement for old age benefits by gender (by scheme or determined by a reference scheme) o reference date / method of calculation by scheme 5

6 Note that at present this information can be provided via an Excel template (modified version of BENEF) that can be used to convert the information to SDMX format (this is an intermediate solution towards the full implementation of SDMX) or, if SDMX is being used, there exists a specific Excel template for collecting this information (BENTEMP). In terms of the dissemination of this information, none are systematically disseminated Net benefits (NET) module ESSPROS net benefits (NET) data is collected annually providing data on the taxes and social contributions applied to cash benefits reported in the ESSPROS Core system in the form of AITR, AISCR and fiscal benefits. Traditionally this information has been transmitted to Eurostat via an Excel questionnaire but SDMX is gradually being adopted and expected to replace the Excel questionnaire in the future. As part of the collection the following qualitative information is provided: Footnotes: Footnotes associated with the AITR, AISCR and fiscal benefits are collected. There appear to be no guidelines on the requirements but the collection template is selfexplanatory. It is quite different from the template used to collect footnotes for other datasets (QUANTI and BENEF). It requests footnotes separately for the AITR, AISCR and residual fiscal benefits. In any cases data providers need to identify the relevant AITR/AISCR/residual fiscal benefit and provide the text of the footnote. Note that at present this information can be provided via an Excel template (modified version of NET) that can be used to convert the information to SDMX format (this is an intermediate solution towards the full implementation of SDMX) or, if SDMX is being used, there exists a specific Excel template for collecting this information (NETTEMP) Quality reports For each of the three ESSPROS data collections - the Core system, the Pension beneficiaries module and the Net benefits module participating countries are required to provide national quality reports whose purpose is to provide information for assessing the quality of the data according to the quality criteria set out by the quality assurance framework of the European Statistical System (ESS) 4. These are Relevance, Accessibility and Clarity, Timeliness and Punctuality, Coherence, Comparability and Accuracy. The national quality reports only provide information in relation to certain criteria which feeds into the more general assessment of all the criteria in the consolidated quality report. Quality reports are provided every year and refer to new data for the latest reference period and revised data for previous periods (if any) delivered during the corresponding data collection. Broadly speaking the templates for the national quality reports of the different data collections follow a similar template. However, each is tailored to collect specific information relevant to the corresponding dataset. For the large part, the information collected is that which is set out in the legislation underpinning the data collections

7 Ultimately the national quality reports for each data collection are routinely disseminated on the ESSPROS dedicated page in exactly the same format as they are transmitted to Eurostat, albeit converted from Word to PDF, and serve as annexes to the consolidated quality report Core system (QUALREP) The information collected in the national quality report for the Core system (QUALREP) is set by Annex II of Commission Regulation (EC) No 1322/2007. Currently quality reports are provided using a word-based template. This collects the following information about national data: - Information on data sources (Table 1.1): Lists the sources used including the name of source, type, frequency, time lag, schemes covered, additional information and geographical coverage. - Benefits covered by sources (Table 1.2): Identifies the benefits of schemes covered by each source. - Geographical comparability (Table 1.3): Identifies cases of partial coverage of schemes, of partial coverage of expenditure, receipts and detailed benefits and of non-compliance with the methodology. - Estimation methods (Table 2): Identifies the schemes and detailed classifications for which estimates are used due to a lack of data or missing breakdown and the estimation method used. - Revision of statistics (Table 3): Identifies the schemes and detailed classifications data for which data for previous years have been revised, the types of revision and details of these. - Revision of statistics (Table 3bs): Identifies specific cases of revision where the methodology for providing the data has changed, the type of change and whether it results in a break in the series. - Other information (Tables 4.1 and 4.2): Other information about the quantitative data and qualitative information delegates deem relevant. - National dissemination policy (Table 4.3): Whether data are published nationally, the timeliness of this, whether it is part of a regular publication and the other details (e.g. links, etc.) Pension beneficiaries module (QRPENB) The information collected in the national quality report for the Pension beneficiaries module (QRPENB) is set by Annex II of Commission Regulation (EC) No 1322/2007. Currently quality reports are provided using a word-based template. This collects the following information about national data: - Information on data sources (Table 1.1): Lists the sources used including the name of source, type, frequency, time lag, schemes covered, additional information and geographical coverage. - Geographical comparability (Table 1.3): Identifies cases of partial coverage of schemes, of partial coverage of beneficiaries and of non-compliance with the methodology. - Estimation methods (Table 2.1): Identifies the schemes and detailed classifications for which estimates are used due to a lack of data, the type of estimation used and the estimation method used. 7

8 - Treatment of double counting (Table 2.2): Identifies cases affected by each type of double counting (types 1-6), whether this has been treated, the type of treatment, details about the treatment and any other comments on double counting. - Revision of statistics (Table 3): Identifies the schemes and detailed classifications or which data for previous years have been revised, the types of revision and details of these. - Revision of statistics (Table 3bs): Identifies specific cases of revision where the methodology for providing the data has changed, the type of change and whether it results in a break in the series. - Other information (Tables 4.1 and 4.2): Other information about the quantitative data delegates deem relevant. - National dissemination policy (Table 4.3): Whether data are published nationally, the timeliness of this, whether it is part of a regular publication and the other details (e.g. links, etc.) Net module (QRNET) The information collected in the national quality report for the net benefits module (QRNET) is set by Annex II of Commission Regulation (EC) No 110/2011. Currently quality reports are provided using a word-based template. This collects the following information about national data: - List of benefits liable to taxation and social contributions (Table 1): Identifies the benefits which are subject to taxation and social contributions. - Information on data sources (Table 2.1): Lists the sources used including the name of source, type, frequency, time lag, AITR/AISCR/Fiscal benefits covered, additional information and coverage in terms of benefit recipients. - Use of multiple sources (Table 2.2): Identifies cases where multiple data sources have to be used to generate either an AITR, an AISCR or a value of fiscal benefit. - Comparability Completeness of coverage (Table 2.3): Identifies cases for which data are missing or which only provide partial coverage of the fiscal system. - Comparability Coherence with methodology (Table 2.3): Identifies cases for which data have not been provided in accordance with the methodology. - Methods used to estimate AITR, AISCR and fiscal benefits (Table 3): Identifies the estimation methods used to derive the AITR, ASICR and fiscal benefits, the assumptions that had to be made and the extent to which the method adequately covers the fiscal system. - Revision of statistics (Table 4): Identifies the schemes and detailed classifications for which the AITR/AISCR or fiscal benefit data for previous years have been revised, the types of revision and details of these. - Other information (Tables 5.1 and 5.2): Other information about the quantitative data delegates deem relevant. - National dissemination policy (Table 5.3): Whether data are published nationally, the timeliness of this, whether it is part of a regular publication and the other details (e.g. links, etc.) 3. PROPOSALS FOR MORE EFFICIENT DATA COLLECTION AND DISSEMINATION This section attempts to identify possible ways in which the qualitative data of ESSPROS identified in the previous section may be more efficiently collected, validated and 8

9 disseminated. At first, considers the issues which need to be taken into account, then it looks at the limitations of the existing system and ultimately proposes solutions for discussion by the Task Force Issues for consideration The qualitative data collected in ESSPROS have two key overarching purposes: (1) to provide technical information that can be used to support assessment of data quality. (2) to provide descriptive information which complements the quantitative data and facilitates correct interpretation of the data. Technical information and descriptive information are not mutually exclusive. Some descriptive information can be to assess the plausibility of data. For example, information on the rules which determine the amounts of benefit paid out to each beneficiary combined with numbers of beneficiaries may raise questions about the plausibility of expenditure for a benefit. Similarly, some technical information can be helpful in the interpretation of data. For example, information on methods used to produce certain values might indicate that coverage could be over- or under-stated and thus impact on the extent to which results can be interpreted reliably in a specific context. Different actors in the production and use of the data have different roles and the qualitative data need to cater to all of their needs: - Data providers: Data providers collect and organise the qualitative data requested in order to deliver it to the Commission in a pre-determined format (SDMX, Excel or Word). The primary concern is the burden of work involved in compiling the required data each year. Typically, a large part of the qualitative data is incremental in the sense that data providers are expected to revise and update the data provided in the previous year rather than providing a completely new set of data each time. The workload thus depends on what information is requested and the extent to which the national situation results in changes to this each year. A balance therefore has to be struck between the level of detail in the information requested and the burden this places on data providers. The format in which the data are transmitted may also influence workload depending on how easy it is for data providers to compile and edit the data to meet the requirements. - Data validators: Data validators review both the quantitative and qualitative data and work with providers in order to ensure quality. Automation of validation checks where possible speeds up the validation process and allows validators to focus attention on more important quality issues ahead of basic input errors and missing information. Automated validation is already applied to the quantitative data but has yet to be extended to the qualitative data. This is primarily because the format in which data are transmitted (primarily Word and Excel) is not conducive to automated validation methods. Checking consistency between quantitative and qualitative data is vital to identifying potential quality issues. However, efficient implementation of such checks is reliant on having the data in formats that allow comparison without extensive manual manipulation. At present, there is no easy way to do this so data have to be cross-checked manually. - Data users: Researchers and other users undertaking detailed analysis of ESSPROS data need to understand the composition of the figures and what the detailed data actually 9

10 cover in each country. This implies making the data available in a manner which allows users to quickly establish links between the quantitative data and the qualitative data. This includes information on the coverage of particular classifications in each country and technical issues (e.g. estimations or partial data) that might impact on the interpretation of the figures. The needs of users are thus very similar to those of the validation team in that they will want to be able to view quantitative and qualitative data side by side without extensive manual manipulation of the data. Recent work to link ESSPROS with other sources of data such as National accounts, MISSOC and LMP has highlighted the importance of the qualitative data, specifically that collected as part of the qualitative information, for being able to establish detailed links. Establishing links is useful for data providers, validators and users in that it allows the data to be crosscompiled, cross-checked and compared with similar sources and for it to be extended with information from other sources. As has been highlighted by recent work to compare ESSPROS and MISSOC (see DOC SP-TF ) it is the names of detailed benefits in English and national language which are key to facilitating links. These need to be complete, clearly distinguished and identifiable in the qualitative data in order to provide the foundation for linking with other sources. It is important to note at this stage that any adjustment to the current system of collecting the qualitative data would require a short-term increase in the burden of work burden for all parties as they adapt to a new system. However, any short-term increases in workload will hopefully be offset by a longer-term reduction. Further, it must be noted that any adjustments proposed need to adhere to what is possible within Eurostat s systems and standards Limitations of current system At present, some parts of the qualitative data are disseminated while other parts are not. The data is disseminated in the same format as it is collected. For instance, the qualitative information is collected and disseminated in an Excel spreadsheet while the national quality reports are collected as Word documents and disseminated as PDF documents. Among the information that is systematically disseminated, there is a clear division between the quality reports which collect technical information and the qualitative information which collects descriptive information. This seems like a sensible division. Data validators will make use of both but users are likely to mostly make use of the latter, except when they are specifically interested in data quality and revisions. However, the footnotes (in QUANTI, BENEF and NET) are not currently disseminated, this represents a gap with no bridge between technical and descriptive information in that there is lack information highlighting key technical issues that may impact the accuracy of the data. For example, where benefits are split between classifications or where estimations are used. Such information is extremely important to ensuring correct use by data users who may have difficulty interpreting the technical information in the quality reports. It is also worth noting that footnotes are not collected in an efficient/consistent manner across the different datasets (QUANTI, BENEF and NET). Meanwhile, the classification of schemes (in QUANTI), classification of pension scheme (in QUANTI) and supplementary information (in BENEF) are all primarily descriptive information but are not disseminated. 10

11 The main benefit of the current system is that the qualitative data are collected and disseminated in formats which are widely compatible, easy to access, handle and compile Excel and Word/PDF files. However, this is also its main weakness as information in this format is not easy to validate (possibilities to implement automatic validation are limited) or to use in detailed analysis as it cannot be easily reorganised and linked with other parts of the data. The structure of the qualitative data and lack of useful common identifiers across the quantitative and qualitative data also play a role. For instance, trying to put together all quantitative data from the Eurostat tables or detailed data by scheme and qualitative data from the qualitative information and quality reports for a certain category of benefit is currently an extremely laborious task that has to be performed almost entirely manually. Linking of ESSPROS data with other sources demands use of the qualitative information and, in particular, the names of detailed benefits. However, all descriptive information about each benefit (name, year of introduction, etc.) is reported together as a single entry in the qualitative information template, which means that the name can only be separately identified or extracted manually. This immediately limits the possibilities to systematically validate that the names of the benefits have been provided and to establish links with other sources (e.g. MISSOC). Indeed, the lack of separation between the different elements of the descriptive information about each benefit is a key weakness to be addressed. There is some overlap in the qualitative data collected. The qualitative information (QUALI) and list of schemes (in QUANTI) both collect information on the classification of schemes. It seems unnecessary to collect this twice but this information also serves to produce breakdowns of ESSPROS data by category of scheme Possible solutions There is room for improving the collection and dissemination of the qualitative data of ESSPROS to address some of the limitations identified in the previous section. Below a number of possible solutions are proposed for discussion by the Task Force. Note that these are proposed without considering in detail existing practical/technical/legal issues. As a result, they should not be interpreted as draft formal proposals by Eurostat, but simply as ideas to trigger discussion (brainstorming) among the Task Force. Division between the different pieces of qualitative data: - The current division between the descriptive qualitative information and the technical quality reports seems sensible and should be maintained, especially as national quality reports play a clear and distinct role in the quality control process and associated production of the consolidated quality report. 5 - The list of schemes (in QUANTI) and classification of pension schemes (in QUANTI) could be merged together or even into the qualitative information to avoid duplication of information. It is important to bear in mind that the list of schemes is mandatory information while the classification of pension schemes is voluntary but this should not 5 Another difference is that the Quality reports refer to the last reference year (and revisions of previous years), while the qualitative cover the complete time series. As a result, structural information concerning the time series should be included in the QUALI rather than in Quality reports. 11

12 prevent them being collected together. Supplementary information (in BENEF) is probably best left separate as it includes more than just qualitative data. Format of data transmission: - The formats in which the qualitative data are transmitted, particularly the qualitative information (QUALI) and the quality reports, are not well suited to validation, particularly automated validation, or use in detailed analysis. An alternative format with appropriate data structures and common identifiers should be considered in order to facilitate more flexible and efficient use and dissemination of the information and, in particular, to support linking with the quantitative data. Footnotes: - Footnotes represent an important gap in the current dissemination of the qualitative data. They represent a useful bridge between the qualitative information and the more technical and less accessible quality reports, picking out technical issues that support correct use and interpretation of data. Note that additional guidance for data providers would be needed to better clarify what should and should not be provided as footnotes and to ensure consistency across countries. - Footnotes could be added to the SDMX files used for transmitting the QUANTI, BENEF and NET using a separate column as was the case for flags. The benefit of this would be that the data, flags and footnotes could be validated against each other. For instance, where there is a flag there should be a footnote explaining it and vice versa. Also, if the quality reports could be transmitted in a more standard format this could open the door to the development of automatic cross validation of flags/footnotes with information in the quality reports about estimated data, partial data etc. - The footnotes should be disseminated so that they are as visible as possible to users. However, Eurostat s current data dissemination system does not currently support the attachment of footnotes directly to the data in the online tables. Other collections, such as the LMP database for example, have overcome this restraint by disseminating footnotes through an excel file which is made available on the ESMS metadata page for the online dataset. Unfortunately, this is not particularly visible as the links to footnotes for data on expenditure and participants are provided only at the end of the metadata in the list of annexes 6. In the case of ESSPROS, footnotes could also be made available through the dedicated page. Qualitative information: - In addition to considering transmission of this data in another format, such as SDMX, the structure of the qualitative information should be reviewed in order to better separate the information requested so that specific pieces of information can be more easily identified, extracted and validated. In theory, it may be necessary to split qualitative information between scheme information and benefit information (and include

13 information from the scheme list and classification of pension schemes in the former) but with clear connections between the two. - It could be useful to disseminate the qualitative information in several formats to cater for different types of users. For instance, data in SMDX format is likely to be favoured by advanced users conducting detailed analysis while a more immediately accessible format would be useful for other users. Indeed, if qualitative information is transmitted in SDMX then PDF reports could be automatically generated for dissemination via the dedicated section. Quality reports: - Collecting this data in a more standard format could open the door to the development of automatic validation of completeness of the quality reports by automatically crosschecking the information provided with that in the quantitative data. - In addition to considering transmission of this data in another format, such as SDMX, the structure of the quality report should be reviewed in order to better separate the information requested so that specific pieces of information can be more easily identified, extracted and validated. However, revising the structure of the information gathered through the quality reports will need considerable reflection and consideration of the needs of the different data collections (QUANTI, BENEF and NET). 4. DISCUSSION BY TASK FORCE The Task Force is requested to discuss the following: Do you agree with the observations concerning the current system for collecting and disseminating the qualitative data? Do you know of any other limitations? Do you agree with the suggestions for improving the system mentioned above? Do you foresee any implementation issues for the ideas/scenarios mentioned above? Do you have any other ideas for improvement? Do you have any comments on possible priorities to start exploring various areas/options? Nota bene: The discussion by the Task Force is intended to serve as brainstorming and to focus more on content-related aspects (e.g. which "instrument" should collect what?) than on technical issues (SDMX vs. excel vs. word format) or legal considerations (in order to do this, we have to change or not to change the legislation). 13