Mn/DOT Market Research Reporting General Guidelines for Qualitative and Quantitative Market Research Reports Revised: August 2, 2011

Transcription

1 Mn/DOT Market Research Reporting General Guidelines for Qualitative and Quantitative Market Research Reports Revised: August 2, 2011 The following guidelines have been developed to help our vendors understand our reporting requirements. These guidelines apply to all kinds of studies, unless specifically noted. Content Areas: The report should contain, at minimum, the following sections: Cover Page Table of Contents Executive Summary Background Objectives Methodology Readers Notes Summary of Findings/Key Findings Conclusions and Recommendations Detailed Findings Appendix These sections may be combined, if appropriate, at the writer s discretion (though subject to final approval from Mn/DOT Market Research). Each is further defined below: Cover Page: The cover page should include o Project title o Study date (i.e., month and year the bulk of the data were collected) o Date report was written o Report prepared by [vendor] o Report prepared by If the project manager is part of Mn/DOT Market Research, the report is prepared for Mn/DOT Market Research If the project manager is not a part of Market Research, do not include Mn/DOT Market Research in this area. o The cover page may include: logos Table of contents: Include all key report sections and list what is included in each appendix. If the report is in two volumes, make sure both are included in the Appendix. Do not include other materials provided (i.e., data tables) unless they are a part of the report. - 1

2 Executive Summary: The report writer will be asked to write a 2-3 page Executive Summary. The Executive Summary is a short non-technical synopsis of the study, and includes a short description of the project, key findings and recommendations. Background: Describe the background of the situation that leads to the study. If this study is a follow up to earlier research, briefly describe that research as well. If this is a tracking study, include information about previous administrations (i.e., when it was administered previously and key differences, if there are any). Objectives: Why the project was done, what was to be learned and how information will be used. All objectives should be covered in the detailed findings section of the report. Methodology: How the research was conducted, including the sampling plan, and basic screening criteria. Also include information regarding the design and include all key dates. Include any limitations in design and whether the results are projectable and, if so, to what group, or who is being represented. Note: This section should be relatively brief nuances or details regarding sampling, weighting, etc. (i.e., things of interest to other researchers) should be included in an Appendix. o Quantitative research: include margin of error on the total sample, and any key sub-groups, and weighting information, if weighting was done. Note that additional methodology information (for example, extended information about weighting) may be included as an Appendix. o Qualitative research: include a caveat regarding the use of the data Readers Notes: This section should include all information that will aid the reader in understanding the report. This may be included with Methodology at the writer s discretion. This may include: o Definitions of terms o Definitions of subgroups o Report organization information o Information regarding trend calculations o Information regarding significance testing and a note indicating that any differences mentioned in the report are significant and/or meaningful. (In quantitative reports, this is required to be in the report in some location.) o Information regarding highlighting used within tables or graphs. Summary of Findings/ Summary of Key Findings: The findings that directly address the objectives. Mn/DOT prefers that this summary not contain too many numbers, and instead use proportions or other semantic terms to relay the message. These should be worded so that they will be clear to the non-research readers. Mn/DOT would not expect to see graphics in the key findings section as a rule; however, if a summary table or graph is the best way to communicate the information (for example in tracking studies), that is acceptable. Ideally, this - 2

3 section will be very easy to read, formatted in bullets and short paragraphs. Do not include demographics, unless that is one of the key objectives. The Summary of Findings is similar to the Executive Summary, but should include more depth and should only address findings (not background, etc.). Conclusions/Conclusions and Recommendations: This section is optional, depending on the study. Conclusions take the key findings a step further. That is, if two or three findings formulate an idea, Mn/DOT would call that a conclusion (sometimes this is only a possible conclusion). Conclusions are not just a restatement of the findings. And although Mn/DOT does not require consultants to make recommendations, it is a plus when you can make well-founded and defensible ones. This section may be combined with Summary of Findings at the writer s discretion. Note that all recommendations should able to be traced back to this study, though may include information drawn from other sources or previous research. (For example, if no one in the survey has mentioned congestion, Mn/DOT would not expect a recommendation regarding this issue.) Detailed findings: Present the results of the study. Use charts and graphs to enhance the readability of this section. Structure the presentation of the data to tell the story, rather than presenting the data in questionnaire order. Note that some questions may not be included in the detailed findings (example: screening questions, demographics), but every question should be in the report in some way. In quantitative studies, report only those differences which are statistically significant. If differences are not significant, don t report them. Please avoid comparisons, such as higher, lower, significant, and so on, except in reporting statistical differences. Appendix: The appendices should include any material necessary for interpretation of the report. This may include: o Survey documents: questionnaires, screeners, discussion guides, handouts, etc. o Demographics: in studies of the general public, where possible, include census data to show how the study parallels the general public o Sample resolution data: incidence, hours on phone, completion ratio, number of calls, etc. o Detailed information regarding advanced statistical testing o Sample of comments included in codes Tracking studies: The point of a tracking study is to understand changes over time. As such, the body of the report should focus on trends and the interaction of the current year data with past data (i.e., if females rate something higher than males in the current year, is this similar to the past or is it unique to this year?). Additional current year information the writer feels should - 3

4 be included in the report, should be included in an Appendix (unless, of course, the questionnaire was changed). Remember to include a note in the body of the report referring the reader to the Appendix. Layout: It is important that each page of the report be able to stand alone. As such, every page should include the following: Report or Study Title Date (date = month and year in which the bulk of the data were collected) Page number Because of the requirement that each page stand alone, it is particularly important that charts and graphs be formatted as follows. Each chart or graph should include the following (footnotes are okay or elsewhere on the page, as long as they are obviously/clearly related): The full text of the given question. [If the question is too long to include, it can be abbreviated and the reader referred to the questionnaire in the Appendix.] Base sizes (n= ) Whenever possible, avoid splitting tables or graphs across pages. Others for open-ended questions should represent only 5% of responses, with 10% being the maximum. (Mn/DOT Market Research can grant exceptions to this rule.) Remember, all code lists need to be approved before using. Color: Be careful when using colors. For example, the color red generally connotes negative or bad results and should not be used as a highlight for positive results. Also, be sure that colors will be able to be distinguished between readers, though try not to make color differences so extreme as to be garish or distracting. Mn/DOT do not dictate the layout or graphics to be used; however, expect that the writer/analyst will present the data in ways that are most reader friendly, and follow generally accepted graphic/reporting standards. (For example, vertical bar charts are preferred for time series data.) In addition, Mn/DOT appreciates well-edited prose and proper grammar. A few reminders: Write concisely. Please self-edit and have someone who is unfamiliar (or less familiar) with the study read for clarity. Watch for common usage errors (affect vs. effect, etc.), and remember that data takes the plural ( data are not data is ). Also avoid the use of persons unless you re writing legal text, people is appropriate. And, when you re talking about people, use the word who rather than that, (i.e., respondents who participated not respondents that participated ). Watch for common grammatical errors (commas and periods go inside quotation marks always). Titles should be short and to the point. Formatting should be consistent throughout the report. This applies to text and titles as well as graphs and charts. - 4

5 Also, please note that a report or page may be printed in black and white. Please try to make sure colors used in graphs show some distinction when not printed in color. Deliverables: Unless otherwise stated in the contract or agreed upon with your project manager, deliverables should include: Clean data file (in Excel or SPSS) with a data map or layout and codes Data tables with statistical testing An electronic version of the report, in MS Word Three bound copies of the final report - 5