IPC Integrated Food Security Phase Classification. Lesson: IPC Quality Assurance

Transcription

1 IPC Integrated Food Security Phase Classification Version 2.0 Lesson: Text-only version In partnership with:

2 In this lesson LEARNING OBJECTIVES... 2 INTRODUCTION... 2 WHERE YOU ARE IN THE IPC PACKAGE... 2 PROTOCOLS FOR ASSURING QUALITY... 3 PROCEDURES FOR ASSURING QUALITY... 3 SUMMARY

3 LEARNING OBJECTIVES At the end of the lesson, you will be able to: explain the purpose and general procedures for assuring Integrated Food Security Phase Classification (IPC) quality; and explain the Technical Working Group (TWG) Self-Assessment Tool and the Peer Review Tool. INTRODUCTION IPC results are based on the convergence of evidence, reasoning and human knowledge. Therefore, these results can be prone to human error and agendas. The purpose of quality assurance throughout IPC is to ensure technical rigour and neutrality of analysis. In this lesson you will learn about Function 4 of IPC which includes a series of quality assurance mechanisms and two additional protocols for quality assurance: TWG Self-assessment and Technical Peer Review. WHERE YOU ARE IN THE IPC PACKAGE Functions Building Technical Consensus Classifying Severity and Causes Communicating for Action Quality Assurance Tools TWG MATRIX ANALYTIC FRAMEWORKS REFERENCES ANALYSIS WORKSHEETS COMMUNICATIO N TEMPLATE SELF ASSESSMENT PEER REVIEW Procedures for Multi-agency stakeholders to do collaborative analysis Understanding evidence with an integrated Analytical Framework Referencing evidence against international standards Transparently, methodically and consensually analysing evidence Transforming analyses into concise information for action Assuring for quality 2

4 PROTOCOLS FOR ASSURING QUALITY The first two IPC functions (building technical consensus and classifying severity and causes) already include several quality assurance mechanisms, such as FUNCTION 1. Building technical consensus Formation of a TWG Documenting evidence 2. Classifying severity and causes Assigning reliability scores Assigning confidence levels 3. Communicating for action 4. Quality Assurance In addition to these mechanisms, Function 4 specifically aims to ensure the quality and the neutrality of the IPC analysis through a series of additional quality assurance mechanisms and protocols. PROCEDURES FOR ASSURING QUALITY In particular, there are general quality assurance procedures and two additional protocols and related tools for quality assurance. Provide source of evidence, reliability scores and confidence level Hold a consultative meeting with stakeholders Complete a Conduct a TWG Self- Technical assessment Peer Review Publish analysis worksheets TWG Selfassessment Tool Peer Review Assessment Tool 3

5 Provide source of evidence, reliability scores and confidence level Ensure that source and reliability scores for each piece of evidence have been assigned and that the overall confidence level of the analysis is clearly indicated on the communication template. The analysis had been carried out with many limitations. In particular, data were often inadequate and obsolete at the time of the analysis on some indicators. The analysis risked being unreliable and of poor quality. The TWG indicated the source and reliability for each piece of evidence and did not include data with a ranking lower than "somewhat reliable". Where confidence level was below this ranking, areas were not classified and were identified in grey on the map. Hold a consultative meeting with stakeholders Hold a consultative meeting with key decision-makers to technically check the findings and make any necessary revisions based on additional evidence before public release. The main obstacle to the analysis had been limited coordination between the TWG and the decisionmaking centres. There was the risk of a possible disagreement on the IPC findings, because of this minimal coordination and information sharing Before public release of the findings, the TWG held a consultative meeting with core TWG IPC partners who engaged in the analysis, including government representatives. In the end, they all agreed on the analysis findings and on how these were portrayed in the communication brief. 4

6 IPC Information Support System (ISS) The TWG Self-Assessment Tool can be also done online using the ISS application. The ISS is a Web-based application that has been designed to facilitate global, regional and national stakeholders in creating, storing, and disseminating IPC analyses and information. To know more about ISS see Lesson Introduction. Complete a TWG Selfassessment TWG Selfassessment Tool Complete the TWG Self-Assessment Tool after every analysis to reflect on how well the TWG followed the IPC protocols and to identify areas for improvement. The TWG Self-Assessment Tool should be completed by the chairperson of the TWG, in consultation with TWG members, and should be circulated to stakeholders. It includes questions on the four IPC functions and allows you to insert comments and indicate areas for future improvement. Conduct a Technical Peer Review Conduct a Technical Peer Review (if necessary) and have reviewers complete the Peer Review Assessment Tool. Peer Review Assessment Tool Many agencies had collected data, but it was soon realized that not all data were made available during the analysis. When the analysis findings were finally shared, key stakeholders strongly disagreed on the analysis. The TWG chairperson called for a Technical Peer Review by technical experts external to the TWG, who disagreed on the evidence provided to support the classification. Based on their feedback, the TWG decided to revise the analysis. 5

7 The Technical Peer Review is conducted by external experts to review findings and process, and is typically conducted by experts who were not involved in the analysis. Reviewers may include professionals from country, regional or global levels. They should complete the Peer Review Assessment Tool to document their feedback in a systematic and comprehensive manner. Make Analysis Worksheets publicly available. National authorities accused the TWG of presenting the situation of some areas as being worse than is actually was. To ensure transparency, the national TWG decided to have a meeting with the authorities and to present the evidence used. Analysis Worksheets were also published online and evidence and data for each indicator were made publicly available. SUMMARY The purpose of Function 4 is to ensure quality and neutrality of analysis. It includes the following quality assurance protocols and general procedures: ensure you indicate the source and reliability score for each piece of evidence and the overall confidence level of the analysis; hold a consultative meeting with key decision-makers; complete Technical Working Group Self-Assessment Tool; conduct a Technical Peer Review (if necessary) and have reviewers complete the Peer Review Assessment Tool; make Analysis Worksheets publicly available. The Technical Working Group (TWG) Self-assessment Tool allows the TWG to reflect on how well the IPC protocols for classifying food security were followed and to identify areas for future improvement. The Peer Review Assessment Tool lists questions to evaluate the technical rigour and validity of the analysis and should be done by technical experts external to the TWG. 6