Thursday, January 19, 2012

Measuring Regulatory Performance: A Practitioners' Guide to Perception Surveys

Now more than ever, OECD countries are investing significant resources in regulatory policies and reforms. At the same time, governments are under increasing pressure to explain such reforms and their benefits to the public. Perception surveys are an important part of this process and they are being used by OECD members to measure how citizens and businesses view regulation in their countries.

The OECD has developed a guide that: helps officials use perception surveys to evaluate and communicate the results of reform processes;clearly explains the challenges involved in the design and use of business and citizen perception surveys – and ways to overcome them and helps to get the most out of survey results, whether conducted internally or by external experts.

Perception surveys are an increasingly integral component of a business- and citizen-centred approach to regulatory reform, as a means to assist governments with better results in an open, democratic system. This guide helps officials planning perception surveys or engaging external expertise to use perception surveys for evaluating and
communicating progress in regulatory reform. It explains the challenges involved in the design and use of business and citizen perception surveys – and ways to overcome them. It will also help officials responsible for writing and evaluating tenders for surveys judge the quality of consultants’ work and get the most out of survey results. The guide is written in non-technical language for a broad audience, drawing on examples from the regulatory field.

The following ten key policy messages are presented in this guide:

1. Understanding and improving the perception of the regulatory environment matters to performance. Positive perceptions of regulations can influence investment decisions and promote respect for and compliance with regulations. (Introduction)

2. Perception surveys are increasingly used in OECD countries to evaluate the performance of regulatory reform programmes, in particular in the area of reducing administrative burdens. Perception surveys are further used to obtain information on the level of awareness and confidence in regulatory reform programmes among businesses and citizens, and as a diagnostic tool to identify areas of concern to business and citizens in order to inform future regulatory reforms. (Chapter 1)

3. If pitfalls in survey design are ignored, survey results become unusable for policy makers. There are a surprising number of pitfalls in designing surveys. For example, even the order and phrasing of questions can affect responses and the quality of survey results. (Chapter 2)

4. Using good practice methodologies will improve the quality of results considerably and help to avoid pitfalls. For example, it is advisable to run pilot surveys to identify questions that respondents have difficulty understanding and then adjust questions accordingly. (Chapter 3)

5. Perceptions and hence survey results are shaped by many factors; the actual quality of regulations is only one of them. For example, perceptions of the quality of regulations can be influenced by trust in government, the current economic situation, experience with front-line service, prior expectations and the content of government (and general media) communication. (Chapter 4)

6. It is necessary to look beneath survey results. The same survey results may be driven by very different underlying factors. In-depth questions and selected qualitative research techniques can prove very valuable in bringing to light the reasons for the results and drawing concrete policy conclusions from survey results. (Chapter 4 and 5)

7. Irritation from experiences with regulation and frontline service can account for a significant degree of business and citizens’ dissatisfaction with regulation. This experience is often more negative than might be suggested by the measurable costs of administrative burdens. (Chapter 4 and 5)

8. Perception surveys also have their limitations. Experience suggests the likelihood of a disparity between the perceived quality of regulations as reported by business and citizens and the measurable results of regulations. For example, in many countries surveys have tended to reveal negative perceptions of the quality of regulations while more fact-based measurements have shown an improvement. This appears to apply particularly to programmes targeted at reducing administrative burdens. (Chapter 5)

9. A comprehensive evaluation system should include different types of indicators, each revealing different information for policy evaluation. Discrepancies in results can highlight the need for deeper analysis to evaluate and inform policies. (Chapter 5)

10. Perception surveys are an integral part of a two-way communication strategy with stakeholders. They can serve as a means to communicate stakeholder views to the government, and discussion of the results can lead to fruitful exchanges between government and stakeholders in the case of regulatory reform. Survey results can also help to evaluate the success of the government’s communication strategy by assessing stakeholders’ level of awareness of recent initiatives. (Chapter 5)


No comments:

Post a Comment