The headline screams “EU lobbyists wary of media, civil society” as the European Centre for Public Affairs (ECPA) present their annual ‘State of Public Affairs’ survey for 2008. Intrigued, I clicked through to learn more. The article highlights that 65 % of the respondents view the European Parliament as “more likely to be persuaded by civil society” although the point is also made that all of the lobbyists rated the Parliament as the most accessible EU institution so presumably even if the MEPs are biased towards NGOs, other interest groups still feel that they can get in the door. It also seems that the people who answered the survey felt that using the media was a gamble because the story cannot be ‘controlled’ and 25 % of the respondents indicated that they would not trust journalists at all.

So what does this survey tell us? What lessons should be drawn from it?

My first issue with this survey is the size of the sample – just 140 people. Can we consider this to be representative of the public affairs community in Brussels? A quick google search reveals a huge different in the estimated size of the lobbying sector in Brussels, ranging from 15,000 to 20,000 people or more. So, if we use 15,000 people as the potential group of people to be sampled, 140 individuals represents 0.93 %. Of those who responded, 50 % work in trade associations or corporations, 5 % in NGOs and 7 % in government bodies. No data is provided on where the remaining 38 % of the respondents work. So, the viewpoint of less than 1 % of the lobbyists in Brussels (half of whom work in the commercial sector) feel that the Parliament listens to civil society more than they do to industry. So no surprises there, hardly seems sufficiently strong material to base a ‘State of Something’ report on.

The survey is a classic piece of hype around a statistically insignificant poll. The real question is why this is being considered as a new item or worthy of being reproduced in European news portals.

Participative democracy initiatives at EU level present special logistical problems of language and geography. However, they share many of the same fundamental issues of usefulness and motivation with small scale, local initiatives. In the end, the success or failure of all such initiatives comes down to an issue of trust. Citizens must trust that their ideas will be used by policy makers. Policy makers much trust citizens to be co-creators of policy. Without mutual trust, these initiatives can actually increase citizen cynicism and the distance between governments and people.

Check out this wonderful podcast discussing issues of trust, motivation, relevance, and much more in participative democracy projects in the UK. Much of this mirrors what’s being said about similar initiatives at EU level. Click here to listen to Andy Howell of Political Futures blog talk with Stella Creasy and Edward Andersson of the UK-based think tank Involve about “Active Citizens and Public Engagement”.

Thoughts from The Centre’s “Polls Apart” Roundtable (20 February 2008) – Part II

Since surveys alone cannot provide a complete or nuanced view of public opinion, they are best preceded or complemented by qualitative research. Samples are small and not necessarily random. So, opinions may not be representative of the broader public. But, since questions are open-ended and flexible, a range of opinions can be expressed and unexpected ideas can emerge. The complex factors and emotions that form opinions can be explored in depth.

Aware of the limits of surveys, the Commission in its paper Communicating Europe in Partnership promises more widespread use of qualitative research tools, both traditional focus groups and new deliberative methods.

Focus Groups

The Commission has conducted focus group studies for many years. Focus groups are a relatively simple, inexpensive, and quick qualitative research tool. A small group of people meet to discuss, in a loosely structured way, a topic for a couple hours. A broad range of opinions can be discovered and both rational and emotional factors that influence opinion explored in detail. However, as with surveys, opinions may be based on a poor understanding of the issues.

It is not clear how many focus group studies are conducted or how they are used by the Commission. Unlike Eurobarometer surveys, few qualitative study reports are made public.

Deliberative Methods

Deliberative methods, such as deliberative polls, citizen juries, and 21st Century Town Meetings, have been used for years by local and national governments to obtain citizen input into policies. They give participants information about issues, several days to weeks to consider them, and the opportunity to discuss issues with others to arrive at carefully considered opinions and preferences. They can be used to engage citizens in policy making, help citizens understand issues, and/or shape policy development.

They are just beginning to be used at EU level and there is still much discussion as to their proper role. The Commission has yet to incorporate them into its regular public opinion research practices.

Two Plan D pilot projects used deliberative methods, for the first time, at EU level. The Tomorrow’s Europe deliberative poll combined a survey with deliberations among randomly selected citizens from all 27 EU member states. Its goal was to determine how opinions about EU policies evolve with adequate information and trans-national deliberation.

European Citizens Consultations used structured discussions at national level among randomly selected citizens, followed by discussions among country representatives at EU level. Its goal was to find common ground, frame issues, and reach a consensus on EU policy priorities for the future.

Critics question the utility of deliberative processes that provide participants with more and/or better information than they’d normally receive. Supporters contend that these methods mirror how public opinion evolves in real life: people learn about an issue, discuss it, and then arrive at their own opinion. That opinions will change with information seems obvious and measuring this change unnecessary to some. However, to others, learning how opinions evolve is valuable.

There seems to be consensus that deliberative techniques are valuable at EU level for complex or technical policy areas. It is virtually impossible to obtain useful citizen input on some subjects without first providing background information and time for reflection. The Meeting of Minds citizen consultations proved that ordinary people could contribute thoughtful, reasoned policy suggestions on topics as complex as neuroscience.

Deliberative methods can also be used to engage citizens in and educate them about EU policy issues. Critics point out that only a small number of people actually participate in these projects, making this approach too costly. However, extensive media coverage can have an important multiplier effect, exposing millions of others to the topics and ideas expressed.

Both of these pan-European deliberative projects were much more expensive than traditional Eurobarometer surveys or focus groups. For that reason, perhaps more than any other, there is some reluctance to currently consider these tools as anything other than “experimental”.

Thoughts from The Centre’s “Polls Apart” Roundtable (20 February 2008) – Part I

One way the “voice of the ordinary European” currently reaches EU policy makers is via opinion polling. The European Commission’s Eurobarometer surveys measure what Europeans know and how they feel about many topics. Sample sizes are large. Respondents are chosen randomly. Theoretically, this means that responses are representative of all Europeans. But, is this really true?

What surveys actually measure

What surveys do best is put opinions and knowledge into perspective. They tell us how many people know this or which kinds of people support that. They can be quite valuable for tracking changes in opinion over time and comparing opinions of one group vs. another. But, they don’t necessarily tell us how people themselves define the issues, how well they actually understand those issues, or the complex and often contradictory feelings and thoughts that form opinions. This is because respondents can only choose from a pre-selected list of answers to a question. They can’t add different answers (or questions). They can’t ask for clarification of questions they don’t understand. So, whoever writes the questions and answers effectively determines the range of public opinion that is measured.

Who writes the questions (and answers)?

In the case of most Eurobarometer surveys, both the questions and answers are written by Commission bureaucrats and reviewed by research experts with no input from regular people. There is rarely preliminary qualitative research to determine how ordinary citizens might frame issues or define the range of possible answers. There isn’t even pre-testing of questions to see if they’re understandable. Questions that may be embarrassing to the EU or a member state simply are omitted.

How they’re used

Eurobarometer surveys can become quite controversial once you consider how they’re used. On one hand, they can give decision makers the political courage to tackle issues that are genuinely important to citizens but politically sensitive, like climate change. On the other hand, they can be misused by politicians to promote agendas that informed citizens might not actually support, like the notorious use of a biased Eurobarometer survey to justify EU funding for Galileo.

Ideally, Eurobarometer surveys would be used early in the policy development process to help set agendas that reflect citizens’ desires. They would also be used to plan communications strategies to best explain policies and programs. In reality, they often take place too late in both of these processes to be used this way. Instead, they’re frequently used to fulfill statutory obligations to measure the impact of policies on citizens or to provide “news value” when promoting EU policies and programs.

Transparency

Unlike most opinion polling by national governments, the results of Eurobarometer surveys are made available to the public. This is a good thing. But, transparency clearly has its limits.

Results of Eurobarometer surveys that support Commission goals tend to be released with great fanfare. Other studies are released quietly, without even a press release.

Reports summarizing the Commission’s interpretation of the data are easily available on its website. Raw results (that could lead to different analyses and conclusions) are also made available to the public. However, they’re difficult to access and require expensive statistical analysis software to use.

Fortunately, the Commission is experimenting with new ways to introduce more true “randomness” into their public opinion research. That will be discussed in part II.