Measuring the impact of citizen media – academic and donor perspectives

Anne Nelson will be speaking at the Summit in a session on Measuring the Impacts of Grassroots Citizen Media in Developing Countries. These are preliminary notes.

I work at an extremely busy intersection between old media and new media, consulting for foundations on media projects and teaching a research course at Columbia called “New Media and Development Communications.”

Citizen media has transformed the media agenda. The word that best describes the reactions of traditional institutions is “overwhelmed.” Both foundations and government donors have begun to recognize the importance of social media and fund leading efforts (including Global Voices). But the new path has its own perils. In the words of one of my colleagues, “Donors have to ask whether what they're doing is making sense.”

The donor and the implementer communities represent two distinct cultures, that can be summed up in the contrast between the idea of the “architecture of knowledge” and the McLuhan-inspired concept of media ecology. Architecture is created from a specific purpose and blueprint, taking shape through defined skills and stages of construction.  An ecosystem, on the other hand, is organic. Species spring up unannounced, including marvelous transplants and hybrids — as well as invasive species and kudzu.

The institutions that have evaluated media in the past have tended to be large, slow, and interlocked: government and intergovernmental agencies, foundations, and their university and publishing counterparts. Career tracks are linear, credentials are a serious matter, and information is subjected to editorial review, sometimes in the interest of accuracy, sometimes to promote conformity.  These factors are the hallmarks of traditional journalism.  As Steve Coll pointed out on an OSI panel last January, traditional journalism developed in parallel with the scientific method, an “architecture” of argument based on a structured process of tested thought and review.

The past decade of growth in citizen media has challenged the old architecture in unprecedented ways. It has yielded a tremendous bounty of both individual expression and data.  Once new media platforms become available, new communities use them in unexpected ways. Many of them have obvious social utility.

So here we have a logical progression:  good project emerges, donors offer seed money, project grows. Some donors accept that in an era of experimentation, they have no choice but to “run toward the confusion,” as Eric Newton of the Knight Foundation has phrased it, and fund on an experimental basis. The Knight Foundation's News Challenge grants are a leading example.

The crunch arrives at the next phase of development. Good project grows, grant expires, next proposal is submitted. Now the bar is raised: it's not enough to have an “interesting idea.” The project, however worthy, must either make the leap to being “sustainable” in donor terms (receiving an ongoing flow of contributions) or develop a “business model” that generates a revenue stream.

This is where the “impact evaluation” rubber hits the road. For any projects that participate in a commercial sphere, the metrics are taking shape, logically enough, around the principles of commerce. It's now easy to measure web traffic, clicks, and sales.  It's hard — if not impossible — to measure “democratization.” I think Lova Rakatomalala has made some terrific points about qualitative and quantitative approaches to impact evaluation in his post.

I'd like to step back a bit. Any process of “impact evaluation” requires an underlying value system — and the premise of a shared value system between donor and implementer. Often, that simply is not the case — sometimes because there is tension, but more often, because the values haven't been clearly defined. For example, what is the role of a citizen media project in measuring public opinion? Participation is one measure. But societies differ wildly as to what proportion of society has online access (and which slices of the population are represented). What is the relationship between the online population and the traditional elite?

Every donor has a set of goals, even if they are often described in vague and generic language. By the same token, every institution that participates in the sphere of research has a value system, and these can vary tremendously. Donors are seeking sources of research to inform their grants, but they encounter a fragmented research community that struggles to keep up with developments. I'll name just a few of the research centers that are active in the field, and welcome comments that add others of note.

The Center for Independent Media Assistance commissions reports and has created a database of international media research. Other Washington-based institutions, including IREX, the International Center for Journalists, Internews, and AudienceScapes, have built out new research on aspects of media development. The Toronto-based Communication Initiative Network publishes extensive updates on global media development projects. The Global Forum for Media Development has begun to publish research drawing on the resources of its member organizations.

U.S. universities are beginning to explore the terrain, though their approaches vary widely. Two major threads are emerging from Harvard these days, with quite different emphases. The Kennedy School of Government's Pippa Norris has worked closely with the World Bank and the UNDP on issues of transparency and governance. In 2009 she published a landmark report for the World Bank's Communications for Governance and Accountability Program called “Public Sentinel: News Media and Governance Reform.” Although the report deals extensively with traditional broadcasting, it also makes forays into the subject of new media. Norris has been publishing additional articles on the role of social media in the British electoral process.

Harvard's Berkman Center for Internet and Society was founded at Harvard Law School, often emphasizing legal and freedom of expression issues. Although it began with a U.S. focus, Berkman has been increasing its international scope over the past few years.

Great Britain hosts other important research projects. One is the Oxford Internet Institute, whose research includes a new project on East Africa, and combines issues of law and governance with studies based in sociology and anthropology.

The London School of Economics’ Department of Media and Communications links its research to work in development.

Another important player in the British research community is the BBC World Service Trust, which has developed a program in “Research and Impact.”

There is no way to begin to register all of media research-related programs underway in the U.S. and Britain, let alone in the rest of the world. The University of Hong Kong, under the leadership of Ying Chan, has contributed important understanding of the workings of media in China, while Guy Berger at Rhodes University has gathered important data on Africa. Not surprisingly, there's often little formal research in countries with weak university systems — though that's changing rapidly.

Given the current accelerated technological and social change, some unexpected research tools have also appeared. One is an outgrowth of Global Voices itself, in the form of Rising Voices’ Technology for Transparency, which surveys a number of citizen media projects around the world to see how they affect civic life.

But the truth is that the various organizations often don't solve the problems of the donor/implementer connection. The donor is asking: does the implementer's work accomplish my goals? The implementer asks “Will the donor's support make it possible for me to grow as I wish?” If the donor is offering apples and the implementer needs oranges, a database of research is rarely flexible enough to broker the relationship.

The world we're examining is full of gaps: in generations, culture, technology, income, and certainly in communication. At the same time, I thoroughly believe there is a world of shared values beneath the differences in terminology. Every mode of architecture begins and survives within an evolving ecosystem. So we may as well roll up our sleeves — anyone for a treehouse?

1 comment

Join the conversation

Authors, please log in »


  • Please treat others with respect. Comments containing hate speech, obscenity, and personal attacks will not be approved.