Tuesday, April 30, 2013

Rubbery Numbers for Chinese Aid to Africa

Today, a new paper and media-based dataset on Chinese aid/finance was released by the AidData project, in an event at the Center for Global Development. AidData collected these numbers over the past 18 months, from media reports. According to AidData, the Chinese have committed $75 billion in official development finance to Africa, 2000 to 2011.

I've been getting a lot of emails asking what I think of this study.

Sigh.

I've already provided my comments to the authors in an earlier draft, and warned them about the pitfalls of this approach. Here's my conclusion: this number is way off. Yes, it's a start, and yes, the goal is a good one, but the approach, and the publication of this data at this early stage, is a problem, for several reasons.

First, reliance on media reports for data collection on Chinese finance is a very dicey methodology. I elaborate below.

Second, as with the land grab databases, publishing the database before it has been adequately cleaned will solidify what look like rubbish numbers. Data-driven researchers won't wait around to have someone clean the data. They'll start using it and publishing with it and setting these numbers into stone. Yet with errors of the order of magnitude of those I describe below, what kind of results will they be getting and what kind of conclusions will they be drawing?

Methodology: We saw this early on with the Warner School student project on Chinese aid to Africa, which was used by the Congressional Research Service. This project defined all state-related activities as aid, collected media reports, and then produced the (preposterous) estimate of $18 billion in aid from China for 2007 alone.  We have also seen this with media reports of Chinese "land grabs" in Africa, which created the false impression that the Chinese were very active in land acquisitions in Africa. (Outside of Zambia, where Chinese companies and individuals have been investing since 1990, and a handful of former state-farm aid projects, now privatized with Chinese involvement, there is very little Chinese farming investment in Africa). We've also seen it with media-based data collection on Chinese involvement in hydropower dams in Africa (click here for my take on these efforts).

So, how does this latest effort fare? Not so well, I'm afraid. The main problem is that the teams that have been collecting the data and their supervisors simply don't know enough about China in Africa, or how to check media reports, track down the realities of a project, and dig into the story to find out what really happened. You can start with media reports, but it is highly problematic to stop there.

Mega-errors.  Table 2 in the paper provides a good example of the problems. It contains 20 Chinese "megadeals" totaling over US$38 billion. But only 6 of these 20 projects -- less than a third -- reflect actual deals (Ghana $3 bn CDB credit; Equatorial Guinea $2 bn credit; Angola Phase 1 $1.5 bn, CDB loan to Angola for agriculture $1.2 bn; Cameroon Memve'ele Dam $674 million; Nigeria light rail $673 million). That's around $9 billion.

Thirteen of the other "deals" never happened, are mistakes, or shouldn't be in the list. Some were under discussion but at least so far, have never happened or were cancelled. Others are simply mistakes of some magnitude. China and South Africa signed a vague agreement on economic cooperation and discussed investments worth $2.5 billion, this was not a loan. China's agricultural demonstration station in Mozambique is listed with a value of $700 million (it was actually 55 million RMB, or about US$ 8 million). Debt cancellation did happen but shouldn't be counted as new finance, or a "megadeal" as this is double counting. And so on.

According to this database, the top recipients of Chinese finance have been Ghana, Nigeria, and Mauritania. Yes, Mauritania. Anyone who knows something about China in Africa -- which these researchers do not pretend to -- would be surprised to see Angola missing from this list. My own data (supplemented by field research) suggest that the top three recipients of firm Chinese financial commitments have been Angola, the DRC, and Ethiopia, with Sudan closely following. Mauritania? Not even in the top fifteen. Nigeria? About the same as Niger.

Bottom Line:  The authors are striving for a database that can be replicated by anyone. But that's the problem. This is not research that can be done by just anyone, and especially not by only looking at media reports. That's why two teams -- both lightly criticized by the AidData authors -- have produced better results:  Derek Scissors and Kevin Gallagher, Amos Irwin, and Katherine Koleski. Their methodologies are eclectic, but they know that there is no substitute for painstaking, informed, expert, bi-lingual investigative research and digging into the cases.

Post script: These numbers, as I expected, are already being "spun" in media reports as "Chinese aid to Africa". For a kindred commentary on this, see Philippa Brant's blog post at the Lowy Institute. The controversy was discussed by Eric Voeten at the Monkeycage blog. A story on it appeared at Think Africa Press, and a lively podcast from the China Africa Project. The Witts China in Africa Reporting Project has a collection of many of the links for this debate.









8 comments:

Anonymous said...

I don't think it's fair to blatantly criticize what is intended to be supplementary to information that doesn't exist publicly in the first place. Given that China is a non-traditional donor and does not formally report its development finance, this project is an interesting start to increasing Chinese financial transparency--especially considering that public users have the ability to update projects on the open website that was launched yesterday. Do you not see any value to the project at all? For a scholar, that seems rather ignorant.

Deborah Brautigam said...

Of course, as I said in my comments, I do see value in the project. I think that starting with media reports is a good start. But I don't think they worked hard enough on cleaning this data, which requires experienced people, going beyond media reports. I predict that these numbers will be widely used by researchers who will not clean the data, and that in one to two years we will see a rush of scholarly journal articles using this data and making conclusions about "Chinese aid." Media reports are already reporting these figures as numbers on "Chinese aid" not finance.

Unknown said...

Madame Butterfly


Thanks for the clarification. Of course, its fair. Even in official data on AID, there are serious problems. And as you noted, the problem is that others will run with this database and start regressing all kind of nonsense with it! You are spot on! Aggregated data, even when cross checked with similar sources, does not change systematic error - just compounds it!

Carlos Oya said...

I actually don't see much value in projects that generate wrong numbers, which are then recycled, cited and abused by users who are not aware of the severe limitations of the exercise. The Guardian in the UK also contributing to the spread of these 'numbers' now http://www.guardian.co.uk/global-development/interactive/2013/apr/29/china-commits-billions-aid-africa-interactive

Basically, the outcome of this project will be yet another wave of GIGO analysis (Garbage In = Garbage Out) as it has happened with datasets on land grabs.

Thank you Deborah for doing the difficult job of digging out the methodological problems with this project.

Unknown said...

Professor Brautigam,

Thank you for your feedback, I’ve been working on the AidData China initiative for over 18 months now and its great to hear from you. Most of my knowledge on the relationship between China and Africa stems from your previous work, while the rest has come from my experience collecting and vetting media-based data. Have you had the opportunity to read our updated methodology, or look at any of the project records you have labeled as “Mega-errors?” I think you’ll be surprised if you do.

Regarding the Agricultural Center in Mozambique, our project page clearly denotes the discrepancy between the $700 million figure and the 55 million RMB http://aiddatachina.org/projects/1215, and actually used your blog as a source to confirm the $700 million figure http://www.chinaafricarealstory.com/2012/01/zambezi-valley-chinas-first.html. Based upon the title alone, it may appear like we’re overestimating, but if you re-read it you may find it’s a combined project record for the center and the “pet-project” you described in your earlier post.

Additionally, how do you come to the conclusion that there are no Chinese funders for the Merowe dam in Sudan? Our project page http://aiddatachina.org/projects/178 has 2 independent sources, plus the PPIAF Building Bridges database, confirming the involvement of the Chinese EXIM bank for a commitment of $519 million. Even the official project website lists the “government of China” as a major funder http://www.merowedam.gov.sd/en/funding.html. If you have additional information we don’t that would help improve this project record, we would love to set the record straight.

Deborah Brautigam said...

Hi Alex,
Briefly, read carefully.

(1)My blog does not give any support for a figure of $700 million as Chinese finance for a demonstration center in Mozambique -- it reports the hopes of a Mozambican minister for this kind of finance, which did not happen. Here's what I wrote:

Could [the idea of an $800 million pledge from China] it be related to a request the Mozambicans made for China to help fund the Moambe Science and Technology Park, a pet project of the Minister of Science and Technology? Together with the agricultural research center in Umbeluzi/Boane, the two projects would have cost $700 million (the Chinese agricultural center itself was projected to cost 55 million RMB, about US$9 million) (10). The Chinese did say they would help out with Moambe, but not fund the entire thing. Mozambique later received a mixed grant/credit of $15.8 million from China to support distance education and "science and technology" See: http://www.clubofmozambique.com/solutions1/sectionnews.php? secao=social_development&id=22558&tipo=one

(2) As for Sudan, my source, like yours, is the project website, which lists the funders of the dam, the power station, and the transmission lines. The Chinese provided finance for the hydromechanics of the power station, and the transmission lines for Merowe, not the dam.

The Building Bridges project data was also media-based and not all of their numbers were fully vetted, or included projects that were not fully confirmed, or offers still under negotiation that were never finalized.

Joshua G said...

I totally agree with Deborah.
We can all agree that it is important to come up with a clear picture of Chinese investments and aid activities in Africa. However, as murky as most information coming from China tends to be, this methodology particularly stands out as ineffective and counter-productive.
There will be numerous projects announced at press conferences which never materialize.

Anonymous said...

There is an interesting response on behalf of AidData to some of Professor Brautigam's criticisms of this database and methodology: http://blog.aiddata.org/2013/05/a-rejoinder-to-rubbery-numbers-on.html?utm_source=buffer

Although we can all be sympathetic to AidData's argument that exposing their data to public scrutiny could help to improve on it, I think it is far more likely that these numbers will be misused and misunderstood, despite whatever disclaimers may be given. Scholars may have a good handle on how to use (or not use) this type of preliminary data, but misinformed journalists are often just looking for numbers to support their alarmist views. For example, The Guardian has already written an article about the database with the headline: "China commits billions in aid to Africa as part of charm offensive: Database reveals government has backed 1,700 projects on continent since 2000 in apparent attempt to win favour. The country's financial commitments are significantly larger than previous estimates." http://www.guardian.co.uk/global-development/interactive/2013/apr/29/china-commits-billions-aid-africa-interactive