Planning to donate? Know what your dollar buys

WHEN we buy stuff, we want to know we're getting the most for our money.

So why don't we demand the same thing with our charitable donations? If we can buy twice as much good for the same money - or three times as much, or a thousand - why don't we?

"There's a very emotional aspect to giving," said Michael Thatcher, the president and chief executive of Charity Navigator, the website most used to evaluate charities. "We are touched by a situation and somehow want to impact it." We give with our hearts. But we should also give with our heads. Every problem we care about has many, many different organisations aiming to solve it. Which should get our money?

Say you care about clean water. A $50 donation to a charity called Give Clean Water can provide someone with a year of clean water. But the same donation to Evidence Action can buy 100 people a year of clean water. Same cause - but with a different charity, your gift does 100 times as much. It's as if one of the charities wasted 99 per cent of its money.

Some causes have even bigger impact gaps. Toby Ord, a senior research fellow in philosophy at Oxford University, likes to cite the cause of helping the blind. For $42,000 you could train a guide dog. But for the same money, you could pay for trachoma surgeries in Africa to restore the sight of 1,344 people.

Such comparisons should be as normal to making donations as comparing prices is to shopping. They aren't. A report on donor behaviour in 2010 found that 90 per cent believed that the performance of a nonprofit was important, but only 3 per cent had done any research about effectiveness before giving.

Which makes sense, as there has been no way to get that information. But as of last month, there is. The clean-water comparison above came from ImpactMatters, a new organisation that offers one-sentence descriptions of the good your money will create.

For example, the entry for Evidence Action says, "$0.50 provides clean water to one person for a year." ImpactMatters finds how much the organisation spends by looking through the charity's 990 tax form, a document every nonprofit must make public. Then it looks through the charity's website for claims about outcomes: How many students graduated? How many trees got planted?

COUNTERFACTUAL

There's one more ingredient going into that impact sentence, called a counterfactual: What would have happened without the charity's help? ImpactMatters researches questions like: How many households in the village would have been able to get clean water another way? What's the normal graduation rate among this group? ImpactMatters subtracts what would have happened anyway from the equation.

Great idea, but as ImpactMatters readily admits, it sometimes doesn't work. The organisation can't always get enough detailed information to calculate the counterfactual. There's lots to improve in ImpactMatters - not surprising for a three-week-old system. Information on outcomes comes from the charities themselves and so could be biased or invented.

The group's president, Michael Weinstein (a former economics writer for The New York Times editorial page), said he expects the A+ charities to demand audits by impartial outsiders so that they can better demonstrate how effective they are. This will improve the ratings.

ImpactMatters has rated about 1,000 charities. All provide services. They help veterans get benefits, help students graduate from college, feed meals to the hungry and do other direct work with clients. It's much harder to measure the good done by other kinds of charities. How do you calculate the impact of a research or advocacy organisation? A museum? A church?

And even with service providers, ImpactMatters can miss nuances. Dr Weinstein gives the example of a seemingly effective homeless shelter. It might serve people cheaply because it scrimps on cleaning and security. He said that in the future, ImpactMatters will ask shelters for information about problems such as robberies and fights.

Despite its rookie problems, ImpactMatters's debut is a big step forward. It's the first broad ratings system for charities that actually measures how much good they do.

Charities love to feature a four-star rating from Charity Navigator on their websites. But that rating isn't proof of effective work. The group's ratings used to be based entirely on financial metrics, mainly on the percentage of a charity's money spent on administration and fund-raising (known as overhead) as opposed to programmes. Overhead is still the measure most people identify as the test of a charity.

But judging an organisation solely by overhead isn't just useless - it can actually cause harm. Organisations that get things done need to spend money on training staff, buying equipment, hiring the right people, gathering data and evaluating their results. All those things drive up overheads.

Charity Navigator and other ratings organisations measure overhead because they can - the percentage is easy to calculate from the 990 form. "Forget that it's the wrong number. It's a number," Dr Weinstein said.

Overhead also addresses a fear shared by many donors: Is this charity stealing my money? Too much spent on overhead and too little on programmes could be an indication of fraud.

But such scandals are rare. (The public imagines they are common because they often get a lot of media attention.) And if there's evidence of fraud, Charity Navigator has other ways to inform donors - there is no need to make this the basis of its ratings. "We run advisories when there is potential wrongdoing: attorney general investigations, cases of fraud, credible news sources doing an investigation," Mr Thatcher said. "On average there are about 400 advisories, out of 1.7 million nonprofits. That's tiny."

The charity sector is well aware of the deficiencies of overhead. Overhead "is a poor measure of a charity's performance", read an open letter to charitable donors from Charity Navigator, Guidestar and the BBB Wise Giving Alliance. "In fact, many charities should spend more on overhead." All three organisations have broadened their evaluations of charities to include other measures of how well they are run. Charity Navigator, for example, looks at how well a charity is governed. Is it accountable and transparent?

An assortment of metrics is better than one. But Charity Navigator still rewards low overhead, a decision that even its close partners criticise. And none of these measures answers what people need to know: Does this charity actually make a difference?

Mr Thatcher said that Charity Navigator plans to include ImpactMatters ratings in the profiles of charities that accompany ratings. Dr Weinstein argues that it's not enough.

"The only way to get donors to consider effectiveness is to incorporate it into the star ratings," he said. "Charity Navigator has 11 million viewers, but virtually none read anything other than how many stars a group has gotten." Mr Thatcher said that Charity Navigator is working on adding impact measurements to the star ratings. "That is our intention, but we want to do that in a thoughtful way," he said.

ImpactMatters is part of a two-decades-old movement to evaluate social programmes. In October, Esther Duflo and Abhijit Banerjee of MIT and Michael Kremer of Harvard were awarded the Nobel Prize in Economics. The Nobel committee said that the three "have considerably improved our efforts to fight global poverty" through their use of experiments to test how well social programmes work.

Which intervention is most effective for keeping children in school? These economists can tell you. (It's telling parents about the benefits that education holds for their children, by the way.) Dr Duflo and her colleagues find answers with randomised control trials. Nicknamed randomistas, these researchers use the same methods of experimentation we use to determine whether medicines and scientific innovations work.

DIFFERENT APPROACH

GiveWell takes a different approach. It carries out exhaustive investigations of a small number of charities, recommending seven where a charitable donation goes farthest. Its top charities - all in developing countries, where money buys more - distribute bed nets for malaria, give out vitamins to prevent blindness and deworm children. One, GiveDirectly, sends poor people cash.

What GiveWell and the randomistas do is crucial. But their evaluations are so in-depth and expensive that they cover only a tiny number of charities. Knowing the best way to keep girls in school or prevent malaria is helpful for governments, aid agencies and big foundations. It's less useful for the large number of individuals giving to hundreds of causes.

Broader guidance is necessary - and ImpactMatters is a first stab at providing it. "We want no donor, however small, to think of making a grant if they had no evidence of the impact of that grant," Dr Weinstein said. "We want to make impact a necessary part of philanthropy." NYTIMES