Counting What Counts: One Foundation's Experiment in Measuring Everything

When the staff of a small community foundation in the Midwest decided to audit its own ripples, the first spreadsheet alone ran to forty tabs. Three years later, the foundation has published its findings — and its missteps — in a candid report that is quickly becoming required reading among funders wrestling with the same question: how much can you measure before measurement itself becomes the mission?

Beyond the Program

Most foundations evaluate what they fund. A job-training program reports placements. A food pantry reports meals served. These numbers matter, but they capture only the first ring of impact. The foundation in question wanted the second ring: did the job-training graduates stabilize their households, vote more often, patronize local shops? Did the food pantry's presence reduce emergency room visits on its block?

The ambition was admirable. The execution, staff now admit, was chaotic.

What Proved Meaningful

Two measurement practices survived the experiment and earned permanent places in the foundation's toolkit.

First, partnering with a regional university to conduct a longitudinal survey of grantee beneficiaries — tracking employment, housing stability, and civic engagement over eighteen months. The university's independence lent the data credibility that self-reporting never could, and the findings genuinely reshaped grantmaking. The foundation shifted nearly a fifth of its discretionary funding toward programs that produced measurable spillover effects in neighborhood commercial corridors.

Second, a simple annual question posed to grantee organizations: what changed in your community that you did not directly cause? The answers were often anecdotal, occasionally unverifiable, and consistently illuminating. One workforce nonprofit reported that a competing regional employer had raised wages simply to retain workers the program was training. That is a ripple no spreadsheet predicted.

What Became Vanity

Not every metric earned its keep. The foundation's early dashboards tracked social media impressions, the raw count of community events attended, and — most infamously — the number of 'touchpoints' per grantee per quarter. Staff spent an estimated ten hours a month feeding a system nobody read.

The lesson, according to the foundation's evaluation lead: if a metric does not change a decision, it is decoration. Numbers that flatter but do not inform crowd out the ones that could. By the second year, the foundation had cut its tracked indicators by more than half.

Measuring the health of neighboring businesses proved the trickiest undertaking of all. Correlation was easy to chart; causation was nearly impossible to defend. A block of storefronts may thrive beside a well-funded community center, but proving the center caused the thriving required methodological humility the foundation initially lacked. The final report describes this effort as 'directionally suggestive, not causally demonstrable' — an honest phrase other funders would do well to borrow.

The Accountability Trap

There is a deeper tension beneath the logistics. Funders increasingly face pressure — from boards, from donors, from the public — to demonstrate impact with precision. But the communities foundations serve did not ask for dashboards. When evaluation demands grow heavy, small nonprofits absorb the burden, diverting staff time from service delivery to data entry. Several national studies have documented this dynamic; the foundation's grantees confirmed it anecdotally before any reform began.

The foundation's answer was to internalize the cost of rigor. Rather than requiring grantees to produce additional reports, it hired its own data staff and pulled from public sources — census records, state labor statistics, county health data — to build the second-order picture itself. Grantees were asked only for what they uniquely knew.

Lessons for Other Funders

The report closes with recommendations that read as a quiet rebuke to sector orthodoxy:

• Measure decisions, not just outcomes. Tie every metric to a choice the foundation could realistically make differently.
• Pay for your own curiosity. Second-order effects are a funder's question; the burden of answering it belongs on the funder's ledger.
• Embrace the word 'suggestive.' Honest uncertainty builds more trust than confident overreach.
• Sunset your metrics. Review the dashboard annually and delete freely.

A Humbler Kind of Rigor

For Fundación SM and organizations like it, the story offers a welcome reframing. Accountability is not the same as comprehensiveness. The most credible funders of the coming decade may not be those with the largest datasets, but those with the discipline to measure a few things well, publish what they learn — including the failures — and return the saved hours to the communities they exist to serve.

The ripples are real. The audit, done right, simply helps you see where they travel.