We have the data. Now what?

England and Wales: This page sets out the position in full. Scotland: the same position — this is practice, not law, and it is UK-wide; data storage rules are reserved. Northern Ireland: the same position, with no difference in principle.

What can survey results from fifty people tell you?

Fifty responses describe fifty people: that is the honest sentence, and it is the whole of this step’s analysis advice.

A charity with fifty responses can describe what those fifty people said. It cannot generalise to everyone it serves, or to a wider population. It should not write as though it can.

This step assumes the data is already in: a baseline, a collection method and any beneficiary survey. It was gathered under the bases you recorded when you got consent and handled beneficiary data.

That rules out some common sentences. “Most of our users feel less isolated” is a claim about everyone. “Most of the people who answered said they felt less isolated” is a claim about the people who answered. Only the second is supported.

Social Value International’s fifth principle says it in three words: “Do not overclaim.” That principle comes from the body that sets the standard for social value measurement. It is not our gloss.

Contribution, not cause

The same discipline applies to cause. Your data may show that people’s lives changed. It will rarely show that you made the change on your own.

NPC’s Code of Good Impact Practice says this, verbatim. “It’s more realistic to talk about the contribution you’ve made rather than to try to attribute impact to your work alone.” NPC calls attribution “the most difficult and contentious aspect of evaluation.”

So you should write “we contributed to” rather than “we caused”, wherever other people or services were plausibly involved. They nearly always were. Work out what you can honestly claim takes this further.

Analysis methods: a gap in this guide

No analysis method was examined for this guide. That covers:

• descriptive statistics
• coding what people wrote
• testing whether a difference is real
• what a charity with no analyst can credibly do with a small sample

We are not going to fill that gap with general advice. What would close it is guidance on analysing small charity samples from a body such as NPC or Evaluation Support Scotland. Until then, the safe course is to count and to quote — and nothing that dresses a count up as an estimate.

One dataset, in one place

Keep one dataset, in one place: storage is not a separate problem from analysis. If the answers sit in three spreadsheets and a pile of paper forms, nobody can say what they show.

Put it somewhere everyone who needs it can find it, and nobody else can. Decide how long you keep it.

Those rules belong to the running hub, and this page does not repeat them:

• what the dataset is, and why you hold it — Map what personal data you hold
• who may see it, and how staff are trained — Write your data protection policy
• how long you keep it, and who may destroy it — Set your data retention schedule

What the summary should say

The deliverable is modest, deliberately. One dataset, and a short written summary of what it shows. The summary can say:

• how many people you asked, and how many answered
• what they said, as counts and in their own words
• what changed between the baseline and now, for the people who answered both times
• what you cannot tell from this, and why

The last line is the one most summaries leave out. It is also what makes the rest credible. The next two steps use this summary: what to do if something did not work, and what you can honestly claim.

Common mistake: generalising from a few dozen responses to a population

Percentages look like findings whatever the sample. But a charity with fifty responses can describe what those fifty people said. It cannot generalise, and it should not write as though it can.

Social Value Principle 5, “Do not overclaim”, applies, and so does the contribution framing. This is good practice, not a legal requirement — and no analysis method beyond it was examined for this guide.

Worked example

Wrenfield Village Hall has a few dozen feedback forms from its lunch-club regulars. The committee wants to put something in the annual report.

What it can say: what those regulars said, as counts and in their own words. Several wrote that the lunch club is the only time in the week they eat with other people. That can be quoted.

What it cannot say: that the lunch club caused a change, or that the finding holds for everyone in the village. The forms came from the people who come. Other things in their lives also changed.

The committee enters the forms into one spreadsheet, kept where the hall’s data map says. It writes half a page: how many forms, and what they said. A closing line says this describes the regulars who answered and no one else. There are no percentages presented as estimates for the village.