Another day, another conversation about the missing nuanced view of stats and data. Despite the saying, statistics is not all lies but neither does it magically produce nuggets of Truth.

Stats is a mathematical box that takes what we give it and crunches the numbers to produce a single set of summary numbers (t(3) = 2.54, p < .05). What we sometimes forget, or never really knew, was that the numbers are related to our decision as researchers & interpretations as readers. The same dataset will always produce t(3) = 2.54,  but p < .05 only occurs if we assume “one-tailed” but would get p > .05 if assuming “two-tailed” (handy calculator). The meaningfulness & usefulness of the difference requires yet another layer of interpretation and decision-making.

For a quick summary of what stats can do, can’t say, and should be questioned about:

The numbers produced by statistics are very tempting. They are consistently produced by a process created by smart people, they are mysterious and are given weighty titles of being empirical, statistically proven, and accurate. And when interpreted as being about what is specifically measured and specifically tested they are true.

The devil, or in this case, the truth is in the details. What exactly is being measured? Which analysis is used? What variables are or are not included in the analysis? How are they positioned, weighted, and ordered? What level of difference or relationship do we set as enough? So many choices must be made in good statistical analysis, and it is these choices that you can judge.

The leap to declaring “Truths” about life’s problems, other contexts and the future is indeed a leap of faith. We can make good decisions, bolster our arguments for why others need to believe us, and in the end there is simply trust and the judgement of reasonableness.