“People with blood group A have a significantly higher risk for acquiring COVID-19 compared with non-A blood groups” proclaims the conclusion of a preprint study.
Now this is from a preprint, it has not gone through peer-review and most people who have read it have some serious concerns about the methodology [scroll to the bottom of the preprint, or here for a taste of those concerns]. However, that doesn’t mean that the story hasn’t been picked up by the media. It’s everywhere. But for the purpose of my illustration, let’s look to the Independent’s take:
“They found patients with blood type A had a “significantly higher” rate of infection and seemed to develop more severe symptoms of the virus.”
And that’s where the problem lies. In the manuscript version, the authors have used significant in the statistical sense to indicate that their data are unlikely to be a false positive. They are emphasising that this observation is unlikely to have occurred solely due to chance. However, in the news report, significant will instead be read by most readers as meaning “a lot” or “an important amount”. The meaning has changed.
Obviously, news writers want to make the story juicy and using more dramatic language suits them just fine BUT as science communicators we have a chance to, at very least, ensure that the storyline the news writers start with is clear.
Focus the narrative on the effect size
For this story; the message really is; people with blood group A appear to be between 2 and 43%over-represented in terms of patients with severe symptoms (note the wide range which reflects the small sample size). It’s not a huge effect, even though they are 98% confident that this observation isn’t due to sampling.
Was there any point in saying significant anyway?
The thing that bothers me the most about this particular example is that throughout the results section they did the right thing and reported the P values, adding significant for emphasis. “Meta-analyses on the pooled data showed that blood group A had a significantly higher risk for COVID-19 (odds ratio-OR, 1.20; 95% confidence interval-CI 1.02~1.43, P = 0.02) compared with non-A blood groups,”
In essence, they have stated their confidence twice: first by saying “significantly” and then again by providing the output from their stats test P=0.02. By choosing to write it this way they are saying that their confidence that their results aren’t a type I error is twice as important in the observation itself.
So, to answer my rhetorical question: no, saying significant in the sentence had zero value. all it did was introduce potential for ambiguity and misunderstanding. The sentence would be better without it and even better still if it was focused on the effect size.
I repeat my plea. Please stop using significant where it can be misinterpreted, especially if you re reporting something newsworthy. In a broader sense, consider whether there is added value in using significant at all in the rest of your writing.
LaNts and Laminins 