Understand you must.

yoda explaining the use of significant in scientific text

When reporting your results, you have two main options regarding stats reporting:

If you selected a critical value that your P value needed to reach for you to reject your null hypothesis then the outcome has to be binary. Data are either significant or not significant.

However, remember that a P value is a measure of probability. Whether the observed result may have occurred due to sampling rather than a true difference (“false positive” / Type I error rate). An arbitrary line that is drawn to indicate “significance” doesn’t tell the whole story. P=0.051 is very similar to P=0.049 in terms of this likelihood.

Therefore, option 2, is to report the P values on a continuous scale. I always prefer this option for any exploratory study. Say what you see.

The only time where the significant/non significant line becomes valuable is when you are performing a confirmatory study, and using the outcome to make a decision. “If P<0.01 that the drug works, then I will prescribe it.” In which case a P of 0.011 is really disappointing.

Either way, you will need to do the correct test – check out our flow chart of stats tests here