Statistical significance

From

Tests of statistical significance are means of determining how much confidence can be had that a particular result is not the product of chance. There are two standard tests of statistical significance that should be reported for each relative risk (RR). You will most likely have to obtain a copy of the actual study to verify statistical significance.

Contents 1 Relative risk p-values 2 Relative risk confidence intervals 2.1 Clinical trial confidence intervals 2.2 Cohort and case-control study confidence intervals 3 Confidence interval game-playing

Relative risk p-values

The first test of statistical significance concerns the p-value of the RR. The p-value will usually be found in the same table in the study as the RR. The p-value indicates how much confidence can be had that the RR is different from the no-effect level of 1.0.

By convention:

• If the p-value is 0.05 or less, then the RR is statistically significant. This does not necessarily mean that the RR is not junk science, it just means that the RR has passed the p-value test. Go to #Step 4: Evaluating the relative risk: Statistical significance (Part 2 of 2)
• If the p-value is greater than 0.05, then the RR is not statistically significant at the 95% confidence level. If the RR is not statistically significant, then it is viewed as a meaningless result and as having been debunked.

If a study does not present the p-values of its RRs, interpret this to mean that the researchers were too embarrassed to publish them because they would expose the RRs as not statistically significant, i.e., meaningless.

Relative risk confidence intervals

The second test of statistical significance for an RR involves its confidence interval.

The confidence interval for an RR is typically found in the same place as the RR and its p-value. It is often indicated with the abbreviation C.I. or CI. The confidence interval is a range of values, typically indicated with parentheses) within which the true RR lies, according to a certain degree of confidence. The standard degree of confidence is 95%, designated as 95% C.I.

Clinical trial confidence intervals

In a clinical trial, for example, if the RR=0.50, and its confidence interval ranges from 0.4 to 0.6, then this may be designated as RR=0.50 (95%CI 0.40, 0.60). The two numbers in the confidence interval are referred to as its lower bound and upper bound. In the above example, 0.40 is the lower bound and 0.60 is the upper bound. In order for the RR of a clinical trial to pass the second test of statistical significance, the upper bound of the confidence interval must be less than 1.0. The reason for this is that if the confidence interval includes 1.0, then we cannot be sure with 95% confidence that the true value of the RR is less than 1.0. In other words we cannot safely exclude the possibility that the true RR isn't 1.0, the no-effect level. For example, if you see a RR and CI presented as RR=0.60 (95% CI 0.35, 1.05), the RR is not statistically significant because its upper bound (1.05) is greater than 1.0 and includes the no-effect level as well as a range (from 1.0+ to 1.05) indicating that the treatment may actually worsen disease. So in the case of a clinical trial, if the upper bound of the confidence interval touches or crosses the 1.0 no effect level, then the RR is not statistically significant, i.e., it is meaningless and debunked.

Cohort and case-control study confidence intervals

For an RR from a cohort study or a case-control study to pass the second test of statistical significance, the lower bound of the confidence interval must be greater than 1.0. The reason for this is that if the confidence interval includes 1.0, then we cannot be sure with 95% confidence that the true value of the RR is greater than 1.0. In other words we cannot safely exclude the possibility that the true RR isn't 1.0, the no-effect level. For example, if you see a RR and CI presented as RR=2.60 (95% CI 0.95, 5.25), the RR is not statistically significant because its lower bound (0.95) is less than 1.0 and includes the no-effect level as well as a range (from 0.095+ to 1.00) indicating that the exposure may not be statistically associated with the occurrence of disease.

So in the case of a cohort study or a case-control study, if the lower bound of the confidence interval touches or crosses the 1.0 no effect level, then the RR is not statistically significant, i.e., it is meaningless and debunked.

Confidence interval game-playing

Confidence intervals are be subject to game-playing by junk scientists. The width of the confidence interval may be adjusted by altering the confidence level. For example, a 90% confidence interval is narrower than a 95% confidence interval, while a 99% confidence interval is wider than a 95% confidence interval. This makes sense if you think about it. You will have less confidence in a narrower range and more confidence in a wide range. So here's the game.

Keeping in mind that the 1.0 or the no-effect level is the third rail of an epidemiology study and means instant death for an RR which confidence interval touches or crosses it, there is incentive to narrow the confidence interval so that the upper bound stays below 1.0. This may be accomplished by choosing to use a lower confidence level than the standard 95% level. This lower confidence level is typically 90%. So if you see a 90% confidence interval, this is a red flag that the researcher is trying to conceal the fact that his CI touches or crosses the no-effect level.