DNP 820 TOPIC 5 DQ 1
Compare statistical and clinical significance. Explain why a project or practice outcome may not yield statistical significance but can still have clinical significance. When can a project’s outcomes be deemed “successful” by only using clinical significance as a measure? Provided examples from current research.
Measures of statistical significance quantify the probability of a study’s results being due to chance. Clinical significance, on the other hand, refers to the magnitude of the actual treatment effect (i.e., the difference between the intervention and control groups, also known as the “treatment effect size”), which will determine whether the results of the trial are likely to impact current medical practice (Ranganathan, P. et al., 2015). The “P” value, frequently used to measure statistical significance, is the probability that the study results are due to chance rather than an actual treatment effect. The conventional cut-off for the “P” value to be considered statistically significant is 0.05 (or 5%). A P < 0.05 implies that the possibility of the results in a study being due to chance is <5% (Ranganathan, P. et al., 2015). In clinical practice, an impact’s “clinical significance” depends on its implications on existing practice-treatment effect size, which is one of the most critical factors driving treatment decisions. Sharma (2021) suggests that the clinical significance should reflect “the extent of change, whether the change makes a real difference to subject lives, how long the effects last, consumer acceptability, cost-effectiveness, and ease of implementation.” While there are established, traditionally accepted values for statistical significance testing, this is lacking for evaluating clinical significance (Sharma H., 2021). Often, it is the judgment of the clinician (and the patient) that decides whether a result is clinically significant or not.
According to Van Cutsem, E. et al. (2018), statistical significance heavily depends on the study’s sample size; with large sample sizes, even minor treatment effects (clinically inconsequential) can appear statistically significant; therefore, the reader must carefully interpret this “significance.” A study published in the Journal of Clinical Oncology compared overall survival in 569 patients with advanced pancreatic cancer who were randomized to receive erlotinib plus gemcitabine versus gemcitabine alone. Median survival was “significantly” prolonged in the erlotinib/gemcitabine arm (6.24 months vs. 5.91 months, P = 0.038). The P = 0.038 means that there is only a 3.8% chance that this observed difference between the groups occurred by chance (which is less than the traditional cut-off of 5%) and, therefore, statistically significant. In this example, the clinical relevance of this “positive” study is the “treatment effect” or difference in median survival between 6.24 and 5.91 months – a mere ten days, which most oncologists would agree is a clinically irrelevant “improvement” in outcomes, especially when considering the added toxicity and costs involved with the combination. In clinical research, statistically significant study results are often clinically meaningful. While statistical significance indicates the reliability of the study results, clinical significance reflects its impact on clinical practice.
I agree with you that clinical significance and measures of statistical significance are different concept of healthcare statistics. However, both clinical significance and measure of statistical significance are important when handing statistical information in healthcare settings. Measures of statistical significance quantify the probability of a study’s outcomes because of chance (Griffiths & Needleman, 2019). Clinic significance is the magnitude of the actual treatment effect. The magnitude can be examined by identifying the difference between the intervention and control groups. Measures of statistical significance are general when addressing healthcare issue. The generalization may make measures of statistical significance unfit to handle clinical issues that requires specialty (Frank et al., 2021). Therefore, clinical significance becomes a right replacement for measures of statistical significance. At the same time, the measures of statistical significance can replace clinical significance. The uniqueness of these two terms allow measures of statistical significant and clinical significance to complement one another in healthcare settings. Statistical significance illustrates the reliability of the study results, clinical significance reflects its impact on clinical practice.
- Frank, O., Tam, C. M., & Rhee, J. (2021). Is it time to stop using statistical significance?. Australian Prescriber, 44(1), 16. doi: 10.18773/austprescr.2020.074
- Griffiths, P., & Needleman, J. (2019). Statistical significance testing and p-values: Defending the indefensible? A discussion paper and position statement. International journal of nursing studies, 99, 103384. https://doi.org/10.1016/j.ijnurstu.2019.07.001