Clinically significant differences: what to choose?

The usefulness of scientific findings in clinical settings is the focus of evidence-based health practices[1]. Therefore, it is necessary to properly see the data from a probabilistic point of view and its application in clinical settings. In this last aspect, there are different strategies their relevance need to be understood.

Initially, it is necessary to understand the meaning of the minimal clinically important difference (MCID), whose concept emerged with Jaeschke et al. in 1989, being a practical application measure for some dimension of care, based on the patient’s perception and that changes the way the patient is managed[2]. The MCID resembles other measurements that do not take into account the patient’s perception as minimal important difference (MID), minimal clinical difference (MCD) or minimal clinically significant difference (MCSD), which are derived from clinical reports, changes in clinical parameters or effectiveness of clinical interventions[3].

There are three basic methodologies for developing clinical impact measures: data distribution approach, anchor-based approach, and Delphi-based approach. There is no unanimity between them, thus requiring consideration or even the use of more than one in research situations so that the information is properly used[4].

Methods based on data distribution take into account the change in the outcome variable, as well as the variability of the set of measurements. To do so, they use the standard error of measurement (SEM), the standard deviation of Benchmark or effect measures that relativize the gain by data variability, such as Cohen's D[5]. These methods are very interesting to measure the effect on the data set, generally on any spectrum of morbidity, with the sample as the baseline element, and also when the outcome is not a measure derived from the participant’s perception. The downside is not taking into account what would be a real impact gain from the patient’s perspective[4].

Anchor-based methods are based on the comparison of clinical measurements in relation to patients’ perceptual aspects regarding their evolution after interventions. The interesting thing about this method is that the improvement references should be elaborated by the patient and previously established for each one of them so as not to produce biases on what would be the ideal improvement. In addition, it is of good application in day-to-day clinical practice to establish treatment change or staging. On the other hand, it is important to emphasize that this approach depends on the staging of the examined health condition and psychosocial factors, and its applicability needs to be evaluated according to the patient’s commitment[4].

Finally, the Delphi method is a consensus approach between subject matter experts and the establishment of impact standards on the specific health condition parameter. It may experience the same problems as data- and anchor-based methods[6].

Based on this brief theoretical framework, it is clear that the study entitled “Effect of Leap Motion-based 3D Immersive Virtual Reality Usage on Upper Extremity Function in Ischemic Stroke Patients”[7], the authors bring results of great impact on the management of post-stroke rehabilitation patients, and they need to have a wider read to apply.

In that study, the authors[7] mentioned that the Functional Independence Measurement (FIM) variable, even with a probabilistically identified change, was lower than the measure of 22 units considered the MCID for this instrument. However, analyzing the reference cited[8], it is clear that the sample used to elaborate this measure of clinical effect consisted of individuals with an average of 7.5 (0–27) days of stroke involvement, that is, patients in the acute stage of stroke. Strokes with a very different evolutionary picture than those of the sample by Ögün et al[7]. These authors used a sample with an involvement time between six and 24 months, which configures the staging of chronic patients and, consequently, with a different prognosis of recovery due to neuroplasticity capacity[9]. This has considerable implications for the health condition studied and requires careful interpretation of the MCID measurement.

Applying a clinical’s measure significance based on data distribution ([Table 1]), such as Cohen’s D, we found that not only FMUE and ARAT but also FIM, PASS-BADL and PASS-IADL revealed relevant effects (D>0.5)[5]. This implies that since there is no MCID measurement anchored in the perceptions of patients or independent of health condition staging, the emissive virtual reality intervention produced an improvement in the functional parameters analyzed to a much greater extent than the findings of the control therapy.

Table 1 Measurement of clinical impact measures from an approach based on data distribution.

	M1before	M2After	SDbefore	SDDepois	Means Difference	SD Means	Cohen’s D
FMUE	39.63	46.54	8.84	7.91	-6.91	8.37	-0.82
	38.56	40.06	8.8	8.33	-1.5	8.56	-0.17
ARAT	32.81	41.15	7.17	7.82	-8.34	7.49	-1.11
	30.84	32.09	6.32	5.94	-1.25	6.13	-0.20
FIM	84.81	89.6	5.96	8.2	-4.79	7.08	-0.67
	84.25	84.96	6.37	6.42	-0.71	6.39	-0.11
PASS-BADL	1.46	1.84	0.27	0.24	-0.38	0.25	-1.49
	1.53	1.56	0.25	0.17	-0.03	0.21	-0.14
PASS-IADL	1.58	1.98	0.16	0.18	-0.4	0.17	-2.35
	1.57	1.61	0.15	0.32	-0.04	0.24	-0.16

Source: Ögün et al.[7].

The major highlight was identified in the PASS-IADL measurements, with an improvement almost five times greater than the group variability, and also ARAT and PASS-BADL, with gains about twice the group variability. The Cohen’s D used was based on the formula: D=(M1-M2)/(SD1+SD2)/2[5], for paired data of the same size as the groups. Its measurement changes depending on the characteristics of the research.

These findings imply an innovative therapeutic approach that shows good results in a chronic morbid group, based on motor imaging fundamentals and supported by mirror neurons that allow neuronal rehabilitation without sensory inputs to the musculoskeletal system. Some questions arise from these findings: Would it have greater impact on acute patients? Would adding peripheral sensory feedback increase the effects? What are the minimum treatment dosages sufficient to produce clinical effects? Can it be used as self-managed therapy with professional follow-up? This is the scene for the next chapters. Shall we get to work?

Conflict of interest:

There is no conflict of interests to declare.

• References

• 1 Winstein CJ, Stein J, Arena R, Bates B, Cherney LR, Cramer SC, et al. Guidelines for Adult Stroke Rehabilitation and Recovery: A Guideline for Healthcare Professionals from the American Heart Association/American Stroke Association. Stroke. 2016 Jun;47(6):e98-e169. https://doi.org/10.1161/STR.20200011202000110098
• 2 McGlothlin AE, Lewis RJ. Minimal clinically important difference: Defining what really matters to patients. JAMA. 2014 Oct;312(13):1342-3. https://doi.org/10.1001/jama.2014.13128 .
• 3 Cook CE. Clinimetrics corner: The Minimal Clinically Important Change Score (MCID): a necessary pretense. J Man Manip Ther. 2008;16(4):82E-83E. https://doi.org/10.1179/jmt.2008.16.4.82E
• 4 Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol. 2008 Feb;61(2):102-9. https://doi.org/10.1016/j.jclinepi.2007.03.012
• 5 Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Front Psychol. 2013 Nov;4:863. https://doi.org/10.3389/fpsyg.2013.00863
• 6 Johnston BC, Ebrahim S, Carrasco-Labra A, Furukawa TA, Patrick DL, Crawford MW, et al. Minimally important difference estimates and methods: A protocol. BMJ Open. 2015;5(10):1-7. https://doi.org/10.1136/bmjopen-2015-007953
• 7 Ögün MN, Kurul R, Yaşar MF, Turkoglu SA, Avci Ş, Yildiz N. Effect of leap motion-based 3D immersive virtual reality usage on upper extremity function in ischemic stroke patients. Arq Neuropsiquiatr. 2019 Oct;77(10):681-8. https://doi.org/10.1590/0004-282X20190129
• 8 Beninato M, Gill-Body KM, Salles S, Stark PC, Black-Schaffer RM, Stein J. Determination of the minimal clinically important difference in the FIM instrument in patients with stroke. Arch Phys Med Rehabil. 2006 Jan;87(1):32-9. https://doi.org/10.1016/j.apmr.2005.08.130
• 9 Langhorne P, Sandercock P, Prasad K. Evidence-based practice for stroke. Lancet Neurol Lancet Neurol. 2009 Apr;8(4):308-9. https://doi.org/10.1016/S1474-4422(09)70060-2

Publication History

Received: 28 November 2019

Accepted: 12 December 2019

Article published online:
13 June 2023