Objective: With the evolution of theory and methods for detecting recalibration, reprioritization, and reconceptualization response shifts, the time has come to evaluate and compare the current statistical detection techniques. This manuscript presents an overview of a cross-method validation done on the same patient sample.
Methods: Three statistical techniques were used: Structural Equation Modeling, Latent Trajectory Analysis, and Recursive Partitioning and Regression Tree modeling. The study sample (n = 3,008) was drawn from the North American Research Committee on Multiple Sclerosis (NARCOMS) Registry to represent patients soon after diagnosis, classified as having either a self-reported relapsing, progressive, or stable disease trajectory. Patient-reported outcomes included the disease-specific Performance Scales and the Patient-Derived Disease Steps, and the generic SF-12v2 measure.
Results: Small response shift effect sizes were detected by all of the methods. Recalibration response shift was detected by Structural Equation Modeling, Recursive Partitioning Regression Tree demonstrated patterns consistent with all three types of response shift, and Latent Trajectory Analysis, although unable to distinguish types of response shift, did detect response shift in less than 1% of the sample.
Conclusion: The methods and their findings were discussed for operationalization, interpretability, assumptions, ability to use all data points from the study sample, limitations, and strengths. Directions for future research are discussed.