Skip to main content

Response to: Simpson’s Paradox is suppression, but Lord’s Paradox is neither: clarification of and correction to Tu, Gunnell, and Gilthorpe (2008) by Nickerson CA & Brown NJL (https://doi.org/10.1186/1742-7622-5-2)

  • The original article was published in Emerging Themes in Epidemiology 2019 16:5

Abstract

We commend Nickerson and Brown on their insightful exposition of the mathematical algebra behind Simpson’s paradox, suppression and Lord’s paradox; we also acknowledge there can be differences in how Lord’s paradox is approached analytically, compared to Simpson’s paradox and suppression, though not in every example of Lord’s paradox. Furthermore, Simpson’s paradox, suppression and Lord’s paradox ask the same contextual questions, seeking to understand if statistical adjustment is valid and meaningful, identifying which analytical option is correct. In our exposition of this, we focus on the perspective of context, which must invoke causal thinking. From a causal thinking perspective, Simpson’s paradox, suppression and Lord’s paradox present very similar analytical challenges.

We congratulate Nickerson and Brown for their exposition of the mathematics behind Simpson’s paradox, suppression and Lord’s paradox [1]. They explain the extent of statistical adjustment obtainable in assessing the \(X\)\(Y\) focal relationship when augmenting the linear model \(Y\sim X\) with \(Z\) to \(Y\sim X + Z\): depending on the relationships amongst \(X\), \(Y\) and \(Z\), the \(X\) coefficient increases, decreases, or changes sign (hence ‘reversal paradox’).

Illustrated as Simpson’s paradox, this phenomenon explores the association between any two binary variables (\(X\) and \(Y\)) that might be reversed if conditioned on any third binary variable (\(Z\)). Illustrated as suppression, this phenomenon involves \(X\), \(Y\) and \(Z\) as any continuous variables. In both instances, the univariable linear model \(Y\sim X\) and the multivariable linear model \(Y\sim X + Z\) are contrasted and ‘reversal paradox’ occurs if the analytical options evaluating the \(X\)\(Y\) focal relationship conflict or differ. In contrast, Lord’s paradox is often illustrated for the specific context based on Lord’s 1967 paper [2], where \(Z\) and \(Y\) are baseline and follow-up measures of student weights, respectively, at the start and end of first year in college. The \(X\)\(Y\) focal relationship is the effect of sex (\(X\)) on weight change (\(Y - Z\)). This common understanding of Lord’s paradox exhibits different analytical options to Simpson’s paradox or suppression; contrasts are made between the model (\(Y - Z\sim X + Z\)) and t test of weight change (\(Y - Z\)) by sex (\(X\)). This is not mathematically identical to Simpson’s paradox or suppression. However, Lord published a note in 1969 in which he clarified that change-score analysis is possible only where such a quantity is calculable; [3] he urged reflection on where \(Y\) and \(Z\) are not on the same scale and options revert to the multivariable (\(Y\sim X + Z\)) verses univariable (\(Y\sim X\)) model, as for Simpson’s paradox and suppression. Whether examining follow-up (\(Y\)) or change-score (\(Y - Z\)), Lord’s paradox asks the same question to that for Simpson’s paradox and suppression: is statistical adjustment of\(Z\)valid and meaningful?

We did not discuss all analytical options in our paper since our focus was whether statistical adjustment is appropriate from a causal inference perspective [4]. Confusion arises if results from different evaluations of the same focal relationship differ or contradict. But there is no ‘paradox’ if the ‘correct’ or ‘meaningful’ analysis can be guided by context. Context need not be linked to the mathematics of analytical options. For Simpson’s paradox, suppression and Lord’s paradox the identical question is: which analytical option is ‘correct’ or ‘meaningful’?

The answer is not obtainable from data or mathematics, but from contextual theory. For instance, we assume that \(X\) precedes \(Y\), else there is no meaningful inference to be had. If \(Z\) precedes \(X\), then \(Z\) ‘confounds’ the \(X\)\(Y\) focal relationship and statistical adjustment is warranted [5]. In contrast, if \(X\) precedes \(Z\), then \(Z\) ‘mediates’ the \(X\)\(Y\) focal relationship and statistical adjustment is not warranted, unless seeking the direct effect of \(X\) on \(Y\) [5]. The mathematics and inferential context are equivalent for Simpson’s paradox and suppression, with variables either entirely categorical or continuous, respectively. For Lord’s paradox, in his 1967 paper [2], sex (at birth) precedes baseline weight (at entry to college), and baseline weight (\(Z\)) mediates the focal relationship between sex and weight change. We either adjust or not for baseline weight (\(Z\)); the same dichotomy for Simpson’s paradox and suppression.

There are additional complications with Lord’s paradox. Change-scores (\(Y - Z\)) with observational data do not generally provide meaningful causal inference, and the preferred approach is to view change as the part of \(Y\) not explained by \(Z\) [6]. The \(X\)-\(Y\) focal relationship is thus more reliably estimated for the effect of sex (\(X\)) on follow-up weight (\(Y\)). The question is again whether to adjust for baseline weight (\(Z\)). The \(X\) coefficients in \(Y - Z\sim X + Z\) and \(Y\sim X + Z\) are mathematically equivalent [7], whereas they differ for \(Y - Z\sim X\) and \(Y\sim X\) [6]. Despite multiple analytical options, there is none that yields total effect of sex on weight change.

Thinking more generally, as encouraged in the 1969 note, different contexts warrant different analytical strategies. Suppose, for instance, students are assigned to (mixed-sex) halls and the research question examines how halls affect weight change during the college year. Baseline weight (\(Z\)) precedes hall assignment (\(X\)), and \(Z\) is a confounder or competing exposure if not causally related to \(X\) (e.g. hall assignment is random). The multivariable models \(Y - Z\sim X + Z\) and \(Y\sim X + Z\) both appropriately (and equivalently) [7] estimate total effect of halls on weight change.

Lord’s paradox may be especially challenging and there are several analytical options [2], but Lord stressed the same interpretational issues as for Simpson’s paradox and suppression [3]. Understanding context is essential. In our exposition of this issue, we focus on the analytical validity according to context [4]. Although Nickerson and Brown provide an excellent exposition of the mathematics of statistical adjustment, analytical validity is more than algebra; robust and meaningful inference needs causal thinking [5], which is complementary.

Availability of data and materials

Not applicable

References

  1. 1.

    Nickerson CA, Brown NJL. Simpson’s Paradox is suppression, but Lord’s Paradox is neither: clarification of and correction to Tu, Gunnell, and Gilthorpe (2008). Emerg Themes Epidemiol. 2019;16:5.

    Article  Google Scholar 

  2. 2.

    Lord FM. A paradox in the interpretation of group comparisons. Psychol Bull. 1967;68:304–5.

    CAS  Article  Google Scholar 

  3. 3.

    Lord FM. Statistical adjustments when comparing preexisting groups. Psychol Bull. 1969;72:337–8.

    Article  Google Scholar 

  4. 4.

    Tu YK, Gunnell D, Gilthorpe MS. Simpson’s Paradox, Lord’s Paradox, and suppression effects are the same phenomenon—the reversal paradox. Emerg Themes Epidemiol. 2008;5:2.

    Article  Google Scholar 

  5. 5.

    Pearl J, Glymour MM, Jewell NP. Causal inference in statistics: a primer. London: Wiley; 2016.

    Google Scholar 

  6. 6.

    Tennant PW, Arnold KF, Ellison GT, Gilthorpe MS. Analyses of ‘change scores’ do not estimate causal effects in observational data. arXiv preprint arXiv:190702764 2019.

  7. 7.

    Laird N. Further comparative analyses of pretest-posttest research designs. Am Stat. 1983;37:329–30.

    Google Scholar 

Download references

Acknowledgements

None.

Funding

This work was not supported by any funding

Author information

Affiliations

Authors

Contributions

Both authors contributed equally to the drafting of this letter. Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Mark S. Gilthorpe.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Both authors of the letter consent to publication.

Competing interests

Neither author has any competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gilthorpe, M.S., Tu, Y. Response to: Simpson’s Paradox is suppression, but Lord’s Paradox is neither: clarification of and correction to Tu, Gunnell, and Gilthorpe (2008) by Nickerson CA & Brown NJL (https://doi.org/10.1186/1742-7622-5-2). Emerg Themes Epidemiol 17, 1 (2020). https://doi.org/10.1186/s12982-020-00089-7

Download citation