Repeatability for Gaussian and non-Gaussian data: a practical guide for biologists
Corresponding Author
Shinichi Nakagawa
Department of Zoology, University of Otago, 340 Great King Street, Dunedin, 9054, New Zealand (E-mail: [email protected] )
Tel: +44-114-222-0113; Fax: +44-114-222-0002; E-mail: [email protected]Search for more papers by this authorHolger Schielzeth
Department of Behavioural Ecology and Evolutionary Genetics, Max Planck Institute for Ornithology, Eberhard-Gwinner-Str. 5, D-82319 Seewiesen, Germany
Department of Evolutionary Biology, Evolutionary Biology Centre, Uppsala University, Norbyvägen 18D, SE-752 36, Uppsala, Sweden (E-mail: [email protected] )
Search for more papers by this authorCorresponding Author
Shinichi Nakagawa
Department of Zoology, University of Otago, 340 Great King Street, Dunedin, 9054, New Zealand (E-mail: [email protected] )
Tel: +44-114-222-0113; Fax: +44-114-222-0002; E-mail: [email protected]Search for more papers by this authorHolger Schielzeth
Department of Behavioural Ecology and Evolutionary Genetics, Max Planck Institute for Ornithology, Eberhard-Gwinner-Str. 5, D-82319 Seewiesen, Germany
Department of Evolutionary Biology, Evolutionary Biology Centre, Uppsala University, Norbyvägen 18D, SE-752 36, Uppsala, Sweden (E-mail: [email protected] )
Search for more papers by this authorAbstract
Repeatability (more precisely the common measure of repeatability, the intra-class correlation coefficient, ICC) is an important index for quantifying the accuracy of measurements and the constancy of phenotypes. It is the proportion of phenotypic variation that can be attributed to between-subject (or between-group) variation. As a consequence, the non-repeatable fraction of phenotypic variation is the sum of measurement error and phenotypic flexibility. There are several ways to estimate repeatability for Gaussian data, but there are no formal agreements on how repeatability should be calculated for non-Gaussian data (e.g. binary, proportion and count data). In addition to point estimates, appropriate uncertainty estimates (standard errors and confidence intervals) and statistical significance for repeatability estimates are required regardless of the types of data. We review the methods for calculating repeatability and the associated statistics for Gaussian and non-Gaussian data. For Gaussian data, we present three common approaches for estimating repeatability: correlation-based, analysis of variance (ANOVA)-based and linear mixed-effects model (LMM)-based methods, while for non-Gaussian data, we focus on generalised linear mixed-effects models (GLMM) that allow the estimation of repeatability on the original and on the underlying latent scale. We also address a number of methods for calculating standard errors, confidence intervals and statistical significance; the most accurate and recommended methods are parametric bootstrapping, randomisation tests and Bayesian approaches. We advocate the use of LMM- and GLMM-based approaches mainly because of the ease with which confounding variables can be controlled for. Furthermore, we compare two types of repeatability (ordinary repeatability and extrapolated repeatability) in relation to narrow-sense heritability. This review serves as a collection of guidelines and recommendations for biologists to calculate repeatability and heritability from both Gaussian and non-Gaussian data.
XI. REFERENCES
- Bakker, T. C. M. (1999). The study of intersexual selection using quantitative genetics. Behaviour 136, 1237–1266.
- Becker, W. A. (1992). A manual of quantitative genetics , 5th edition. Academic Enterprises, Pullman, WA.
- Bell, A. M., Hankison, S. J. & Laskowski, K. L. (2009). The repeatability of behaviour: a meta-analysis. Animal Behaviour 77, 771–783.
- Berteaux, D., Thomas, D. W., Bergeron, J. M. & Lapierre, H. (1996). Repeatability of daily field metabolic rate in female meadow voles (Microtus pennsylvanicus). Functional Ecology 10, 751–759.
- Biro, P. A., Beckmann, C. & Stamps, J. A. (2010). Small within-day increases in temparature affects boldness and alters personality in coral reef fish. Proceedings of the Royal Society B-Biological Sciences 277, 71–77.
- Boake, C. R. B. (1989). Repeatability: Its role in evolutionary studies of mating behavior. Evolutionary Ecology 3, 173–182.
- Bolker, B. M., Brooks, M. E., Clark, C. J., Geange, S. W., Poulsen, J. R., Stevens, M. H. H. & White, J.-S. S. (2009). Generalized linear mixed models: a practical guide for ecology and evolution. Trends in Ecology & Evolution 24, 127–135.
- Bolund, E., Schielzeth, H. & Forstmeier, W. (2007). Intrasexual competition in zebra finches, the role of beak colour and body size. Animal Behaviour 74, 715–724.
- Bolund, E., Schielzeth, H. & Forstmeier, W. (2009). Compensatory investment in zebra finches: females lay larger eggs when paired to sexually unattractive males. Proceedings of the Royal Society B-Biological Sciences 276, 707–715.
- Brommer, J. E., Rattiste, K. & Wilson, A. J. (2008). Exploring plasticity in the wild: laying date-temperature reaction norms in the common gull Larus canus. Proceedings of the Royal Society B-Biological Sciences 275, 687–693.
- Browne, W. J., Subramanian, S. V., Jones, K. & Goldstein, H. (2005). Variance partitioning in multilevel logistic models that exhibit overdispersion. Journal of the Royal Statistical Society Series A-Statistics in Society 168, 599–613.
- Carrasco, J. L. (2009).A generalized concordance correlation coefficient based on the variance components generalized linear mixed models with application to overdispersed count data. Biometrics, in press, DOI: 10.1111/j.1541-0420
- Carrasco, J. L. & Jover, L. (2003). Estimating the generalized concordance correlation coefficient through variance components. Biometrics 59, 849–858.
- Carrasco, J. L. & Jover, L. (2005). Concordance correlation coefficient applied to discrete data. Statistics in Medicine 24, 4021–4034.
- Carrasco, J. L., Jover, L., King, T. S. & Chinchilli, V. M. (2007). Comparison of concordance correlation coefficient estimating approaches with skewed data. Journal of Biopharmaceutical Statistics 17, 673–684.
- Carrasco, J. L., King, T. S. & Chinchilli, V. M. (2009). The concordance correlation coefficient for repeated measures estimated by variance components. Journal of Biopharmaceutical Statistics 19, 90–105.
- Clark, J. S. (2005). Why environmental scientists are becoming Bayesians. Ecology Letters 8, 2–14.
- DeWitt, T. J. & Scheiner, S. M. (2004). Phenotypic plasticity: functional and conceptual approaches. Oxford University Press, Oxford.
- Dingemanse, N. J., Both, C., Drent, P. J., Van Oers, K. & Van Noordwijk, A. J. (2002). Repeatability and heritability of exploratory behaviour in great tits from the wild. Animal Behaviour 64, 929–938.
- Dingemanse, N. J., Kazem, A. J. N., Reale, D. & Wright, J. (2009). Behavioural reaction norms: animal personality meets individual plasticity. Trends in Ecology & Evolution 25, 82–89.
- Dohm, M. R. (2002). Repeatability estimates do not always set an upper limit to heritability. Functional Ecology 16, 273–280.
- Donner, A. (1986). A review of inference procedures for the intraclass correlation coefficient in the one-way random effects model. International Statistical Review 54, 67–82.
- Falconer, D. S. & Mackay, T. F. C. (1996). Introduction to quantitative genetics, 4th edition. Prentice Hall, Harlow, U.K.
- Faraway, J. J. (2006). Extending the linear model with R. Chapman & Hall/CRC, Boca Raton, FL.
- Fleiss, J. L. & Cuzick, J. (1979). The reliablility of dichotomous judgments: unequal numbers of judges per subject. Applied Psychological Measurement 3, 537–542.
10.1177/014662167900300410 Google Scholar
- Forstmeier, W. & Birkhead, T. R. (2004). Repeatability of mate choice in the zebra finch: consistency within and between females. Animal Behaviour 68, 1017–1028.
- Garamszegi, L. Z., Calhim, S., Dochtermann, N., Hegyi, G., Hurd, P. L., Jèrgensen, C., Kutsukake, N., Lajeunesse, M. J., Pollard, K. A., Schielzeth, H., Symonds, M. R. E. & Nakagawa, S. (2009). Changing philosophies and tools for statistical inferences in behavioral ecology. Behavioral Ecology 20, 1363–1375.
- Garamszegi, L. Z., Hegyi, G., Heylen, D., Ninni, P., De Lope, F., Eens, M. & Mèller, A. P. (2006). The design of complex sexual traits in male barn swallows: associations between signal attributes. Journal of Evolutionary Biology 19, 2052–2066.
- Gelman, A. & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press, Cambridge, U.K.
- Gill, J. (2007). Bayesian methods: a social and behavioral sciences approach. CRC, Boca Raton, FL.
- Goldstein, H., Browne, W. & Rasbash, J. (2002). Partitioning variation in multilevel models. Understanding Statistics 1, 223–231.
10.1207/S15328031US0104_02 Google Scholar
- Hadfield, J. D. (2010). MCMC methods for multi-response Generalised Linear Mixed Models: the MCMCglmm R package. Journal of Statistical Software 33, 1–22.
- Hadfield, J. D. & Nakagawa, S. (2010). General quantitative genetic methods for comparative biology: phylogenies, taxonomies and multi-trait models for continuous and categorical characters. Journal of Evolutionary Biology 23, 494–508.
- Isberg, S. R., Thomson, P. C., Nicholas, F. W., Barker, S. G. & Moran, C. (2005). Quantitative analysis of production traits in saltwater crocodiles (Crocodylus porosus): I. reproduction traits. Journal of Animal Breeding and Genetics 122, 361–369.
- Lee, Y., Nelder, J. A. & Pawitan, Y. (2006). Generalized linear models with random effects: unified analysis via H-likelihood. Chapman & Hall/CRC, Boca Raton, FL.
- Lessells, C. M. & Boag, P. T. (1987). Unrepeatable repeatabilities: a common mistake. Auk 104, 116–121.
- Littell, R. C., Milliken, G. A., Stroup, W. W., Wolfinger, R. D. & Schabenberger, O. (2006). SAS ® for Mixed Models. SAS Institue Inc., Cary, NC.
- Lynch, M. & Walsh, B. (1998). Genetics and analysis of quantitative traits. Sinauer, Sunderland, MA.
- Manly, B. R. J. (2006). Randomization, Bootstrap and Monte carlo Methods in Biology, 3rd edition. Chapman & Hall/CRC ,Boca Raton, FL.
10.1111/j.1365-2745.2005.01082.x Google Scholar
- McCarthy, M. A. (2007). Bayesian methods for ecology. Cambridge University Press, Cambridge.
- McCulloch, C. E. & Searle, S. R. (2002). Generalized, linear and mixed models. Wiley, Chichester.
- McGraw, K. O. & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods 1, 30–46.
- Merilä, J. & Sheldon, B. (2000). Avian quantitative genetics. Current Ornithology 9, 179–255.
- Nakagawa, S. & Cuthill, I. C. (2007). Effect size, confidence interval and statistical significance: a practical guide for biologists. Biological Reviews 82, 591–605.
- Nakagawa, S., Gillespie, D. O. S., Hatchwell, B. J. & Burke, T. (2007a). Predictable males and unpredictable females: sex difference in repeatability of parental care in a wild bird population. Journal of Evolutionary Biology 20, 1674–1681.
- Nakagawa, S., Ockendon, N., Gillespie, D. O. S., Hatchwell, B. J. & Burke, T. (2007b). Does the badge of status influence parental care and investment in house sparrows? An experimental test. Oecologia 153, 749–760.
- Namboodiri, K. K., Green, P. P., Kaplan, E. B., Morrison, J. A., Chase, G. A., Elston, R. C., Owen, A. R. G., Rifkind, B. M., Glueck, C. J. & Tyroler, H. A. (1984). The collaborative lipid research clinics program family study: IV. Familial associations of plasma-lipids and lipoproteins. American Journal of Epidemiology 119, 975–996.
- Nelder, J. A. (1954). The interpretation of negative components of variance. Biometrika 41, 544–548.
- Nussey, D. H., Wilson, A. J. & Brommer, J. E. (2007). The evolutionary ecology of individual phenotypic plasticity in wild populations. Journal of Evolutionary Biology 20, 831–844.
- O’Hara, R. B. (2009). How to make models add up - a primer on GLMMs. Annales Zoologici Fennici 46, 124–137.
- Pigliucci, M. (2001). Phenotypic plasticity: beyond nature and nurture. The Johns Hopkins University Press, Baltimore, Maryland.
- R Development CoreTeam. (2009). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria.
- Réale, D., Reader, S. M., Sol, D., McDougall, P. T. & Dingemanse, N. J. (2007). Integrating animal temperament within ecology and evolution. Biological Reviews 82, 291–318.
- Richards, S. A. (2008). Dealing with overdispersed count data in applied ecology. Journal of Applied Ecology 45, 218–227.
- Schielzeth, H. (2010). Simple means to improve the interpretability of regression coefficients. Methods in Ecology and Evolution 1, 103–113.
- Schielzeth, H. & Bolund, E. (2010). Patterns of conspecific brood parasitism in zebra finches. Animal Behaviour. 79, 1329–1337.
- Schielzeth, H., Bolund, E. & Forstmeier, W. (2010). Heritability of and early-environmental effects on variation in mating preferences. Evolution 64, 998–1006.
- Schielzeth, H. & Forstmeier, W. (2009). Conclusions beyond support: overconfident estimates in mixed models. Behavioral Ecology 20, 416–420.
- Shrout, P. E. & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin 86, 420–428.
- Sih, A., Bell, A. & Johnson, J. C. (2004a). Behavioral syndromes: an ecological and evolutionary overview. Trends in Ecology & Evolution 19, 372–378.
- Sih, A., Bell, A. M., Johnson, J. C. & Ziemba, R. E. (2004b). Behavioral syndromes: an integrative overview. Quarterly Review of Biology 79, 241–277.
- Snijders, T. A. B. & Bosker, R. (1999). Multilevel analysis: an introduction to basic and advanced multilevel modeling. Sage Publications, London, U.K.
- Sokal, R. R. & Rohlf, F. J. (1995). Biometry: The principles and practice of statistics in biological research, 3rd edition. W.H. Freeman and Company, New York.
- Solomon, P. J. & Taylor, J. M. G. (1999). Orthogonality and transformations in variance components models. Biometrika 86, 289–300.
- Stamps, J. & Groothuis, T. G. G. (2010). The development of animal personality: relevance, concepts and perspetives. Biological Reviews 85, 301–325.
- Stamps, J. A. (2007). Growth-mortality tradeoffs and ‘personality traits' in animals. Ecology Letters 10, 355–363.
- Van de Pol, M. V. & Wright, J.(2009). A simple method for distinguishing within- versus between-subject effects using mixed models. Animal Behaviour 77, 753–758..
- Venables, W. N. & Ripley, B. D. (2002). Modern applied statistics with S, 4th edition. Springer, New York.
- Venzon, D. J. & Moolgavkar, S. H. (1988). A Method for Computing Profile-Likelihood-Based Confidence-Intervals. Applied Statistics-Journal of the Royal Statistical Society Series C 37, 87–94.
- Verbeke, G. & Molenberghs, G. (2001). Linear Mixed Models for Longitudinal Data. Springer, New York.
- Visscher, P. M., Hill, W. G. & Wray, N. R. (2008). Heritability in the genomics era - concepts and misconceptions. Nature Reviews Genetics 9, 255–266.
- Whittingham, L. A., Dunn, P. O. & Stapleton, M. K. (2006). Repeatability of extra-pair mating in tree swallows. Molecular Ecology 15, 841–849.
- Wilson, A. J. (2008). Why h2 does not always equal VA/VP? Journal of Evolutionary Biology 21, 647–650.
- Wilson, A. J., Pemberton, J. M., Pilkington, J. G., Clutton-Brock, T. H., Coltman, D. W. & Kruuk, L. E. B. (2007). Quantitative genetics of growth and cryptic evolution of body size in an island population. Evolutionary Ecology 21, 337–356.
- Zou, G. Y. & Donner, A. (2004). Confidence interval estimation of the intraclass correlation coefficient for binary outcome data. Biometrics 60, 807–811.