Go right to CalR, a web application for indirect calorimetry analysis.
Development and Background to the CalR project:
Indirect calorimetry: great potential behind messy datasets and thorny statistical issues. Indirect calorimetry is a non-invasive method of Energy Expenditure (EE) determination based on gas exchange allowing for rapid and extensive datasets to be collected. Alternatives, such as direct bomb calorimetry, require sacrificing the animals and harvesting organs—eliminating the possibility for serial measurements or for human research. In contrast, indirect calorimetry allows for flexible and sophisticated experiments that can be repeated in the same subjects over time. Although the use of indirect calorimetry has become widespread, controversies have emerged about the appropriate treatment of the data generated by these experiments, fundamentally challenging some published conclusions (1). Because analysis of these large data sets is onerous, there is a need for a tool to assist with appropriate analysis and interpretation of results in a comprehensive and standardized manner. The absence of such tools has allowed conflicting analyses to spread throughout the community (2, 3). A recent effort to promote transparency and increase the rigor of the scientific process, especially regarding biostatistical analysis and improving reproducibility, has created an atmosphere receptive towards novel tools that assist in achieving these goals (4-7).
Decades of controversy and eventual statistical recommendations. Starting in 1949, position papers regularly denounced the common practice of ratio-based normalization of metabolic rates (e.g. kcal/kg/hr) in favor of regression-based approaches (8-30). A key observation is the different methods used when comparing metabolic rates across varying species, assuming similar body compositions (31), versus comparing animals of the same species with different body compositions (8). A group of influential senior scientists issued a statement concluding that analysis of covariance (ANCOVA) has emerged as the consensus method for the analysis of indirect calorimetry data using body mass as the covariate (24). ANCOVA was chosen for its ability to handle the comparison of animals with different body composition (e.g., increased fat mass in obesity, or sarcopenia in aging). Because neither lean body mass (LBM) nor fat mass (FM) is metabolically inert and have different metabolic activities, ANCOVA may include LBM or FM as a covariate in the analysis of EE further extending is use in analysis of calorimetry data.
Beyond ANCOVA. ANCOVA was originally developed to extend the precision of analysis of variance (ANOVA) by adjusting for a continuous variable (a covariate) that correlates with group means and/or variances, thus increasing the power of the study (32). ANCOVA is included as a special case of the generalized linear model (GLM), which encompasses a great many classical statistical analysis techniques (e.g., ANOVA, linear regression, and logistic regression) (33). When designing CalR, we realized that many metabolic experiments violate the assumptions of ANCOVA. A key assumption of ANCOVA in EE studies is that the effect of the body mass covariate on EE is the same for all groups (i.e., parallel slopes in an EE vs. mass regression plot (33)). While ANCOVA brings many benefits to the interpretation of energy balance, this assumption can be overly restrictive under non-standard experimental conditions, including those of non-shivering thermogenesis. This includes activation of Brown Adipose Tissue (BAT) increasing EE through heat generation which is strongly dependent on BAT mass (34). Experiments of thermogenesis in mice with greater BAT mass often violate assumptions of the ANCOVA with a differential interaction of mass and EE (i.e., non-parallel slopes of an EE vs. mass plot). A generalized linear model (GLM) can be used to analyze experiments where a significant interaction effect exists between body mass and EE (27, 35). In GLM models, the magnitude of the adjusted difference between groups depends on the value of the body mass covariate, whereas in ANCOVA, the adjusted difference between groups is the same across the range of the mass covariate. CalR is designed to automatically determine the relationship between the slopes and select the appropriate statistical test.
Barriers to implementation: Quality control of large datasets and programming knowledge. In many cases, constraints exist for the widespread implementation of ANCOVA or GLM in EE analysis. These barriers include “wrangling” large data sets to prepare the raw data for analysis, unfamiliarity with statistical software packages, and the lack of an available software package to perform statistical analysis of indirect calorimetry experiments. In one day, a typical experiment might produce more than 60,000 data points. Quality control is especially important in longer experiments to eliminate instrument noise (e.g., opening cages to refeed animals produces spurious gas exchange readings). Ideally, each investigator would be familiar with statistical software including Python, R, or Julia. In actuality, not all calorimetry end-users have sufficient programming knowledge. Impartial, automated, reproducible quality control features are therefore required to handle these types of data. Without accessible tools, neither consistent quality control nor appropriate statistical tests are being implemented in the analysis of energy balance experiments (1).
Preliminary Results: CalR: A pilot toolbox for metabolic analysis. We have spent 5 years in iterative development of the CalR pilot software project, which is an easy-to-use web-based software tool freely available to the scientific community. CalR is developed to enable investigators to thoroughly and reproducibly perform statistical analyses of indirect calorimetry data. CalR allows users to import large data files, evaluate the experiment’s validity through quality control checks, examine data for outliers from experimental artifacts, and compare statistical differences between groups using ANCOVA. Results and workflow are exportable as files that can be shared in a centralized repository or as supplementary data accompanying publications. Within minutes, the experimental data is visualized and analyzed according to consensus methodology. More than 18,500 experimental sessions have been run on the CalR server since its launch in Dec 2017. CalR provides a useful statistical tool that is intuitive for biologists with a wide range of statistical knowledge. We now hope to expand the information extracted from each subject to include predictions of metabolic flexibility, predictions of body weight change, and calculation of resting metabolic rates. We will open access to our source code, expand our analysis pipeline, and add further quality control.
Standardization: CalR provides a universal data file format along with a set of standard measurement units, plotting standards, and statistical cutoffs. Prior to this, specific investigators were strongly influenced by the default output settings from the different calorimeter manufacturers. Currently, the three major corporate entities that make preclinical indirect calorimetry systems (Sable Systems International, TSE International, and Columbus Instruments) each produce different data analysis methods in different data formats. CalR created a new open format; based off comma separated values (CSV) in the “tidy data” arrangement (36). CalR files can be automatically read and processed by CalR, or any programming language. When loading manufacturers’ proprietary formats into CalR, automatic detection of file formats is performed to identify the type of system, reverse the manufacturers’ default data treatments (e.g., ratio-based normalization), resulting in platform-independent analyses. When exported from CalR, a separate metadata file may also be produced. By sharing CalR data and metadata files, investigators can fully and reproducibly share experimental analysis.
Development and ‘dogfood’. The user interface is typically, and has been, the most challenging aspect to developing software that was powerful yet intuitive for this application. There have been hundreds of specific design choices ranging from specifying metadata, selecting the time period of interest, the visual appearance of plots, interactivity vs speed decisions, and which functions are essential vs distractions. Initially, our vision of this program was to distribute R scripts or python scripts to users. We quickly realized that the majority of users are not comfortable in a programming environment. To address this issue, we implemented the Shiny graphical user interface for R programming (37). To test usability, we next held a user group meeting where we invited interested parties to bring their data for analysis. While there was considerable excitement for the project, we quickly discovered several essential features that were not supported, as well as confusing aspects of the user interface. Subsequent testing of CalR was performed on our own experiments run by the indirect calorimetry core facility that I directed. We “ate our own dogfood” by running CalR analysis on a broad variety of experimental analysis, including effects on age, temperature challenges, pharmacological treatments, genetic overexpression or knockouts, optogenetics, chemogenetics, sleep, circadian biology, torpor, microbiome, activated immunity, immunodeficiency, cytokine stimulation, and dietary challenges (38-51). We continue to operate a support email account where CalR users can ask questions about the program and their analyses which continually reveals gaps in CalR’s functionality.
1. Butler AA, Kozak LP. A recurring problem with the analysis of energy expenditure in genetic models expressing lean and obese phenotypes. Diabetes. 2010;59(2):323-9. doi: 10.2337/db09-1471. PubMed PMID: 20103710; PubMed Central PMCID: PMC2809965.
2. Himms-Hagen J. On raising energy expenditure in ob/ob mice. Science. 1997;276(5315):1132-3. doi: 10.1126/science.276.5315.1132. PubMed PMID: 9173544.
3. Pelleymounter MA, Cullen MJ, Baker MB, Hecht R, Winters D, Boone T, et al. Effects of the obese gene product on body weight regulation in ob/ob mice. Science. 1995;269(5223):540-3. doi: 10.1126/science.7624776. PubMed PMID: 7624776.
4. Flier JS. Irreproducibility of published bioscience research: Diagnosis, pathogenesis and therapy. Molecular metabolism. 2017;6(1):2-9. doi: 10.1016/j.molmet.2016.11.006. PubMed PMID: 28123930; PubMed Central PMCID: PMC5220388.
5. Jarvis MF, Williams M. Irreproducibility in Preclinical Biomedical Research: Perceptions, Uncertainties, and Knowledge Gaps. Trends Pharmacol Sci. 2016;37(4):290-302. Epub 2016/01/19. doi: 10.1016/j.tips.2015.12.001. PubMed PMID: 26776451.
6. Drucker DJ. Never Waste a Good Crisis: Confronting Reproducibility in Translational Research. Cell Metab. 2016;24(3):348-60. doi: 10.1016/j.cmet.2016.08.006. PubMed PMID: 27626191.
7. Peng RD. Reproducible research in computational science. Science. 2011;334(6060):1226-7. doi: 10.1126/science.1213847. PubMed PMID: 22144613; PubMed Central PMCID: PMC3383002.
8. Tanner JM. Fallacy of per-weight and per-surface area standards, and their relation to spurious correlation. J Appl Physiol. 1949;2(1):1-15. Epub 1949/07/01. doi: 10.1152/jappl.19184.108.40.206. PubMed PMID: 18133122.
9. Weil Jr WB. Adjustment for size-a possible misuse of ratios. American Journal of Clinical Nutrition. 1962;11:249-52.
10. Dinkel C, Wilson L, Tuma H, Minyard J. Ratios and percents as measures of carcass traits. Journal of Animal Science. 1965;24(2):425-9.
11. Atchley WR, Gaskins CT, Anderson D. Statistical properties of ratios. I. Empirical results. Systematic Zoology. 1976;25(2):137-48.
12. Devlaming V, Grossman G, Chapman F. On the use of the gonosomatic index. Comparative Biochemistry and Physiology Part A: Physiology. 1982;73(1):31-9.
13. Packard GC, Boardman TJ. The use of percentages and size-specific indices to normalize physiological data for variation in body size: wasted time, wasted effort? Comparative Biochemistry and Physiology Part A: Molecular & Integrative Physiology. 1999;122(1):37-44. doi: https://doi.org/10.1016/S1095-6433(98)10170-8.
14. Reist JD. An empirical evaluation of several univariate methods that adjust for size variation in morphometric data. Canadian Journal of Zoology. 1985;63(6):1429-39.
15. Gould SJ. Allometry and size in ontogeny and phylogeny. Biological Reviews. 1966;41(4):587-638.
16. Prothero J. Methodological aspects of scaling in biology. Journal of Theoretical Biology. 1986;118(3):259-86.
17. Packard GC, Boardman TJ. The misuse of ratios, indices, and percentages in ecophysiological research. Physiological Zoology. 1988;61(1):1-9.
18. Jackson DA, Somers KM. The spectre of ‘spurious’ correlations. Oecologia. 1991;86(1):147-51.
19. Raubenheimer D, Simpson S. Analysis of covariance: an alternative to nutritional indices. Entomologia experimentalis et applicata. 1992;62(3):221-31.
20. Kronmal RA. Spurious correlation and the fallacy of the ratio standard revisited. Journal of the Royal Statistical Society: Series A (Statistics in Society). 1993;156(3):379-92.
21. Albrecht GH, Gelvin BR, Hartman SE. Ratios as a size adjustment in morphometrics. American Journal of Physical Anthropology. 1993;91(4):441-68.
22. Allison D, Paultre F, Goran M, Poehlman E, Heymsfield S. Statistical considerations regarding the use of ratios to adjust data. International journal of obesity and related metabolic disorders: journal of the International Association for the Study of Obesity. 1995;19(9):644-52.
23. Raubenheimer D. Problems with ratio analysis in nutritional studies. Functional Ecology. 1995;9(1):21-9.
24. Tschöp MH, Speakman JR, Arch JR, Auwerx J, Brüning JC, Chan L, et al. A guide to analysis of mouse energy metabolism. Nature methods. 2012;9(1):57-63. PubMed Central PMCID: PMC3654855
25. Speakman JR, Fletcher Q, Vaanholt L. The ’39 steps’: an algorithm for performing statistical analysis of data on energy intake and expenditure. Disease models & mechanisms. 2013;6(2):293-301. Epub 2013/03/23. doi: 10.1242/dmm.009860. PubMed PMID: 23520145; PubMed Central PMCID: PMC3597012.
26. Kaiyala KJ, Schwartz MW. Toward a more complete (and less controversial) understanding of energy expenditure and its role in obesity pathogenesis. Diabetes. 2011;60(1):17-23. doi: 10.2337/db10-0909. PubMed PMID: 21193735; PubMed Central PMCID: PMC3012169.
27. Arch J, Hislop D, Wang S, Speakman J. Some mathematical and technical issues in the measurement and interpretation of open-circuit indirect calorimetry in small animals. International journal of obesity. 2006;30(9):1322-31.
28. Kaiyala KJ, Morton GJ, Leroux BG, Ogimoto K, Wisse B, Schwartz MW. Identification of body fat mass as a major determinant of metabolic rate in mice. Diabetes. 2010;59(7):1657-66. Epub 2010/04/24. doi: 10.2337/db09-1582. PubMed PMID: 20413511; PubMed Central PMCID: PMC2889765.
29. Kaiyala KJ. Mathematical model for the contribution of individual organs to non-zero y-intercepts in single and multi-compartment linear models of whole-body energy expenditure. PloS one. 2014;9(7):e103301. doi: 10.1371/journal.pone.0103301. PubMed PMID: 25068692; PubMed Central PMCID: PMC4113365.
30. Poehlman ET, Toth MJ. Mathematical ratios lead to spurious conclusions regarding age-and sex-related differences in resting metabolic rate. The American journal of clinical nutrition. 1995;61(3):482-5.
31. Kleiber M. Body size and metabolism. Hilgardia. 1932;6(11):315-53.
32. Fisher RA. The analysis of covariance method for the relation between a part and the whole. Biometrics. 1947;3(2):65-8. Epub 1947/06/01. PubMed PMID: 20255205.
33. McCullagh P, Nelder JA. Generalized linear models. 2nd ed. Boca Raton: Chapman & Hall/CRC; 1998. xix, 511 p. p.
34. Stanford KI, Middelbeek RJ, Townsend KL, An D, Nygaard EB, Hitchcox KM, et al. Brown adipose tissue regulates glucose homeostasis and insulin sensitivity. J Clin Invest. 2013;123(1):215-23. Epub 2012/12/12. doi: 10.1172/JCI62308. PubMed PMID: 23221344; PubMed Central PMCID: PMC3533266.
35. Mina AI, LeClair RA, LeClair KB, Cohen DE, Lantier L, Banks AS. CalR: A Web-Based Analysis Tool for Indirect Calorimetry Experiments. Cell Metab. 2018;28(4):656-66 e1. Epub 2018/07/19. doi: 10.1016/j.cmet.2018.06.019. PubMed PMID: 30017358; PubMed Central PMCID: PMC6170709.
36. Wickham H. Tidy data. Journal of Statistical Software. 2014;59(10):1-23.
37. Chang W, Cheng J, Allaire J, Xie Y, McPherson J. shiny: Web Application Framework for R. 1.5.0 ed2021.
38. Wara AK, Wang S, Wu C, Fang F, Haemmig S, Weber BN, et al. KLF10 Deficiency in CD4(+) T Cells Triggers Obesity, Insulin Resistance, and Fatty Liver. Cell reports. 2020;33(13):108550. doi: 10.1016/j.celrep.2020.108550. PubMed PMID: 33378664; PubMed Central PMCID: PMC7816773.
39. Walker RG, Barrandon O, Poggioli T, Dagdeviren S, Carroll SH, Mills MJ, et al. Exogenous GDF11, but not GDF8, reduces body weight and improves glucose homeostasis in mice. Scientific reports. 2020;10(1):1-13. PubMed Central PMCID: PMC7067781.
40. Krisko TI, Nicholls HT, Bare CJ, Holman CD, Putzel GG, Jansen RS, et al. Dissociation of Adaptive Thermogenesis from Glucose Homeostasis in Microbiome-Deficient Mice. Cell Metab. 2020;31(3):592-604 e9. Epub 2020/02/23. doi: 10.1016/j.cmet.2020.01.012. PubMed PMID: 32084379.
41. Hu B, Jin C, Zeng X, Resch JM, Jedrychowski MP, Yang Z, et al. γδ T cells and adipocyte IL-17RC control fat innervation and thermogenesis. Nature. 2020;578(7796):610-4. PubMed Central PMCID: PMC7055484.
42. Hrvatin S, Sun S, Wilcox OF, Yao H, Lavin-Peter AJ, Cicconet M, et al. Neurons that regulate mouse torpor. Nature. 2020;583(7814):115-21. Epub 2020/06/13. doi: 10.1038/s41586-020-2387-5. PubMed PMID: 32528180; PubMed Central PMCID: PMC7449701.
43. Harris DA, Mina A, Cabarkapa D, Heshmati K, Subramaniam R, Banks AS, et al. Sleeve gastrectomy enhances glucose utilization and remodels adipose tissue independent of weight loss. Am J Physiol Endocrinol Metab. 2020;318(5):E678-E88. Epub Feb 24, 2020. doi: 10.1152/ajpendo.00441.2019. PubMed PMID: 32069072; PubMed Central PMCID: PMC7395476.
44. Hall JA, Ramachandran D, Roh HC, DiSpirito JR, Belchior T, Zushin PH, et al. Obesity-Linked PPARgamma S273 Phosphorylation Promotes Insulin Resistance through Growth Differentiation Factor 3. Cell Metab. 2020;32(4):665-75 e6. Epub 2020/09/18. doi: 10.1016/j.cmet.2020.08.016. PubMed PMID: 32941798; PubMed Central PMCID: PMC7543662.
45. Adhikari AA, Seegar TCM, Ficarro SB, McCurry MD, Ramachandran D, Yao L, et al. Development of a covalent inhibitor of gut bacterial bile salt hydrolases. Nat Chem Biol. 2020;16(3):318-26. doi: 10.1038/s41589-020-0467-3. PubMed PMID: 32042200; PubMed Central PMCID: PMC7036035.
46. Yao L, Seaton SC, Ndousse-Fetter S, Adhikari AA, DiBenedetto N, Mina AI, et al. A selective gut bacterial bile salt hydrolase alters host metabolism. eLife. 2018;7. doi: 10.7554/eLife.37182. PubMed PMID: 30014852; PubMed Central PMCID: PMC6078496.
47. Mills EL, Pierce KA, Jedrychowski MP, Garrity R, Winther S, Vidoni S, et al. Accumulation of succinate controls activation of adipose tissue thermogenesis. Nature. 2018;560(7716):102-6. Epub 2018/07/20. doi: 10.1038/s41586-018-0353-2. PubMed PMID: 30022159; PubMed Central PMCID: PMC7045287.
48. Long JZ, Roche AM, Berdan CA, Louie SM, Roberts AJ, Svensson KJ, et al. Ablation of PM20D1 reveals N-acyl amino acid control of metabolism and nociception. Proceedings of the National Academy of Sciences. 2018;115(29):E6937-E45. PubMed Central PMCID: PMC6055169
49. Kohlgruber AC, Gal-Oz ST, LaMarche NM, Shimazaki M, Duquette D, Koay HF, et al. gammadelta T cells producing interleukin-17A regulate adipose regulatory T cell homeostasis and thermogenesis. Nat Immunol. 2018;19(5):464-74. Epub 2018/04/20. doi: 10.1038/s41590-018-0094-2. PubMed PMID: 29670241.
50. Hong S, Song W, Zushin PH, Liu B, Jedrychowski MP, Mina AI, et al. Phosphorylation of Beta-3 adrenergic receptor at serine 247 by ERK MAP kinase drives lipolysis in obese adipocytes. Molecular metabolism. 2018;12:25-38. Epub 2018/04/18. doi: 10.1016/j.molmet.2018.03.012. PubMed PMID: 29661693; PubMed Central PMCID: PMC6001906.
51. Palmer CJ, Bruckner RJ, Paulo JA, Kazak L, Long JZ, Mina AI, et al. Cdkal1, a type 2 diabetes susceptibility gene, regulates mitochondrial function in adipose tissue. Molecular metabolism. 2017;6(10):1212-25. Epub 2017/10/17. doi: 10.1016/j.molmet.2017.07.013. PubMed PMID: 29031721; PubMed Central PMCID: PMC5641635.