Problems with reproducibility in scientific results is a topic of major concern to most stakeholders. Fraud such as “dry-labbing” is one cause, but is probably very rare. More often the difficulty is due to differences in the experimental technique. Some differences are unavoidable since instrumentation can evolve, as can understanding. However, in my experience, uncontrolled variables, which are often unrecognized at the time of the experiment, are the most frequent and frustrating gremlins in experimental reproducibility. Difficulty in getting the same results from an assay run in two labs in different locations is an expectable problem. This contrasts with the ideal world expectation that experiments and data should be FAIR (Findable, Accessible, Interoperable, and Reusable).1
Phillip Morris International (PMI) has a large, ongoing research program developing modified risk tobacco products (MRTPs).2 This research involves many labs around the world. Program management involves comparing experimental protocols, software, and data over time from different labs. To facilitate comparability, reproducibility, and replicability, and in recognition of the need to have its research scrutinized by external, independent scientists, PMI has developed an open online platform called INTERVALS for sharing data, software, and experimental protocols. INTERVALS initially was an acronym for Inhalation Toxicology rEpositoRy for noVel tobAcco and aLternative productS. Recently, the scope of the platform has expanded from a sharp focus on tobacco toxicology to include more general clinical and population studies.
INTERVALS provides full standardized and annotated data sets managed by PMI as part of the MRTPs programs. Scientists can use in vivo and in vitro studies on humans to investigate the pharmacokinetics of several candidate nicotine-delivery systems. They are invited to add their own research data to INTERVALS. The databases will be assigned unique DOIs. By sharing, contributing scientists will facilitate open-access sharing of their data with regulatory agencies such as the FDA. Society should benefit, since the data can be used or repurposed to support grant applications.
To date, comparison and integration of data files has not lived up to the naïve initial expectations. Recognizing the problems and mitigating them over the decades should improve performance in reduction of tobacco harm, in particular, but much more broadly in data-driven life science research and decision-making. PMI claims that INTERVALS can be applied to any field that deals with large and complex data. This is precisely what the National Academies of Science, Engineering and Medicine recently advocated in its report, Reproducibility and Replicability in Science.3
PMI has decades of experience comparing data from different labs. Results were sometimes suspect because of public controversy over smoking. Experiments undertaken to prove a point are not conducive to the FAIR status. However, INTERVALS may be a useful platform for modern, data-rich science. Let’s not get caught up in the origins. If INTERVALS works, let’s use it.
The INTERVALS Platform is accessible at www.intervals.science.
- Wilkinson, M.D.; Dumontier M. et al. The FAIR guiding principles for scientific data management and stewardship. Sci. Data 2016, 3, 160018; https://www.nature.com/articles/sdata201618
- “Philip Morris International advocates scientific transparency and creates an online community platform for scientific data.” Press release from PMI Science, Neuchatel, Switzerland, July 16, 2019.
- Reproducibility and Replicability in Science. The National Academies Press: Washington D.C.; doi: https://doi.org/10.17226/25303.
Robert L. Stevenson, Ph.D., is Editor Emeritus, American Laboratory/Labcompare; e-mail: firstname.lastname@example.org