idw – Informationsdienst Wissenschaft
Nachrichten, Termine, Experten
Nachrichten, Termine, Experten
The Number One confounding factor in animal research is the person conducting the experiment. A team headed by behavioural biologists Dr. Vanessa von Kortzfleisch and Prof. Helene Richter examined this factor in behavioural experiments involving mice at different locations.
For more than ten years now, scientists have been discussing the so-called reproducibility crisis: often, scientific findings cannot be reproduced at a later time and/or in other laboratories, although the studies are carried out under highly standardised conditions. Thereby, standardisation includes for example the use of genetically identical animals, keeping the animals in identically equipped cages, and carrying out the experiments in always the same way. To uncover sources of poor reproducibility, researchers usually try to identify potential confounding factors in the experimental conditions. Thereby, confounding factor Number One is the experimenter – in other words, the person conducting the experiment. A team headed by behavioural biologists Dr. Vanessa von Kortzfleisch and Prof. Helene Richter from the University of Münster has now studied precisely this factor in behavioural experiments on mice carried out simultaneously at three different locations. Their study has now been published in the journal PLOS Biology.
To the researchers’ surprise, the influence of different experimenters on the test results was not as pronounced as earlier studies suggested. By contrast, the researchers detected other confounding factors. Thus, what plays a much greater role than the experimenter was the factor “laboratory”. Most importantly, however, most variation was explained by inexplicable differences between the individual mice. More precisely, this proportion of “unexplained variance” in the data was between 41 and 72 percent. “This is especially surprising,” says lead author Vanessa von Kortzfleisch, “when you consider that the animals were tested under highly standardised conditions within the same testing cohort – in other words, by the same experimenter in the same lab and under exactly the same conditions.”
The results do certainly not mean that the experimenter do not represent a decisive factor. What they do indicate, though, is that the different test conditions in the labs – despite standardised conditions –have a considerably greater influence on the outcome than the experimenter. These conditions might include for example small differences in ambient sounds or smells. “But what our results show above all is that biological variation plays a key role in animal research – even when the animals come from inbred lines. In future, we will need better strategies for integrating this variation in a controlled way into the experimental design,” says Vanessa von Kortzfleisch.
Twelve experimenters at three locations
The background: Contrary to the dogma of strict standardisation, there are alternative suggestions for integrating variation systematically into the experimental design to improve reproducibility. In order to investigate whether involving multiple experimenters in a single study can increase the external validity, and hence improve the reproducibility of the outcome, this latest study was conducted by twelve different experimenters in Münster, Osnabrück and Bern, all carrying out the same behavioural test battery with mice of two inbred strains. Such phenotyping experiments are widely used in biomedical research to study the effects of different genotypes on the animals’ behaviour and, thereby, draw conclusions about the genetic basis of certain human diseases. For example, in a so-called Open-Field test, researchers check whether a mouse is more or less anxious when exploring a new environment.
Specifically, the team of researchers investigated whether a strictly standardised experimental design, in which all the animals are tested by one experimenter, differs in terms of reproducibility from an experimental design in which the animals are tested by multiple experimenters. The team compared the experimental designs to see which of them yielded the more consistent results across the three different laboratories. In addition, the researchers investigated which other influencing factors might explain the variation in the data. One result was that at all three locations the researchers were not able to reproduce some of the results, regardless of whether the experiment was conducted by just one or several experimenters.
Besides the team from the Department of Behavioural Biology at Münster, other researchers involved in the study are from the Universities of Osnabrück and Bern (Switzerland), the University of Veterinary Medicine in Vienna (Austria) and the AstraZeneca company in Cambridge (UK).
Funding
The German Research Foundation (DFG) provided financial support for the study as part of the project entitled “Standardisation, Heterogenisation and Reproducibility of Results from Animal Testing” (Grant Number: RI 2488/3-1)
Dr. Vanessa von Kortzfleisch
University of Münster
Department of Behavioural Biology
Mail: v_vonk01@uni-muenster.de
Prof. Dr. Helene Richter
University of Münster
Department of Behavioural Biology
Mail: richterh@uni-muenster.de
Vanessa Tabea von Kortzfleisch, Oliver Ambrée, Natasha A. Karp, Neele Meyer, Janja Novak, Rupert Palme, Marianna Rosso, Chadi Touma, Hanno Würbel, Sylvia Kaiser, Norbert Sachser and S. Helene Richter (2022): Do multiple experimenters improve the reproducibility of animal studies? PLOS Biology 20(5): e3001564
Criteria of this press release:
Journalists
Biology
transregional, national
Research results, Scientific Publications
English
You can combine search terms with and, or and/or not, e.g. Philo not logy.
You can use brackets to separate combinations from each other, e.g. (Philo not logy) or (Psycho and logy).
Coherent groups of words will be located as complete phrases if you put them into quotation marks, e.g. “Federal Republic of Germany”.
You can also use the advanced search without entering search terms. It will then follow the criteria you have selected (e.g. country or subject area).
If you have not selected any criteria in a given category, the entire category will be searched (e.g. all subject areas or all countries).
