idw – Informationsdienst Wissenschaft
Nachrichten, Termine, Experten
Nachrichten, Termine, Experten
When writing program code, software developers often work in pairs—a practice that reduces errors and encourages knowledge sharing. Increasingly, AI assistants are now being used for this role. But this shift in working practice isn’t without its drawbacks, as a new empirical study by computer scientists in Saarbrücken reveals. Developers tend to scrutinize AI-generated code less critically and they learn less from it. These findings will be presented at a major scientific conference in Seoul.
When two software developers collaborate on a programming project—known in technical circles as 'pair programming'—it tends to yield a significant improvement in the quality of the resulting software. ‘Developers can often inspire one another and help avoid problematic solutions. They can also share their expertise, thus ensuring that more people in their organization are familiar with the codebase,’ explains Sven Apel, professor of computer science at Saarland University. Together with his team, Apel has examined whether this collaborative approach works equally well when one of the partners is an AI assistant. In the study, 19 students with programming experience were divided into pairs: six worked with a human partner, while seven collaborated with an AI assistant. The methodology for measuring knowledge transfer was developed by Niklas Schneider as part of his Bachelor’s thesis.
For the study, the researchers used GitHub Copilot, an AI-powered coding assistant introduced by Microsoft in 2021, which, like similar products from other companies, has now been widely adopted by software developers. These tools have significantly changed how software is written. 'It enables faster development and the generation of large volumes of code in a short time. But this also makes it easier for mistakes to creep in unnoticed, with consequences that may only surface later on,' says Sven Apel. The team wanted to understand which aspects of human collaboration enhance programming and whether these can be replicated in human-AI pairings. Participants were tasked with developing algorithms and integrating them into a shared project environment.
'Knowledge transfer is a key part of pair programming,' Apel explains. 'Developers will continuously discuss current problems and work together to find solutions. This does not involve simply asking and answering questions, it also means that the developers share effective programming strategies and volunteer their own insights.' According to the study, such exchanges also occurred in the AI-assisted teams—but the interactions were less intense and covered a narrower range of topics. 'In many cases, the focus was solely on the code,' says Apel. 'By contrast, human programmers working together were more likely to digress and engage in broader discussions and were less focused on the immediate task.
One finding particularly surprised the research team: ‘The programmers who were working with an AI assistant were more likely to accept AI-generated suggestions without critical evaluation. They assumed the code would work as intended,’ says Apel. ‘The human pairs, in contrast, were much more likely to ask critical questions and were more inclined to carefully examine each other’s contributions,' explains Apel. He believes this tendency to trust AI more readily than human colleagues may extend to other domains as well. ‘I think it has to do with a certain degree of complacency—a tendency to assume the AI’s output is probably good enough, even though we know AI assistants can also make mistakes.’ Apel warns that this uncritical reliance on AI could lead to the accumulation of 'technical debt’, which can be thought of as the hidden costs of the future work needed to correct these mistakes, thereby complicating the future development of the software.
For Apel, the study highlights the fact that AI assistants are not yet capable of replicating the richness of human collaboration in software development. ‘They are certainly useful for simple, repetitive tasks,’ says Apel. ‘But for more complex problems, knowledge exchange is essential—and that currently works best between humans, possibly with AI assistants as supporting tools.' Apel emphasizes the need for further research into how humans and AI can collaborate effectively while still retaining the kind of critical eye that characterizes human collaboration.
Alisa Welter, a PhD student in Apel’s group and first author of the article, will present the findings at the 40th IEEE/ACM International Conference on Automated Software Engineering—one of the top three conferences in the field. The conference will take place from November 16 to 20 in Seoul, South Korea. Out of the approximately 1,200 papers submitted to the conference, only 150 were accepted for presentation. The study was funded by the European Union through the ERC Advanced Grant ‘Brains On Code’ (see press release from April 26, 2022): https://saarland-informatics-campus.de/piece-of-news/brains-on-code/
Further information:
Empirical study:
Professor Sven Apel
Professor of Computer Science at Saarland University
Tel.: +49 681 302-57211
Email: apel@cs.uni-saarland.de
Alisa Welter
PhD student in the Software Engineering research group
Email: welter@cs.uni-saarland.de
https://www.se.cs.uni-saarland.de/publications/docs/WSD+.pdf
https://conf.researchr.org/home/ase-2025 - 40th IEEE/ACM International Conference on Automated Software Engineering
https://conf.researchr.org/details/ase-2025/ase-2025-papers/122/An-Empirical-Stu...
https://www.se.cs.uni-saarland.de - Software Engineering research group at Saarland University
Sven Apel, professor of computer science at Saarland University
Source: Oliver Dietze
Copyright: Universität des Saarlandes
Alisa Welter, PhD-Student in the Software Engineering research group
Source: private
Copyright: Universität des Saarlandes
Criteria of this press release:
Business and commerce, Journalists, Scientists and scholars
Information technology, Media and communication sciences, Social studies
transregional, national
Research results, Scientific conferences
English
Sven Apel, professor of computer science at Saarland University
Source: Oliver Dietze
Copyright: Universität des Saarlandes
Alisa Welter, PhD-Student in the Software Engineering research group
Source: private
Copyright: Universität des Saarlandes
You can combine search terms with and, or and/or not, e.g. Philo not logy.
You can use brackets to separate combinations from each other, e.g. (Philo not logy) or (Psycho and logy).
Coherent groups of words will be located as complete phrases if you put them into quotation marks, e.g. “Federal Republic of Germany”.
You can also use the advanced search without entering search terms. It will then follow the criteria you have selected (e.g. country or subject area).
If you have not selected any criteria in a given category, the entire category will be searched (e.g. all subject areas or all countries).
