Using state-of-the-art artificial intelligence techniques did not speed up experienced software developers’ work when they were working in codebases they were familiar with, as is commonly believed.
The comprehensive study, which was carried out by the AI research foundation METR, opens a new tab on a group of seasoned coders earlier this year while they worked on open-source projects they were familiar with using Cursor, a well-known AI coding helper.
The open-source developers thought AI would speed them up prior to the study, expecting a 24% reduction in work completion time. The developers thought they had reduced task times by 20% even after using AI to complete the tasks. However, according to the study, applying AI had the opposite effect, increasing task completion time by 19%.
Joel Becker and Nate Rush, the study’s principal authors, expressed their surprise at the findings, pointing out that Rush had previously stated that he had anticipated “a 2x speed up, somewhat obviously.”
The results cast doubt on the notion that AI invariably boosts the productivity of costly human engineers, a notion that has drawn significant investment into businesses that offer AI tools to support software development.
Additionally, entry-level coding jobs are anticipated to be replaced by AI. In the next one to five years, AI may eliminate half of all entry-level white-collar employment, according to Dario Amodei, CEO of Anthropic, who recently told Axios.
Previous research on increasing productivity has shown notable benefits. For example, one study indicated that utilizing AI sped up programmers by 56% and opened a new tab. Another study found that developers were able to do 26% more work and open a new tab in a given amount of time.
