"The study examined 52 software engineers learning a Python library and found productivity gains came at the cost of learning, particularly in debugging abilities. The randomized controlled trial divided participants into two groups while learning the Trio asynchronous programming library. One group used AI assistance, the other coded by hand. The results were striking: AI users averaged 50 percent on a subsequent quiz, compared to 67 percent for manual coders."
"How developers use AI matters The study identified six distinct patterns of AI usage. Three led to quiz scores below 40 percent, where AI was seemingly detrimental to the results. These were complete AI delegation, progressive reliance on AI (1 human-led task, then AI-led), and iterative AI debugging. Participants who wholly relied on AI completed tasks fastest but learned least."
52 software engineers learning a Python library were randomized into two groups: one used AI assistance and the other coded manually. AI users averaged 50 percent on a subsequent quiz, while manual coders averaged 67 percent. AI users finished tasks only about two minutes faster on average, a non-significant difference, and some participants spent up to 11 minutes composing AI queries. Six usage patterns emerged: three correlated with poor learning (<40%): complete AI delegation, progressive reliance, and iterative AI debugging; three preserved learning (65–86%): conceptual questioning, explanations with code, and follow-up questioning after generated code. Debugging ability showed the largest deficit.
Read at Techzine Global
Unable to calculate read time
Collection
[
|
...
]