Trained on buggy code, LLMs often parrot same mistakes

"Researchers have discovered that large language models often replicate flaws in buggy code instead of correcting them, leading to higher error rates when tasked with code completion."

"In a recent study, scientists tested various large language models and found they frequently mirror known code defects found in buggy code snippets, raising concerns about their reliability."

A study conducted by researchers from various institutions analyzed how large language models (LLMs) respond to buggy code. They found that these models, including OpenAI's GPT-4 and GPT-3.5, often repeat known mistakes instead of correcting them when asked to complete flawed code snippets. This replication of errors is particularly problematic, as it suggests a higher likelihood of generating erroneous code outputs. The findings raise serious concerns about the reliability of LLMs in practical coding scenarios, particularly given the prevalence of bugs in programming.

Read at Theregister