Comment by byko3y

No, they are not the same, and they require different tools: you cannot write code in gdb or valgrind. Even in IDE-s debug and editing interfaces differ significantly. You can literally separate gdb session from any code editing, do debugging with zero code writing and then relay the findings to someone who will make the changes.

The fact that you personally do both things does not mean everybody and everything does it this way. And certainly LLM-s don't function same way people do.

And no, debugging and code writing are not in same datasets. More to it, debugging and architectural planning is mostly represented in fine-tune datasets (prompt-response, agentic step prediction), while code writing is mostly (like 99%) pre-training data (refined internet scraps and regenerated code).

>Not really, they are primarily trained on LLM-generated trajectories from executable training environments.

Training on purely LLM-generated data is known to give horrible results. Big AI companies already have thousands of debugging sessions at hand — those bring high novelty/variability into the training sets. It is possible to multiple those fully supervised sets via LLM-generated bugs or bugs from github reproduced in a sandbox. With all that said, feeding lots of very similar bugs and solution trajectories does not help much in training — so it's rather "combinations of known human skills tested in a sandbox", not "totally random efforts transformed into datasets of successful debugging outcomes".

>So they're only similar now?

`c = a + b` and `c = a + d` are similar, but the difference is very small — it's about variable rename. It's not a unique code. And I never told LLM has to reproduce 1 Mb of source code byte-to-byte without any change. Just in case some of my words sounds like I told — I did not mean it.

>in the way that I learned how to solve bugs while learning to program and now I can fix similar ones? I guess we can now agree that LLMs are capable of learning and extrapolating knowledge to new problems?

No, LLM does not learn to program same way people do, it generally does not learn things the way humans do. Why would they? The training of a human looks nowhere close to how LLM-s are trained. Even the way people translate similarities is not the same as LLM's: LLM does precise literal swap, human extracts meaning and unfolds the solution into concrete words. Humans chase goals and plan things, LLM-s pattern-match and reproduce superficial shape, treating steps in complete isolation.

Human can try to reproduce actions with zero understanding of the subject — ­which usually leads to very mediocre results even after lots of effort. Basically, LLM can be seen as a stupid and blindly-diligent office worker x100.