I wonder if this could partially be a result of training on code found in question/answer environments like Stack Overflow. It sees "How do I do X, here's what I've tried" with broken code and then an answer "This is incorrect because Y, here is the correct answer" with the correct code.

Intuitively it makes sense to me that broken code would often be very close to questions about how to achieve something in code.