fubarx , 1 month ago

I’vd tried multiple times to convert existing code or createnew ones using LLMs. The first attempts are OK, but once you start refining the prompts, they all go off-the-rails.

Most of the time, the generated code uses old or deprecated libraries or APIs. You point that out and they correct it. But a few iterations later, you’re refining something else and the old, deprecated calls come back. Once again, you point it out and it gets corrected.

Forget trying to correct it yourself by hand, because now it’s diverged from the LLM context. And this can happen in multiple places in the code. Rinse. Repeat.

At some point you just give up. Either it’s wrong or it will be wrong in different ways later. You have to read through every line to find strange, divergent errors. Over and over. It gets exhausting.

At the end, it feels like maybe you could have done it faster and more quickly yourself, but the time has already been sunk.