I’ve had 100% failure rate on simple requirements that require a simple spin on well known solutions
“Make a pathfinding function for a 2d grid” - fine
“Make a pathfinding function for a 2d grid, but we can only move 15 cells at a time” - fails on lesser models, it keeps clinging to pulling you the same A* as the first one
“Make a pathfinding function for a 2d grid, but we can only move 15 cells at a time, also, some cells are on fire and must be avoided if possible, but if there is no other path possible then you’re allowed to use fire cells as fallback” - Never works
There for that last one, none of the models give a solution that fits the very simple requirement. It will either always avoid fire or give fire a higher cost, which is not at all a fitting solution
High costs means if you’ve got a path that’s 15 tiles long without fire, but way shorter with fire, then sure, some fire is fine! And if you could walk 15 tiles and go to your destination but need to walk on 1 fire, then it will count that as 15-something and that’s too long.
Except no, that’s not what you asked.
If you try and tell it that, gpt4 flip flops between avoiding fire and touching the price of tiles
It fails because all the literature on pathfinding talks about is the default approach, and cost heuristic functions. That won’t cut it here, you have to touch the meat of the algorithm and no one ever covers that (because that’s just programming, it depends on what you need to do there are infinite ways you could do it, to fit infinite business requirements)