Even though I regularly read papers in this field and I generally try to keep up with the state of the art, I keep finding myself exclaiming “wtf” whenever something like this comes out. Like, you can see it made the same mistakes other generatie models make, but then it corrects them (mostly) once we get closer. It wasn’t the sharpest or completely flawless, but I’m in awe about the stability across such a long video and about how lifelike it all looks.
Edit: ah it’s using world models just like PlaNet did for reinforcement learning. I suspected something like that because of the stability of the generation. Absolutely amazing results.