There are ways this can happen even without the technicalities of date tracking on the Linux file system. Take, for example, Microsoft’s decision to store local time in the system clock. If you dual boot, and don’t configure either Linux or Windows to be consistent with the other, your clock will be off by one or more hours, unless you happen to live at UTC+00:00. Every modern computer users NTP to automatically correct itself, but it’s not uncommon to see tons of files with weird timestamps after booting Windows.
Even without dual booting, it’s possible your computer’s clock has drifted into the future when it was off, and got corrected later. That would explain seconds or minutes of differences.