Yes, but we’re talking about “seconds” and “nanoseconds” rather than hours.
Networks move much faster than we do.
There’s also no hierarchy of IP addresses, and that matters for lookups.
So the 1 second it takes to do a dns lookup is WAY too long for continuous ip lookups, and the size of the database and chains requires explaining where to find ip address X is too long and updates WAY too much to be accurate and/or kept.
Lookups are easiest if you know “I lookup .uk addresses at this particular server in England” because that particular “ authoritative DNS server” only really handles its own little segment of lookups.
There is no such hierarchy in ip addresses, and they can’t really be cached for long.
You would have to continually know and update all of them. And we sorta do in the larger routers, but keeping that up to date at the edges would require a TON of bandwidth.