The basic gist is that archeologists will be able to date not just the find itself, but the soil it’s in and other potential finds within the same layer of soil/around the site.
So you’re right, if you just found two Roman swords on a bench in the Cotswolds, you probably couldn’t infer from that that the Roman military had definitely been present. But if you find them in a specific layer of soil along with other Roman artifacts, that starts to build up context around the find. The more evidence you collect, the better able you are to put a date on when the swords arrived at your site.