You are right that things would still look like we’re accelerating away from us, even if we were actually contracting.
Interesting hypothesis! How do we investigate?
What could we expect from a large central gravitational point? We should have other signs of the gravity well:
We would expect a point that we contract towards (and that seems ill fitting, as we see the expansion moves as the observer (including earth) moves), we would expect some kind of mass or similar effect, which would also have a size to fit it in (we know that gravity works different when you’re inside the mass, and we would be able to see it, much like black holes or dark matter), we would expect things to orbit the gravity well (which we know that at least our galaxy doesn’t orbit us).
You might want to actually check on these things to make sure they apply and are true, but at least at first glance it seems the expansion is better explained without a central gravity.