The Universe probably wouldn’t fit in a sphere of any size and it very well may be infinite.
But looking at a very large spherical region like the visible universe from our perspective here on Earth, everything is moving away from us. If objects were being pulled towards one particular edge of our imaginary sphere, it would look very different. We’d see a clear drift in that direction, but it all looks pretty even across every direction we look.