So long as the stream that the pi is receiving can be decoded in HW you’re fine. But if you’re streaming e.g. a raw VC1 Blu-ray rip, I think things will work much better on a pi 3 than 4 (though you still need to purchase the codec license for HW decode on a 3).
But yeah, if the server is transcoding to HEVC or h264 in real time (or again, the source material is already in that format), then the 4 should be good.