Odd consideration, but… I use 16Gb of ram and I have zero swap space and I’ve never seen a freeze in the three years this system has been assembled.
You won’t see big problems until you use most of the ram, then you’re toast. Also Linux (if that’s what you’re using) prefers to have more cache at the expense of swapping out pages. There’s a lot of rarely used code on many apps that can be safely swapped out to get you more cache.