It certainly can be noticeable, depending on where and at what you’re looking. The issue can be that making a general “yes, you’ll see a difference” statement is usually difficult; opinions of what counts as “a difference” vary wildly from person to person.
What you experience could be as small as shaving a few KiB off your executable or getting an extra 2-3 FPS in a game all the way up to “it just won’t work unless I compile it this specific way because of <insert_variable>.”