Everything has a negative health impact, literally even water does. The question is always about cumulative and spontaneous dosage. At what point does it become bad for you.
One common explanation I’ve seen for aspartame is that it makes your body think you’re drinking sugar while no sugar is being absorbed. This is then potentially harmful for those with a predisposition for diabetes.