I would say that rule should only apply to purposefully taking something for an unfair advantage to juke the competition, like most steroids.
For the ones where the only reported reasoning is being harmful to the user or has a potential for abuse, I would say the decision be on a case by case basis.
My reasoning is that there are also people who’ve turned their lives around and quit the substances in question. Being banned when you are fully “legit” would be extremely demotivational and can increase relapsing. It could also be controversial to one of the ideals of the Olympics; overcoming adversity to be the best you can be.