I agree with everything you said in the second half. Better non-car infrastructure should be the primary focus!
But I don’t see why traffic calming and slower car speeds would make things miserable for “everyone”, like cyclists or pedestrians. I wouldn’t ride my bike nearly as often if cars could go 45 mph on Canadian streets. Making cars uncomfortable is part of encouraging other modes of transportation.