You’re being downvoted because you’re asking another “I want everything, but works exactly to my needs, only the way I want it, and cheap.” kind of question.
Cloudflare exists for a reason, as does every other DDOS mitigation platform. If there was a better or cheaper solution, they would be out of business already.
Best you’re probably going to do for self-hosting is going to be blackholing abusive connections, but even then you’re only going to be able to mitigate so much. Differentiation of mass amounts traffic still takes a massive amount of time and compute.