Willpower is just an abstraction. It is not a “real” thing, consumed once you do something you wouldn’t.
That said yes, you can train it. Or rather, condition your behaviour so you do things that you’d otherwise avoid. The how-to is actually simple:
Choose a task to perform. Push it a tiiiny bit harder.
Did you manage to push it harder? If yes, reward yourself. If not, skip.
Repeat 1 and 2 for some time.
Gradually decrease reward frequency, make it a bit random. Oddly enough this makes the desired behaviour to stick further.
Eventually you won’t need the reward, but the behaviour is still there.
What you consider a reward is up to you. For example, for me snacking on cheese is a reward, but it might not be for you. With the right mindset, even mundane things can be a reward, like taking a comfy shower or playing some games.