Just go for it! It’ll be fun. You will find yourself navigating your digital life (or at least: most of it) at relative ease with less distractions and graphical fuzz you otherwise get.
Getting to know terminal multiplexer (like screen) or vi-keys are crucial skills for anyone who is not afraid from working on “headless” machines!