I care that the text remains readable (both to me and also software) and that I don’t balloon my storage out of control.
JPEG (even at higher levels) subjectively degrades text in particular to a degree that I worry about the former and PNG makes me worry about the latter.
My current plan is to go with the latter since storage is a relatively cheap issue to fix while data loss is pretty much permanent.