Literally, you’re right - in Latin it means “not following”. But in conventional usage, non-sequitur is more for things that are so completely out of place for the conversation.
Not a non-sequitur: “Okay, so based on this finding, [insert something topical but wrong]”.
Non-sequitur: “Okay, so that’s great, but Michigan beating Ohio State means this is irrelevant”.
(edit because I did not realize the formatting I used for my non-sequitur example caused it not to render)