It’s not “people vs persons” but “those people vs they”.
Conversationally, “those/these” distances you from the group you are talking about, which is humorously weird when it’s your family you’re talking about.
It’s not the meaning of the words, but habitual (and often fleeting) attribution around them that tripped you up.
PS: “People” are uncountable, “persons” are countable. That’s basically the whole difference between the two plurals. Although it’s rapidly disappearing, as “ten people” won’t raise a single eyebrow in a conversation.