Polygamy just means that you marry more than one person. It's not related to patriarchy and there are many polygamous people around the world who are not patriarchal at all.
Polyamory means loving more than one person and can take many of the same forms as polygamy does, including patriarchal structures.