Thank you everyone, was rather sure that variable refresh rates of different monitors were a hard problem for X11.
So, in nvidia-settings I made my 120Hz one the primary and set the refresh rate from auto to the highest for both monitors (120, 60)
The only difference I noticed was that easyeffects window is now completely unreadable, somehow the text and frames bleeds out/blurs and only gets sharp when hovered with mouse but still buggy. Force-compositon-pipeline did not help there.
Will have to diagnose tomorrow.