In you case setting bind-address-ipv4 and bind-address-ipv6 to your VPN IPs should work. According to this under macOS settings are stored in $HOME/Library/Preferences/org.m0k.transmission.plist. Not sure if the format is the same tho and don’t forget that editing a plist on macOS isn’t just always just editing a text file, it might be encoded and cached by the system, do your research.