If I start the client without being connected through the VPN, my IP does not show up. It is only when the VPN connection is disconnected and I transition from one network interface to another. This is also if I have the kill switch enabled, so I imagine that if the connection is lost, I am safe as all internet access would be blocked then. And that it is only if I manually disconnect myself that this happens.
Could this be a Proton VPN issue? I am pretty sure I checked this previously, and didn’t see it before the recent Linux client GUI update, but I am not entirely sure.