Try to exclude it from being restored on KDE startup. There could be something running in the background that keeps getting saved and restored. Go to System Settings, System, Desktop Session, put virtualbox in the ignored applications list. You can also just choose “Start with an empty session” and then relog to see if that stops it from coming back, then you’ll know if this is causing your problem or not. Good luck!