Correlation does not imply causation. It could be that the same thing that is lowering your internet speed is affecting LAN, perhaps the router not being enough to handle the traffic, or something in the network occupying a lot of bandwidth which is only active when there is internet (e.g. a download client, or worse a download client accessing a NAS).
In any case you need to give more info into what your setup looks like, e.g.
is the Jellyfin server wired or wireless?
what is the maximum speed you reach when disconnected from the internet?
are you accessing via a computer or phone? And if a computer is it wired?
does the Jellyfin server has other services running that could occupy bandwidth?
is there a NAS for that Jellyfin server?
is the TP-Link acting like a router or a switch?
how are you measuring speed?
have you monitored the Jellyfin server bandwidth usage during those tests? Does it drop or remains constant while you’re testing and disconnect the internet? (If it remains constant it means you’re saturating your server’s connection, so while it is connected to the internet it L’s only using a limited bandwidth for you and using the rest for something else, if you disconnect the internet it allocates everything for you because it’s not doing anything else anymore. If on the other hand the usage increases it signifies the router is not being able to handle the traffic)