I’ve used a very similar setup in the past (J1900 CPU, 4x1 Gbps network ports) and I only replaced it due to reasons. Not noticed any performance bottle necks with that setup.
The latest N100/N200/N300/N305 CPUs from Intel looks really interesting, similar performance as my workstation but at a 10th of the power usage. N305 also has 8 cores in a passively cooled case, amazing stuff!