Ethernet Consortium Shoots For 1 Million Node Clusters That Beat InfiniBand

drdabbles , 10 months ago

The Ultra Ethernet consortium work is interesting to me, but I’m still on the fence about some of the design intentions or at least considerations. One of the major bullet points they list on their introductory white paper is that they want whatever UltraEthernet becomes to be reverse compatible with standard Ethernet. IMO, if you’re going to go through the pain of addressing the entire protocol stack from PHY up to application level, then you might as well rip the bandage off all at once.

These massive scale networks that suffer from Ethernet’s “inefficiencies” aren’t likely to deploy a mixed environment. They’re much more likely to build an entirely new environment, decommission an old environment, and build another new environment in its place. The odds of overlapping the two seem really remote to me. But then again, having 1M device mostly flat broadcast domains also seems like an odd choice to me.

enu , 10 months ago

Impressive that Nvidia has found a price point for Infiniband that’s so bad that it’s actually more cost effective for companies to create an open standard to ditch them. The initiative sounds really interesting and I’m curious to see what comes out of it. The article mentions latency several times, but they really need to beat Infiniband’s deterministic latency. That’s going to be a challenge to do, especially while carrying all Ethernet’s baggage.

sky , 10 months ago

This looks really interesting! proprietary networking has always rubbed me the wrong way, though infiniband is very impressive.