Jellyfish: Networking Data Centers Randomly
Summary: We present Jellyfish, a high-capacity network interconnect which, by adopting a random graph topology, yields itself naturally to incremental expansion. Somewhat surprisingly, Jellyfish is more cost-efficient than a fat-tree, supporting as many as 25% more servers at full capacity using the same equipment at the scale of a few thousand nodes, and this advantage improves with scale. We describe approaches to resolve the challenges Jellyfish's unstructured design poses to routing, physical layout, and wiring.
Why does Jellyfish work?Random graphs have provably short path lengths. Short path lengths imply that each flow in the network consumes a small amount of network capacity, and this translates to random graph networks being able to provide high capacity. The picture below shows a path length comparison with the fat-tree. The servers are all drawn at the edge in each network. Any server in the fat-tree can only reach 3 other servers in less than 6 hops. In contrast, an arbitrary origin server can reach almost all other servers in a smaller number of hops. The primary reason the fat-tree has longer paths is that many edges do not shorten path lengths between any pair of servers. (For instance, the two edges marked 'X'.) For detailed comparisons refer to our paper.
| NSDI 2012 |
Jellyfish: Networking Data Centers Randomly |
| HotCloud 2011 |
Jellyfish: Networking Data Centers, Randomly |
(Chi-Yao Hong and I share equal credit for the above papers.)