Google Sheds Some Light On Its Data Center Network Tech
More than any other tech company in the world, Google's business revolves around the shuffling of gargantuan amounts of data across almost innumerable data centers. Given the nature and demands of such complex networks, Google really had very little choice but to embrace distributed computing. Unfortunately for Google, back when it started, data center network technology, both hardware and software, wasn't yet at a level that it could use to meet its requirements. So like any enterprising Silicon Valley company, it made its own.
The latest iteration of its in-house network technology, which started with Firehose, is called Jupiter, which hints at its massive proportions. Virtually, of course. A single datacenter can boast of 100x in capacity compared to their first generation counterparts. It's bandwidth reaches 1 Petabit per second. If that sounds too vague, Google claims that it means being able to read the entire scanned Library of Congress at a tenth of a second. It is impressive definitely, but it also shows the burden that Google must support on its own.
Rolling out your own datacenter network definitely has its advantages. It means tighter control over everything that goes in, and more fine tuning for specific performance. It also means that Google has to do a lot of the heavy lifting itself. It had to, for example, build not just the software but even their own hardware, direct from the silicon provided by vendors instead of relying on ready-made solutions. It means not relying on industry standard protocols and cooking up your own to meet your peculiar needs.
The good news, even for the industry as a whole, is that Google isn't keeping things to itself. Or at least some of them. It has, for example, shared its SPDY protocol which eventually made its way into the new HTTP/2 standard. It has also opened up its datacenter network to outsiders through the Google Cloud Platform. And, being a share infrastructure, GCP users are assured that the data center network technology they are using is the very same one that Google uses internally, with all the benefits that a well-tested, redundant, fast, and stable platform can bring.
SOURCE: Google