Edge Infrastructure requires more than you might think

Edge computing will be 100X what public cloud is – Michael Dell*

I was recently discussing Edge build out during one of Rob Hirschfeld’s Cloud2030 sessions and came to what should have been an obvious observation - If Edge is going to be 100X or even just 5X what public cloud is, we’re going to have to rethink how the infrastructure is built, maintained and maybe more importantly, how efficiently it’s utilized.

Public Cloud is roughly 60 million servers in 600+ hyperscale facilities and assorted smaller data centers around the world. Imagine putting 300 million servers (5X public cloud) and the assorted storage, network, racks, cement, steal, etc., that you’d need to support it at the edge.

What does 300 million servers mean:

1. Roughly 10 million standard racks, requiring 30SQF (3SQM) each for servers**
2. A corresponding 10 million racks for storage and networking gear***
3. The average cloud data center is roughly 65% IT and 35% supporting equipment. So, for every 30SQF of IT space, we need another 10SQF for supporting ME (Mechanical & Electrical equipment).
4. There are approximately 10,000 cities in the world****

Doing the math from the above:

• 10+ 10 + 7 (ME Space) equal 27 million racks
• 27 million X 30SQF per rack equals 810 million Square Feet (SQF) or 80 million Square meters
• If we divide 810 million square feet by 10,000, we get 81,000 SQF average for every city

What does the above mean?

27 million racks mean that each of the 10,000 cities around the world would need to add approximately 2700 racks of IT gear, just for Edge. “Just for Edge” means, I’m not including organic on-going growth in more traditional on premises, hosted or cloud-based IT workloads and applications. Also, 27 million racks are only 5% of what Michael Dell believes we’ll need. Even if Mr. Dell’s estimate is 100% higher than actual? That would mean we must multiply my estimates by 10X?

The numbers make clear

To me scaling out the edge will require a rethinking of what a server needs to be, of how data is managed, how the equipment is protected and how we provide support. This is the next scale dilemma. We either solve this or Edge doesn’t happen. Think about Google or Amazon. If they were still attempting to build out new servers the same way they were in 2002, they would have gone broke or a Google search would cost you $5 a crack and the company would be much, much smaller.

OK, what do we do then?

We need to rethink our assumptions about what a data center or a server is and what it “needs” to operate. The majority of industry folks whom I speak with about Edge, talk about deployments in terms that sound very familiar to traditional modern infrastructure thinking. They are considering the need for better automation and security etc., but it generally stops at “It should install quickly and update easily” and then they talk about more traditional requirements and usage models. To those reading this who aren’t familiar with what it takes to operate a server or it’s big brother, a data center here’s a quick primer of assumptions:

A server generally has:

• A case
• Fans
• CPU
• Motherboard
• Memory (RAM)
• Storage (disk drive)
• Bus
• Network interface
• Display panel
• Management software
• Security and environment monitoring
• Batteries
• Power Supply
• Etc.

A data center is much the same only a little bigger

• A building (case)
• Air Conditioning (Fans)
• Servers (CPU)
• Storage arrays (Disk drive)
• Network (internal and external with external being your connection to the world)
• Management tools like Data Center Infrastructure Management (DCIM) (Management Software)
• Network Operations Center (Display)
• Security & Environment Monitoring
• UPS (batteries)
• Generators/Transformers/PDUs (Power Supply)
• Etc

Some of the things in each of the above list are there because, “that’s the way servers and data centers have always been built”. To some degree industry leaders have broken the “always been built that way” paradigm, but where they’ve broken it it’s been in support of operating at massive, centralized scale or hyperscale. Edge requires a different perspective of scale and distribution. Without new perspectives and corresponding assumption busting, Edge just isn’t feasible, any more than building Google today using server and data center deployment strategies of 2002 would be feasible. We can easily appreciate the scale of “5X” public cloud from a pure numbers perspective. However, the way Edge is likely to evolve means the approach to building that kind of scale will be very, very different. 10,000 data centers sounds like a huge number, but the reality of data centers for edge is that it’s more likely to be 200,000 small to medium sized data centers, 300,000 telco central offices and 20 million POPs around the world.

Considerations for enabling the growth and successful exploitation of Edge

We shouldn’t think about edge deployments from the perspective of “what do I need to make this first site or even handful of sites work”. Scale has a way of surprising us and you never really appreciate that until you get there. Working on 1, 5, even 20 sites likely won’t drive that appreciation for scale either. There is benefit with faster time-to-value, so each decision of what to build and how to deploy should be considered against that notion. However, your eye must always be on the future:

I.e., How do I -

• replace hardware?
• add more hardware?
• ensure that my patching (firmware and up) can be done the same day as a Zero Day is announced?
• scale infrastructure versus people? It can’t be comparable linear lines.
• build to maximize the value of each piece of equipment? determine the best KPIs to use?
• build the correct incentives with staff?
• evaluate location selection and ownership?
• minimize the risk of painting myself into a corner vs consumer/developer trends?
• deploy with speed?
• Etc., etc..

Edge isn’t for the weak of mind and spirit

Edge is the new frontier of IT and identifying the best way to exploit its opportunities will be what determines the winners and losers over the next five years. Whether you’re an enterprise looking to offer improved or new services or a supplier hoping to grow by enabling this market, don’t be afraid to think bigger than what you can accomplish on your own and don’t be afraid to constantly challenge assumptions.

Details/References:

*. It’s not clear in Michael Dell’s estimate whether he’s saying 100X of what public cloud is today or at some point in the future. I decided to use todays numbers because they are fantastic enough.

**. Rack density will play a huge part in the actual number of racks. In my estimate I’ve used mid density of 10kW per rack.

***. 10 million racks for storage and networking might be light by as much as 50%. Storage is commonly taking more rack space than servers these days and data is the currency at the Edge.

****. https://nextcity.org/daily/entry/there-are-10000-cities-on-planet-earth-half-didnt-exist-40-years-ago