AWS Series: Hyperscale Data Centers, Physical Moats, & Server Lifetime

Content Published Last Week

1. AWS: Running a Hyperscale Data Center

2. HCA Healthcare: US Hospital Operations

3. Netflix Engineering & AWS Relationship

4. Next PLC: Sourcing Capabilities & UK Apparel Competition

5. Webflow: Enterprise Customer Angle

6. Greatview, Tetra Pak & Aseptic Packaging

AWS Hyperscale Data Centers, Physical Moats, & Server Lifetime

Over the last month, we’ve been interviewing former AWS executives to answer a specific question: in the long run, are AWS’ compute and storage services commoditised?

We’ve covered a range of angles, including:

1. Why GCP lags behind AWS / Azure with a Former GCP Director

2. Kubernetes and the risk of porting workloads between clouds

3. AWS Graviton and hardware-software integration as a competitive advantage

Last week, we published an interview with a Former AWS executive who was responsible for managing data centers that contributed ~33% of AWS’ revenue. The interview explores how to run a hyper scale data center, capacity planning, and server lifetime.

In this analysis, we explore a few interesting points from the interview that address the question of hyperscaler commoditization.

Firstly, the scale of an AWS data center is incredible: each center is ~20,000 square meters, takes 3 years to build, and costs ~$1bn to prepare without the servers:

it costs about $1 billion to build out a hyperscale data center just to get it ready to start taking in your first rack. Our hyperscale data centers were about 30 megawatts, and we had about 700 racks per hyperscale data center. Each position was around 25 to 30 kVA or kilowatts, about 20,000 square meters each. - Former Director at AWS

Then you need to add 700 racks, pay staff, and power the servers. This amounts to ~$1bn in opex per year to run a hyperscale center:

you put about 700 racks in one of these hyperscale data centers. Each rack is $400,000 to $500,000. It is not cheap. When we have semis come up to roll out 12 racks to be installed, you're looking at $4.8 million worth of racks in one semi. That's an enormous cost, your compute. However, we depreciate that over five years…You're spending about $1 million in salary and benefits to run an entire region. You also have your facilities, probably your biggest cost, because that's where the power comes from. The more servers you put in, the more power you consume and the higher your cost. Having a full data center will cost a billion dollars a year because of your racks. - Former Director at AWS

Realistically, how many companies on earth can afford such a cost for one data center?

The colocation players like Equinix and Digital Realty barely spend ~$3bn in capex per year. We estimate AMZN will spend ~$35bn on AWS infrastructure in 2022.

Consensus suggests the hyperscale market is an oligopoly of AWS, Azure, and GCP. Possibly even a duopoly. No other company can afford to spend billions building out data centers, and companies like Dropbox who run their own centers are increasingly rare.

Maybe the physical assets are the advantage, not the software services on top?

Could the hundreds of AWS microservices be a classic ‘commoditize your complement’ strategy? By bringing the cost of software microservices down to zero, AMZN can capture economic profit via EC2 and S3.

This also highlights why it's difficult for open-source companies like Confluent or Hashicorp to monetise its technologies; AWS effectively commoditizes Apache Kafka via its MSK managed service, for example. The value Confluent needs to create on top of its Kafka open source tier and Amazon MSK service is forever increasing.

Maybe the durable moat of AWS is the fact it owns the physical rails of cloud computing and less so about capturing margin from the PaaS layer on top of compute and storage?

But is it really as simple as just owning the assets?

One bear case of AWS hinges on IaaS becoming commoditized and independent best-of-breed players like Snowflake displacing AWS-native Redshift to capture the economic profit on top. This will leave AWS with a lower-margin, low ROIC compute and storage layer with third-party ISV's capturing more of the economic profit in the industry.

close

Zoom InZoom OutReset

The telco industry in the late 90’s could be one analogy for the future of cloud computing.

In 1997, a former Bell Labs engineer wrote an interesting memo called ‘The Stupid Network’. It’s worth a read in its entirety. In short, he explains how the telephone companies built the infrastructure to offer circuit-switched voice communication that was soon disintermediated by the internet. A once ‘Intelligent Network’ offering premium voice telco services and controlled by the Bell System legacy was to become a ‘Stupid Network’ merely tranporting data.

The telcos were left in the tricky position of disrupting itself to build higher-value services on top of its now ‘stupid network’. This proved difficult. Instead, the industry resorted to using The 1996 Telecom Act and 1997 WTO Telecom Agreement to limit VOIP and protect its oligopolistic positioning.

In hindsight, adding broadband was profitable but it was third-party providers like Netflix that captured the higher-value services on top.

Could AWS follow a similar path as the telephone companies? Could AWS fail to capture all of the high-value PaaS services and be left with a compute and storage ‘stupid network’?

A few reasons come to mind why this may not be the case:

1. Switching costs are higher in cloud computing than telco: configuring workloads to AWS or Azure is stickier than for phone or broadband.

2. Circuit-switched voice communication was disintermediated by the internet; there are no signs of cloud computing being disintermediated by a new technology.

3. Amazon’s customer obsession: AWS is more focused on improving the service and price for customers more than the telcos in the 90’s; for eg, proven AWS microservices, Graviton, ML chips, and even its own racks and servers.

4. AMZN microservices success: Kinesis, MSK, Lambda, EKS, etc all prove AMZN can add value on the PaaS layer.

But even if AMZN did lose the PaaS services to best of breed ISV’s, maybe EC2 and S3 will be highly profitable as an oligopoly anyway. Digital Ocean earns 60%+ gross margins selling compute at a much smaller scale.

It's also hard to grasp the change in unit economics when Redshift loses share to SNOW. If we assume Redshift operates at 80%+ gross margins, and EC2 / S3 at 65%, AMZN would lose higher gross margin revenue to SNOW. However, if it drives higher utilisation of EC2, it may be lower margin but higher-quality, more durable revenue.

Maybe it's the 'stupid network' of EC2 and S3 and AMZN's hyperscale fixed assets that are the durable advantage, not the 'differentiated' microservices on top?

Bears would highlight a risk that the storage or compute layer could be competed away from AWS by new technologies. Maybe Intel and AMD’s new processors could reduce ARM’s advantage? Who knows what will happen as AI evolves. Could we also see disruption to store primitives that impacts S3?

These are all ‘unknown unknown’ risks for AWS. And challenges that likely scare off non-techy generalists.

However, generalists can take comfort in that a crucial variable which they can analyse is AMZN’s culture; as long as AWS maintains its Day 1 and customer obsession culture, it’s hard to argue it won’t be first to innovate.

AMZN’s ability to innovate combined with the physical rails of both general commerce and cloud computing is the potential moat.

Another interesting insight from the interview are details of the useful life of servers. Since January 2020, AWS server lifetime has increased from three to five years and networking equipment from 5 years to 6 years. This has been a tailwind to AWS margins.

close

Zoom InZoom OutReset

This is mainly because the server racks are more reliable and efficient with more parts available to repair:

If you looked at racks built from 2010 to 2015, you wouldn't get the life out of them because they just weren't as reliable. We're making racks a lot more reliable with standard components. Parts are becoming more interchangeable with racks; for example, a fan or a memory module will be the same across different types of racks. We're getting to where you are not as specific on your replacement parts. For example, a motherboard will be compatible against three different rack types instead of one, or memory modules will be the same for 80%. It comes down to the component level. - Former Director at AWS

The useful server life could also be further extended to six years:

I managed eight-year-old racks in one of the regional PDX regions. It was very touchy. It was trying to run it at a steady state, and we'd have a special parts area. I can't imagine that we would let anything go past eight. You're taking way too many availability risks. We would raise an alert to the application group; once it gets to five years, you start to raise red flags. It would have to be such a custom app that had to run on a certain technical rack platform to stay past that, and there’s not much of that out there. I don’t see it ever going over eight, and I don't see it much going over six. - Former Director at AWS

If we assume compute and networking assets are ~60% of AWS PPE, and land is 25%, at 6 and 7 year life of servers and networking equipment, the normalised depreciation rate would add ~450bps to 2021 EBIT margin.

close

Zoom InZoom OutReset

AMZN also only runs ~60% through its own hyperscale centers, the rest through colocation facilities. As more compute moves to AWS facilities, and increasingly running on Graviton instances, efficiency and margins will increase. Greater scale and efficiency through AWS' own centers plus useful life extensions suggests AWS is underearning at 30% EBIT margin today. The normalised underlying EBIT margin of AWS could be closer to 40% than 30%.

However, once again, this doesn’t consider the margin profile of the various revenue lines included in AWS (compute, storage, databases, managed services, etc).

If S3 and EC2 is indeed a commoditized ‘stupid network’, and many more Snowflake-equivalent ISV’s take share from higher-margin AWS-native PaaS services, it may well be over-earning. Disconfirming this hypothesis and understanding the nature of AWS’ durable competitive advantage is a deepening part of our work at In Practise.

Do let us know if you have a better analogy to the future of cloud computing and please reach out if you'd like us to source AWS executives for you!