History has a pattern of mass-market products eventually decimating low-volume high-end products. The cost savings from economies of scale overpower the benefits of specialized solutions. This has happened to word processing (e.g. Wang Labs), mainframes, high-end CPUs (SGI versus Intel), post production systems (ADSK, ADBE, AVID, etc.), film/video cameras, and many other industries.
Back in Dec 2012, I wrote a post on enterprise storage. Now, I’m starting to be more confident that enterprise storage will indeed become increasingly commoditized.
Mass market versus the high-end
If a combination of hardware and software is designed for a single purpose, it can have characteristics that are far superior to what a general-purpose system can do. For example, the graphics card in a desktop computer is far superior at graphics than a general-purpose CPU. The RAM in a computer is much faster than its storage drives.
Enterprise storage can offer the following benefits over commodity hardware:
- More IOPS.
- High bandwidth.
- For network-attached storage, high network bandwidth.
- Low latency.
- Consistent performance.
- Higher reliability.
Something commodity solutions struggle to do is offering several of these benefits at the same time.
Probably the biggest thing holding back commodity hardware is IOPS. IOPS refers to the number of Input/output Operations Per Second. An enterprise-grade SCSI drive can deliver significantly higher IOPS due to its higher rotational speed and tricks such as Tagged Command Queuing (see Wikipedia for an explanation of TCQ). This gap in performance has mostly stayed constant (with the exception of Western Digital’s Raptor drives, which narrowed the gap somewhat). One reason why IOPS is so important is because many enterprises have massive databases where IOPS is the performance bottleneck.
The SSD paradigm shift
The Solid State Drive (SSD) will become increasingly popular. With traditional rotational hard drives, IOPS performance did not get better with Moore’s Law (computer hardware gets better roughly every 2 years). The IOPS performance of SSD does get better with Moore’s Law. The downside to SSD drives is that the cost per amount of data stored ($/GB) will likely always be several times that of a rotational hard drive. The price gap will likely continue in the future
As time goes on however, the performance gap between rotational hard drives and SSDs will get bigger and bigger. This is driving the shift in the enterprise storage from rotational hard drives to SSDs.
The problem for high-end enterprise storage is that the IOPS performance gap between enterprise and mass-market drives is getting very narrow. Going forward, we will likely enter a world where IOPS will not be the bottleneck. For most applications, the IOPS performance from a consumer drive will be overkill. I think that enterprise users will care more about:
- Consistent performance.
- Ultra-high reliability (e.g. a track record of high reliability).
Not all enterprise users will care about #2 and #3. All of them will care about the overall cost of the system. If the battle comes down to $/GB, mass-market commodity storage will be the obvious winner.
In practice, the maximum latency of a consumer SSD drive can get as high as a second. Benchmarks from StorageReview.com for several mass-market SSDs shows a range from 0.047 seconds to 0.7+ seconds. (Some SSDs get over one second.)
While I don’t understand this too well, I believe that the worst-case effect of very high maximum latency is that a server will lock up for a second. If multiple consumer drives are used in a drive array, then maybe the server will lock up for a few seconds in very rare cases. For most applications, this is not a big deal. The user can wait an extra second every once in a while.
Because of this, enterprise SSDs will only be needed for niche applications. This means lower volumes and a lower market size for enterprise SSDs.
R&D costs for enterprise SSD drives will go up
The trend in technology is that the market will increasingly shift towards flash memory that wears out quickly. As the cells in flash storage get smaller, each cell will trap fewer and fewer electrons. The laws of physics will limit the durability of this form of storage. The storage may only be reliable for a few thousand read write cycles (say 2000-3000) rather than say 20,000 read/write cycles.
In practice, there is a mini-computer on every SSD drive. This computer can apply various tricks to get around the durability problem. Wikipedia has descriptions of various wear leveling techniques. These techniques try to spread out (or reduce) the “wear and tear” across all of the cells on a drive. Enterprise SSD storage solutions will increasingly need to spend more R&D dollars to implement these techniques. This gives mass-market solutions a small advantage because the higher R&D costs are spread over a dramatically higher volume of units.
Pooled infrastructure (public cloud) and Infrastructure-as-a-service
For background on pooled infrastructure, see part 1 of my series of blog posts on the subject.
I expect pooled infrastructure to take a small amount of market share from enterprise storage.
- In some cases, the pooled infrastructure provider is offering something very close to what the customer wants. Granted, this isn’t true in most case because the pooled infrastructure providers cannot offer a wide range of specialized hardware.
- In other cases, the value-added software is so compelling that the customer is willing to put up with hardware that is not ideal for their needs.
In the future, pooled infrastructure should offer more flexibility and a wider range of hardware choices.
Infrastructure that can be rented out may help drive the adoption of commodity hardware because consumers can try before they buy. They can determine the suitability of a particular hardware setup at a very low cost. With enterprise storage, it can be difficult for small customers to obtain demo units. There is also less price transparency in the enterprise market. Salespeople may also charge certain customers higher prices than other customers because they can get away with it. Being able to rent out commoditized storage will allow customers to get away from the shenanigans of enterprise markets.
Politics and biases
With high-end markets, there is a very limited number of high-end systems available worldwide. Gaining expertise with such hardware generally requires the person to work for companies that own high-end systems. This means that expertise is difficult to gain and there aren’t many people in the world with such specialized expertise. In turn, this means that people who work in specialized high-end markets can demand high salaries. Because of their high pay versus doing the same work in low-end markets, they generally have a vested interest in perpetuating the status quo. Salespeople also have an interest in spreading FUD (fear, uncertainty, doubt) about commodity mass-market solutions because they have an incentive to talk trash about the competition.
To be fair, experts in high-end storage are often right about the benefits of high-end storage solutions over commodity solutions. They may be correct about the present advantages of high-end solutions over commodity solutions. However, they may not be that forthcoming about where the future is headed. I’m not trying to say that most experts are trying to be charlatans. I’m just saying that you need to be careful about the biases and incentives that human beings have.
Unfortunately for investors, high-end markets are difficult to understand because there isn’t a lot of information out there because the size of these markets is so small.
Some people (myself included) have a thing for very expensive high-end cutting-edge systems. But just because these things are sexy does not mean that they will make money for shareholders.
Examples of customers choosing commodity SSD
Stack Overflow/Exchange has a 2012 blog post on their decision to use mass-market SSDs in their database servers. They noted that their vendor Dell does not approve the use of the drives that they went with. Dell wanted to charge insane prices for “approved” drives. Over time, I think that the enterprise shenanigans will go away (which may mean lower profits for Dell, HP, and other enterprise server companies). Over time, competition with companies like Super Micro will force the vendors to change their ways.
The excellent Brent Ozar blog has an August 2013 post with lots of details on a client server setup with commodity SSDs. The blog notes that 16 Samsung 840 Pro 512GB SSDs cost $8k while SQL Server Enterprise Edition licensing for 8 cores cost $56k. It also explains why they did not go with a solution from FusionIO.
Both Amazon and Google use mass-market SSDs in their data centers. Google designs its own SSDs. Due to Google’s sheer size, the company can essentially create its own economies of scale when it designs its own hardware.
The past and the present
Since my Dec 2012 post, four of the stocks mentioned (NTAP, STEC, FIO, EMC) underperformed the S&P 500 index. VMEM did not IPO yet. During this timeframe, operating income at these companies stayed flat (NTAP, EMC) or declined (STEC, FIO, VMEM). It vaguely looks like the trend towards commoditization is underway.
However, Dot Hill Systems (HILL) had an incredible run in its stock price. Unfortunately I started shorting Dot Hill around December 2013.
Thoughts on specific stocks
EMC – I don’t really understand all of their business lines. The company generated a lot of growth under Joseph Tucci’s leadership. It could be that his successor is not as skilled as a CEO.
NTAP – Netapp was formerly a scrappy newcomer in the enterprise storage market. It was a pioneer in using commodity (rotational) hard drives in its storage arrays. While I don’t understand enterprise storage that well, I believe that there are some low-cost low-margin commodity solutions that will chip away at NetApp’s profits. NetApp will face less disruption than the companies with lots of exposure to low-volume enterprise flash/SSD.
VMEM – I think that their market will be badly disrupted by commodity solutions. Their year-over-year drop in revenues may be a sign of things to come.
HILL – Unfortunately I have been shorting this stock. While the company has a track record of destroying shareholder capital, its operations have turned around and now the company is profitable. What attracted me to this short position is because I ran across stuff like this:
But to management’s credit, this company is currently making money. I really don’t know what happened. I think my process was correct but I got unlucky.
*Disclosure: No positions other than HILL. I have shorted STEC, FIO, and VMEM in the past.
Storage Search.com – Both Storage Mojo and Storage Search don’t necessarily agree with my viewpoints. I think that the authors there are much more bullish on enterprise SSDs and enterprise flash than I am.
Storage Review.com – More of a focus on consumer and low-end server storage.