RFD 12 Host CPU Evaluation

While we laud alternative instruction set architectures, server-side computing remains dominated by x86. This dominance stems from multiple historical factors, not least that it triumphed in personal computing nearly four decades ago. Now, there is reason to believe that this dominance is more fragile than ever: with ARM’s Ares core (the basis for Amazon’s Graviton 2) there is at last a non-x86 core that can meaningfully compete with (Intel) x86 on the server — but it is not clear to what degree this will get adoption in the long tail of software development. (For example, will software developers who are increasingly returning to a binary model embrace cross compilation and two different instruction set targets?) In our target market of the enterprise space, it seems likely that any adoption of hybrid instruction sets will be slower still; Oxide will have x86-based host CPUs for the foreseeable future.

The most obvious criteria for the host CPU is performance. While quantifiable, performance is also incredibly nuanced: with so many different ways of describing the performance of a system, it’s easy to measure (or emphasize) an aspect of the system that is practically irrelevant. Ultimately, the performance that matters is the performance of a customer workload — which may or may not be what vendors measure or design around! That said, there remain obvious axes of microprocessor performance: clock rate; number of cores; cache size; cache architecture. (In terms of quantifying these, SPECrate is generally reasonable.) These axes are directly affected by density/process: barring architectural differences, one cannot expect (say) a 14nm part to ever be broadly competitive with a 7nm one. There are more architectural aspects to performance as well: on-die acceleration (e.g., SIMD extensions), compiler optimization, cache coherence architecture, etc. And then there are system aspects of performance that can be implied by processor choice: bus speed, memory speed, memory sizing etc. Performance certainly isn’t our only criteria, and its importance must be kept in perspective, but it also must be viewed as central: Oxide racks will compete with extant infrastructure; the higher their performance (however measured, observed or perceived), the more those racks will enjoy a tail wind in our customer base.

The second obvious criteria for any microprocessor is price. Historically, the CPU is the most expensive single component in the BOM, but the price can vary widely within a single family. For example, when it launched, Intel’s Cascade Lake ranged in price from $1600 to $17900 for a single socket — and that only includes the "Gold" and "Platinum" classes of Xeon. At the quantities that we are seeking (on the order of ~96 sockets per rack), these price differences have enormous impact on the economics of the entire rack. In this regard, price almost always needs to be denominated — e.g., as $/core or $/SPECrate.

Critical for us (or anyone looking to achieve any kind of density) is the thermal design point (TDP). This is — more or less — the draw of the part that the needs to be cooled. Historically, dense form factors like the OCP Tioga Pass form factor have allowed TDP per socket of around ~165W; beyond that, a different form factor would need to be considered. Note that 250W is generally the threshold for air cooling; beyond TDP of 250W, a CPU must be water cooled. Moreover, power for the rack must be considered: assuming an Open Rack, we should assume no more than ~15kW at ~32OU — or about 450W per OU. This must be all in (DRAM, NVMe, NIC, etc.), so the CPU power budget must be less than this, although to what degree is to be determined.

Transistor density has many implications for the system, in terms of performance, power and price. Regrettably, for historical reasons, transistor density (which ideally should be expressed in terms of transistors per fixed unit area) is expressed by the industry in terms of feature size — which doesn’t actually correspond to anything. (That is, in a 7nm process — and especially in the forthcoming 5nm processes — there is nothing that is in fact as small as the putative feature size.) This has given rise to absurd process nodes like Intel’s "14nm+++" (yes, three plusses). Making things worse, vendors making claims that are difficult to verify ("our 10nm is equivalent to their 7nm"). Density is critically important (there is unquestionably a density difference between, say, 14nm and 7nm), but it is likely best measured through its manifestation in the artifact: core count, cache sizing, TDP, etc.

Closely related to TDP and transistor density is core density. We are seeking a dense product offering, with more than 1344 cores in a 15 kW rack. Core density should be thought of per U — and if achieving that desired core density necessitates multiple sockets, issues around multiple sockets must be considered (see, e.g. "Cross-connect", below).

Related to transistor density is yield. We have no control over yield and yield often isn’t disclosed, but there are architectural decisions that can materially affect yield. In particular, chiplet designs can result in (much) higher yield because of their (much) smaller size. Similarly, tiled designs can circumvent some of the yield problems that may be present in a large monolithic design.

CPUs do not exist on their own; they exist in a larger system that must support them. CPUs are mated to specific platforms in several key attributes, but especially via I/O (i.e., generation of PCIe) and memory (i.e., generation of DDR SDRAM). This is particularly germane for us because we happen to be targeting a product that is arguably at the tail end of one generation or the leading edge of the next. Specifically, both Intel and AMD have roadmap CPUs (Sapphire Rapids and Genoa, respectively) that are targeting PCIe Gen 5 and DDR5; to build on either of these parts is to build a dependency on both of these technologies — each of which has economic, performance, complexity, and schedule risk implications.

Microprocessors take a long time to develop — especially given dense processes like 7nm. A roadmap can therefore be a concrete way of predicting the future in that one can often known years ahead of time if a particular CPU is going to be competitive or not. Roadmaps should be taken as a best case; schedules certainly can slip. In general, late projects get later, and when there is a history of slip, more can be reasonably expected.

Our strategic relationship with a microprocessor vendor can have an outsized effect: the ability to get the support we need when we need it can often come down to the depth of our relationship. As a startup, it is a challenge for us to be strategically relevant to much larger companies.

Microprocessor security was brought to the fore with Spectre and Meltdown — which presaged a deluge of microprocessor vulnerabilities. There are many aspects to security — some practical (degree of historical vulnerability, history of effective embargoed communication) and some more architectural (role of secure enclaves, secure virtualization, etc.)

Microprocessors require firmware to boot and operate correctly. Firmware considerations include: degree of openness, degree of support, degree of platform specificity, etc. The most important single access for us with respect to firmware is our ability to control and attest our entire stack: we want to minimize our dependency on third-parties (that is, firmware delivered neither by us nor by the component vendor). In particular, nearly all computing systems have a dependency on third-party BIOS vendors such as AMI (formerly American Megatrends, Inc.); we view it as a constraint to avoid this spurious dependency, for myriad reasons.

x86 architectures contain software that runs in a hyperprivileged state: for Intel, this is the Intel Management Engine (ME); for AMD, it is the Platform Security Processor (PSP). Understanding this software — its scope, its firmware, the conditions under which it operates — can be critical for understanding many different aspects of the system (not least its security).

Where multi-socket systems are to be considered, cross-connect architecture (protocol, bandwidth, latency) should be understood. Intel and AMD have different architectures for cache coherence (PCIe Gen 4 vs. UPI in Rome and Cascade Lake, respectively) — which can introduce asymmetries in the system. Specifically, in two socket systems, the I/O path must be understood to know if sockets are asymmetric with respect to I/O.

Both x86 microprocessors extend the ISAs in different ways. Some of these can be significant (e.g., AVX-512) and can have an outsized impact on performance.

The Oxide rack integrates a top-of-rack controller. It is desirable for this controller to have the same CPU architecture as the rest of the rack. Some microprocessors may lend themselves better to this integration than others.

Different microprocessors have different mechanisms for executing the first instruction. We believe that our root-of-trust will be implementable with either Intel or AMD, but the effort involved in each path will surely not be identical.

Features for reliability, availability and serviceability differ in ways that can affect the ability for system software to be resilient to certain kinds of failure.

Microprocessors have microcode that the host can dynamically load. How the microprocessor is able to load microcode can have ramifications for security and availability.

This table compares publicly available CPUs: AMD’s Rome (EPYC) to Intel’s Cascade Lake (CLX):

Intel’s forthcoming Ice Lake will likely have a 6262V equivalent (dubbed 6362V) with better density (32 or 36 cores), but with no better TDP/core than Cascade Lake. In essentially every conceivable way of looking at this data, Rome is superior to Cascade Lake: in price, in performance, in density, in TDP — and in all of these things when viewed in terms of the other.

The following table depicts known vulnerabilities, and the degree that they historically affected Intel CPUs vs. AMD CPUs circa February 2020: