The hardware behind the diagrams
Most of my week is spent in catalogs, configuration rules and spreadsheets. Underneath all of that is very physical hardware, and spending time with the systems themselves makes the product decisions much less abstract.
Where the memory sits, what an accelerator does to the chassis around it, how storage expands, and the point where power and cooling stop being a line item and start being the limit. I took these at an HPE hardware showcase, where released servers, drives and accelerators were opened up so you could see inside.
Everything shown is released hardware. Part numbers, serial numbers, bench annotations and labels have been blurred.
Server
What you see as one server is really a collection of constraints: processor sockets, memory channels, drive bays, expansion lanes, power and cooling. Almost every configuration argument I have is about how those compete.
A single-processor server with the lid off. The processor sits under the large heatsink, with memory slots on either side, a row of fans across the middle and expansion risers at the back.
Memory
Memory decisions are about more than capacity. How a capacity is reached, with fewer large modules or more small ones, changes speed, cost and how much room is left to grow.
A two-processor server. Each processor has its own banks of memory slots on either side of it, which is why the number of slots follows the number of processors rather than the number of cores.
Accelerators
An accelerator does not just occupy slots. It pulls power, cooling and often the network into the same decision, and the server ends up shaped around it.
A GPU-dense server. Six double-width GPUs fill one half of the chassis, beside two processors and their memory in the other.
Drives
Drive format sounds like a small choice until it changes density, chassis layout and which configurations can exist at all. The industry move from 2.5-inch drives to the smaller EDSFF formats is the current example: a different drive needs a different cage, so the drive choice becomes a chassis choice.
NVMe solid-state drives in hot-plug carriers. The drive, the carrier and the bay it slides into all have to match, which is why a format change is never only about the drive.
Storage enclosure
Shared storage is where the access model becomes physical. Capacity grows by adding carriers and enclosures, and that expansion path is part of the product decision rather than an afterthought at install time.
Three drive carriers part-way out of an enclosure. Each drive rides in a carrier that mates with a specific bay, and the enclosure behind them is what turns a stack of drives into something a server can be told to use.
Cooling and power
At rack scale, power and cooling become part of the product decision. A denser configuration only helps if the rack can power it and carry the heat away, and the room usually decides that before the equipment does.
A removable fan module with four large fans and eight small ones.