Writing

Cost per million tokens is not one number

A working thesis on what actually determines inference cost.

Inference gets specified in tokens and latency and paid for in dollars, watts and rack units. This is the argument behind InferSpend: what varies, what a model would have to capture, and what would show the argument to be wrong.

Storage starts with how the application wants its data

Capacity is one of the last questions, not the first.

Block, file and object are three different contracts between an application and its data. Which one an application expects settles most of the platform choice, and it is the thing most often assumed rather than checked.

Why the cheaper server can cost more

The price is the deposit. Licensing, power and support are the bill.

Processor choice is usually settled on the quoted price difference, months before anyone works out what the core count does to software licensing. Here is the mental model I use instead, and the cases where it stops being true.