Memory prices are the quiet driver of your inference bill

High-bandwidth memory is the component most people outside hardware have never heard of and most inference economics depend on. Contract pricing for the current generation has repriced upward as supply is allocated to accelerator manufacturers on multi-quarter agreements, and the spot market has followed.

The transmission into enterprise pricing is indirect and lagged, typically around two quarters. It shows up not as a headline price rise, which providers avoid for competitive reasons, but as firmer renewal quotes, reduced discounting on committed volumes, and quieter changes to what is included in a tier. Watch the inclusions, not the headline rate.

The offsetting force is real and should not be understated. Model efficiency has improved faster than hardware has become expensive in every year of this cycle, and the cost of a given capability has fallen consistently. The finance-relevant question is which curve dominates for your workload, and the answer usually depends on whether you are routing everything to a frontier model out of habit.

That habit is the most common avoidable cost we see. Document extraction, classification and summarisation — the bulk of finance workloads — run acceptably on substantially smaller models at a fraction of the unit cost. Very few organisations have ever measured the quality difference on their own data. It is a half-day exercise with a measurable payback.