8.3 FM (Foundational Model)-as-a-Service & Shared Infrastructure

Introduction:

Hosted foundation-model APIs run on shared, highly-utilised GPU pools, spreading the embodied and operational carbon cost of that hardware across many customers and workloads. Dedicated or reserved GPU capacity provisioned for a single product, by contrast, tends to sit under-utilised most of the time, industry reporting consistently puts average cloud workload utilisation below 20% which means a far higher carbon cost per useful inference. Default to shared FM-as-a-service APIs unless you have a specific, sustained-volume reason, cost at scale, latency, data residency; to justify running dedicated capacity instead.