The uncomfortable fact about an AI product is that success can make the numbers worse. A subscriber who discovers long documents and agent loops still pays $30 a month, and can now cost $90 a month to serve.
The reason is mechanical. Every request carries a real compute cost, so the more people use the product (and the more each of them leans on long context, multi-step reasoning and large outputs), the faster the cost line climbs alongside the revenue line. Traditional SaaS could ignore marginal cost because it rounded to zero. An AI product cannot, and a pricing plan that assumes it can will look healthy in the demo and bleed at scale.
Financial scalability is therefore decided in the architecture, well before the first finance report: caching, prompt size, output caps, routing and the pricing unit all get chosen before the first heavy user arrives.
The gap between growth and profit is usually a distribution problem. A small tail of power users runs long-context or agentic workflows that cost many times the average, while a flat subscription lets them do it for the same price a light user pays.
Add a generous free tier served by the same GPUs, and onboarding more users can move gross margin the wrong way. Closing that gap means getting product, engineering, finance and go-to-market to look at one cost-per-task number rather than four disconnected dashboards.
A small tail of heavy users sets the compute bill
AI products add several cost layers that have to be tracked separately, and inference is the one everything else bends around. Every interaction generates cost that scales with prompt length, output length, model family, retrieval calls and multi-step reasoning. There is no near-zero marginal cost to hide behind.
The numbers worth modeling are cost per request, per user and per workflow, plus the best- and worst-case envelopes, because the worst case is exactly what the power-user tail will find.
Here is how the tail works, with illustrative numbers. A $30-a-month plan has 1,000 subscribers. Light users, 950 of them, cost $3 a month each to serve: $2,850. The other 50 run long documents and agent loops and cost $90 each: $4,500. Revenue is $30,000, compute is $7,350, and compute is a comfortable 24.5% of revenue.
Now the product succeeds the way you hoped, and the heavy share doubles to 10%. Compute becomes $2,700 + $9,000 = $11,700, or 39% of the same revenue. Nobody churned and nobody downgraded. Margin fell 14.5 points because the best customers used the product more.
Underneath inference sits infrastructure you pay for whether or not anyone logs in: storage and search indexes for your documents, GPU capacity, logging and monitoring. It grows with data volume, with the number of users at once and with how fast answers must arrive.
Then comes the cost of keeping a model current: spotting when it gets worse, retraining it, re-running tests and watching for safety problems. It is easy to leave out of the plan. Yet you pay to build once and this cost keeps coming, so over time it often decides the total.
Finally there is support load, which rises precisely as the product succeeds. When outputs fail or workflows break, humans absorb the difference, so support belongs inside unit economics rather than in a separate line.
Bringing the marginal cost down, in the right order
Optimizing inference is the fastest path to a scalable margin. The order matters more than any single lever, and the rule for ordering is simple: start with the changes that cut cost without changing the answer, and move to the ones that change the answer only after you can measure quality.
Caching comes first. Provider pricing now rewards repetition directly: the major APIs bill cached input tokens at a steep discount to the normal input price, sometimes with a small premium on the first write.
For a product that sends the same long system prompt or document on every call, restructuring prompts so the stable part comes first and stays byte-identical can cut input cost more than switching to a smaller model.
A 6,000-token system prompt plus 500 tokens of user input, at a list price of $3 per million input tokens, costs about $0.0195 per call. If the cached prefix is billed at a tenth of the input price, the same call costs 600 + 500 = 1,100 token-equivalents, about $0.0033. At a million calls a month, that is $19,500 against $3,300, with byte-for-byte the same answers.
Prompt and output discipline come next, for the same reason: they do not change which model answers. Shorter prompts, less padding, answers in a fixed format and tighter system instructions cut cost without changing quality, which is why prompt optimization is the closest thing AI economics has to growth hacking.
Output is the mirror image of the prompt, and on many price lists output tokens cost more than input tokens, so response length, summary tiers and answer-depth levels are economic guardrails that PMs define and engineers enforce.
Routing is the biggest lever, and it comes third. Send each request to the cheapest model that still answers well enough: try a small model first and pass the request up only when the answer looks unsure. This can cut inference cost by 50 to 90 percent. But routing changes the answer a user gets, so it is only safe once you have an evaluation set that tells you, per request type, where the small model stops being good enough. Without one, you are trading margin for quality you cannot see.
Routing also has a failure mode worth pricing out. Suppose a small model costs $0.002 per request, a mid-tier model $0.006 and a large one $0.03. A cascade sends everything to the small model first. If 85% of requests stop there and 15% fail and retry on the large model, the average is $0.002 + 0.15 × $0.03 = $0.0065. That is more than sending everything to the mid-tier model, and the retried requests are slower too.
Tighten the threshold so only 8% escalate and the cascade drops to $0.0044. Watch the escalation rate and the cost of a wrong route; the headline share of traffic that the cheap tier handles says very little on its own.
Whatever can wait should. Batch endpoints charge less for jobs that can accept a slow turnaround, which suits bulk analysis, evaluation runs and content generation. Moving heavy tasks off the user-critical path keeps the interactive experience fast without paying interactive rates for work nobody is watching.
Searching your own documents before answering has its own bill: indexing the text, searching it and ranking the results. Caching the index and pulling fewer documents per question trims it.
Pricing that does not punish your heaviest users
Price has to reflect both cost-to-serve and value delivered, which is where a flat subscription eventually breaks. It assumes a stable cost per user and predictable usage, and AI violates both, as the 1,000-subscriber example showed.
Usage-based pricing ties revenue to consumption, so revenue tracks cost and scales with value. It still does not guarantee a positive margin, and the extra complexity adds friction that can slow adoption.
The arrangement that fits most AI products is a hybrid: a subscription for baseline value plus usage credits beyond it. Take the same $30 plan and include $15 of compute. A heavy user who consumes $90 pays for the $75 above the allowance at 1.5 times cost, $112.50, so that account brings in $142.50 against $90 of compute instead of $30. Light users never see the meter.
The hybrid works provided the credit allowance and overage rate are simulated on real usage curves rather than guessed. Set the allowance too low and light users hit overage, which feels like a penalty; set it too high and you are back to a flat plan.
The most durable version prices the outcome rather than the tokens: tasks completed, hours saved, revenue per AI-driven workflow. It is also the hardest to sell, because it needs a metric both sides trust and can audit.
Billing on hours saved works only when the customer agrees on the baseline; billing on tasks completed works only when "completed" is defined tightly enough that a half-right answer does not count. Where that shared definition is missing, the hybrid is the pragmatic floor, and teams earn their way toward outcome pricing one well-instrumented workflow at a time.
Holding on to margin once you have scale
Profitability shows up at the intersection of usage, retention and cost control, and the metrics have to be read net of compute. Lifetime value only means something after you subtract compute, support and infrastructure, and count churn and expansion. In short: net LTV is revenue LTV minus total variable cost.
Acquisition deserves the same honesty: onboarding complexity, early support spikes and free-tier power usage all raise the real cost of adding a user well above the ad spend.
Payback then governs how fast it is safe to scale. AI products need the classic SaaS comparison of LTV against CAC, but run on net LTV, after compute has taken its share. As rough benchmarks, consumer AI should recover a customer in under six months, prosumer AI in six to nine, and B2B AI SaaS in twelve to eighteen; beyond those windows, growth is borrowing against a return that may not arrive.
Averages lie. Split customers into groups by how they use the product, how complex their tasks are, their industry and which model they need, and you see which groups pay for the others.
Cohorts also change what retention means. In classic SaaS a retained user is pure upside, because serving them costs almost nothing. Here a retained power user who deepens usage every month raises revenue and can lower margin at once, so a cohort that looks loyal on a logo-retention chart can be unprofitable on a net-LTV chart.
Reading retention and cost-to-serve on the same axis stops a growth team from celebrating engagement that the compute bill is paying for. The single number that captures it is cost per successful task: it folds value, model behavior and economic viability into one figure a whole team can act on.
Won't falling model prices fix this anyway?
Token prices for a constant level of capability have dropped steeply since the first public APIs, and every price cut makes the engineering above look a little less urgent. If the trend continues, why spend engineering time on caching and routing when next year's price list will do the work?
For an early product, waiting is often the right call. If inference is under 15% of revenue and growing more slowly than revenue, the margin problem is real but distant, and shipping features that win customers is the better use of the team. Optimizing a product with 200 users is optimizing a rounding error.
At scale the argument breaks down twice. First, the cheaper tokens get spent: each price drop invites longer contexts, more agent steps and features that were too expensive last year, so tokens per task rise while price per token falls. Second, every competitor gets the same price cut on the same day, so falling prices raise the floor for everyone and confer no advantage. The margin edge comes from the architecture that uses fewer tokens per outcome. The decision rule follows: below 15% inference share, watch the trend and ship; above 30%, or rising for two consecutive quarters, the levers above come before the next feature.
Modeling the quarters that do not go to plan
Scenario work is what prevents a scale surprise from becoming a crisis. Before you need them, build the cases that could break the margin: compute-cost inflation, growth faster than forecast, long-context misuse, RAG overload and free-tier abuse. The OWASP Top 10 for LLM Applications 2025 lists "unbounded consumption" (LLM10) as a security risk in its own right, because a user or a script that can send unlimited long requests is both an attack and a bill.
Alongside the scenarios, change one assumption at a time and see which one breaks the model first: model size, context length, answer length, how many documents are pulled per question, and retries after bad answers.
None of this survives as a one-off spreadsheet. It becomes durable only when a PM personally owns cost modeling, pricing design, experimentation, ML literacy, and forecasting, and when the numbers are shared across engineering, finance, data science and go-to-market rather than living in one team.
Concretely, that means guardrails set in advance: a maximum cost per request, a free-tier ceiling, per-account rate limits, fallback thresholds and acceptable payback windows, so a bad quarter cannot compound into a bad year. And go-to-market spend should scale only once net LTV is stable, payback is predictable and cost per task is trending down.
Fix caching and output caps before you touch price
Of the levers in this guide, the first to pay back are instrumenting cost per successful task, restructuring prompts for caching and putting output caps in place. None of them changes what the user sees, and together they usually recover more margin, faster, than any pricing change. Routing comes next, once an evaluation set can show where the small model stops being good enough.
Then let the cost-per-task number pick the next step. If it falls while usage grows, pricing can wait, and the hybrid credit allowance can later be sized from real consumption data instead of guesses. If it rises with usage, the power-user tail is outrunning your architecture, and that is the moment to revisit the escalation threshold and the free-tier ceiling, before any price change reaches customers.