Gross margin: 72% in year one, 45% in year two. That is the 60 USD support copilot in the worked example below, and nobody on its team made a bad decision. The product got better, and better answers meant more context, more steps and more billed calls.
Traditional software pushes marginal cost towards zero. AI keeps it above zero and lets it drift upwards with success: every answer is a billed call, and better answers tend to need more of them. A model that treats inference as a rounding error looks profitable on a slide and loses margin at scale.
That one fact reorders the usual business-plan chapters. Strategy, architecture, pricing, cost, data, and validation stop being separate boxes on a canvas and become a chain. What you sell decides what the system has to do. What the system does decides what each call costs. And you can only set a price once you know that cost.
Teams that start from the price end up re-pricing after every tab of the spreadsheet already depends on the old assumption. What follows walks the chain in the order the money moves.
Charge for the layer a rival cannot copy by Friday
Before any cost can be judged too high, you need to know what the customer is paying for, and in AI the answer comes in three layers. At the bottom is the core job the model does instead of a person: reading a contract, summarising a thread, classifying a ticket. Above it sits amplification, the same workflow done faster or with more capacity per person. At the top is a workflow that was not practical before, such as a semi-autonomous agent that closes a refund end to end.
The demo almost always sells the bottom layer. The durable money comes from the upper two, because raw capability is the easiest thing in the stack to copy: a rival with the same API key can summarise as well as you can by Friday.
A workflow your customer's team has rebuilt itself around is much harder to dislodge.
Naming the layer you charge for also tells you which cost fight you are in. A summariser is priced against tokens and compared, call for call, with the cheapest alternative. A copilot that changes how a team works is priced against the hours it saves, which leaves room for a cost line that drifts.
Say the copilot saves an agent 25 minutes a day over 21 working days. At a loaded cost of 30 USD an hour, that is about 262 USD of time a month. A 60 USD seat captures 23% of the value it creates. Priced as a summariser, the same product would be benchmarked against a raw API bill of roughly 11 USD a month and squeezed towards it.
The moat question follows from the same split. Models alone rarely differentiate; systems do.
What compounds is a data pool that grows with usage, retrieval tuned to one domain, an evaluation harness that lets you ship changes safely, and workflows deep enough that switching means retraining a team.
One test cuts through most pitch decks. Suppose a well-funded competitor copied your prompts and called the same base model tomorrow: what would still be hard for them? If the honest answer is "nothing", the moat exists only on the roadmap, and your margin will be set by whoever runs the same calls cheapest.
Every feature hides a cost line in orchestration
Once you know what you sell, the system that delivers it writes the bill. It has four layers. The data layer (pipelines, feature stores, vector databases, labelling and quality monitoring) holds most of the fixed monthly spend. The model layer sets the lowest possible cost of one call: you choose between a model bought through an API, a version tuned on your own data, or a set-up that looks up your documents before it answers. The orchestration layer is the logic that strings calls together: prompts, multi-step agents, rules for which model gets which request, backups when a call fails, and safety checks. This is where one design decision can multiply the tokens you pay for.
The experience layer is the only part the customer sees.
Read the layers from the top and they tell you what to build. Read them from the bottom and they tell you what it costs. The two answers rarely match.
Take two hypothetical features that score equally well in user interviews. "Summarise this ticket" is one call at about 0.026 USD all-in. "Resolve this ticket" retrieves the account history, drafts a reply, checks it against policy and retries once in five cases: call it six calls, or 0.158 USD per use. At 30 uses per seat a month, the first costs 0.79 USD and the second 4.74 USD. On a 60 USD seat that is the difference between 1.3% and 7.9% of revenue, from one roadmap line.
That is why demand is a poor sort key. Before a feature is scheduled, ask the same five questions of every candidate: how much value it creates, whether the data to support it exists, what one use costs, what happens when it is wrong, and how it behaves under load.
A workable rule: if a feature's expected monthly cost per seat exceeds 5% of the seat price, it ships with its own price lever (a higher tier, a credit cost or a cap) or it waits. Treat the 5% as a setting to tune against your margin target.
The roadmap is the cheapest place to catch this, while the cost is still a line in a plan. The next place it shows up is the pricing page.
The billing unit decides who absorbs usage drift
Pricing starts with a choice of unit, and the unit decides who pays when usage grows. Bill by seat and revenue no longer follows usage. That is easy to sell, but every extra call eats margin. Bill by task or request and gross margin steadies while revenue swings with the customer's internal volume, which moves your forecasting error from the cost side to the revenue side.
Bill by outcome (a resolved ticket, a qualified lead) and you need an attribution layer and a reserve for disputes, but you are selling the thing the buyer values.
The arithmetic makes the trade concrete. A seat at 60 USD with 420 tasks a month is equivalent to 0.143 USD a task. Price the task at 0.18 USD instead and the same customer pays 75.60 USD, 26% more. A customer who runs only 120 tasks pays 21.60 USD.
Per-task pricing protects margin: at the year-one cost of 0.026 USD a task, inference margin is 85%, and at the year-two cost of 0.043 USD it is still 76%. What you lose is a predictable revenue line, and with it some of the cash buffer you can plan around.
So the choice reduces to one question. If you can forecast usage per customer within about 20% and your cost per task is rising, sell tasks or outcomes. If buyers need a fixed budget line (most enterprise procurement does) sell seats, but with a fair-use cap and an overage rate written into the contract from day one.
Credits and tiers are packaging on top of the unit. Credits let one balance cover images, analyses and text, but the customer then has to translate what they pay into what they use. Tiers split customers by which models they get, how much text the model can read at once, how much is automated and which compliance controls are included. Enterprise deals combine a platform fee with usage, reserved computing capacity and a service-level agreement. In that negotiation, who carries the cost risk matters as much as the price.
A year of success can cost 26 points of margin
Whichever unit you pick, inference is the marginal cost that settles gross margin, and a worked pass shows how fast it moves (illustrative figures, list-style API rates).
Year one. A support copilot bills 60 USD per seat for 800 included tasks, but seats average 420 tasks a month. Each task sends 3,500 input tokens and receives 600 output tokens, at 3 USD and 15 USD per million. One call costs 0.0105 + 0.009 = 0.0195 USD. Multiply by 1.35 to fold in retrieval, logging, retries and offline evaluation: 0.0263 USD. Per seat that is 420 × 0.0263 = 11.05 USD of inference. Add 6 USD of hosting and support, and the seat costs 17.05 USD to serve: a gross margin of 71.6%.
Year two. The product now carries more conversation history, adds a self-check step and gets used more. Input grows to 6,200 tokens, output to 900, usage to 620 calls. One call: 0.0186 + 0.0135 = 0.0321 USD, or 0.0433 USD after the 1.35 factor. Per seat: 620 × 0.0433 = 26.85 USD, plus 6 USD, against the same 60 USD. Gross margin: 45.3%.
Two levers pull it back, and they behave differently. The first is prompt caching: the provider stores the part of the request that never changes and charges much less to reuse it. Much of those 6,200 input tokens is exactly that kind of fixed opening (system prompt, tool definitions, policy documents) that repeats on every call. Providers that support caching bill a cache read at a small fraction of the normal input rate and add a premium on the first write; assume reads at a tenth of the base rate here. If 3,800 of the 6,200 tokens hit the cache, input cost falls from 0.0186 to 0.0083 USD per call; with output and the 1.35 factor the call costs 0.0295 USD, the seat 18.28 USD of inference, and margin recovers to about 60%.
The second lever is batch processing: a cheaper rate for calls that can wait until the next day. It suits nightly test runs and bulk summaries, but never the live chat window.
Both levers have failure modes you can see in the logs. Caching depends on a stable prefix: change the system prompt weekly or inject fresh retrieval results at the top of every call, and the hit rate drops. The symptom is a cost per call that creeps back up while usage stays flat. Batching fails when someone routes an urgent job through it to save money and support tickets start to age.
Upstream price cuts are the third lever and the least controllable. Token prices for a given level of capability have fallen quickly since the first public APIs, but they fall in steps that you do not schedule. A 40% cut in token prices would bring the year-two inference bill to 16.11 USD and margin to 63.2%.
Keep that line separate from usage drift. Usage climbs month by month as the product improves; the price per token drops in steps when you switch models, and every switch needs a re-run of evaluations and prompt fixes that can take a quarter. Fold the two into one "net cost falls 20% a year" constant and you lose the question that matters most: what happens if the price steps stop for half a year.
Inference is the cost that moves fastest, but others recur too. Data upkeep (cleaning, annotation, drift monitoring, retraining) often exceeds the one-time cost of training over a product's life. Unreliability sends its own bill, and those hours belong in unit economics alongside inference.
Say 8% of answers fall below the confidence threshold and go to a reviewer who spends 90 seconds on each at 30 USD an hour. That is 33.6 reviews and 25.20 USD per seat a month, almost as much as the year-two inference bill. Cutting the review rate to 3% brings it down to 9.45 USD.
Sending easy calls to smaller models, training a small model on a big model's answers (distillation) and shortening prompts can win back points too. Give each one an owner and a date like any feature. Left as a someday clean-up, they never ship.
Data is the cost line that can turn into a moat
The data layer appeared at the bottom of the stack as a cost. It is also the one line that can pass the copy test from the first section. The most defensible sources accrue from operation rather than purchase: workflow exhaust, user corrections, integration data. A rival can buy the same public corpus. Nobody can buy two years of your customers' corrections.
Volume is not the goal. A small, well-labelled corpus trains and evaluates better than a large noisy one, and you can measure its quality through freshness, labelling consistency and an error taxonomy. Each shows up in model reliability, and therefore in the review rate above. That is the direct line from data quality to gross margin: every point of review rate you remove is worth about 3.15 USD per seat a month in the example (420 tasks × 1% × 0.75 USD).
Governance belongs in the same budget. Consent tracking, lineage, redaction, PII-safe embedding flows and audit logs separate a product a regulated buyer can adopt from one their security team blocks at the first review. For a bank or an insurer, that list is a purchase criterion, and missing it costs the deal outright.
Prove four things before you scale
Everything so far is still assumption, and each assumption fails in its own way. An AI idea has to clear four doubts: that people want it, that it can be built, that the economics hold, and that the model performs. The model is first tested before launch on a fixed set of questions with known good answers, with explicit checks for made-up facts and unsafe output. Real usage is judged on task success, retention and willingness to pay, and on whether the workflow is adopted rather than tried once.
Pricing gets tests of its own: the tier structure, how demand changes with price, credit packs and the effect of caps. The economics are then stress-tested with cost simulations, usage load models and checks on how far margin moves when each input changes. This is where the worked example earns its place, because unit economics that work at a hundred seats can invert at ten thousand once usage drifts as it did above.
A founder watching token prices fall may fairly ask why anyone should sweat a 26-point swing that the market could erase by next spring.
Sometimes the market does erase it. If your product's quality is already saturated at an older capability tier (classification, extraction, routing), you can ride the curve and spend your effort elsewhere. The more common case runs the other way. The product moves up to newer models as it improves, usage per seat keeps rising, and each price step arrives a quarter late because switching models means re-running evaluations. Nobody can promise that the pace of price cuts will hold, and a business plan that only works if it does is a bet on the curve.
Measure the outcome with one metric and one guardrail: gross margin per seat by monthly cohort, guarded by task success rate. Margin rising while task success falls means you are saving money by making the product worse.
Build the cost tab one driver per row
A flat "cost of revenue: 25%" line hides exactly the trend this article is about. The tab that survives year two gives every cost layer its own driver, so that when margin moves you can see which row moved it:
- Inference: input and output tokens per call × calls per seat × current rates × an overhead factor for retrieval, retries and evaluation (1.35 in the example).
- Hosting and support: a fixed amount per seat (6 USD above).
- Human review: share of answers below the confidence threshold × minutes per review × loaded hourly rate.
- Data upkeep: cleaning, annotation, drift monitoring and retraining as a monthly line, never as a one-off project.
- Price: per seat, per task or per outcome, with the fair-use cap and overage rate that the contract actually contains.
Two columns per row are enough to start: this month's value and the value you expect a year from now. The row whose two columns differ most is where the year-two margin will be won or lost. Re-run the tab whenever usage or unit prices move: unit economics computed once describe a product that no longer exists.
Fill in the economic layer before the rest
The 26-point slide in the worked example came from success, not error, and caching recovered more than half of it. Before polishing the value proposition or the moat slide, run one cost pass on real usage logs and price the year-two version of your product. If margin stays above your target only when token prices fall on schedule, change the billing unit or the architecture first and the pitch deck second.