What percentage of revenue should AI costs represent?

For most SaaS products, AI costs should stay below 10-15% of the revenue generated by AI-powered features. If costs exceed 20%, the feature may not be viable without optimization. Consumer products with lower ARPU have tighter margins and should target 5-10%.

When should we self-host models instead of using APIs?

Self-hosting becomes cost-effective when your request volume exceeds roughly 1 million requests per month for a given model, and your team has the ML infrastructure skills to manage deployment. Below that volume, API providers are almost always cheaper when you factor in engineering time.

How do we estimate costs for a feature that has not launched yet?

Use the cost projections section with conservative estimates. Multiply expected daily active users by estimated requests per user by cost per request. Add a 30% buffer for edge cases, retries, and usage patterns you did not anticipate.

Should we optimize costs during development or after launch?

Build cost awareness into development from the start. Set a cost per request target in your PRD. After launch, use the first 30 days of real usage data to calibrate your projections and prioritize optimizations based on actual cost drivers.

How do we handle cost spikes from prompt injection or abuse?

Set hard rate limits per user and per API key. Monitor for anomalous usage patterns (sudden spikes in token count or request volume). Include abuse-related cost controls in your governance rules section.

AI Cost Optimization Template for AI Products

What This Template Is For

AI infrastructure costs can grow faster than revenue if nobody is tracking them. A single LLM-powered feature might cost $0.02 per request at launch and look affordable at 1,000 daily users. At 100,000 daily users, that same feature costs $60,000 per month. Multiply that pattern across several AI features and the numbers become painful quickly. Training costs, data storage, GPU compute, and third-party API fees all compound in ways that traditional software budgets do not account for.

This template gives product and engineering teams a structured way to track AI costs by category, identify the highest-impact optimization levers, and calculate the ROI of specific cost reduction initiatives. The AI ROI Calculator provides an interactive way to model the financial impact of your AI features. For teams building cost projections into their initial planning, the AI Product PRD Template includes a cost estimation section. The LLM Cost Estimator helps estimate per-request costs across different model providers.

Most teams optimize reactively after a surprise cloud bill. This template supports proactive cost management by creating visibility into where money is going, which optimization levers have the highest payoff, and how to build cost awareness into the product development lifecycle. If your team is evaluating whether to build or buy AI capabilities, the AI Build vs Buy assessment can help frame the total cost of ownership decision.

When to Use This Template

When AI infrastructure costs exceed 15% of feature revenue. That is a signal that cost optimization needs dedicated attention.
Before scaling an AI feature to a larger user base. Model what costs look like at 10x current volume before you get there.
During quarterly planning or budget reviews. Use the template to present AI cost trends and optimization opportunities to leadership.
After switching model providers or architectures. Track the cost impact of technical changes.
When evaluating build vs. buy decisions. Compare the total cost of an in-house model against API-based alternatives.
When onboarding a new AI vendor. Establish baseline cost tracking from day one.

How to Use This Template

Inventory all AI cost categories. List every source of AI-related spend: model inference, training compute, data storage, labeling, API fees, monitoring tools, and human review.
Fill in current monthly costs. Pull actual numbers from cloud billing dashboards, vendor invoices, and internal cost allocation.
Identify the top 3 cost drivers. Focus optimization efforts where the spend is highest.
Evaluate optimization levers. For each cost driver, list specific actions (model distillation, caching, batching, prompt compression) and estimate the savings.
Track month-over-month. Update the cost tracker monthly to measure progress and catch new cost spikes early.

The Template

## AI Cost Optimization Tracker

**Product / Feature**: [Name]
**Period**: [Month / Quarter]
**Owner**: [PM or Engineering Lead]
**Total AI Budget**: $[Budget]
**Total AI Spend This Period**: $[Actual]

---

### 1. Cost Breakdown by Category

| Category | Monthly Cost | % of Total | Trend (MoM) | Notes |
|----------|-------------|-----------|-------------|-------|
| Model Inference (API) | $[Amount] | [%] | [Up/Down/Flat] | [Provider, model] |
| Model Inference (Self-hosted) | $[Amount] | [%] | [Up/Down/Flat] | [GPU type, instances] |
| Model Training / Fine-tuning | $[Amount] | [%] | [Up/Down/Flat] | [Frequency, duration] |
| Data Storage (Training data) | $[Amount] | [%] | [Up/Down/Flat] | [Volume in TB] |
| Data Storage (Embeddings / Vectors) | $[Amount] | [%] | [Up/Down/Flat] | [Vector DB provider] |
| Data Labeling / Annotation | $[Amount] | [%] | [Up/Down/Flat] | [Vendor, volume] |
| Third-party AI APIs | $[Amount] | [%] | [Up/Down/Flat] | [Service names] |
| Monitoring / Observability | $[Amount] | [%] | [Up/Down/Flat] | [Tools used] |
| Human Review / QA | $[Amount] | [%] | [Up/Down/Flat] | [Headcount or vendor] |
| **Total** | **$[Total]** | **100%** | | |

---

### 2. Unit Economics

| Metric | Current | Target | Gap |
|--------|---------|--------|-----|
| Cost per inference request | $[Amount] | $[Amount] | [%] |
| Cost per active user / month | $[Amount] | $[Amount] | [%] |
| Cost per 1K tokens (input) | $[Amount] | $[Amount] | [%] |
| Cost per 1K tokens (output) | $[Amount] | $[Amount] | [%] |
| AI cost as % of feature revenue | [%] | [%] | [pp] |
| AI cost as % of COGS | [%] | [%] | [pp] |

---

### 3. Cost Projections

| Scenario | Monthly Users | Requests/User | Cost/Request | Monthly Cost |
|----------|-------------|---------------|-------------|-------------|
| Current | [N] | [N] | $[Amount] | $[Total] |
| 3-month forecast | [N] | [N] | $[Amount] | $[Total] |
| 6-month forecast | [N] | [N] | $[Amount] | $[Total] |
| 10x scale | [N] | [N] | $[Amount] | $[Total] |

---

### 4. Optimization Opportunities

| # | Optimization Lever | Category | Estimated Savings | Effort | Priority |
|---|-------------------|----------|-------------------|--------|----------|
| 1 | [e.g., Switch from GPT-4 to GPT-4o-mini for simple tasks] | Inference | $[Amount]/mo | [Low/Med/High] | [P0/P1/P2] |
| 2 | [e.g., Add response caching for repeated queries] | Inference | $[Amount]/mo | [Low/Med/High] | [P0/P1/P2] |
| 3 | [e.g., Reduce prompt length by 40% via compression] | Inference | $[Amount]/mo | [Low/Med/High] | [P0/P1/P2] |
| 4 | [e.g., Batch non-real-time requests] | Inference | $[Amount]/mo | [Low/Med/High] | [P0/P1/P2] |
| 5 | [e.g., Distill large model to smaller fine-tuned model] | Training | $[Amount]/mo | [Low/Med/High] | [P0/P1/P2] |
| 6 | [e.g., Archive old training data to cold storage] | Storage | $[Amount]/mo | [Low/Med/High] | [P0/P1/P2] |

---

### 5. Optimization ROI Tracker

| Initiative | Start Date | Baseline Cost | Current Cost | Savings | Status |
|-----------|-----------|-------------|-------------|---------|--------|
| [Initiative 1] | [Date] | $[Amount]/mo | $[Amount]/mo | $[Amount]/mo | [Active/Complete] |
| [Initiative 2] | [Date] | $[Amount]/mo | $[Amount]/mo | $[Amount]/mo | [Active/Complete] |

---

### 6. Cost Governance Rules

- [ ] AI cost reviews happen [weekly / monthly] with [Engineering Lead / PM]
- [ ] New AI features require a cost projection before approval
- [ ] Cost per request is monitored with alerts at [threshold]
- [ ] Model upgrades require cost impact analysis
- [ ] Vendor contract renewals include usage audit

Filled Example: Customer Support AI Assistant

## AI Cost Optimization Tracker

**Product / Feature**: SupportBot AI Assistant
**Period**: February 2026
**Owner**: Alex Rivera, Senior PM
**Total AI Budget**: $45,000/month
**Total AI Spend This Period**: $52,300 (16% over budget)

---

### 1. Cost Breakdown by Category

| Category | Monthly Cost | % of Total | Trend (MoM) | Notes |
|----------|-------------|-----------|-------------|-------|
| Model Inference (API) | $31,400 | 60% | Up 22% | GPT-4o via OpenAI, ~2.1M requests |
| Model Inference (Self-hosted) | $0 | 0% | - | Not using self-hosted models |
| Model Training / Fine-tuning | $3,200 | 6% | Flat | Monthly fine-tune on new support tickets |
| Data Storage (Training data) | $1,800 | 3% | Up 5% | 4.2 TB in S3 |
| Data Storage (Embeddings) | $4,100 | 8% | Up 12% | Pinecone, 8M vectors |
| Data Labeling / Annotation | $2,500 | 5% | Flat | Labelbox, 5K tickets/month |
| Third-party AI APIs | $3,800 | 7% | Up 8% | Whisper (voice), text classification |
| Monitoring / Observability | $1,200 | 2% | Flat | Langfuse + Datadog |
| Human Review / QA | $4,300 | 8% | Down 5% | 3 contractors reviewing edge cases |
| **Total** | **$52,300** | **100%** | **Up 14%** | |

---

### 2. Unit Economics

| Metric | Current | Target | Gap |
|--------|---------|--------|-----|
| Cost per inference request | $0.015 | $0.008 | 47% reduction needed |
| Cost per active user / month | $1.74 | $1.00 | 43% reduction needed |
| Cost per 1K tokens (input) | $0.005 | $0.003 | 40% reduction needed |
| Cost per 1K tokens (output) | $0.015 | $0.008 | 47% reduction needed |
| AI cost as % of feature revenue | 18% | 10% | 8pp |
| AI cost as % of COGS | 34% | 20% | 14pp |

---

### 3. Cost Projections

| Scenario | Monthly Users | Requests/User | Cost/Request | Monthly Cost |
|----------|-------------|---------------|-------------|-------------|
| Current | 30,000 | 70 | $0.015 | $52,300 |
| 3-month forecast | 42,000 | 75 | $0.015 | $73,500 |
| 6-month forecast | 60,000 | 80 | $0.015 | $108,000 |
| With optimizations | 60,000 | 80 | $0.008 | $57,600 |

---

### 4. Optimization Opportunities

| # | Optimization Lever | Category | Estimated Savings | Effort | Priority |
|---|-------------------|----------|-------------------|--------|----------|
| 1 | Route simple queries to GPT-4o-mini (est. 55% of volume) | Inference | $12,000/mo | Medium | P0 |
| 2 | Cache responses for top 200 FAQ-type queries | Inference | $4,500/mo | Low | P0 |
| 3 | Compress system prompts from 2,100 to 900 tokens | Inference | $3,100/mo | Low | P1 |
| 4 | Batch nightly analytics and summary jobs | Inference | $1,800/mo | Low | P1 |
| 5 | Migrate Pinecone to self-hosted Qdrant | Storage | $2,600/mo | High | P2 |
| 6 | Reduce fine-tuning frequency to biweekly | Training | $1,600/mo | Low | P1 |

---

### 5. Optimization ROI Tracker

| Initiative | Start Date | Baseline Cost | Current Cost | Savings | Status |
|-----------|-----------|-------------|-------------|---------|--------|
| Model routing (simple to mini) | 2026-03-01 | $31,400/mo | TBD | Est. $12,000/mo | Active |
| FAQ response cache | 2026-03-10 | $31,400/mo | TBD | Est. $4,500/mo | Planned |

---

### 6. Cost Governance Rules

- [x] AI cost reviews happen monthly with Engineering Lead and PM
- [x] New AI features require a cost projection before approval
- [ ] Cost per request is monitored with alerts at $0.020 threshold
- [ ] Model upgrades require cost impact analysis
- [x] Vendor contract renewals include usage audit

Key Takeaways

Model inference is typically 50-70% of total AI cost. Start your optimization efforts there.
Unit economics (cost per request, cost per user) matter more than total spend. A growing total is fine if unit costs are declining.
Model routing (sending simple queries to cheaper models) is usually the highest-ROI optimization with the lowest effort.
Response caching and prompt compression are quick wins that most teams overlook.
Project costs at 10x current volume before scaling. If the numbers do not work at scale, optimize before you grow.
Monthly cost reviews with a documented tracker prevent surprise bills and build cost awareness across the team.

AI Cost Optimization Template for AI Products