SaaS Revenue & Churn Strategy Copilot: Part 2 — Production Considerations

In Part 1, I built a revenue copilot that connects to Stripe, product usage data, and support systems, then uses a four-agent architecture to score churn risk and generate retention recommendations. The implementation works. Now the real questions start.

What does this actually cost to run at 200 sessions per month? How do you build observability that goes beyond "did the LLM call succeed" to "did the recommendation actually lead to a retained account"? What framework should you choose — Python with LangGraph or C# with Semantic Kernel — for a SaaS analytics workload? And, critically, when should you not build this at all?

This article covers all of it with real numbers and honest trade-offs.

Cost Analysis

Let me break down the actual token consumption per session type. All figures use GPT-4o pay-as-you-go pricing: $2.50 per million input tokens, $10.00 per million output tokens.

The most expensive component by far is churn scoring, which is exactly why the pre-filtering pattern from Part 1 matters. Without it, scoring all 500 accounts directly would cost ~$0.48 per full session — before the strategy and response steps.

Monthly Infrastructure

Observability & Debugging

Standard AI observability — logging each LLM call, tracking latency, alerting on errors — is the easy part. The hard problem is outcome tracking: did the recommendations the copilot generated actually lead to retained accounts?

Without outcome tracking, you have a copilot that produces recommendations and no way to know if they're any good. Here's the pattern I use.

Every recommendation is stored in Cosmos DB with a unique ID, the account_id it targets, the session that generated it, and a status field that starts as surfaced. When a CS rep acts on it (logs a call, creates a task, sends an email), the status moves to actioned. At the next renewal date, the status becomes either retained or churned. This gives you a feedback loop that tells you which signal types and recommendation strategies actually work.

import uuid
from datetime import datetime

async def store_recommendations(
    session_id: str,
    at_risk_accounts: list,
    cosmos_container
) -> list:
    """Persist recommendations so we can track outcomes later."""
    stored = []
    for account in at_risk_accounts[:20]:   # top 20 only
        rec = {
            "id":                 str(uuid.uuid4()),
            "session_id":         session_id,
            "account_id":         account["account_id"],
            "risk_score":         account["risk_score"],
            "risk_tier":          account["risk_tier"],
            "primary_signal":     account["primary_signal"],
            "recommended_action": account["recommended_action"],
            "urgency_days":       account["urgency_days"],
            "status":             "surfaced",   # → actioned → retained | churned
            "created_at":         datetime.utcnow().isoformat(),
            "actioned_at":        None,
            "outcome_at":         None,
            "outcome":            None,
        }
        await cosmos_container.upsert_item(rec)
        stored.append(rec)
    return stored

async def record_outcome(recommendation_id: str, outcome: str, cosmos_container):
    """outcome: 'retained' | 'churned' — called at renewal date."""
    item = await cosmos_container.read_item(recommendation_id, recommendation_id)
    item["outcome"]    = outcome
    item["outcome_at"] = datetime.utcnow().isoformat()
    item["status"]     = outcome
    await cosmos_container.replace_item(recommendation_id, item)

public class RecommendationTracker(CosmosClient cosmosClient, string databaseId)
{
    private Container Container =>
        cosmosClient.GetContainer(databaseId, "recommendations");

    public async Task StoreRecommendationsAsync(
        string sessionId,
        IEnumerable<ChurnSignal> atRiskAccounts)
    {
        // Store top 20 at-risk accounts for outcome tracking
        foreach (var account in atRiskAccounts.Take(20))
        {
            var rec = new RecommendationRecord
            {
                Id                = Guid.NewGuid().ToString(),
                SessionId         = sessionId,
                AccountId         = account.AccountId,
                RiskScore         = account.RiskScore,
                RiskTier          = account.RiskTier,
                PrimarySignal     = account.PrimarySignal,
                RecommendedAction = account.RecommendedAction,
                UrgencyDays       = account.UrgencyDays,
                Status            = "surfaced",   // → actioned → retained | churned
                CreatedAt         = DateTime.UtcNow
            };
            await Container.UpsertItemAsync(rec, new PartitionKey(rec.Id));
        }
    }

    public async Task RecordOutcomeAsync(string recommendationId, string outcome)
    {
        // outcome: "retained" | "churned" — called at account renewal date
        var response = await Container.ReadItemAsync<RecommendationRecord>(
            recommendationId, new PartitionKey(recommendationId));

        var rec        = response.Resource;
        rec.Outcome    = outcome;
        rec.OutcomeAt  = DateTime.UtcNow;
        rec.Status     = outcome;

        await Container.ReplaceItemAsync(rec, recommendationId);
    }
}

Beyond outcome tracking, the metrics that actually matter for day-to-day debugging are: agent call latency (churn scoring batches should complete in under 8 seconds each), cache hit rate on the DataConnector (should be above 70% during business hours), and the pre-filter ratio (what percentage of accounts are being filtered — if it drops below 40%, your signal thresholds may need tuning).

Why Choose Python or C# for This

Both implementations work. The choice depends less on technical capability and more on your team's existing context.

My recommendation: if you have a SaaS backend that's already Python, use Python — the data pipeline integration will be smoother, and LangGraph's graph model maps naturally to the orchestration pattern in Part 1. If your backend is .NET, use C# with Semantic Kernel — the type system will catch financial data errors at compile time that Python only catches at runtime, which matters when you're dealing with MRR figures.

Azure Infrastructure

The minimal production setup for this copilot uses five Azure services:

• Azure OpenAI — GPT-4o deployment. Use pay-as-you-go for low volumes (<500 sessions/month). Move to provisioned throughput (PTU) if you're running 2,000+ sessions/month — the economics flip around that point.
• Azure Cosmos DB (NoSQL API) — Session state, the 4-hour account data cache, and recommendation outcome tracking. Serverless mode works well here; the access pattern is bursty during business hours and quiet overnight.
• Azure Container Apps — Host the copilot API. Scale-to-zero handles the overnight quiet period; scale-up handles concurrent CS team usage during the day. No always-on VMs needed at this volume.
• Azure Monitor + Application Insights — Standard telemetry. Add a custom metric for recommendation outcome rates — it's the most important business metric to track.
• Azure Key Vault — Store Stripe API keys, database connection strings, and the OpenAI API key. Don't put these in environment variables in production.

When NOT to Build This

Five specific scenarios where you shouldn't build a revenue copilot:

1. Under $500K ARR. At this stage, you know all your customers personally. A Stripe dashboard and a weekly Notion doc with your top 20 accounts is more than sufficient. Adding AI at this point is complexity without leverage.

2. You already have Gainsight or ChurnZero. These platforms have dedicated CS workflow integrations, purpose-built churn scoring models trained on millions of SaaS accounts, and years of UX refinement. Don't rebuild a worse version. Use the existing tool properly instead.

3. Your churn is a product problem. If customers are leaving because the product doesn't solve their core problem, no amount of proactive outreach will sustainably improve retention. The copilot surfaces at-risk accounts — it can't fix missing features, poor onboarding, or product-market fit issues. Diagnose the cause of churn before building tooling to manage its symptoms.

4. Your data is incomplete or inconsistently collected. Churn risk scores generated from partial usage data, missing NPS records, and untagged support tickets are worse than no scores — they give false confidence in assessments that aren't grounded in reality.

5. Your CS team is already at capacity. If your CS team doesn't have bandwidth to act on additional at-risk flags, generating more of them creates anxiety without improvement. Fix capacity first.

Key Takeaways

Across both articles, the lessons that actually matter in production:

• Pre-compute all financial figures. Never ask the LLM to do arithmetic on revenue data. It will hallucinate, and the errors will look plausible.
• Pre-filter before scoring. The 65–73% reduction in token costs from filtering healthy accounts before LLM scoring is the single most impactful optimisation in this system.
• Track outcomes, not just outputs. A recommendation log that tells you whether CS acted and whether the account retained is the only feedback loop that matters for improving the copilot over time.
• Cost is not the barrier. At $90–200/month infrastructure and $0.07–0.17 per session in AI costs, the question isn't whether you can afford to run this — it's whether your data and team are ready for it.
• Know when to stop. The five "when not to build this" scenarios are not caveats. If any of them apply, fix the underlying problem first.

You can read the architecture and core implementation in Part 1.

Cost figures based on Azure OpenAI GPT-4o pay-as-you-go pricing and Azure service list prices as of March 2026. Infrastructure costs will vary by region and usage patterns.