Financial Advice Agent: Personal Portfolio & Scenario Planner — Part 2: Production Considerations

What Nobody Tells You About AI Finance Tools

The demo works. You've got GPT-4o routing between four tools, fetching portfolio data from Cosmos DB, running compound growth calculations, and explaining the results in plain English. It's genuinely impressive.

Then someone asks: "What does this cost to run per user per month?" And you realise you haven't thought about it.

This part is about the questions that come after the demo: token costs, debugging a system that touched three tools to answer one question, choosing between Python and C#, and — most importantly — the situations where you should step back and not use AI at all.

Cost Analysis: Real Numbers

I tracked actual token usage across 50 test sessions. Here's what a typical session looks like:

At $0.06 per session, 1,000 active sessions/month costs roughly $60. That's reasonable for an internal tool. It becomes notable if you're scaling to tens of thousands of users.

Observability: Debugging a Three-Tool Chain

The hardest debugging problem with multi-tool agents isn't finding bugs in individual tools — it's understanding why the orchestrator made the routing decision it did, and which tool produced the data the LLM narrated.

In practice, you'll find that an unexpectedly bad response is almost always one of three things: wrong intent classification, stale market data, or a scenario calculation with parameters the user didn't intend to set.

import logging
from azure.monitor.opentelemetry import configure_azure_monitor
from opentelemetry import trace

configure_azure_monitor(connection_string=os.environ["APPLICATIONINSIGHTS_CONNECTION_STRING"])
tracer = trace.get_tracer("portfolio-agent")
logger = logging.getLogger("portfolio-agent")

def traced_tool_call(tool_name: str, inputs: dict):
    """Decorator/context manager for tracing tool invocations."""
    with tracer.start_as_current_span(f"tool.{tool_name}") as span:
        span.set_attribute("tool.name", tool_name)
        span.set_attribute("tool.inputs", str(inputs)[:500])  # truncate for safety
        logger.info(
            "Tool invoked",
            extra={
                "tool": tool_name,
                "inputs_summary": {k: type(v).__name__ for k, v in inputs.items()},
                "session_id": inputs.get("session_id", "unknown"),
            }
        )
        yield span  # caller fills in outputs after execution

# Usage
async def run_portfolio_analyser(state: PortfolioState) -> PortfolioState:
    inputs = {"user_id": state["user_id"], "holdings_count": len(state["holdings"])}
    with traced_tool_call("portfolio_analyser", inputs) as span:
        result = await _do_portfolio_analysis(state)
        span.set_attribute("tool.result_keys", str(list(result.keys())))
        return result

using Azure.Monitor.OpenTelemetry.Exporter;
using OpenTelemetry;
using OpenTelemetry.Trace;
using Microsoft.Extensions.Logging;

public class AgentTracer
{
    private static readonly ActivitySource Source = new("portfolio-agent");
    private readonly ILogger<AgentTracer> _logger;

    public AgentTracer(ILogger<AgentTracer> logger) => _logger = logger;

    public IDisposable TraceToolCall(string toolName, Dictionary<string, object> inputs)
    {
        var activity = Source.StartActivity($"tool.{toolName}");
        activity?.SetTag("tool.name", toolName);
        activity?.SetTag("tool.session_id", inputs.GetValueOrDefault("sessionId", "unknown"));

        _logger.LogInformation(
            "Tool invoked: {ToolName} with {InputCount} inputs",
            toolName,
            inputs.Count);

        return activity ?? Disposable.Empty;
    }
}

// Registration in Program.cs
builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing
        .AddSource("portfolio-agent")
        .AddAzureMonitorTraceExporter(o =>
            o.ConnectionString = config["ApplicationInsights:ConnectionString"]));

The most useful signal I added was logging the intent classification decision alongside the raw user message. When the system routes incorrectly — say, classifying "what's my risk exposure?" as a portfolio analysis rather than a risk assessment — you can see exactly what the classifier received and what it returned. That makes tuning the system prompt surgical rather than guesswork.

Python vs C#: Which One to Use

Python Implementation

Why choose Python: If your team writes Python, you get access to the richest AI/ML ecosystem available right now.

• LangGraph — mature graph-based agent framework with built-in checkpointing and state persistence
• Rapid iteration — Jupyter notebooks let you test individual tools interactively before wiring them into the graph
• Financial libraries — numpy, pandas, and scipy are readily available if your scenario calculations need statistical analysis
• Community — most new Azure OpenAI patterns appear in Python first

C#/.NET Implementation

Why choose C#: If you're building this as part of a .NET backend — or if your team is primarily a C# shop — Semantic Kernel gives you first-party Microsoft support and familiar patterns.

• Native Azure integration — first-party SDKs, Microsoft-maintained, strong typing throughout
• Enterprise patterns — dependency injection, middleware pipeline, familiar to .NET teams
• decimal precision — .NET's decimal type is built for financial calculations; no need to import a separate library
• ASP.NET Core integration — drop the agent into an existing .NET API with minimal ceremony

Azure Infrastructure

The minimum viable setup for running this in Azure:

• Azure OpenAI (GPT-4o deployment) — orchestration and narration
• Azure Cosmos DB (NoSQL API) — portfolio data and scenario history persistence
• Azure Cache for Redis — market data caching with 15-minute TTL
• Azure App Service or Container Apps — hosting the API layer
• Azure Monitor + Application Insights — tracing and cost visibility
• Azure Key Vault — connection strings, API keys

Azure AI Foundry Agent Service

Azure AI Foundry Agent Service is now generally available, providing managed orchestration for AI systems.

• Built-in routing and workflows — you could use this instead of LangGraph/Semantic Kernel for the orchestration layer
• Managed state persistence — replaces the MemorySaver / session dictionary pattern
• Native Azure OpenAI integration
• Observability through Azure Monitor out of the box

Check Azure AI Foundry Agent Service for current pricing.

When NOT to Use AI for Financial Planning

This is the section most tutorials skip. The architecture works. The costs are manageable. But there are situations where you should not deploy this kind of tool.

Simpler alternatives to consider:

• Static calculator — if users just need a projection with fixed inputs, a JavaScript calculator with no AI is faster, cheaper, and auditable
• Rule-based system — if your "risk assessment" is actually a scoring model with fixed rules, implement it as a rules engine, not an LLM
• Human adviser + AI summary — for regulated contexts, use AI to prepare a summary for a human adviser to review, not to replace the adviser

Key Takeaways

If I were starting this project again:

• Separate calculation from narration from day one. The LLM's job is language, not maths. Every dollar figure comes from deterministic code.
• Cap your context window deliberately. Sliding window of 6 turns is not a hack — it's a cost control mechanism that also keeps responses focused.
• Trace intent classification separately. Routing errors are your biggest source of bad responses. Knowing what the classifier decided (and why) makes debugging 10x faster.
• Start with serverless Cosmos DB. Switch to provisioned when you can predict load. Paying for RUs you don't use in development adds up.
• Know your regulatory context. Personal tool? Fine. Public product? Get legal involved early.