I built an AI-powered resume-to-job-description matching platform solo, across roughly three months, in three distinct eras: raw Python scripts, a Flask web product, and finally a multi-tenant .NET SaaS on Azure. Each rewrite was triggered by a real, documented limitation I ran into along the way — not by a desire to use newer technology for its own sake.
Jump to a section
- The problem and where it started — how it began, and the categorization mistake that got corrected early.
- The build, era by era — the three rewrites, in order:
- Key production decisions along the way — the Azure OpenAI quota fix, and the cost audit that reversed an earlier call:
- Current state — an honest account of what's still incomplete.
- Skills and technologies this demonstrates
- Closing
The problem and where it started
The project started as offline Python scripts that categorized resumes and matched them against job descriptions using an LLM. I tried keyword-based categorization first, and abandoned it almost immediately — it visibly misclassified resumes, tagging a cook's resume as "frontend developer." I moved to LLM-based categorization instead, with a written rationale attached to every decision. That became the template for everything that followed: the LLM makes the judgment call, and the reasoning behind it is always inspectable, never a black box.
The build, era by era
Era 1 — Python prototype
The scripts grew into a real Flask web app with role-based login (admin/recruiter), per-category resume "datastores," and LLM-driven semantic matching with scoring and reasoning attached to every match. Over its lifetime this app picked up:
- Category-filtered search — job descriptions are auto-categorized, and searches are scoped to the matching resume category by default, cutting the number of files processed (and LLM calls made) by 70–80%.
- A "grounded" prompt rewrite, after discovering the model would invent job requirements that weren't actually in the JD text. The fix was to constrain the prompt to only the stated JD content, not to bolt on more instructions asking it to "be careful."
- A move from a flat
users.txtfile to MySQL-backed auth with roles, and later from MySQL to SQL Server specifically to align with an eventual Azure deployment. - Hand-rolled backup/restore tooling (complete and delta backups) — a solo developer's substitute for a real release/rollback process.
By the end of this era, the app's own architecture doc listed its own limitations candidly: synchronous request handling blocking the UI, tight coupling to one LLM provider, one storage backend per deployment, in-memory sessions that wouldn't scale horizontally. That self-assessment is what justified the rewrite — not a desire to use newer tech.
Era 1 architecture: the Flask app, its per-category datastores, and the grounded-prompt LLM matching step.
Era 2 — .NET rewrite
The rewrite started deliberately narrow: a generic document-search proof of concept, not resume-matching-specific, built to de-risk the .NET 9 + Azure stack before betting the whole product on it. That POC settled two open questions:
- Vector search: Chroma (the vector store from the Python era) doesn't host well on Azure outside of Docker, so Azure AI Search replaced it — a managed service with built-in vectorization, chosen over Azure Cosmos DB's vector search as "overkill" for what was actually needed.
- LLM provider: the code was written to support both Azure OpenAI and direct OpenAI from day one. That decision mattered a lot once a real quota problem showed up later.
Once the stack was proven, I rebuilt the actual resume matcher on it with a clean/onion architecture (Core/Infrastructure/Web/Api/Shared), EF Core + SQL Server, and ASP.NET Core Identity. The defining decision of this stage was a 3-layer content split: the database holds only metadata, Azure Search holds all searchable content and AI analysis, and Blob Storage holds the original files. This was a real fix after a production bug where the code assumed content still lived in the database after the schema had already been simplified to metadata-only. This era also marks the first point in the project's history with real automated test projects — unit, integration, UI, frontend — rather than ad hoc test scripts.
Era 2 architecture: the onion-architecture rebuild, the 3-layer content split first introduced here, and dual LLM provider support built in from day one.
Era 3 — Multi-tenant SaaS
The final stage converted the single-tenant .NET app into a multi-tenant platform: database-per-tenant SQL Server, per-tenant Blob containers, per-tenant Azure Search indexes, and route-based (/tenant/{id}/...) tenant resolution, with a full security hardening pass — rate limiting, audit logging, CSRF protection, tenant-boundary authorization, and input validation against SQL injection/XSS.
Era 3 architecture: tenant-scoped requests hit the 3-layer content split, with the LLM layer able to switch providers via configuration.
Key production decisions along the way
1. The Azure OpenAI quota problem, solved and documented
Azure OpenAI's S0 tier capped out at 50K tokens/minute, threw frequent 30-second timeouts, and required a company email address just to request a quota increase. Switching to OpenAI direct kept the same GPT-4 pricing but raised the effective rate limit from roughly 10K to 2M+ TPM, and cut response times from timeouts down to 2–5 seconds — while keeping Azure Search for vector search and adding local ONNX embeddings (all-MiniLM-L6-v2) to cut API calls further.
Why this wasn't a rip-and-replace
The system supports both providers via a single config switch (LLM:Provider). This was a configuration-level pivot, not a rebuild — made possible specifically by the Era 2 decision to support both providers from day one, before the quota problem ever showed up.
2. A rigorous, numbers-driven cost audit that walked back an earlier decision
A self-audit (SQL_GPT_vs_Vector_GPT_Consideration.md) found that GPT was doing roughly 95% of the actual matching intelligence — categorization, skill extraction, scoring, synonym handling — while Azure AI Search had effectively degraded into an expensive filtered database query: $75–250/month for what a WHERE category = X SQL clause could do.
The audit modeled five alternative architectures on real infrastructure cost:
| Architecture | Monthly cost | Trade-off |
|---|---|---|
| Pure SQL + GPT | $0 | Simplest option; candidate pre-selection is random/recency-based, not semantic |
| SQL + keyword filtering + GPT | $0 | Better pre-selection than random, but keyword matching misses synonyms |
| PostgreSQL + pgvector + GPT | ~$100 | Semantic pre-selection at a fraction of Azure Search's cost; best economics at scale |
| SQL Server (native vector support) | $0 | Not available yet at the time of the audit — still in preview |
| Azure Search + vector (status quo) | $75–250 | Already built and working, but paying for search-engine features the pipeline barely uses |
The audit then modeled how the two realistic contenders — keep Azure Search, or migrate to PostgreSQL + pgvector — scale with tenant count:
| Deployment size | Azure Search | PostgreSQL + pgvector | Annual savings |
|---|---|---|---|
| 1–5 tenants | $375/mo | $36/mo | $4,068/yr |
| 10 tenants | $750/mo | $152/mo | $7,176/yr |
| 20 tenants | $1,500/mo | $200/mo | $15,600/yr |
| 50 tenants | $3,750/mo | $309/mo | $41,292/yr |
| 100 tenants | $7,500+/mo | $618/mo | $82,584/yr |
PostgreSQL + pgvector wins on cost at every scale modeled; the audit's own conclusion was that the 5–10% search-quality difference isn't worth $4,000–$80,000+/year for most deployments this size.
This is the kind of build-vs-buy, cost-vs-value reasoning that transfers directly to forward-deployed engineering work — not just "can it be built," but "should it be built this way at this scale."
Current state
The multi-tenant SaaS stage is the most architecturally advanced but is documented as still under active hardening, not a finished product: the tenant-admin UI is incomplete, some manual Azure SQL provisioning is still required, and there's a self-identified high-risk gap around security/compliance logging — auth-failure logging, tenant-isolation-breach detection, and audit trails for data access aren't yet implemented. Stating this plainly here rather than glossing over it, because the gap analysis itself is part of the same practice described below.
Skills and technologies this demonstrates
- Iterative architecture under real constraints — every rewrite was triggered by a documented failure or limitation (miscategorization, hallucinated JD requirements, a schema/code mismatch, an Azure quota wall, an unjustified cost), not by chasing new technology.
- LLM application engineering — prompt grounding to prevent hallucination, categorization-then-matching pipelines, prompt-version comparison, provider-agnostic LLM integration.
- Azure platform breadth — Blob Storage, Azure AI Search, Azure OpenAI, Key Vault/App Configuration, ASP.NET Identity, multi-tenant resource provisioning via ARM templates.
- Cost-conscious engineering — per-tenant cost tracking via a decorator pattern around every Azure service call, and a willingness to reverse an earlier architecture decision once the numbers didn't hold up.
- Multi-tenant SaaS fundamentals — database-per-tenant isolation, route-based tenant resolution, tenant-scoped authorization, dynamic resource provisioning.
- Honest self-assessment as a practice — nearly every stage documents its own known gaps candidly, rather than overstating completeness.
Closing
The one thing none of these three builds ever had was a formal eval/scoring harness — matching quality was judged by eye, not measured. That gap became the explicit first target of the project that followed this one.