Production-Grade AI Compliance for EY Virtual Compliance Officer

The inherited AI layer suffered from hallucinations, false "no access" errors, and inconsistent answers. Hardening the chatbot across multiple modules and pushing end-to-end query latency below a conversational 3-second threshold were the baseline requirements for enterprise readiness.

The architecture had to be GDPR-compliant and survive an external penetration test. That meant enforcing backend-level isolation between modules and strict per-report permissions to close off data leakage, ID guessing, and prompt injection vectors.

Document indexing time couldn't be meaningfully reduced, but the experience felt broken because users had no visibility into processing status. The fix was real-time progress indicators and alerts, making an opaque background process feel predictable.

The AI infrastructure needed to support monetization and admin control: per-request token tracking to lay the groundwork for usage-based pricing, and superadmin access to tune system prompts and model parameters.

 Replacing the legacy multi-hop orchestration with a direct Azure OpenAI integration, without breaking existing API contracts or disrupting live users, had to land within 3 months, governed by a strict MoSCoW prioritization framework.

An orchestration framework optimized to validate an initial business hypothesis is rarely built to sustain enterprise scale. Orchestration latency, opaque debugging layers, and cross-system permission handling compound rapidly once an application graduates into a regulated environment. Productization should be planned as an intentional refactoring pass.

Moving data access and authorization checks directly into individual tool implementations makes context separation a code-level property rather than an emergent property of an external router. It shifts security from an abstract risk conversation into a concrete engineering checklist.

When an application experiences hallucinations or erratic responses, the LLM itself is frequently blamed. Once the orchestration layer was self-contained and tools became strongly typed and observable, remaining bugs were identified as standard prompt-tuning tasks, which are much cheaper and faster to fix.

Building with a native tool-based agent design ensures that introducing subsequent capabilities (e.g., advanced reindexing, document summarization, or alternate data sources) becomes a self-contained task of adding an isolated tool rather than redesigning the foundational AI core.