AI Integration Into Existing Systems: A Practical Playbook For Engineering Leaders

AI integration into existing systems is the work of fitting AI capabilities, most often generative AI and large language models, into the software, workflows, and data a company already runs. Roughly 97% of enterprise AI pilots fail to deliver measurable value, according to a study cited by James LePage, Director of Engineering AI at Automattic, and the choke point is almost always integration work, not model choice.

This piece is a working playbook for CTOs and engineering leaders making those bets. The patterns below come from interviews with practitioners at Wix, Automattic, MindsetMed, UserDocs, and other companies that have shipped AI inside large, live systems.

Why most AI integrations stall before they ship value

James LePage frames the failure mode directly: "97% of pilots are ineffective in enterprise environment... because these pilots are using tools that are inflexible to the workflows." A new AI tool gets introduced next to the existing system. People are asked to switch context to use it. Adoption flatlines. The pilot quietly gets paused, then killed at the next budget review.

The downstream cost shows up in three places. Budget gets stranded on tools no one opens. Engineers and product people who were enthusiastic about AI lose patience and either disengage or leave. The board, which approved the pilot expecting a clear win, starts treating future AI proposals with new skepticism.

The pattern is consistent enough to tabulate:

The practical takeaway for engineering leaders: when you scope an AI initiative, scope the integration first. The model can be swapped later anyway.

How AI integrations actually create value

Foundation models are converging in capability. The differentiation has moved one layer up. Adam Ben-David puts it plainly: "we are moving from obsessing over which model is best and testing all the different versions of the model to context engineering." Context engineering covers the tools you give the agent, the data you put in front of it, and how you manage what it knows at any moment.

Anna Barnacka, working in MedTech where regulatory consequences are real, makes the same case from a different angle. Her teams have found that "deep knowledge of human physiology proves more crucial than simply deploying language models." Quality of data and domain understanding outperforms raw model capability whenever the application has any specificity.

For most mid-market companies, this points to a clear strategic answer that LePage spelled out separately: "We win in applied AI. That's where we win because we have the distribution, we have the users, we have the data." Training a proprietary foundation model is rarely the right bet at mid-market scale. Your moat is the user behavior data flowing through the product you already operate, paired with your team's domain knowledge.

A worked example: UserDocs and the legacy-code value chain

Chris Rickard's UserDocs shows the value chain end-to-end. The product takes a legacy codebase, runs it through an AI pipeline that reverse-engineers structure and intent, and produces queryable documentation that humans and other AI tools can both consume.

Rickard describes the workflow in plain terms: "downloading the code, and then coming into UserDoc and creating a new project and saying, let's translate code into documentation... It reverse engineers how it all works."

The chain looks like this. Input: a codebase no one fully understands anymore. Workflow: an AI pipeline that lives inside the documentation tool the team already uses. Behavior change: design, dev, and QA query the same living artifact instead of pinging the one engineer who remembers how the system works. Outcome: what used to be a multi-month archaeology project takes days. Constraint: the generated documentation needs an owner, otherwise it decays back into the stale artifact it replaced.

Six integration patterns that work

Apply these as concurrent design decisions made on day one of any integration project.

Embed AI inside workflows people already use

What it does: surfaces an AI capability inside the tools, screens, and pipelines a team already opens every day.

Where it fits: any workflow with established habits and tooling.

Why it matters: adoption follows the path of least friction.

Evidence: companies that pushed AI into existing workflows materially outperformed those running standalone pilots, per the study LePage cites. Wix calls their approach "AI-friendly, not AI-binary": the easy option in workflows where AI helps, with no mandate where it breaks things.

Constraint: requires engineering ownership of the existing tool, or a clean integration surface such as an API, plugin model, or IDE extension.

Choose meld over build vs. buy

What it does: pairs a third-party foundation model or AI product with your own code, data, and UX.

Where it fits: any feature where a general-purpose model already handles the core task well.

Why it matters: training or hosting your own model rarely pays off at mid-market scale.

Evidence: Steve Brown describes the call: "The smart companies are going to meld bought code and built code... If there's a foundational model that works really well... someone else is there to sustain and maintain it. Add value around that." A third-party model wrapped in your business logic, your data, and your UX powers most successful AI features in production today.

Constraint: pick vendors with clear pricing, model versioning, and SOC 2 or ISO posture. Lock-in is a real risk.

Treat context engineering as the core engineering work

What it does: invests engineering effort in the data, tools, and runtime context you give an agent rather than in model selection.

Where it fits: any agentic system, retrieval pipeline, or LLM feature where output quality is uneven.

Why it matters: this is where the real engineering work now sits, per Ben-David.

Evidence: in domain-specific applications, Barnacka's team has shown that "a well-labeled, meticulously curated dataset is more valuable than a massive array of questionable reliability."

Constraint: this is an investment in evaluation, retrieval quality, and tool design. Prompt tweaks alone will not produce it.

Layer capabilities onto existing systems

What it does: adds AI as a new layer on top of existing systems rather than replacing components.

Where it fits: any environment with established workflows, data flows, and legacy code.

Why it matters: Steve Brown frames AI evolution as additive, observing that "they're just new capabilities that layer in over time." Generative AI, agentic AI, and spatial AI stack rather than obsolete each other.

Evidence: incumbents are adding AI capability incrementally to existing products instead of launching AI-first rebuilds that compete with their own user base.

Constraint: requires honest evaluation of your architecture's integration points. If the legacy system has no clean seams, cost climbs fast.

Turn legacy code into living documentation

What it does: uses AI to reverse-engineer legacy systems into structured, queryable documentation that humans and other AI tools can both consume.

Where it fits: environments where institutional knowledge has decayed, ownership has churned, or the codebase predates current staff.

Why it matters: the documentation becomes a shared substrate for design, dev, QA, and the AI tools your team uses next. See the UserDocs worked example above for the full chain.

Evidence: compresses what used to be a multi-month archaeology project into days.

Constraint: the generated documentation needs an owner. Without one it decays back into the stale artifact it replaced.

Build LLM guardrails on day one

What it does: assumes every LLM output is potentially hostile and validates it across multiple layers before it can act on your system or reach a user.

Where it fits: any LLM-powered feature that touches customer data, takes action, or is exposed to user input.

Why it matters: the attack surface spans prompt injection through user inputs, retrieved documents, third-party tools, and the model itself.

Evidence: Donato Capitalla is blunt about the threat model: "all LLM outputs should be treated as 'untrusted input' to the system." His open-source Spikee testing framework generates large volumes of automated prompt injection attempts so teams can measure guardrail effectiveness.

Constraint: guardrails need continuous testing. New attacks ship constantly, and a one-time audit will not hold the line.

What must be true for any of this to work

The patterns above assume four organizational conditions that often do not exist by default.

First, the data has to be good enough for the application. Barnacka's framing again: prioritize quality over quantity, well-labeled over abundant. A small curated dataset with strong domain relevance outperforms a large unlabeled one for most real applications.

Second, you need a centralized enablement function. Wix, at 5,500 employees, ran their transformation through a 50-to-60 person team called "AI Core" that owns tooling decisions, training, and shared infrastructure (per Asaf Yonay, GM of AI-Native Transformation). At mid-market scale this might be three to five people. The point is the same: stop having every product team re-solve evaluation, prompt design, and governance from scratch.

Third, plan for tool redundancy from day one. Wix learned this in production: "teams who over-rely on a single LLM/IDE stop working when that service has downtime. Keep 2-3 viable alternatives in your tooling stack." An outage at one provider should not freeze the company.

Fourth, AI integration is the whole org's job. Steve Brown is firm on this: "AI transformation... is the job of everybody, including the IT department, to figure out... what are the places you're gonna put AI in your organization." John Frankel makes the parallel argument from the staff side, advocating for "AI point solutions" where individual employees identify and apply AI in their own workflows. Both motions require leadership to make adoption a real priority with structural support behind it.

Quick self-test before you greenlight the next integration:

Measuring success: the KPIs that actually matter

Most teams instrument AI projects with traditional output metrics: tickets closed, lines of code, features shipped. Those metrics under-fit the work AI does.

Oji Udezu argues for a higher bar: "I looked at time to value. If I could not do like five to 10x acceleration, then AI was pointless." His companion metric is tweak time: how much effort the team spends cleaning up AI output before it becomes usable. If tweak time approaches the time saved, the integration is net-neutral.

Asaf Yonay frames the headline metric differently for organizations: velocity. "If I can create velocity, that's great. It's up to the company to decide how they use it." When tooling shifts every quarter, the cost of a slow decision is higher than the cost of a wrong one.

Targets to aim for in the first 90 days of an integration:

Where to focus next

Companies shipping useful AI integrations treat the work as 80% organizational design and 20% engineering. The model is the cheapest part. They embed AI inside workflows that already work, invest seriously in context engineering, layer new capabilities onto existing systems, secure model outputs by default, and pair every adoption push with structural support like a centralized enablement team.

Monterail's AI integration team has shipped guarded LLM features into production systems across healthcare, fintech, and SaaS. If you are scoping an integration this quarter and want a second read on the architecture, the data, or the rollout, we can pressure-test the plan before you commit budget.

Strategic takeaway

Treat AI integration as a systems problem first and a model problem last. The product surface, the data pipeline, the workflow it sits inside, the enablement function around it, and the guardrails that contain its outputs all need to work together. Each layer compounds the next. Optimize any one layer in isolation and the integration will look impressive in a demo and fall apart in production. Build the system and the model becomes interchangeable, which is the point.