What Is Security by Design, and Why Does It Matter More in the Agentic AI Era?

Security by design means treating cybersecurity as a priority through the whole software lifecycle, from the first architecture sketch to long-term maintenance, instead of bolting it on after launch.

By 2025, that idea had moved from security conferences into boardrooms. JPMorganChase's CISO was publicly calling out SaaS vendors, and regulators in the US and EU were writing it into policy.

Then AI agents went into production, and software that once displayed data started reading it, deciding what to do with it, and acting on it with real credentials.

What Is Security by Design?

Security by design means that cybersecurity is a priority throughout the whole software product lifecycle, from the design phase all the way to long-term maintenance.

According to Microsoft's CEO Satya Nadella, security by design means that "security comes first when designing any product or service."

Another useful definition comes from the influential report by America's cyber defense agency, CISA:

"'Secure by design' means that technology products are built in a way that reasonably protects against malicious cyber actors successfully gaining access to devices, data, and connected infrastructure."

Christoph Kern, Principal Engineer focused on information security at Google, offers yet another take on security by design in a research publication:

"Many safety and security hazards can only be mitigated when product developers consider them during design of a product—they must incorporate mitigations into its shape and basic structure."

Security by design goes hand in hand with security by default, meaning software that is shipped to the end user with a secure default configuration.

The same three sources describe security by default as:

• Microsoft: "Security protections are enabled and enforced by default, require no extra effort, and are not optional."

• CISA: "'Secure by default' means products are resilient against prevalent exploitation techniques out of the box without added charge. These products protect against the most prevalent threats and vulnerabilities without end-users having to take additional steps to secure them."

• Google: "For many physical products, customer expectations (and in some cases formal standards and regulations) call for the product to mitigate common hazards. [..] At Google, we think that software systems should be offered with a similar mindset, with basic safety and security features included in every version of the product, and enabled by default."

None of these definitions mention AI, and none of them need to. Kern's point about building mitigations into a product's "shape and basic structure" is exactly the question teams now face with AI agents: which actions should the architecture make impossible, no matter what the model decides?

What Are the Benefits of Security by Design?

Security by design is the foundation of protection against cybercrime. It works a bit like herd immunity: the more software products are secure by design, the less likely it is that a single breach or attack damages a large chunk of the technology ecosystem.

The actual global annual cost of cybercrime is unclear. One number you might see thrown around online is $10.5 trillion in 2025, but it appears to be highly exaggerated.

So instead of the cost, it helps to look at how often serious attacks happen. Since 2006, the Center for Strategic and International Studies (CSIS) has maintained a live list of significant cyber incidents, which it defines as "cyber attacks on government agencies, defense and high tech companies, or economic crimes with losses of more than a million dollars." The list logged 57 such incidents in 2025 and 26 more in the first half of 2026. Add the countless attacks on individuals and SMBs that never make a list like this, and it's easy to see why security by design is necessary.

In many cases, security by design means preventing software vulnerabilities that have been understood for years. Jen Easterly, CISA's Director at the time, pointed out that 10 out of 13 software defects deemed "unforgivable" by security experts in 2007 were still prevalent in software.

The Common Weakness Enumeration project ranks the most dangerous software weaknesses every year. Cross-site scripting (XSS) topped the 2024 list and held the number one spot again in 2025. It can lead to multiple harmful outcomes, such as the leakage of private user data stored in cookies.

For most vulnerabilities, XSS included, the solutions are known and sometimes trivial. They just aren't prioritized in the software lifecycle, because it's not always feasible to spend extra money and development time on them, especially for new products where product-market fit isn't guaranteed.

Major tech providers have the biggest responsibility, need, and ability to implement secure-by-design principles. And when they do, the results can be astonishing. In a February 2025 blog post, Google engineers reported that their commitment to secure-by-design principles helped them reduce cross-site scripting vulnerabilities in their web apps from nearly 100 to just 1 per year.

That figure covers all of Google's applications that adopted the secure-by-design web framework, combined. Google got there by making the dangerous pattern hard to write in the first place, and the same principle carries over to AI agents.

Why Did Security by Design Become a Business Priority Before AI Agents Arrived?

Security by design became a business priority because buyers, peers and regulators started demanding it at the same time. Cybersecurity had long been treated as a cost to cut. The echoes of the "move fast and break things" mentality, and the pressure of an extremely competitive software market, backed the industry into a corner.

That was changing fast. A Cloud Security Alliance survey of 420 IT and security professionals, published in April 2025, found that 86% of organizations were prioritizing SaaS security and 76% were increasing their cybersecurity budgets. More than half of companies shared too much data externally, with employees uploading sensitive information to unauthorized SaaS apps. Oversight was difficult, identity and access management remained a challenge, and GenAI tools were expanding the attack surface.

Why Did JPMorganChase's CISO Call Out SaaS Providers?

One of the biggest cybersecurity stories of 2025 was an open letter penned by Patrick Opet, Chief Information Security Officer at JPMorganChase.

It was addressed to enterprise SaaS providers, but it sent shockwaves across the whole software industry. It begins with:

"The modern 'software as a service' delivery model is quietly enabling cyber attackers and creating a substantial vulnerability that is weakening the global economic system."

Opet goes on to describe the biggest issues. As organizations converge on a small set of leading SaaS providers, an attack on one major SaaS or PaaS provider can disturb global markets. Fierce competition leads providers to focus on rapidly releasing features in lieu of strong security, which exposes their entire client ecosystems to risk.

Modern SaaS integration, with OAuth singled out as one of the main culprits, leads to oversimplified interactions between internal enterprise resources and external systems on the internet:

• Poorly secured authentication tokens are stolen and reused

• Software providers gain privileged access without explicit consent or transparency

• Lack of transparency about fourth-party vendor dependencies expands the risk upstream

• AI and automation services amplify and rapidly spread these risks

To realize the vision of secure-by-design software, Opet calls on SaaS providers to:

• Reprioritize security to be at least as important as launching new products

• Provide continuous, demonstrable evidence that controls work effectively (not just annual compliance checks)

• Ship with secure-by-default configurations for clients

• Be transparent about risks

Finally, he proposes several modern technical solutions to make SaaS integrations safer:

• Confidential computing

• Customer self-hosting options

• Bring Your Own Cloud (BYOC) models

• Sophisticated authorization methods (beyond simple OAuth)

• Advanced detection capabilities

• Proactive measures to prevent abuse of interconnected systems

Read that list again with AI agents in mind. Stolen tokens, privileged access without consent, opaque dependencies, AI amplifying risk, authorization beyond OAuth, confidential computing... the letter reads like an early draft of the agentic AI security agenda.

How Did the Software Industry React to the JPMorganChase Letter?

Even before Opet posted the letter, industry leaders had been calling for a higher priority on cybersecurity.

In one interview, senior leaders from domain registrar Verisign pointed out that security is becoming a competitive advantage:

"Cybersecurity is evolving. It's no longer about just protecting the company from security risks; it's becoming a new source of competitive advantage. Customers synonymize security with quality and value; hence it's crucial to build security upfront into products and services, in order to gain and retain customer confidence."

Charanjit Chana, Head of Development at UK-based marketing tech company Maglabs, responded to the CISO letter on his blog, saying that prioritizing security is necessary and that the topic deserves more attention in universities:

"Patrick Opet, the Chief Information Security Officer at J.P. Morgan sent an open letter that outlines the need for a shift in how much companies focus on features over security. It is not only necessary, but refreshing.

Everything is connected now, whether we like it or not. Web development is a state of bloat that has persisted into a second decade. There's not enough crafting on the web, it's all about delivery. Often at speed at the cost of security and definitely at the cost of privacy.

[..] Universities should spend as much time focusing on OPSEC as a concept as much as they would OOP and whatever else they push students towards these days."

Amir Khayat, CEO of SaaS security provider Vorlon, wrote that the letter brought to the forefront the frustration security leaders had been feeling for years:

"His message was blunt: Get your act together. Opet's letter called out the lack of reliability, accountability, and transparency from too many cybersecurity and SaaS vendors. For many security leaders, it put words to a frustration that's been simmering for years: we're spending more than ever on tools, and yet we still can't answer basic questions during an incident."

Mike Schwartz, founder of an Austin-based cybersecurity firm and leader of the Linux Foundation project Janssen, agreed with the letter in his LinkedIn article:

"As your letter posits, we need to 'modernize security architecture.' In particular, you write that many SaaS 'integration models collapse authentication (verifying identity) and authorization (granting permissions) into overly simplified interactions.' I could not agree more. This brittle approach has proved insufficient. It creates an illusion of security which doesn't protect our institutions."

When Patrick Opet posted his letter on LinkedIn, it generated over 1,700 reactions and 150+ comments. Most commenters agreed:

• "I agree! I was surprised by how many unsanctioned SaaS apps were outside central governance at my last company — nearly 70% (almost 600 apps) of the SaaS apps the company used were not within our SSO/MFA capabilities. The convenience for random business people to adopt SaaS apps without IT or Security is out of control."

• "Industry-wide collaboration is critical, one company following best security practices isn't enough when the components and dependencies they rely on might not. The entire ecosystem is showing serious cracks under the current velocity of threat actors."

• "Important points Pat. The industry's dependency on a few SaaS platforms has outpaced the risk frameworks needed to govern them. Integration convenience often bypasses core principles like least privilege and network segmentation. It's time providers and customers realign on secure by design models instead of layering security on after the fact."

• "Well said, Pat Opet. Both powerful and timely. As both a long-time enterprise CISO and Trust Officer and now providing identity security as SaaS, this deeply resonates. The cascading fourth-party dependencies and the explosion of machine identities and AI agents, all requiring access to sensitive resources, are especially concerning."

Even Redditors were surprisingly positive. In one popular thread discussing the letter, many commenters were cynical (as usual on Reddit), but plenty agreed with Opet:

• "Great letter to share with senior leadership! This hits all the key points and is concise enough for people to actually read the whole thing. Key take away is to make the required investments in security before investing in more new features. Granted, new features = profits. BUT ... Downtime due to a security incident results in lost profits and reputational damage."

• "Couldn't agree more. We spend more time managing vendor integrations and f**k ups than warranted. Systems engineering roles shifted to 'bouncer' roles"

• "Every significant incident we've managed in the past four years hasn't been in our own back yard. It is such a pain in the butt."

From the largest organizations to small firms, the industry agreed that software needed to get more secure across the board. In the middle of it all are regulators, using their top-down power to speed up the adoption of security by design.

How Are US and EU Regulators Pushing Security by Design?

Jen Easterly, then Director of the US Cybersecurity and Infrastructure Security Agency (CISA), described the software industry's relationship with cybersecurity better than anyone:

"We're in the 'before the seat belts' era of software."

It's a clear reference to the automotive industry. Today, you don't think about seat belts; you fasten them when you get in the car. That wasn't always the case. The push to put seat belts in all cars began around 1970 across the Western world, and it took several decades to become a universal standard.

For Easterly, it was clear that software companies, just like car manufacturers in the 20th century, won't create a secure digital ecosystem unless the public demands it. Dangerous software that leads to system compromise can no longer be business as usual, and blaming users for poor security habits is no excuse. The focus belongs on the design flaws that enable poor security in the first place.

CISA bet on industry cooperation. Its Secure by Design Pledge, launched in 2024, has been signed by hundreds of companies, including AWS, Palantir, GitHub, Google and Scale AI. CISA also encourages companies to become Common Vulnerabilities and Exposures (CVE) Numbering Authorities (CNAs) to transparently disclose vulnerabilities in their own products.

In the European Union, the NIS2 directive sets the baseline for securing network and information systems. NIS2 is a legal framework to "uphold cybersecurity in 18 critical sectors across the EU," and it encourages cross-border enforcement between EU countries. It expands the scope of its predecessor, NIS1, to more sectors, like social platforms and the space industry.

NIS2 also introduced accountability for the leadership of companies that don't comply with cybersecurity risk management requirements, as a way to bring "cybersecurity to the attention of the boardroom." It strengthened the CSIRT network, which improves the flow of cybersecurity information between EU member states.

Then there's the Cyber Resilience Act (CRA), which entered into force at the end of 2024. The CRA "enhances cybersecurity standards of products that contain a digital component, requiring manufacturers and retailers to ensure cybersecurity throughout the lifecycle of their products." In other words, it's the EU's version of a push for security by design, with requirements that apply at every stage of the product lifecycle.

Part of the CRA already applies. Its vulnerability and incident reporting obligations have applied since September 11, 2026: manufacturers must report actively exploited vulnerabilities and severe incidents, starting with an early warning within 24 hours. The rest of its requirements apply from December 11, 2027.

How Does Agentic AI Change the Threat Model for Security by Design?

Agentic AI changes the threat model because the software now makes decisions and takes actions, and the instructions guiding those decisions can come from any data the agent reads. Take this simple example from Arm: "a generative AI system might suggest booking a flight, but an agentic system could search fares, use stored payment details, and complete the booking on a user's behalf."

Agents have spread quickly. In August 2025, Gartner predicted that 40% of enterprise applications would feature task-specific AI agents by 2026, up from less than 5% in 2025. An IDC forecast sponsored by Microsoft, cited on Microsoft's blog, puts the number of AI agents at 1.3 billion by 2028. Controls haven't kept up. According to Cisco's 2025 AI Readiness Index, only 24% of organizations that have deployed AI felt they had the controls in place to control agent actions.

The classic secure-by-design concerns haven't gone away. Each one has a new, harder version:

The input validation row is the one that changes the most. With XSS, the fix is to keep untrusted data from being executed as code. With agents, untrusted data is processed as instructions by design. As NVIDIA's Daniel Rohrer put it during a Linux Foundation webinar on agentic AI security, "in agents, data is your code."

That turns prompt injection into an access control problem. A July 2026 paper by 25 security and privacy researchers describes the classic confused-deputy problem in agentic form: "an AI agent granted broad privileges to function effectively becomes the access mechanism as well as the access control and a user may persuade it to retrieve data that the user was not meant to access." AWS's agentic AI security guidance makes the same point from the builder's side: agents can act "with the effective permissions of the authenticated end user," and "these permission sets may allow for mass export, modification or removal of critical data."

The same paper flags memory as a new attack surface. "An adversary who can alter or selectively shape what an AI agent holds in its memory can effectively gaslight humans or distort future AI agent behavior." And because agents talk to other agents, "a single misbehaving AI agent can trigger a domino effect that affects many parties downstream."

These risks now have a shared vocabulary. In December 2025, the OWASP GenAI Security Project published the OWASP Top 10 for Agentic Applications, which starts with agent goal hijack, tool misuse and exploitation, and identity and privilege abuse, and also covers memory and context poisoning, cascading failures and rogue agents.

What Does Security by Design Look Like for AI Agents?

Security by design for AI agents means the architecture, not the model, decides what an agent is allowed to do.

Teams that already practice security by design have a head start, because the security fundamentals remain the same. What's changed is the operating model.

How Should Teams Threat-Model an AI Agent Before Building It?

Threat modeling an AI agent starts with listing everything it can touch (data sources, tools, credentials, other agents, actions it can take without asking anyone). AWS recommends doing this early: "Conduct threat modeling during system design to implement mitigations during development. This approach reduces overall development time by identifying security requirements early in the process."

For agents, the model needs to cover more than a classic web app. The researchers behind the July 2026 paper call for a "structured pre-deployment process to understand how the system works, construct threat models, and conduct systems-level cause-and-effect analysis before sign-off rather than only after incidents occur," including risks from autonomy, tool use, non-determinism and agent-to-agent communication. The OWASP agentic Top 10 works well as a checklist for that exercise, and the NIST AI Risk Management Framework gives it an organizational frame.

The most useful output of the exercise is often a list of things the agent shouldn't do at all. Every tool you don't connect is an attack path you don't have to defend.

Why Does Every AI Agent Need Its Own Identity and Least-Privilege Access?

Every AI agent needs its own identity because you can't restrict, audit or revoke something you can't name. Cisco's Peter Bailey writes that "every agent needs a digital identity that can be authenticated, monitored, and revoked. If an agent starts behaving oddly, the system needs to be smart enough to pull its credentials immediately."

Microsoft's guide for security leaders goes further and recommends managing agent identities through "onboarding, access reviews, and offboarding for agents, managed with the same rigor applied to human employees." Some organizations already register agents in HR systems and assign each one a human manager, which forces someone to be accountable for what the agent does.

Identity only helps if permissions are narrow. In practice, that means scoped, short-lived credentials per agent and per task, rather than letting the agent inherit a user's full session. It's the agentic version of the "sophisticated authorization methods (beyond simple OAuth)" Opet asked SaaS providers for.

Which AI Agent Actions Should Require Hard-Coded Rules or Human Approval?

Any action that is irreversible, moves money, exports data in bulk, or changes permissions should sit behind a deterministic check or a human approval, not the model's judgment. The July 2026 paper describes this as "hybrid architectures that combine deterministic safety gates with non-deterministic decision-making, locking certain actions behind hard-coded rules while leaving the rest flexible." For high-sensitivity decisions, the authors suggest treating consent "as a security primitive," which means requiring authentication rather than a yes/no prompt.

AWS recommends adaptive, risk-based authentication for agent actions, and for high-risk activities, "requiring additional authentication factors or implementing a user-approval chain." Microsoft calls for "human-in-the-loop checkpoints for decisions that carry real-world consequences."

The design challenge is making that oversight real. The researchers warn that human oversight "should not be reduced to placing a human nominally 'in the loop.'" A reviewer who sees fifty approval prompts an hour will approve all fifty. Gate the few actions that matter, and give the reviewer enough context to make an actual decision.

How Do You Secure Prompts, Skills and the AI Agent Supply Chain?

You secure them the same way you secure code, because in an agent they effectively are code. AWS recommends that production prompts go through version control and pull requests so they "are subject to the same standards of peer review, automated testing, and approval processes before being deployed."

The same discipline applies to everything else an agent loads. The arXiv paper notes that a deployed agent "may combine a base model, an orchestration framework, plugins or skills, a retrieval infrastructure, user-specific memory, and sub-agents authored or operated by various contributors." Each of those is a dependency, so it belongs in your software bill of materials (SBOM), your static analysis and your review process. AWS points out that many agent frameworks are open source, "which makes supply chain security critical for system integrity."

Where Do Confidential Computing and Continuous Monitoring Fit in Agentic AI Security?

Confidential computing protects data while it's being processed, and continuous monitoring catches agents that drift from expected behavior. Both address risks that design-time controls can't fully remove.

Encrypting data at rest and in transit isn't enough when an agent decrypts sensitive records in memory to work on them, which is why the Linux Foundation webinar panelists from NVIDIA, Intel and Edgeless Systems focused on protecting data in use. Hardware-backed attestation lets one agent verify another before acting on its output. Felix Schuster of Edgeless Systems advised teams starting fresh "to make the decision to use confidential computing from day one." That's another item from Opet's 2025 list, now showing up as standard advice for agentic workloads.

Isolation has limits, though. NVIDIA's Rohrer noted that "sandboxing by itself isn't sufficient," and that the same runtime evidence should drive policy enforcement that keeps an agent within its expected workflow. Microsoft flags a related gap: "monitoring outputs, not just inputs." Most teams filter what goes into a model. Far fewer watch what comes out, and with agents, the output is often an action.

What Are the Limits and Trade-Offs of Security by Design for Agentic AI?

Security by design reduces agentic AI risk, but it doesn't eliminate it, and some of the tools are still immature. Anyone selling a fully solved agent security stack is ahead of the evidence.

The July 2026 paper reviews proposed containment measures, from sandboxing to runtime monitoring and "smart firewalls," and concludes that "none of these mechanisms yet exist at the maturity needed for confident real-world deployment." Testing has a structural ceiling too: "The inability to test AI agents as deterministic systems is a structural feature of their design rather than a limitation that better testing methods will overcome." That's why the paper argues "one-time certification is insufficient for AI agents," and why continuous evaluation has to be part of the budget, not an afterthought.

Controls can also backfire. AWS warns that "overly complex or granular access controls can lead to human error through misunderstanding or workarounds that compromise security." Confidential computing has its own boundary; as Rohrer put it, "We can't save you from yourself if you put buggy or malicious code inside the box."

Compliance is a moving target, especially in Europe. Microsoft's guide notes that the EU AI Act, GDPR, DORA and the Cyber Resilience Act "overlap, interact, and frequently conflict." The CRA's reporting duties are already live. The EU AI Act's obligations for high-risk systems were pushed back in July 2026 by the Digital Omnibus: stand-alone Annex III systems now face them from December 2, 2027, and AI embedded in regulated products from August 2, 2028. The delay moves the dates; it doesn't change what will be required. For teams in FinTech or HealthTech, that means mapping which agent features touch which regime before writing code, not after an auditor asks.

Finally, not every workflow needs an autonomous agent. Cisco observes that "many organizations are blocking agentic experimentation altogether, choosing stagnation over risk they can't measure." There's a middle path: start with agents that read and recommend, add write access one tool at a time, and keep a deterministic workflow where a deterministic workflow does the job. Sometimes the most secure agent is a well-designed form.

What Does Security by Design Require From Teams Building AI Agents Now?

Security by design was already becoming non-negotiable before AI agents. Enterprise buyers, CISA and EU regulators made sure of that, and Google's XSS results showed how much it pays off when the architecture removes a whole class of bugs. Agents make that lesson more urgent, because a design flaw in an agent can become an action someone else triggers with your credentials. The organizations that get this right treat agent permissions, identities, approval gates and monitoring as architecture decisions made on day one, and they budget for continuous evaluation instead of a one-time audit. Smaller companies once had more leeway here, but that leeway shrinks when your agent acts inside a customer's systems. Early design decisions are also the cheapest ones to get right.

If you're planning an AI agent or an agentic feature and want security built into the architecture from the start, Monterail's team can help you scope it, decide what the agent shouldn't do, and build it. Get in touch to talk it through.