Proven Shortcuts to Enterprise AI ROI

Key takeaways:

• Only 25% of enterprise AI initiatives generated expected ROI as of 2025, with just 16% scaling organization-wide.

• 72% of CEOs identify proprietary data as the critical ingredient for unlocking GenAI value.

• Most AI failures stem from rushing deployment without proper data preparation, clear business objectives, or adequate team training.

• Successful AI strategies focus on proven, low-risk use cases rather than flashy demonstrations or complex implementations.

• Modern AI product development can be done by business analysts with AI training, not PhD-level data scientists, making custom solutions more accessible than ever.

• Companies should prioritize data infrastructure investment and foster experimentation culture while accepting that 40-50% of AI proof-of-concepts might be abandoned.

Ever since ChatGPT broke into the mainstream, every executive has been pressured to integrate Generative AI into enterprise operations. But if your organization hasn’t been able to generate jaw-dropping ROI from GenAI initiatives, you might be glad to hear that it’s not just your company’s problem. It’s a global issue.

The Economist called 2025 AI’s “trough of disillusionment,” saying that companies struggle to find ways to make the technology useful (what one expert calls “capability overhang”). In May 2025, based on a survey of 2,000 CEOs around the world, IBM reported that:

• Only 25% of AI initiatives over the last few years generated expected ROI, and just 16% have scaled across the whole organization.

• Two-thirds of surveyed CEOs plan to keep increasing their AI investments, but their main focus now is defining clear metrics for ROI and leaning into use cases that support those metrics. 

• 72% CEOs see their company’s proprietary data as the key ingredient to unlock ROI from GenAI.

Another study from Accenture confirms the importance of data. Their analysis shows that, among companies that struggle with AI initiatives, 61% don’t have AI-ready data assets and 70% have difficulties scaling projects that use proprietary data.

Clearly, now is the time of reckoning for enterprises that don’t have an effective data environment and integrated, enterprise-wide data architecture. It’s an investment that may have seemed over the top in the past, but is now universally viewed as a crucial stepping stone towards creating value with generative AI.

Why Do Enterprise AI Initiatives Fail?

2023’s AI boom was so powerful that the world is still feeling shockwaves from it. One of the worst consequences of it was that many executives pushed their organizations into AI projects without the right planning or preparation. 

For industry experts that have been at the frontlines of the AI revolution, the key causes of enterprise AI project failure were:

• Jumping into AI because it was “trendy,” without first identifying a core business issue to fix.

• Prioritizing deployment speed over product quality.

• Moving forward with AI without preparing necessary resources – most importantly, without cleaning up their internal data.

• Failing to correctly estimate the time and investment needed to achieve ROI.

• Focusing on difficult use cases instead of low-hanging, least-risky ones.

• Underestimating the effort needed to move from a promising AI prototype to a production-level system delivering value at scale.

Then there’s the elephant in the room – skill issues. In January 2025, BCG reported that less than one-third of companies have upskilled one-quarter of their workforce to use AI. Accenture found that 78% of executives believe that AI is evolving too quickly for their company’s training to keep up. McKinsey discovered that 48% of employees realize they need training for faster Gen AI adoption, but many of them feel like they’re not receiving it.

The result of all these issues? Enterprise analysis from S&P Global Market Intelligence has shown that 42% of companies end up abandoning most of their AI initiatives – up from 17% in 2024. The average company scrapped almost half of their AI proof-of-concepts before they reached production. Cost, data privacy and security risks came up as the biggest roadblocks. 

It’s not all bad. Scrapping an AI PoC before it reaches production is fine as long as it’s part of a smart strategy of experimentation focused on cherry-picked use cases. 

After all, it’s a new technology ecosystem, where things change on a daily basis. The tooling, the infrastructure, even the AI models themselves are constantly evolving. Organizations that want to be AI-ready need to foster a culture of experimentation. Accepting – even celebrating – failures is a big part of it.

What Are Examples of Successful Enterprise AI Initiatives?

According to Deloitte, in order to accelerate the ROI of AI, companies should:

• focus on a small number of high-impact use cases in proven areas,

• integrate GenAI into existing processes and centralized governance to promote adoption and scalability.

A great example that fits this bill comes from SAP, the world’s largest ERP vendor. When Jared Coyle, SAP's Chief AI Officer for the Americas, reflects on their strategy, he doesn't focus on chasing the latest large language models or flashy AI demos. Instead, he emphasizes something more fundamental: "We are running a huge portion of the world's economy through our systems, and we need to bring AI to the business processes in those solutions."

This confirms the first crucial element of successful enterprise AI: leveraging your existing data ecosystem as a competitive moat. SAP's decades of enterprise resource planning data, from sales orders to purchase orders to obscure legacy fields, became the foundation for their AI copilot Joule and specialized AI agents for expense validation, supply chain management, and customer service. Rather than starting from scratch, they're amplifying what they already know about how businesses actually operate.

But SAP's approach also illuminates a second key principle: specificity over scale. Instead of deploying massive general-purpose models, Coyle's team focuses on smaller, specialized foundation models enhanced with knowledge graphs that understand business context. 

"One of the best things we've done is creating a knowledge graph, basically a graph engine that can understand, what's a sales order, what's a purchase order," he explains. This targeted approach cuts down on hallucinations and delivers what Coyle calls "boring" but highly efficient AI use cases – exactly the kind that generate measurable ROI.

All in all, SAP’s recipe seems to be combining robust data infrastructure with surgical precision in their AI applications, focusing on augmenting existing business processes rather than revolutionary transformation.

How to Accelerate the ROI of Enterprise Generative AI

Based on what we’ve covered so far, the general approach for achieving fast ROI from enterprise genAI initiatives is starting to unfold:

• Create an AI-friendly culture – encourage employees and other executives to explore genAI, but make sure to teach them how to do it in a safe way

• Clean up internal data – make sure you can build datasets for training or fine-tuning AI models quickly and securely

• Get the low-hanging fruit first – focus on low-risk and high-impact use cases for AI 

• Adopt a culture of experimentation – be prepared to abandon AI proof-of-concepts quickly, don’t try to push every single one into production by any means necessary

• Explore the genAI tooling and infrastructure ecosystem – the solutions in this space are imperfect and keep evolving, but you shouldn’t wait until everything is perfect, because you’ll miss out on the early adopter advantage and waste time on change management later on.

Organizations that have little to no experience with machine learning and AI can achieve decent quick wins from adopting packaged solutions from providers like Google or Microsoft. These don’t require costly implementation or experimentation, and offer an easy way to add features such as:

• Chatbots for customer service or internal support,

• Workflow automation,

• Real-time insight detection from large amounts of data,

• Marketing and sales support,

• Predictive analytics,

• Fraud detection,

• Coding support.

For teams that prefer a more hands-on approach, there are plenty of enterprise-ready platforms to choose from for building custom AI solutions in a safe, simplified environment:

• Platforms like Vellum, H2O, DataRobot, or the Hugging Face Enterprise Hub offer different ways to develop AI solutions without having to build everything from scratch, with features like no-code or low-code builders, pre-built components, and a wide range of tools necessary to build AI PoCs and take them to production.

• If you want to go deeper into fine-tuning models or even building your own, options range from Hugging Face AutoTrain (train and deploy machine learning models without code), AutoRAG (to power AI applications with proprietary data through retrieval-augmented pipelines), Weights & Biases (for building AI models from scratch), or Neptune (experiment tracker for foundation models).

Even if you think that developing custom AI is beyond your organization’s capabilities, the reality is that it’s much easier than it has ever been. The expertise requirements have shifted radically in the last few years.

Traditional AI development required PhD-level data scientists, ML engineers with years of experience, specialized infrastructure teams, months of model architecture design, and extensive MLOps expertise.

Modern platform-based development needs business analysts with AI training, developers with prompt engineering skills, platform-specific certification taking weeks not years, pre-built model fine-tuning capabilities, and automated MLOps through platforms.

Roadmap for Unlocking ROI From Enterprise AI

The next action you should take depends on the current stage of your company’s AI readiness. This roadmap will help you visualize the necessary steps to expand your AI capabilities whether you’re starting from scratch, or further along in your AI journey.

Deploy productivity tools if you haven’t done it yet – encourage the usage of tools like ChatGPT, Claude, or Cursor to build organizational confidence in AI capabilities, and offer a safe way to use them.

Establish governance framework – create an AI committee, develop usage policies, implement security protocols, and define success metrics. Early governance prevents costly mistakes and regulatory issues.

Focus on proven high-ROI use cases – deploy customer service chatbots, implement content generation for marketing, establish basic fraud detection rules, or improve inventory forecasting. These build momentum while delivering measurable business value.

Invest in data foundation – implement data quality monitoring, establish data governance processes, integrate data silos, and create centralized data repositories. Quality data enables all subsequent AI initiatives.

Experiment and scale successful pilots – identify opportunities where AI can positively impact your operations, create proof-of-concepts, quickly abandon those that don’t show promise, and scale successful PoCs when they deliver desired results.

Build AI-native capabilities – develop proprietary datasets, train custom models for core business functions, implement agentic AI systems for complex workflows, and create AI-driven business models.

Enterprise Strategy to Accelerate ROI from AI

The novelty of genAI has worn off and reality has set in. It’s time to get down to the brass tacks. Enterprise investment in AI isn’t slowing down, but the rules of the game have changed, with executives everywhere prioritizing ROI above all else.

Technology is only one part of the equation. Organizational readiness needs to be established with AI literacy training and AI-ready infrastructure. Then, executives need to identify the right problems to solve with AI. This way, it’s easier to decide on the technology stack, whether it’s a general-purpose API for broad tasks or a specialized AI model for core business functions.