You can’t FinOps your way out of cloud AI costs

Scan any industry publication and the same story emerges: Cloud costs are out of control, and businesses are scrambling.

The technology that everyone is betting on to boost growth is making the problem worse: Some 55% of respondents to a recent PricewaterhouseCoopers International Ltd. survey. they say they have yet to see a profit from artificial intelligence tools.

A commonly proposed fix is ​​FinOps that focuses on better dashboards, stronger governance and smarter forecasting. But waste continues to grow. Enterprises are still burning more than a quarter of their cloud budgets, and while tools measure the bleeding, they don’t stop it.

What is missing is an honest look at what actually drives the bill. I spent two decades at Microsoft Corp. and SAP SE I watch companies develop the physical parts of the stack while the underlying engine struggles. Cloud costs today are a similar issue.

Cloud debts do not rise in a vacuum. They map to data processing costs. AI has transformed data processing into something cloud architecture was never designed to handle.

Why AI is breaking the cloud model

Before AI took over boardrooms, businesses spent a decade building cloud computing. Data sets were organized, and workloads moved in batches on predictable schedules. Data processing was manageable because the economics worked.

The AI ​​blew that up. Clustering has progressed, sampled data into complete datasets and planned operations into real-time pipelines over multimodal data. The volume, frequency and complexity of data processing has changed, but the underlying architecture has not.

Here’s the part that no one talks about: Even after spending a lot of money every year, most businesses process only a small part of their data in the cloud because using everything there can open up budgets too much. They pay more for the cloud and work harder, but much of the data that is really needed for AI remains untouched.

FinOps maintenance is out of reach

Companies bending the cost curve aren’t doing it with FinOps; they prepare the data processing layer.

Today’s engines are built around similar clusters of central processing units, but modern infrastructure includes CPUs, graphics processing units, arrays of programmable gates and custom accelerators scattered across the cloud. The software has not been caught. Workloads still run on one-size-fits-all setups that can’t move tasks to the right hardware, leaving expensive accelerators sitting idle while CPU clusters max out.

GPUs rip through certain tasks 10 to 100 times faster than CPUs, but only if the software knows where to send the work. When an enterprise data processing engine takes CPU parallelism to a different world, they pay for next-generation hardware just to get legacy performance.

The solution is to correct that mismatch. Rebuild the foundation of what AI needs so workloads can be moved to logical hardware. The result is that costs are significantly reduced. I have seen a large e-commerce platform that processes half a petabyte of daily costs cut by 80% with no code changes and no migration. A social network that serves 350 million users has cut costs by 50% using the same pattern.

Which actually works

FinOps has a role, but dashboards, management and forecasting are tools to fix a working model, not fix a broken one.

As long as AI pipelines run on infrastructure designed for cluster analysis, costs will rise regardless of how strong governance is. You can predict it, you’re a dashboard and provide cost centers and regression teams, but the underlying engine is still wasting money.

Businesses solving the economic data problem can process complete data sets at a cost that doesn’t require quarterly budget battles. Others will continue to watch costs rise while returns shrink, staring at FinOps dashboards that show exactly where the money went but don’t tell them how to hang on to it.

Photo: kalhh/Pixabay

JG Chirapurath is the president of DataPelago Inc. and former vice president at Microsoft Corp.’s Azure cloud. He wrote this article for SiliconANGLE.