Most AI deployments measure what's easy to count, not what actually matters.
3 minutes de lecture
This is Part One of Making Every Conversation Count, an ongoing video series featuring 8x8 CEO Sam Wilson and Liz Miller, VP and Principal Analyst at Constellation Research. Miller covers customer engagement, CX strategy, and AI-driven business outcomes.
When customer success organizations are shopping around for an AI vendor, the conversation is usually built around outputs: deflection rates, handle times, and tickets closed. And that solution is priced accordingly: the payment is for delivery of the exact metrics by which the solution measures its own quality.
It’s like going to an expensive restaurant that’s become popular on social media for an over-the-top dish. By the restaurant’s standards, they slapped some caviar on a fettuccini alfredo and charged you $60 for it. You got a showy dish with an expensive ingredient. Did you actually enjoy the food or your dining experience? Doesn’t matter; that’s not what the 60 bucks was for.
That’s the crux of the issue: organizations that are “doing AI” are paying for outputs rather than business outcomes. A lot of the time, they’re not even pausing to consider which specific business outcome they want from the new tool.
Leaders are paying to add that lump of caviar on top of the dish, rather than figuring out whether the dish actually needs it or if it makes the food taste better.
In part, it’s a data issue. It’s hard to optimize for outcomes when you don’t have enough information to rank them in the first place. Deploying AI in this situation is not an accomplishment in
and of itself. A lot of solutions may look like they’re delivering value, but they’re largely just for show.
It’s no surprise, then, that according to a 2025 survey from Orgvue, a workforce and organizational design platform, 27% of enterprises admitted they still didn’t have a clear AI strategy. Nearly the same percentage said they weren’t even sure which teams and employees would benefit from its implementation.
And if we’re talking hard numbers, a McKinsey report notes that only 39% of companies that made AI investments have seen a measurable upshift in revenue as a direct result.
The AI gold rush has snowballed in part because of this fear of missing out; enterprises are leaping on AI here and AI there because they’re worried about falling behind. But if all they do is buy solutions for outputs instead of shifting to outcome-based measurement in their evaluations, then they’re going to fall behind anyway.
When it comes to customer success, many are forgetting that a conversation lives in context. Thinking in terms of customer journeys is nothing new, of course, but that’s exactly where we get back to the core point. One-off customer interactions rated as good or bad are outputs.
So many AI solutions don’t account for the conversation itself, and even more importantly, how the relationship between a customer and a business develops over time. Improving that relationship, understanding how it impacts retention, that’s an outcome to aim for.
And finally, if you’re adding AI to your organization and think, wait, have we accounted for the human(s) that are a part of this workflow? Did the customer actually get what they needed? Did the agent actually benefit from the tool? If it’s a no, then it’s a great sign you’ve bought the wrong thing.
Stay current on what matters in CX and IT. Subscribe to our LinkedIn newsletter for regular analysis on the decisions shaping customer experience, technology, and AI. Clarity you can act on.
