Who Gets to Judge AI's ROI?

Ever wonder who really gets to decide if AI is worth the investment? Well, it’s not just about the tech itself. Byrne Hobart points out that the true value depends on who’s measuring it — and that can be pretty complicated. Companies often focus on direct returns, but Hobart suggests we should also consider how AI creates new agents and complements, changing the game entirely. Now, here’s where it gets interesting — these agents can go halfway dark, making it tougher to track their true impact. Plus, Hobart highlights that out-of-sample testing, or seeing how AI performs in new situations, is crucial but often overlooked. And get this — feedback loops can inflate AI’s perceived ROI because the system self-reinforces its value, even if it’s not actually improving. So, what does all this mean for you? It’s not just about the AI itself, but about understanding who’s judging its worth and how they’re doing it. Byrne Hobart reminds us — question the metrics, because the true ROI might be hiding in plain sight.