A Board Director’s Guide to Measuring What Actually Matters in AI Investments
In the rush to embrace artificial intelligence, many organisations have fallen into the trap of measuring success by the wrong metrics. Board papers showcase impressive statistics – number of AI pilots launched, and processes automated, millions invested in AI talent – yet these figures mask a troubling reality: they tell us little about actual business value. As both a board director and someone who has led teams of AI researchers and engineers, I’ve witnessed firsthand the consequences of this misalignment. Companies celebrate AI milestones while shareholders question returns. Technology teams report high adoption rates while business leaders struggle to identify tangible improvements. This fundamental disconnect between AI activity and business impact too often leads to substantial investment with disappointing returns, creating AI initiatives that consume resources but fail to deliver meaningful competitive advantage or shareholder value.
The Problem with Current AI Metrics
AI initiatives are frequently evaluated by metrics such as:
- Number of AI pilot projects launched
- Percentage of processes using AI
- Speed of AI deployment
- Size of AI teams
While these metrics offer comfort that “something is happening,” they reveal nothing about whether AI is meaningfully impacting the business. They measure activity, not outcomes.
Shifting to Impact-Based Measurement
To truly understand AI’s value, organisations must measure its impact on core business objectives. This requires a fundamental shift in evaluation frameworks – moving from technology-focused metrics to business outcome metrics. Forward-thinking boards are now demanding evidence that AI initiatives directly strengthen competitive advantage, enhance customer value, or improve operational efficiency. This evolution in measurement approaches separates organisations merely experimenting with AI from those transforming their business through it. The most successful companies establish clear linkages between AI capabilities and measurable improvements in strategic, financial, and operational performance indicators.
Financial Impact Metrics. The most direct measures link AI to financial performance:
- Revenue growth: Has AI demonstrably increased sales, enabled new market entry, or improved customer retention?
- Cost reduction: Beyond the headline automation savings, are there measurable reductions in error rates, rework, or overhead costs?
- Return on AI investment: For every dollar invested in AI, what is the quantifiable return?
Research from McKinsey suggests organisations that tie AI investments directly to business outcomes achieve up to 3-4 times greater ROI than those focused primarily on technological advancement.
Operational Impact Metrics. AI should measurably improve how the business functions:
- Process efficiency gains: Beyond simple time savings, measure improvements in throughput, quality, and consistency
- Decision quality: Evaluate whether AI-augmented decisions outperform human-only decisions over time
- Resource optimisation: Measure improvements in resource allocation efficiency, whether human capital, inventory, or other assets
Strategic Impact Metrics. The most valuable AI applications drive strategic advantage:
- Customer experience enhancement: Measure improvements in customer satisfaction, loyalty, and lifetime value
- Innovation acceleration: Track new products or services enabled by AI capabilities
- Market adaptability: Assess improvements in responsiveness to market changes and competitive threats
Implementing Effective AI Measurement
1. Start with Clear Business Objectives
Every AI initiative should begin with explicit business goals, not technological aspirations. The board should insist on understanding what specific business problem each significant AI investment aims to solve.
2. Establish Meaningful Baselines
Before implementing AI, document current performance thoroughly. This creates the foundation for meaningful before-and-after comparison.
3. Design Controlled Experiments
Where possible, implement AI in ways that allow for proper A/B testing against traditional approaches. This scientific method brings rigour to impact assessment.
4. Look Beyond Direct Effects
The most significant AI benefits are often found in second-order effects. For example, AI chatbots may initially be measured on cost savings, but their greatest value might emerge from improved customer insights that drive product development.
5. Measure Human-AI Collaboration
The most successful AI implementations often enhance human capabilities rather than replace them. Measure how human-AI teams perform compared to either humans or AI systems operating independently.
The Board’s Critical Role
As directors, our responsibility is to ensure that AI investments deliver actual value, not just technological advancement. This requires:
- Challenging the executive team to define clear success metrics before approving significant AI investments
- Ensuring regular reporting includes impact measures, not just adoption statistics
- Allocating resources based on demonstrated impact rather than technological promise
- Setting the cultural tone that values outcomes over technological novelty
Conclusion
The organisations gaining competitive advantage through AI aren’t necessarily those with the most projects or the latest technology. They’re the ones rigorously measuring how AI transforms their business performance through enhanced decision-making, operational excellence, and customer value creation.
Shifting focus from adoption metrics to impact metrics enables boards to ensure AI investments deliver genuine value rather than merely keeping pace with technological trends. This approach not only improves return on investment but also builds organisational capability to deploy AI strategically rather than reactively. When boards demand evidence of impact, they naturally drive more thoughtful AI implementation and foster a culture of outcome-oriented innovation.
The question we should be asking isn’t “How much AI do we have?” but rather “How is AI changing our business for the better?” Only by answering the latter can we truly understand the value of our AI investments. As stewards of organisational resources, board directors have both the responsibility and opportunity to reframe AI conversations around business impact – elevating AI from a technological curiosity to a strategic imperative with measurable returns.
The most successful boards will be those that can distinguish between AI activity and AI achievement, between adoption statistics and business transformation. In doing so, they’ll guide their organisations toward AI investments that don’t just impress with their technical adoption, but deliver lasting competitive advantage going forward.
About Gary Morgan: Gary Morgan is an experienced board director, chief executive, consultant, and corporate advisor with extensive experience in strategy, innovation, and growth across various deep tech sectors including sustainable energy, health tech, aged care, and information security. He is a Fellow at the Governance Institute of Australia and serves on the Griffith University Industry Advisory Board for the ICT School. Gary has co-authored papers and reports published in entrepreneurship and medical journals.
Acknowledgment: This article was crafted with the assistance of AI technology.
Sources
Deloitte (2024), “The State of Generative AI in the Enterprise. 2024 year-end Generative AI report” https://www2.deloitte.com/us/en/pages/consulting/articles/state-of-generative-ai-in-enterprise
McKinsey Digital (2023), ” The economic potential of generative AI: The next productivity frontier”https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/the-economic-potential-of-generative-ai-the-next-productivity-frontier
World Economic Forum (2025), “How AI’s impact on value creation, jobs and productivity is coming into focus”https://www.weforum.org/stories/2025/01/how-ai-impacts-value-creation-jobs-and-productivity-is-coming-into-focus/
Nature (2023), “Artificial intelligence-driven scalability and its impact on the sustainability and valuation of traditional firms” https://www.nature.com/articles/s41599-023-02214-8
Harvard Business Review (2022), “Why Companies That Wait to Adopt AI May Never Catch Up” https://www.zinio.com/au/publications/harvard-business-review-onpoint/3079/issues/508128
Boston Consulting Group (2025), ” From Potential to Profit: Closing the AI Impact Gap” https://www.bcg.com/publications/2025/closing-the-ai-impact-gap
English