Adoption is activity. Transformation is a goal.
Tech companies are now tracking and enforcing AI use. Why measuring adoption is the wrong metric, and what the best leaders are doing instead.
Tuesday was a big day for me: I helped orchestrate Charter’s first San Francisco-based event, our Leading with AI Summit. It was my mom’s birthday and, yes, I gave her a shout-out from the stage. I also got quoted in a WSJ story about major tech companies (Google, Meta, Amazon, Salesforce) moving past encouraging AI use to tracking it, scoring it, and tying it to performance reviews. I stand by what I said: these companies are spending billions building AI tools, and if they can’t demonstrate ROI inside their own walls, it’s a lot harder to sell them to customers.
WSJ article by Katherine Bindley and Katherine Blunt
But we need to get past “carrots and sticks.” There’s a massive gulf between growing adoption numbers and real transformation. In the two weeks since Charter’s Leading with AI Summits — first in NYC, then SF on Tuesday — I’ve heard enough from leaders actually doing this work to know the difference matters enormously.
What we consistently heard: adoption is an activity metric. Transformation shows up in outcomes, for employees and businesses alike. The leaders seeing real results have stopped confusing the two.
The problem with measuring the wrong thing
Microsoft’s Katy George said it as plainly as anyone at the NYC summit: “We used to pay attention to adoption, now we just pay attention to performance.”
When you measure adoption, you get people using AI to generate activity that looks like progress, but it’s the same issue as measuring office attendance at the end of the day: you’re moving a metric, not an outcome.
When you measure performance, you start asking harder questions: Is the work quality better? Are customers happier? Is the business growing?
Many organizations are tracking usage dashboards and AI competency scores. That might be a useful measure of group progress, but some are factoring them into performance reviews. At its worst, this is the equivalent of monitoring keystrokes or mandating badge swipes: it signals surveillance, not trust, and it gets you compliance theater rather than genuine capability building. People perform to the metric, but they don’t necessarily do more.
As Guild COO Jonathan Marek put it in New York: “Don’t teach AI. Teach how to improve the business using AI.”
What outcomes actually look like
I’ve been talking with Brandon Sammut at Zapier since last fall, and he shared in SF his team’s experience with customer support. After training, time for experimentation and a lot of support for the CS team, Zapier saw a 50% reduction in average ticket handle time, customer satisfaction scores went up, and employee engagement rose 20 to 30 points on nearly every measured topic. That last number is the one most companies aren’t tracking at all.
Brandon Sammut, Hannah Pritchett and Charter’s Jacob Clemente
Anthropic’s Hannah Pritchett offered up a memorable story about JFK visiting NASA and asking a janitor what he does there. The janitor’s answer: “I’m helping send a man to the moon.” Pritchett’s point for their CS team is the same: their job isn’t to close tickets, it’s to help customers. That also opened up CS agents to use AI tools to learn and take on new skills: qualifying leads and fixing bugs, not just identifying them. Skills that will help them advance their careers, as well as make customers happier, faster.
Walmart CPO Donna Morris brought this to the frontline: Walmart store associates who speak English as a second language are using real-time AI translation tools to communicate with customers. “That’s a real unlock,” she said. Simple, specific, and meaningful to the people doing the work.
Learning (and change) take time
In SF, Helen Lee Kupp, founder of the non-profit Women Defining AI ran a real-time audience poll and asked the room what their biggest barrier to using AI more actually was. The answer wasn’t fear or lack of training, it was time.
BCG’s Gabriella Rosen Kellerman in New York said she sees this broadly being true: “The single most important pain point isn’t ‘AI is scary, I’m worried for my job.’ It’s ‘when am I going to find time for this?’”
It’s nearly impossible to see how leaders expect people to learn new ways of working while simultaneously delivering ever-increasing existing workloads: more priority projects, de-layered organizations, and layoff fears compound stress and reduce time and willingness to even test new ideas. Something has to give, and the leaders seeing real results are the ones deliberately creating that space.
Chief people officer Barb Cadigan shared that Affirm paused all 950 software engineers from non-essential work for a full week: learn to use agentic coding tools, participate in demos, and submit something you built by Friday. “We’re intentionally pausing,” she said, “because as we’re flying the plane, we need to be figuring out how to build it and rebuild it.” Her people team immediately asked to copy the model.
For Box employees, CPO Jessica Swank describes structured certification programs, added workshops, Friday forums, and is making AI training mandatory. But that’s paired with a focus that’s beneficial to employees: “How do we take away the drudgery, how do we take away the yuck?” as Swank put it. The training isn’t an added burden, it’s part of the path forward.
Swank also made a governance point worth underscoring: being the “chief work officer” is everyone’s responsibility, not just the CHRO’s or CIO’s. Swank, her CIO, and COO jointly lead the company’s AI transformation office. At Workday, CIO Rani Johnson chairs an AI executive committee that includes every business function at the most senior level.
Rani Johnson, Jessica Swank and me
What matters are human skills
Johnson also said something that stuck with me in the prep and on stage. Three years ago, her infrastructure team had the most critical skills in IT: deep technical expertise, systems architecture knowledge. Now? “The most critical skill set are people with consultative skills: those that can bridge between the technology and the business.”
Workday has responded by building champions across IT and HR, creating “Everyday AI” programs to make experimentation feel safe rather than threatening, and measuring skill development alongside business outcomes.
They’re working to create builders: people who use AI to change how work gets done. Not just people using the tools as a search replacement, image generator or editor
Kupp’s framing: “Building isn’t about coding. It’s about turning one-offs into systems, repetition into automation, and implicit thinking into explicit instructions.” The distance between “I have a problem” and “I have a fix” gets reduced. That’s accessible to anyone willing to engage. But it requires nurturing curiosity, not rewarding activity.
Leading from the front
Airbnb’s Iain Roberts, global head of people and culture, demonstrated this from the C-suite. One weekend, he built a team gathering tool using AI: something that previously would have required a product spec, waiting in an IT queue, and probably months of time. He showed his willingness as the CHRO to put in the work, not just the benefit of skipping the queue. His take was that we need “leaders building with tools, not just delegating the learning.”
Which brings us to Stanford economist Nick Bloom’s research which shows that 69% of C-suite executives are using AI less than an hour a week, and 28% are not using it at all, which strikes me as insanely low given how hours they spend weekly talking and hearing about AI. (I’m covering Nick’s insights in more depth next week in Charter!)
You can’t build a culture of AI adoption, let alone experimentation, if leadership isn’t willing to try, fail, and try again themselves. As Zapier’s Sammut put it: “If we’re going to lead our teams to adopt new ways of working, we darn well better be leading from the front.”
Opportunity not fear, trust not monitoring
Klarna CEO Sebastian Siemiatkowski may be characterized in the press at times as about displacing humans with AI, but his evolution on AI is a lot more nuanced and instructive: even after reducing headcount 50% through attrition and growing revenue per employee from $300K to $1.3M, he frames the remaining employees’ situation in terms of expanded scope and 60% higher compensation, not just productivity metrics. The employees who stayed took on more; they got more in return.
The examples from Klarna, Zapier, Anthropic, Box, Workday, Airbnb and the rest all start to follow similar patterns: we will give you real skills, real career growth, and work that matters more. That’s far more important than adoption rates being a factor in performance evaluations (shades of “this will go on your permanent record”) or, worse yet, used as a threat around layoffs.
I loved how Robert David responded to me on this theme overall: “AI adoption should be driven by empowering employees to work smarter and grow their careers, not by surveillance or compliance mandates. Because when people feel supported rather than coerced, you get genuine transformation, not just checkbox behavior. The real leadership challenge isn’t enforcing AI use—it’s building the trust and capability that makes people want to embrace it.”
What to do starting this week
Stop measuring AI adoption as a primary metric. Track it if you want a leading indicator, but make clear both internally and publicly that what actually matters is performance: customer outcomes, quality, speed, innovation, and employee skill development. If your dashboards show 80% adoption but customer satisfaction is flat, you’ve built a compliance program. Congrats?
Create protected time for learning. Affirm paused 950 engineers for a week. Microsoft built an AI-first incubator insulated from regular work demands. Box runs mandatory training programs with real dedicated time. You cannot ask people to rewire how they work while also demanding they maintain full (often increasing) output. The initial productivity dip is the investment, not the failure.
Redefine the job before you roll out the tools. Anthropic’s Hannah Pritchett asks her customer service team to think about making customers happy, not closing tickets. Walmart’s Donna Morris asks store managers to use AI to build better business plans and gives front-line workers translation tools that unlock entirely new customer interactions. The question isn’t “are my people using AI?” It’s “what can my people do now that they couldn’t do before?”
Lead by building, not just by mandate. If 69% of C-suite executives are using AI less than an hour a week, the message going down the organization is obvious. Iain Roberts built a product over a weekend. Brandon Sammut shares publicly when his own experiments fail. Hannah Pritchett gives her team permission to experiment without waiting for permission. The leadership unlock is modeling, not enforcement.
The companies mandating AI use will get their adoption dashboards and compliance check-marks. The companies investing in employees will get better future outcomes.
What’s your take on measuring and rewarding adoption?