substack.com - The Senior Manager’s Guide to AI From Hype to Real Impact - Part 3
created: 2026-02-04 description: “Practical playbook on Scaling AI across the big organisation”
Section titled “created: 2026-02-04 description: “Practical playbook on Scaling AI across the big organisation””Practical Playbook on Scaling AI across the Big Organisation
Section titled “Practical Playbook on Scaling AI across the Big Organisation”This is Part 3 of my Senior Manager’s Guide to AI. In Part 1 we started with personal use. In Part 2 we activated your team. Today we go org-wide. As a senior manager you must influence up and across. Level 4 is Replicate best practice. Level 5 is Kindle the AI-first transformation.
If you are finding this useful, share it with a senior leader in your organisation who is driving AI change. I love learning from others. 🙂
Level 4: Replicate Best Practice across the Org
Section titled “Level 4: Replicate Best Practice across the Org”Once you’ve built the flywheel in your team, it’s time to expand it across the org. This is where “how to” artifacts earn their keep. At Level 3 you were doing. Now you are teaching.
Imagine your team solves a specific problem, say an agent that resolves trouble tickets. The solution may be local. The process is portable. Document the shortcuts and dead ends, record a concise demo, publish the prompt set. Teach people how to fish.
Early on, tools rarely work out of the box. I keep hearing from colleagues, “It should connect to the MCP server, but I can’t make it work,” and “I tried Bedrock, but my permissions block me”. The nitty-gritty “how to” guides are gold at this stage: prompt libraries, short “build an agent” videos, “install this tool” wikis. Share and borrow. The whole org will move faster.
One team creates a proof of concept. A second ships a simple solution to production. A third team maps AI-able opportunities and builds a real roadmap - rather than throwing stuff at the wall like everyone else.
At this level, focus on three things. First, properly ring-fence some resource for AI. Second, set goals and KPIs. Third, invest in security and quality control. It is no longer play. Measure what you get and protect how you ship.
Ring-fencing resource does not mean “everyone gives 10% of their time”. Make AI someone’s day job. One named owner with a roadmap beats 50 people at “ten percent”. At Amazon I have seen senior leaders reassign a 12-person team out of the 500-people org to focus purely on AI transformation. The other 488 still upskill and innovate, but without a dedicated core the transformation stalls. For team design read the book The Other Side of Innovation by Vijay Govindarajan and Chris Trimble.
Second, goals and KPIs. Early metrics can be simple: tool adoption, number of AI-backed launches, onboarding time reduced, time to first contact resolution, SOPs automated, time from idea to launch. Choose by context. Some orgs skew to productivity, others to product innovation. As you ship the first few wins and learn about adoption and maintenance costs, refine your goals. AI gives quantity.
Speed is cheap now. Quality is not.
A 250 words per minute typist who types gibberish is still useless.I can now ask my team to write a PR FAQ for an ideation session tomorrow - the speed expectation has changed. Pre-AI PR FAQ draft took two weeks, now producing the artefact itself takes two hours. But, unless the team has already spent time researching the customer problem in detail (the aforementioned two weeks), the output will not be very good. We need to set the standard and the expectations on quality.
A prototype is not a product. Set a minimum bar: data classification in prompts, human in the loop for anything customer-visible, preand post-launch evaluations, and a clear incident path. Then ship with proper security. Use your internal experts or hire the right ones.
At this level, the same as before, your job as a leader is to broadcast the pain, the solution, and the benefit. In Kotter’s change management process from my last post step 7 is “sustain acceleration”. Share wins and the learning behind them. Once ROI appears, acceleration becomes common sense.
The hallmarks of level 4:
You reused two playbooks from adjacent teams and scaled them.
You automated a handful of repetitive SOPs with audit trails.
You shipped at least one AI feature to real customers with measurable lift (faster, cheaper or better than before).
How to level up:
- Stop only “saving time”. Redesign one end-to-end process with AI-native mindset.
- Upskill your leaders in problem framing and human-agent orchestration.
- Celebrate experimentation and failure, not just wins. You fail and learn many times before you start winning.
Level 5: Kindle AI-first Transformation
Section titled “Level 5: Kindle AI-first Transformation”This level is where AI-first transformation will start to happen. What will it look like?
Start with real customer problems and follow the pull.
Our life flows are already changing.
My cleaner talks to ChatGPT about second world war history as she goes about her business.
My GP says that ChatGPT is replacing Google in diagnosing his patients.
I already shop using AI instead of Amazon-own website. Because the experience is better - and they haven’t even started improving it!
Imagine this scene at bedtime.
One child wants me to buy them rainbow socks “right now”.
Old flow: twenty minutes of website scrolling and negotiations. By the end we forget what we came in for.
New flow: “ChatGPT, find me rainbow socks, 80 percent cotton or more, size EU 27, on Amazon.de.” Twenty seconds later: three options, a yes, and checkout. Back to story time!
Companies and roles will start changing too. Let me tell you two startup stories for inspiration.
First one is Every.to. They are a 15 people startup with $1.3M ARR and 2 developers. They lean on AI-assisted coding. Even more interesting is that one of these developers wasn’t an engineer before joining Every, but a generalist. They say that Claude Code does most of the code writing work, they orchestrate the agents and audit the results.
Watch the video here for more:Uncharted | Startup growth without the fairytales
[
Mental model 🧠…
Why is this amazing? Well, smaller teams can build more. Becoming “more technical” now pays off even if you do not write code yourself. Describe the outcome, let the agent write code, audit the result.
A small example: I struggled with a Q Command Line Interface (Q CLI) task the other day. After two failed tries, Q asked for the wiki I was following and fixed the issue itself in seconds. It felt like magic.
Second story is about Mercor. Mercor is a marketplace for domain experts, i.e. lawyers, doctors, investment bankers, consultants. AI labs hire experts to teach AI models “what good looks like” through reinforcement learning. Mercor went from zero to $400M in revenue in 16 months. What’s amazing is that based on training from these experts and scoring by other experts, the models of today (October 2025) are achieving 60-70% test scores in these domains. They still require substantial human oversight, but - if AI performance in chess and Go is any predictor - they will get to 100% pretty quickly.
Watch this story on Lenny’s podcast:Lenny’s Newsletter
[
Also on Spotify and Apple Podcasts…
What this tells me is that many experts will spend part of their future teaching and scoring models.
Marc Andreesen wrote in 2011 that software is eating the world. AI will too.
Technology development cycle goes like this:
- define a problem that you want to solve
- precise the requirements of what success looks like / and what failure looks like
- build the solution
- test it
- launch, measure, improve
Building the solution has been a bottleneck in technology for a while. And this bottleneck is shrinking. Big companies are slower, but we can already see that in the internet/startup ecosystem, not tied by legacy infrastructure, the focus shifted to “evals”. Evals will eventually get automated like software testing had been, and the work will shift upstream: to the precise problem definition.
The best senior leaders I have seen are really good at three things: 1) simplifying 2) defining the problems precisely and 3) focusing their teams on achieving results. We are all getting promoted to being these senior leaders real soon. Isn’t it exciting?
The hallmarks of level 5:
You are able to achieve more than you thought possible six months ago.
Roles have shifted. What you hired for and what people do now overlap only in part.
Your org runs on layers of agents: doing work, orchestrating, evaluating and building other agents.
The future is here. Welcome.
Interlude - Working backwards
Section titled “Interlude - Working backwards”We walked the SPARK ladder step by step. This is a good way to do. But execs are also impatient. Do you really have to start at the bottom? Use their favourite move and solve backwards from Level 5.
Pick one AI-able process, like vendor onboarding. It currently takes 21 days because it requires touch points with multiple teams, including legal teams on both sides, some entries in a three different systems, etc etc.
Set a goal to cut this process down to 1h. (Really? Yes.)
Get a group of 3-5 people in the room who deal with the current process and let them huddle it out on a whiteboard. They will need to cut it to the bone, examine the value of each step and come up with creative use of agents.
The end result may get you down to 2 days, not 1 hour, but you still win.
The trick with this new goal is that it needs to be aggressive enough to throw out the old construct and design a new one from scratch.
In fact, managers have been using this method for a long time. Elon Musk has an “idiot index” metric where he divides the cost of the final product by the cost of raw materials and if it is >100 then he questions the final product and makes his team to simplify and redesign it.
AI is a perfect opportunity to look at bottlenecks with a new lens. Use it.
In summary:
- Start by experimenting yourself and learning the tools.
- Partner with others to share discoveries and raise collective skill.
- Activate your team by creating mechanisms and expectations for daily AI use.
- Replicate best practices across orgs, formalize learnings, and set measurable goals.
- Kindle the AI-first mindset: redefine problems, reimagine processes, and rebuild how work gets done.
Each stage compounds on the last and helps you get to tangible results.
Over to you: How far along are you on your AI journey? What’s been your biggest unlock, or your biggest bottleneck? Reply to this email or leave a comment to share your story. Your insights might help another leader move faster.