Skip to content
Phil's Library

Notes on experimentation from the 2024 Microsoft Partner Forum

Hi folks - 

Last week I was able to join the annual Partner Forum. This is an internal event for all MSFT Partners/VPs that’s designed to help leaders across the company discuss and meet the company’s most pressing challenges.

This year’s event was focused on “Leading in the AI era - excellence in the core business and fostering innovation”. Since some of the messages were so clearly aligned with the work we do, and since anyone in this company can be a leader - regardless of org position! - I thought it would be useful to share some thoughts.

I don’t have a copy of the materials yet, so in the meantime I scanned some of the ‘pocket guide’ they shared with attendees. Please look through this, with a particular focus on the later sections around adaptation and innovation success, and the approach to experimentation and risk-taking: Partner Forum 2024 - Leading Ambidextrously - Pocket Guide.pdf

As you read the slides, you’ll frequently see the term “ambidextrous leadership”. This is about being able to walk and chew gum at the same time 🙂. About ensuring that we demonstrate execution excellence in the ‘steady state’, day-to-day activities that keep the lights on, while also showing adaptability and the ability to experiment to find the new waves of innovation that will keep MSFT at the forefront of the technology industry long into the future. That balance is difficult - and we had some great small group discussions about how different teams across the company solve for this - but also a critical skill for many of the leaders across MSFT who are juggling the needs of existing huge businesses while being told to find the next big thing with AI.

OCTO, however, operates in a somewhat unorthodox way when compared to that standard operating model. It’s true, of course, that we need to demonstrate excellence in our regular tasks. That’s table stakes. But our day jobs skew so much more heavily towards the “adapt and innovate” end of the spectrum that I’d like us to spend some time as a team thinking about learning and the growth mindset, the role of experimentation and intelligent failure, and context and consequences.

For some background it’s worth watching this short video from HBS’s Amy Edmondson, who also presented at the event: It doesn’t matter if you fail. It matters how you fail. | Amy Edmondson.

In short, a key area I’d like us to work on as a team is our ability to design, execute, and learn from experiments. Since there’s so much ambiguity in thinking about the future, a framework for good experimentation is a key tool for chipping away at that uncertainty and clarifying which areas of technology we should pursue.

And when I say ‘good experiments’, I mean a very specific thing. I mean designing a set of scoped, time-bound activities that:

  • help us learn specific, valuable, new information;
  • are worthwhile (taking into account the scale of MSFT’s business);
  • are predicated on a reasonable set of expectations / a hypothesis about what might happen;
  • are as small as possible to deliver enough the required information;
  • provide a way to scale the learnings to all of our stakeholders; and, critically
  • allow us to take advantage of our team’s unique superpower: namely, that we’re not bounded by the collective intelligence of MSFT! Being a partner team in MSFT’s Office of the CTO comes with a unique gift: we can take advantage of the intelligence, insight and hard work of, well, practically any company on the planet. We must take advantage of that!

Once we’ve designed good experiments, we then must execute well. Some of the key behaviors we know well, and I’m proud to see you folks exhibit them on a daily basis: trustworthiness, alignment, persistence, and fearlessness. But we also need to be good at error awareness: the ability to catch and correct errors as we experiment. And to be clear: designing good experiments and executing well does not guarantee you’ll get the outcome you want! It wouldn’t be an experiment if the result were pre-ordained: we need experiments to be able to fail, and if everything’s a success then, well, we certainly aren’t being ambitious enough. Intelligent failure provides valuable information.

That said, I recognize all that I’ve said requires an environment of high psychological safety, and that’s something both that we should strive for as a team, and that you should hold me accountable for.

I know I’ve spoken with some of your about this in our 1:1s. And one thing I want to be clear is that experiments can be - usually should be! - relatively small. I don’t want us to get writers’ block trying to polish some proposal for the next huge idea that will be the future of MSFT. To have good ideas you need to have lots of ideas, and being able to iterate quickly on an experiment idea is going to be a very useful skill for us to have.

To demonstrate, I’ve pasted below a couple of examples to show how, even for reasonably-sized experiments, the framing really doesn’t need to be that complicated. (Note: these are all ideas from 2018. And, in the spirit of Reid Hoffman’s “if you’re not embarrassed by the first version of your product, you’ve launched too late” 🙂 probably seem very simplistic given all that’s happened since then! But, I hope they’ll give you a flavor for the type of thing that can frame an experiment. Just to get started.)

  1. Cloud AI Platform – GSI Partnership Program

  • Thesis:

  • Cloud AI products demonstrate significant pull-through of other services (e.g. compute/storage). During the early stages of enterprise AI adoption, GSIs disproportionately drive the enterprise adoption of cloud AI technology.

  • Modest partnership investments at this early stage allow us to shift GSI data & AI practices onto Azure, increase platform stickiness, and drive significant Azure revenue.

  • Targets: Accenture, Tata Consultancy Services, Cognizant, Wipro

  • Deal terms:

  • MSFT Gives: Exec sponsorship of partnerships, allocation of Azure credits, funding & resources for AI training, joint sales & marketing GTM activities.

  • MSFT Gets: GSIs shift their AI practices & technologies to the Azure Cloud AI platform, commit to developing MSFT priority solutions, commit to training staff on MSFT technologies, commit to pursuing MSFT’s T400 strategic accounts.

  1. Enterprise Knowledge Graph – Early Adopter Program

  • Thesis:

  • We want to democratize & drive adoption of AI technologies. Enterprise Knowledge Graph aggregates a company’s disparate structured & unstructured data sources, and exposes that data through a natural language interface to help make information workers be more efficient.

  • We’re setting up an early adopter program to drive clarity on product definition, gauge market demand & sign lighthouse partners to showcase MSFT events.

  • Targets:

  • Nestle, Centrica, Kaiser Permanente, Publicis

  • Deal terms:

  • MSFT Gives: MSFT develops a data pipeline & ontology for a customer’s industry & use-case, then deploys an instance of EKG, natural language API, and Bot Framework interface to answer questions. Time-limited license for the customer to test in advance of GA.

  • MSFT Gets: Access to high-value industry scenarios & common use-cases, ability to build industry ontologies & data schemas, PR opportunities around digital transformation. Azure consumption + EAP fees (value to be confirmed).

  1. Ambient Intelligence – Early Adopter Program

  • Thesis:
  • Ambient Intelligence is a platform to instrument & query physical spaces. Through the use of cameras & other sensors, Ambient Intelligence provides a way to ask questions about the presence, movement, activities & interactions between humans, physical spaces & objects.
  • We’re setting up an early adopter program to drive clarity on product definition, gauge market demand & sign lighthouse partners to showcase MSFT events.
  • Targets:
  • Currently targeted at retail, but also investigating manufacturing & other industry use-cases: Kroger, Starbucks, Carrefour, Tesco, Schwarz Gruppe, Marks & Spencer
  • Deal terms:
  • MSFT Gives: 18-month engagement to define retail customer use-cases (pick & go shopping, shelf availability, shrinkage, etc.), and deploy & operate an ‘alpha’ version of the Ambient Intelligence solution in a single retail store.
  • MSFT Gets: EAP fee per customer of between 1M & 4M depending on use-case & store complexity. Real-life testing in a physical retail store. Ability to use retailer data to build common ML models. PR opportunities.

I would love to hear your thoughts on this - what do you think? What am I missing? Who has a different perspective? Perhaps we can set aside a good chunk of next week’s team meeting to discuss. If anyone has any private questions / concerns, please don’t hesitate to ping me or raise in our 1:1s.

Thanks,

Phil

Phil Waymouth phil.waymouth@microsoft.com Office: +1 (425) 707-6266 Mobile: +1 (206) 288-9672