Interview Preparation (DeepMind, CoreAI)-Phil’s MacBook Air

:::note] :::

• The Copilot product is in a good place
• The cross-company branding is a mess
• Need to focus on:
  • Distribution
  • Memory
  • Use emerging open protocols (MCP) to take advantage of others’ capabilities (M365 data, etc.)
• Drive the cost of experimentation to zero

Google DeepMind Preparation

• definitely have the end-to-end
• fine balance for rigor commerciality and science

[!Who I am]

• Intersection of: new technology x business strategy
• Enjoy working on different ambiguous, interesting projects and bringing clarity
• Experience across software engineering, program management, strategy and partnerships.
• Worked as an engineer, business development & strategy, program management
  • OpenAI, AMD, ASML / Disney / Lenfest Institute
• Designing and nurturing partnerships, bringing technology to market.
  • NOW LOOKING FOR AN EXCITING ROLE WITH A MISSION-DRIVEN ORGANIZATION WHERE I CAN WORK AT THE FOREFRONT OF TECHNOLOGY, BUT HELP BRING IT TO MARKET IN A REAL WAY: INVENTION → INNOVATION

What I Can Bring to a Team

• Experience working at the intersection of new technology and business strategy.
• Demonstrated success leading multi-disciplinary, cross-company teams and programs.
• Understanding of how large companies (Microsoft) work

What I want to Contribute to

• A mission driven effort that helps improve the world in which we live.

What I Care about

• Honesty, team work, solving for the global maximum

What I Think is Interesting

• ‘Inhuman’ intelligence
• Focus on utility and identify human tasks that technology can help with.
• 5 second tasks → 5 minutes → 5 hours / days / months
• Memory is key → people will forgo 10 IQ points if only a system remembered you better.

What I Think GDM Can Do Better

• Concern about subscription business.
• Concern about impact on core business.

What Do I Think is Most Interesting about GDM’s Current Work?

• David Silver’s “Era of Experience” paper
• World Models

Why GDM? What’s the Unique Draw for PJW?

• End-to-end focus on solving intelligence, and delivering is safely and responsibly.
• Demis: understand the world around us and help advance human knowledge.
• Act in the world!

---

My point of view

Today’s ML:

• There is a capability overhang with current models.
• You can’t predict the future, only chart out the physical limits. You have to be hands-on. - Set a suitably scary North Star, and have evals that demonstrate whether you’re making progress towards that North Star.
• People often assume there’s a ‘what’ problem, when it’s really a ‘how’ problem
• Find valuable use cases, understand them deeply, train models to complete those tasks.
• It’s OK to fast-follow, but you need to be able to follow.
• Things move fast! Don’t make hard to unwind decisions based on today’s capabilities.

Open: image.png

4 partnership principles:

1. Build trust – we will always have disagreements. Trust is the medicine.
2. Maintain alignment – we will define and redefine shared goals and desired outcomes often
3. Show persistence – we tackle complex problems together
4. Have fearlessness – we will be optimists, and won’t be afraid to do hard things or take risks

Setting goals and running projects:

• People often assume there’s a ‘what’ problem, when it’s really a ‘how’ problem
• Process goals > performance goals > outcome goals
• Creativity is keeping going after the first “good enough” answer
• Building a team is recognizing individuals while managing for team goals
  • hire the best → solve for ability to learn and improve
  • lead so you can push effective decision making to the edge
  • walk the walk
• Drive ambition → develop the platform → differentiate

Run good experiments:

• what should I do → what should I learn?
• help learn new, specific, valuable information
• are worthwhile
• predicated on a reasonable hypothesis about what will happen
• is as small as possible to deliver
• can scale those learnings
• takes advantage of any unique capabilities you have

Run good projects:

• focus
• keep a detailed plan for victory
• run a fast observe / orient / decide / act loop
• overcommunicate
• break off sub-projects

---

OCTO

Me:

• I provide strategic insights and advice on AI initiatives
• I drive cross-company programs to connect our horizontal AI strategy with our vertical product strategy (Build event, Metis, Microsoft Research Publishing, AI Models series)

Mission:

• Inform and guide Microsoft’s strategic and technical direction by ensuring we have the right technology, talent, and culture across the breadth of research and engineering.

Values:

• Servant leadership
• No ego → more important to do the right thing than prove we’re right
• Listen first
• Collaborate and be curious
• Objective and balanced
• Compassionate and respectful
• Build trust with all stakeholders through transparency and confidentiality
• Deliver success through others, solving for Microsoft’s global maximum

---

Why AI Product Management Is Different

Playbook Tip:

• Lead with output quality.
• In AI, the model is both the compiler and the product.
• Ensuring it meets user needs and quality benchmarks is the foundation upon which great UX can then be layered.

AI product management differs from traditional software in three critical ways:

1. Probabilistic Outputs: AI generates variable results influenced by training data, prompts, and real-world usage.
2. Quality First: If model outputs are poor, no UI/UX polish can compensate.
3. Rapid Evolution: AI frameworks, best practices, and models change quickly, requiring constant adaptation.

Traditional PM vs. AI PM

	Traditional Product Management	AI Product Management
Definition of “Good”	Features are defined by a set of functional requirements and deterministic logic. If the feature meets specs, it’s “good.”	Quality is probabilistic; “good” is defined by metrics like accuracy, relevance, clarity, or user satisfaction. Continuous measurement and clear criteria (golden sets, test sets) are essential.
Spec & Requirements	Specifications center on predefined features, acceptance criteria, and deterministic logic. Requirements are mostly about how the system should behave under various conditions.	Specs must explicitly define what good looks like through sample prompts, golden sets, and evaluation metrics. AI PMs must provide annotated examples, success benchmarks, and clear criteria for acceptable vs. unacceptable outputs.
Empirical Mindset	Validation relies on predefined use cases, acceptance criteria, and manual QA.	Demands a data-driven, experimental approach. Product teams must continuously test, measure output quality, and refine based on real-world feedback and metrics.
Core Focus	The UI/UX and workflow design often take precedence. If the feature’s logic is correct, a polished experience is enough.	AI output quality is paramount, overshadowing UI design. A subpar model output can negate even the best-designed interface.
Feature Crew Disciplines	Primary collaboration: Product Managers, Engineers, UX Designers, and Copywriters.	Deep collaboration is needed with applied research (for AI model development, prompt engineering, data pipelines) and technical writers (to craft prompts, refine model responses), in addition to classic disciplines (UX, copy, eng).
Data Requirements	Mostly static requirements and configurations; data typically is for analytics or minimal business logic.	Robust, high-quality datasets drive output evaluation and improvement.
Iteration	Iteration is usually tied to feature roadmaps and version releases; updates are less frequent once feature logic stabilizes.	An ongoing cycle of prompt tuning, model retraining, and evaluation. AI features often see continuous updates as the model and data evolve.
Evaluation & Testing	Test cases and QA checklists ensure deterministic outcomes match the specification.	Golden sets, automated metrics, LLM-as-judge pipelines, and human reviews. Success is assessed against empirical benchmarks and user feedback loops.
Stakeholder Collaboration	Product, marketing, and user research typically align on messaging once core feature functionality is locked.	Tight cross-functional alignment is critical. Marketing must understand AI’s capabilities and limits; user research must inform ongoing prompt/model refinements.
Risk of Failure	Bugs or mismatched features can lead to user frustration, but issues are often binary and more predictable.	AI outputs can fail in subtle ways—incorrect facts, biased or confusing responses. Failures may be less predictable and require robust risk mitigation (e.g., human-in-the-loop evaluations).
User Expectations	Consistent functionality once a feature “works.”	Variable output quality; must manage expectations and clarify limitations.
Safety & RAI	Privacy & Security requirements focus on data protection, regulatory compliance, and standard code-of-conduct.	Goes beyond privacy/security to include algorithmic bias detection, content moderation, ethical usage guidelines, and frameworks for responsible AI (e.g., fairness, transparency, governance).