AI for Product Managers: 10X Growth with Smart Experimentation

AI has completely transformed how we run experiments. What used to take weeks can now happen in minutes.

Frederic De Todaro, CPO at Kameleoon with 12+ years helping thousands of teams, reveals exactly how AI is revolutionizing experimentation from ideation to analysis.

⏰ Timestamps:
00:00 How AI Changed Experimentation Overview
01:54 The 4 Steps of Experimentation Framework
14:12 ADS
16:00 How AI has Changed Experimentation
21:08 User Behaviour Models
26:56 Multi-Armed Bandit vs Contextual Bandit
30:05 ADS
31:55 AI Content Genration
35:13 How Vibe Coding Changes Experimentation
41:35 Live Demo From Idea to Running Experiment in 2 Minutes
43:36 Two-Minute Build Achievement
51:49 How to Measure AI Features Properly
54:17 Measuring RAG Systems 3 Key Metrics
01:07:18 Best Experimentation Company Booking.com
01:10:10 Biggest PM Mistakes in Experimentation
01:13:52 Ending

Transcript: https://www.news.aakashg.com/p/frederic-de-todaro-podcast

--

🏆 Thanks to our sponsors:
1. Mobbin: Discover real-world design inspiration https://mobbin.com/aakash
2. Jira Product Discovery: Build the right thing, reliably https://www.atlassian.com/software/jira/product-discovery
3. Product Faculty: Get $550 off https://maven.com/product-faculty/ai-product-strategy-certificate?promoCode=AAKASH550C1
4. Maven: Get $100 off my curation of their top courses - http://maven.com/x/aakash

--

Key Takeaways:

1. The Build Bottleneck is Dead. Most product ideas never get tested because building takes weeks. AI just killed this constraint - you can now go from idea to live experiment in 2 minutes using plain English prompts.

2. Prompt Your Way to Tests. Type "change sorting to price low to high" and AI builds the variation in 2 minutes. Still run it through design, engineering, and data reviews - but now you're reviewing the actual live variation, not specs.

3. Beyond Text: Draw Your Ideas. Upload mockups or sketch rough concepts. AI transforms drawings into live experiments you can actually review - newsletter popups, onboarding flows, layout changes. Share preview links with stakeholders before going live.

4. AI Reads User Intent. Like a digital sales rep, AI scores every visitor's conversion likelihood in real-time. Show discounts only to users who need them to buy, not everyone who visits your site.

5. Failed Tests Become Wins. 80% of experiments fail overall, but AI automatically finds segments where they succeed. "Failed globally but increased mobile conversions 25%" - insights that would take hours manually.

6. Speed vs Accuracy Trade-offs. Multi-armed bandits optimize news headlines in hours, not weeks - perfect when time beats perfect measurement. Contextual bandits personalize every individual user's experience.

7. Humans Still Drive Strategy. PMs bring business context AI doesn't have - customer constraints, strategic priorities, success metrics. Data scientists validate statistical approaches. Designers review brand compliance. AI handles building variations fast.

8. Measure What Actually Matters. Track business metrics, not just usage: prompts needed per experiment, time from idea to live test, developer dependency rate. If you still need developers 80% of the time, AI isn't solving your bottleneck.

9. Discovery Meets Testing. User interviews reveal what people say they want. Experiments show what they actually do. Combine both for complete insight - validate problems through discovery, solutions through testing.

10. Experimentation Culture Wins. Harvard Business Review found direct correlation between experiments run annually and revenue growth. More experiments = faster growth. AI finally makes this accessible to every team.

---

👨‍💻 Where to find Fred:
LinkedIn: https://www.linkedin.com/in/fdetodaro/
Kameleoon: https://kameleoon.com

👨‍💻 Where to find Aakash:
Twitter: https://www.twitter.com/aakashg0
LinkedIn: https://www.linkedin.com/in/aagupta/

#ai #experimentation #abtesting #productmanagement

🧠 About Product Growth:
The world's largest podcast focused solely on product + growth, with over 180K listeners. Hosted by Aakash Gupta, who spent 16 years in PM, rising to VP of product, this 2x/week show covers product and growth topics in depth.

🔔 Subscribe and turn on notifications to master AI-powered experimentation!

Видео AI for Product Managers: 10X Growth with Smart Experimentation канала Aakash Gupta