The Agent Paradox - Why Moderna's Most Productive AI Systems Aren't Agents

Eric Ma, who leads Research Data Science in the Data Science and AI group at Moderna, joins Hugo on moving past the hype of autonomous agents to build reliable, high-value workflows.

We discuss:

- Reliable Workflows: Prioritize rigid workflows over dynamic AI agents to ensure reliability and minimize stochasticity in production environments;
- Permission Mapping: The true challenge in regulated environments is security, specifically mapping permissions across source documents, vector stores, and model weights;
- Trace Log Risk: LLM execution traces pose a regulatory risk, inadvertently leaking restricted data like trade secrets or personal information;
- High-Value Data Work: LLMs excel at transforming archived documents and freeform forms into required formats, offloading significant “janitorial” work from scientists;
- “Non-LLM” First: Solve problems with simpler tools like Python or ML models before LLMs to ensure robustness and eliminate generative AI stochasticity;
- Contextual Evaluation: Tailor evaluation rigor to consequences; low-stakes tools can be “vibe-checked,” while patient safety outputs demand exhaustive error characterization;
- Serverless Biotech Backbone: Serverless infrastructure like Modal and reactive notebooks such as Marimo empowers biotech data scientists for rapid deployment without heavy infrastructure overhead.

Check out more here: https://hugobowne.substack.com/p/episode-66-the-agent-paradox-why

Want to learn more?

Join the final cohort of our Building AI Applications course in Q1, 2026 (25% off for listeners):
https://maven.com/hugo-stefan/building-ai-apps-ds-and-swe-from-first-principles?promoCode=vgyt

00:00 Defining Agents and Workflows
02:04 Challenges in Regulated Environments
04:24 Eric Ma's Role at Moderna, Leading Research Data Science in the Data Science and AI Group
12:37 Document Reformatting and Automation
15:42 Data Security and Permission Mapping
20:05 Choosing the Right Model for Production
20:41 Evaluating Model Changes with Benchmarks
23:10 Vibe-Based Evaluation vs. Formal Testing
27:22 Security and Fine-Tuning in LLMs
28:45 Challenges and Future of Fine-Tuning
34:00 Security Layers and Information Leakage
37:48 Wrap-Up and Final Remarks

Видео The Agent Paradox - Why Moderna's Most Productive AI Systems Aren't Agents канала Vanishing Gradients