This video was edited with AI agent. But how?
The talk is about world’s first open-source video editing agent!
Diffusion Studio x Re-Skill technology proposal:
Our Python-based agent starts a browser session using Playwright and opens operator.diffusion.studio.
This web app is a video editing UI optimized for agents, providing access to Diffusion Studio Core—a JavaScript-based engine that renders videos directly in the browser using WebCodecs (fully hardware-accelerated).
🖥 How it works:
1️⃣ A VideoEditingTool generates code based on user prompts and runs it in the browser.
2️⃣ If additional context is needed, DocsSearchTool uses RAG to pull information from operator.diffusion.studio/llms.txt.
3️⃣ After each execution step, the composition is sampled (currently 1 frame per second) and analyzed using VisualFeedbackTool via a multi-modal model.
4️⃣ The feedback system decides whether to proceed with rendering or refine further.
📡 File transfers between the browser and Python happen via Chrome DevTools Protocol, and for scalability, the agent can connect to a GPU-accelerated remote browser session via WebSocket (WIP: wss://chrome.diffusion.studio).
---
https://github.com/diffusionstudio/agent
https://re-skill.io/
slides: https://docs.google.com/presentation/d/1eipINYiwx3vjwvJXrv4QA0-9t4-uIVPuh112X9pElkM/edit?usp=sharing
Видео This video was edited with AI agent. But how? канала AI Engineer
Diffusion Studio x Re-Skill technology proposal:
Our Python-based agent starts a browser session using Playwright and opens operator.diffusion.studio.
This web app is a video editing UI optimized for agents, providing access to Diffusion Studio Core—a JavaScript-based engine that renders videos directly in the browser using WebCodecs (fully hardware-accelerated).
🖥 How it works:
1️⃣ A VideoEditingTool generates code based on user prompts and runs it in the browser.
2️⃣ If additional context is needed, DocsSearchTool uses RAG to pull information from operator.diffusion.studio/llms.txt.
3️⃣ After each execution step, the composition is sampled (currently 1 frame per second) and analyzed using VisualFeedbackTool via a multi-modal model.
4️⃣ The feedback system decides whether to proceed with rendering or refine further.
📡 File transfers between the browser and Python happen via Chrome DevTools Protocol, and for scalability, the agent can connect to a GPU-accelerated remote browser session via WebSocket (WIP: wss://chrome.diffusion.studio).
---
https://github.com/diffusionstudio/agent
https://re-skill.io/
slides: https://docs.google.com/presentation/d/1eipINYiwx3vjwvJXrv4QA0-9t4-uIVPuh112X9pElkM/edit?usp=sharing
Видео This video was edited with AI agent. But how? канала AI Engineer
Комментарии отсутствуют
Информация о видео
23 февраля 2025 г. 1:00:06
00:05:00
Другие видео канала
Understanding AI Stakes to Break Production Code: Philip Rathle
Agent Evals: Finally, With The Map
Where AI is superhuman: The right jobs to automate with LLMs
Ensure AI Agents Work: Evaluation Frameworks for Scaling Success — Aparna Dhinkaran, CEO Arize
Building efficient hybrid context query for LLM grounding: Simrat Hanspal
Scaling AI in Education: A Khanmigo case study: Shawn Jansepar
Building and evaluating AI Agents — Sayash Kapoor, AI Snake Oil
Cohere: Building enterprise LLM agents that work (Shaan Desai)
From model weights to API endpoint with TensorRT LLM: Philip Kiely and Pankaj Gupta
How to Build Your Own AI Data Center in 2025 — Paul Gilbert, Arista Networks
Building security around ML: Dr. Andrew Davis
Lessons from the Trenches: Building LLM Evals That Work IRL: Aparna Dhinkaran
The AI Evolution: Mario Rodriguez, GitHub
BotDojo Launch: Enhancing AI Assistants with Evaluations and Synthetic Data
Build, Evaluate and Deploy a RAG-Based Retail Copilot with Azure AI: Cedric Vidal and David Smith
EyeLevel Launch: Your RAG is Tripping, Here's the Real Reason Why
Mission-Critical Evals at Scale (Learnings from 100k medical decisions)
OpenLLMetry is all you need
E-Values Evaluating the Values of AI: Sheila Gulati and Nischal Nadhamuni
Beyond APIs: How AI Web Agents Are Automating the "Long Tail" of Knowledge Work
How to Improve Your Agents: Academic Lit Review