ComfyUI animation with segmentation: TrackAnything with SAM models

***************************************************
It seems there is an issue with gradio. I have updated the requirements.txt file.
If it does not work, install gradio version 3.48.0
Thanks to @sairampv1 to notice and make it work
****************************************************
#animatediff #comfyui #stablediffusion
============================================================
💪 Support this channel with a Super Thanks or a ko-fi! https://ko-fi.com/koalanation
☕ Amazing ComfyUI workflows: https://tinyurl.com/y9v2776r

🚨 Use Runpod and access powerful GPUs for best ComfyUI experience at a fraction of the price. https://tinyurl.com/58x2bpp5 🤗

☁️ Starting in ComfyUI? Run it on the cloud without installation, very easy! ☁️
👉 RunDiffusion: https://tinyurl.com/ypp84xjp 👉15% off first month with code 'koala15'
👉 ThinkDiffusion: https://tinyurl.com/4nh2yyen

🤑🤑🤑 FREE! Check my runnable workflows in OpenArt.ai: https://tinyurl.com/2twcmvya
============================================================
Control your Stable Diffusion animations using TrackAnything and Comfy UI. In this video you will learn how to install and use TrackAnything.

TrackAnything uses tracking segmentation to separate objects from a video. You can then convert the segments to masks and change them as you want.

In this video, we will track the sweater and the pants from our character, change their colors and use our standard animation workflow in Comfy UI to create a new img2img animation.

00:00 Intro
00:10 Animation process steps
01:16 Install and run TrackAnything
01:52 Masking with TrackAnything
02:50 Correcting segments in TrackAnything
04:19 Preparations for Comfy UI
04:49 Workflow to change colors: separating the masks from TrackAnything
06:15 Changing colors workflow: changing the sweater (mask 1)
06:47 Changing colors workflow: changing the pants (mask 2)
07:32 Changing colors workflow: recombining masks 1 and 2
07:41 Changing colors workflow: Mask inverting the rest of the image and composing the complete image
08:12 Workflow to change colors: Running the workflow for all the frames
09:53 Animation workflow: Loading animation template
10:23 Animation workflow: Changing models, loras and prompts for the new animation
10:54 Animation workflow: Using DW Pose Preprocessor for OpenPose ControlNet
11:10 Animation workflow: Adding Tile ControlNet for better image consistency
11:24 Animation workflow: Final settings adjustments and running of the test image
11:45 Animation workflow: Using FaceDetailer to increase the quality of the face
12:53 Animation workflow: Running the workflow for all the frames
13:40 Frame Interpolation workflow: Inspection of frames
13:51 Frame Interpolation workflow: Installation of custom nodes: VideoHelperSuite and Frame Interpolation. Preparation of frames.
14:30 Frame Interpolation workflow: creating the frame interpolation workflow in ComfyUI
15:19 Postprocessing and editing the video
15:35 Outro

TrackAnything Github (modification): https://tinyurl.com/j4jmsjyr
TrackAnything Github (original): https://tinyurl.com/4pbpbfzv
ComfyUI: https://tinyurl.com/24srsvb3
ComfyUI Manager: https://tinyurl.com/ycvm4e29
Vast.ai: https://tinyurl.com/5n972ran

Workflows
Working with masks: https://comfy.icu/c/JeJaSt2c
Animation: https://comfy.icu/c/ttOqYg
Video Interpolation: https://comfy.icu/c/8aqDGqQ

Custom nodes
WAS node Suite: https://tinyurl.com/2ajuh2mx
Fannovel16 CN preprocessors: https://tinyurl.com/23tbcnnj
ImpactPack: https://tinyurl.com/2fn2su6z
VideoHelper Suite: https://tinyurl.com/yckt6r5y
Interpolation: https://tinyurl.com/2updvvwm

Models
RevAnimated: https://tinyurl.com/45xt4vwe
ControlNet: https://tinyurl.com/je85785u
TemporalNet: https://tinyurl.com/56dypbfv

My other tutorials:
https://youtu.be/Spc_F57FmK8
https://youtu.be/B9va_h1olkk

Animations/Clips:
https://youtu.be/v3oAhZuAXgw
https://youtu.be/rlQngY6SG2A

Videos: Mixkit
Music: Youtube Music Library
Edited with Runway.ml and ClipChamp

Make sure to subscribe to Koala Nation Channel:
https://cutt.ly/OZF0UhT

© 2023 Koala Nation
#comfyui #stablediffussion #trackanything

Видео ComfyUI animation with segmentation: TrackAnything with SAM models канала Koala Nation