The Ramona Flowers - Dismantle and Rebuild [FanArt Video]

Sometimes, we stumble upon a song that stays with us, as if it’s etched into the fabric of our memories. Do you remember that moment when you found a song that resonated with you on a deeper level? For me, that track was "Dismantle and Rebuild" by The Ramona Flowers.

Over the years, the inspiration from this song surfaced in many of my projects, and now – years later – I decided to return to this story. This is my way of giving thanks and paying tribute to the music that moved me so deeply back then.

Today, I’m sharing with you my version of the music video, the one I've carried in my mind for a long time. I hope you feel the same energy that accompanied me during the creation. Thanks for being here – this is for you, and perhaps for those inspirations that each of us holds close.

From a technical curiosity perspective – an experiment:

The idea was to generate almost everything within the ComfyUI environment.

Everything essentially happens within a single workflow in ComfyUI. This required writing several plugins, including Whisper, which processes song lyrics from mp3 into text. The plugin swiftly compiles the entire song into an editing list, returning in Comfy a list of durations for each scene and a list of words. You can define the maximum and minimum number of frames per scene to either speed up or slow down the editing, introduce a framerate, and it also saves to an EDL file, allowing the project to be opened in editing software. The plugin also saves text files for the next stage.

Next, the ChatGPT plugin from Mixlab steps in, which, based on the input prompt + song lyrics + the user's overall concept, generates a list of prompts and saves them on disk in the appropriate folder (this allows for potential review and editing of prompts for images in each scene).

Then each prompt is generated and saved with proper numbering in the right folder using, in this case, the Flux Dev model.

The images are then fed into the Luma API to create a video by adding in and out frames (achieving the effect of smooth continuous transitions). Each video clip at this stage is 5 seconds long at 24 fps. Using LiquidTime, the clip lengths are automatically adjusted smoothly by extending with a frame interpolator. Each generated shot is saved in the designated folder.

At this point, you just open the generated EDL file in, say, Premiere and polish the clip.

Here's the final result with some minor touch-ups in After Effects, where I didn’t change or alter any of the proposed shots.

Видео The Ramona Flowers - Dismantle and Rebuild [FanArt Video] канала Paul Lazniak / Paweł Łaźniak