How do you make an AI music video?

Vlad Budanov, founder of CATS Digital Studio. Published , updated .

There are two ways to make an AI music video: generate it from prompts, or shoot live footage and rework it with AI and VFX. The second gives a steadier picture, because the artist’s face and movement stay real and the effects sit on top.

How the work goes

  1. The idea: which effects, where in the track, how the image changes between scenes.
  2. The footage: a new shoot or a finished video.
  3. AI generation to transform the frames.
  4. Tracking and VFX compositing, so the effects follow the movement in the shot.
  5. Delivery of the video and, if you want one, a short breakdown.

How we did it

The artist @every_day_rain came with a finished video. We kept the live footage and built effects on top: shards and particles bursting out of the frame, flashes of light, colour shifts between scenes. The shots were tracked and composited in After Effects. The video passed 3 million views, and there is a layer-by-layer VFX breakdown. Case.

What to prepare

The track, reference videos you like, and the source footage if there was a shoot. Without footage, we start from the idea and decide what to shoot live and what to generate.

More: AI music video production for artists.

Telegram@vladbudanov Discuss a project