Stable Video Diffusion
Stability AI's open video generation model
Stable Video Diffusion is an open-weight image-to-video generation model from Stability AI that animates a still image into a short video clip by extending the latent diffusion approach used in Stable Diffusion image models into the…
Definition
Stable Video Diffusion is an open-weight image-to-video generation model from Stability AI that animates a still image into a short video clip by extending the latent diffusion approach used in Stable Diffusion image models into the temporal dimension.
Overview
Stable Video Diffusion was trained by taking a latent diffusion model, initialized from Stable Diffusion's image-generation architecture, and extending it with temporal layers so that it learns to generate coherent sequences of frames rather than a single static image. Stability AI trained the model in stages: an initial large-scale pretraining phase on a broad video dataset, followed by fine-tuning on a smaller, higher-quality curated video dataset to improve visual fidelity and motion coherence. At release, Stable Video Diffusion primarily supported image-to-video generation, taking a single still image as input and producing a short video clip (typically a few seconds, around 14 to 25 frames depending on configuration) that animates the scene with plausible camera motion or subject movement, rather than accepting a free-text prompt directly as its primary interface in the initial release. This distinguished it from purely text-to-video systems like Sora, positioning it more as an animation tool for existing images. As an open-weight release, Stable Video Diffusion can be self-hosted and run on capable consumer or prosumer GPUs, making it accessible to developers and researchers who want to experiment with or build on video diffusion without relying on a closed API. It has been used as a research base for further fine-tunes and as a building block within larger creative pipelines, and it sits within Stability AI's broader family of open generative models alongside Stable Diffusion for images and Stable Audio for sound.
Key Concepts
- Image-to-video generation animating a still image into a short clip
- Built by extending Stable Diffusion's latent diffusion architecture temporally
- Trained in stages: broad pretraining followed by curated fine-tuning
- Generates short clips of roughly 14-25 frames per configuration
- Open-weight release enabling self-hosting on consumer/prosumer GPUs
- Distinct focus on image animation rather than direct text-to-video generation
- Part of Stability AI's broader open generative model family
- Used as a research and fine-tuning base for downstream video tools