ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Sampler

Recent advancements in text-to-video (T2V) and image-to-video (I2V) diffusion models have improved video generation, particularly for keyframe interpolation. However, adapting current I2V models for two-frame conditioning (start & end) remains challenging, often leading to artifacts and off-manifold issues. We introduce a novel bidirectional sampling strategy that sequentially samples along forward and backward paths without extensive re-noising or model fine-tuning, ensuring more coherent and on-manifold generation of intermediate frames. Advanced guidance techniques, CFG++ and DDS, are also employed to enhance interpolation.

By integrating these, our method achieves state-of-the-art performance, efficiently generating high-quality, smooth videos between keyframes. On a single 3090 GPU, our method can interpolate 25 frames at 1024x576 resolution in just 195 seconds, establishing it as a leading solution for keyframe interpolation.

Comparison of denoising processes between (a) Time Reversal Fusion method and (b) bidirectional sampling (Ours). Our key innovation lies in the sequential sampling of the temporal forward path and the temporal backward path by integrating a single re-noising step between them.

In the geometric view of diffusion models, the sampling process is typically described as iterative transitions moving from the noisy manifold to the clean manifold. From this perspective, (a) fusing two intermediate sample points through linear interpolation on a noisy manifold can lead to an undesirable off-manifold issue, where the generated samples deviate from the learned data distribution. (b) Bidirectional sampling effectively addresses this issue by sequentially sampling both the temporal forward and backward paths, with a single re-noising step in between. This approach enables on-manifold sampling, ensuring that the generated samples stay close to the learned data distribution.