Video super-resolution (VSR) is a classic but challenging task in computer vision and graphics that attempts to recover high-resolution videos from low-resolution counterparts. VSR faces two significant issues. The first challenge is ensuring temporal consistency across output frames. The second challenge is to create high-frequency information in upsampled frames. Previous techniques addressed the first obstacle and demonstrated outstanding temporal consistency in upsampled videos. 

However, these methods frequently result in blurry images and fail to provide high-frequency appearance details or realistic textures. An effective VSR model must generate believable new content that does not exist in low-resolution input videos. However, current VSR models have limited generating capabilities and cannot create detailed appearances. 

Generative Adversarial Networks (GANs) have demonstrated excellent generative capabilities in the image super-resolution. These approaches can accurately simulate the distribution of high-resolution images and produce fine-grained detail in upsampled images. GigaGAN enhances the generative capabilities of image super-resolution models by training a large-scale GAN model on billions of images. 

The dawn of VideoGigaGAN 

Adobe researchers created VideoGigaGAN, a new generative VSR model that can generate videos with high-frequency features and temporal consistency. VideoGigaGAN builds on GigaGAN, a large-scale image upsampler. 

Adding temporal modules to a GigaGAN video model results in severe temporal flicker. Adobe uncovers many significant difficulties and provides approaches for improving the temporal consistency of upsampled videos. 

VideoGigaGAN for fine-grained details 

According to the researchers' findings, VideoGigaGAN produces temporally consistent videos with fine-grained appearance features than earlier VSR methods. The authors validated VideoGigaGAN's effectiveness by comparing it to cutting-edge VSR models on public datasets and displaying video findings in 8× super-resolution. 

In a paper published on April 18th, Adobe claimed VideoGigaGAN is superior to previous Video Super Resolution (VSR) approaches because it can give more fine-grained features without introducing "AI weirdness" into the footage. 

Top-notch image or video quality 

In the simplest terms, Generative Adversarial Networks (GANs) are successful at upscaling still images to higher resolutions but struggle to do it with video without introducing flickering and other undesirable artifacts. Other upscaling methods can prevent this, but the results are less sharp and detailed. VideoGigaGAN promises to deliver the best of both worlds: more excellent image/video quality from GAN models, with less flickering and distortion across output frames. The company has offered various examples that demonstrate its work in full resolution. 

This is simply a research preview, so there is no certainty that Adobe will make VideoGigaGAN available to consumers through Creative Cloud software such as Premiere Pro. During their MAX event in October 2023, the business revealed a diffusion-based upsampling experiment called Project Res-Up, improving the quality of low-resolution GIFs and video recordings. 

• Adobe Research
• Photo by Rubaitul Azad on Unsplash