You are reading the documentation for the in-development version of VSGAN.

VSGAN#

Release v1.6.4-post.116+86eb8af. (Installation)

https://pepy.tech/badge/vsgan https://img.shields.io/pypi/l/vsgan.svg https://img.shields.io/pypi/wheel/vsgan.svg https://img.shields.io/pypi/pyversions/vsgan.svg

PyTorch-based Super-Resolution and Restoration Image Processing Module for VapourSynth.

Short example:

from vsgan import ESRGAN
clip = ESRGAN(clip, device="cuda").\
    load(r"C:\Users\PHOENiX\Documents\Denoise_x1.pth").\
    apply().\
    load(r"C:\Users\PHOENiX\Documents\PSNR_x2.pth").\
    apply(overlap=16).\
    apply(overlap=32).\
    clip

This runs an ESRGAN denoising model at x1 scale on the input clip using the Nvidia CUDA GPU as the device. It then loads a new PSNR-optimized model at 2x scale to the same GPU device and applies it twice over. Finally, it returns the internal clip that had the models applied to it.

Note that by chaining the application of the 2x scale model twice-over, the resulting clip is now four times the size of the original clip. Normally, this is a very expensive operation. However, with VSGAN, this is possible thanks to its automatic tiling system.

For more information, see (Getting Started).

Features of VSGAN#

  • VapourSynth — Transform, Filter, or Enhance your input video, or the VSGAN result with VapourSynth, a Script-based NLE.

  • Model Chaining — Apply multiple models on the input video, or apply models multiple times over.

  • Seamless Tiling — Reduce up-front memory usage by automatically applying the model on quadrants of the input image at a time, without leaving any noticeable seams.

  • Supports All RGB formats — You can use any RGB video input of any bit depth.

  • No Frame Extraction Necessary — Using VapourSynth you can pass a Video directly to VSGAN, without any frame extraction needed.

  • Repeatable Edits — Any edit you make in the VapourSynth script with or without VSGAN can be re-used for any other video.

  • Freedom — VSGAN is released under the MIT License, ensuring it will stay free, with the ability to be used commercially.

Supported Model Architectures#

ESRGAN

Enhanced Super-Resolution Generative Adversarial Networks.

ESRGAN+

Further Improving Enhanced Super-Resolution Generative Adversarial Network.

Also supports other models with only differences during training, i.e., blind degradation models like A-ESRGAN and BSRGAN.

Real-ESRGAN

Training Real-World Blind Super-Resolution with Pure Synthetic Data.

EGVSR

Real-Time Super-Resolution System of 4K-Video Based on Deep Learning.

SwinIR

Image Restoration Using Swin Transformer.

HAT

Activating More Pixels in Image Super-Resolution Transformer.

Quick shoutout to MPGG#

If you plan to work with DVD-Video files, or general MPEG-1/2 media, you should take a look at my other project MPGG: a Streamlined MPEG-1 and MPEG-2 source loader and helper utility for VapourSynth.

I always recommend deinterlacing and detelecining sources before using VSGAN, and MPGG will greatly assist in doing so, as well as a lot more! It is designed specifically for DVD-Video files, but can work on other MPEG-1/2 media, e.g., Camcorders, VHS digitizers, old console capture cards, and more.

Example Results#

Mickey’s Christmas Carol#


There wasn’t really any issues with the original source to fix, but it did improve the sharpness while not over doing it or causing other artifacts, unlike the official Disney Blu-ray, which did a terrible job.

Comparison against the official Disney Blu-ray:

As you can see they over-sharpened, posterized the image, removed detail, caused Halo’ing, added a sort of green tint across the picture, and cropped in to 16:9 from a 4:3 source. Yikes!

I did not mix up the labels of these images. The one labeled as the Official Blu-ray release, really is like that!

(VapourSynth Script)

American Dad S01E01#


Fixed Halo’ing/Glow around outlines, and a bit of Chroma Droop. However, the colors between images in the dataset pair had a mistake so it learned to modify the colors incorrectly.

Family Guy S01E01#

Fixed slight inaccuracies in the Color, Halo’ing/Glow around outlines, Chroma Location Misplacement, as well as a bit of Chroma Droop (on some scenes only). This is an almost perfect result (in my opinion). Do note that the warping/stretch on the edges is an animation/dvd edit and not caused by VSGAN or the model.