More models!

In addition to bundled models, vs-mlrt can also be used to run these models:

• anime-segmentation/isnet_is.onnx: anime character segmentation at a0a563c, RGBS -> GRAYS, requires mod64 input

• oidn/rt_ldr{_small}.onnx: image denoising from Intel® Open Image Denoise library, RGBS, requires mod16 input

• ppocr/ml_PP-OCRv3_det.onnx: multilingual text detection model from PaddleOCR, RGBS -> GRAYS, requires mod32 input

• waifu2x swin_unet: waifu2x's swin_unet models. It's supported by the Python wrapper with vsmlrt.Waifu2xModel.{swin_unet_art,swin_unet_art_scan,swin_unet_photo{v2}}.

  • file list:

    • waifu2x/swin_unet_art/{scale2x, scale4x, noise0, noise0_scale2x, ..., noise3_scale4x}.onnx
    • waifu2x/swin_unet_art_scan/{scale4x, noise0_scale4x, ..., noise3_scale4x}.onnx
    • waifu2x/swin_unet_photo{_v2}/{scale4x, noise0_scale4x, ..., noise3_scale4x}.onnx]
• RIFE (ensemble) (source):

  • 4.0~4.6 ensemble
  • 4.0~4.6 v2
  • 4.7
  • 4.8
  • 4.9
  • 4.10
  • 4.11
  • 4.12
  • 4.12 lite
  • 4.13
  • 4.13 lite
  • 4.14
  • 4.14 lite
  • 4.15
  • 4.15 lite
  • 4.16 lite
  • 4.17
  • 4.17 lite
  • 4.18
  • 4.19
  • 4.20
  • 4.21
  • 4.22
  • 4.22 lite
  • 4.23
  • 4.24
  • 4.25
  • 4.25 lite
  • 4.25 heavy
  • 4.26
  • 4.26 heavy
  v2 models handle paddings internally and reduce PCIe traffic flow.

• safa/{safa_{v0.1,v0.2,v0.3,v0.4,v0.5}_{non_adaptive,adaptive1x,adaptive}.onnx: SAFA video enhancement models. Individually packaged.

• scunet: SCUNet denoisig models.

• ArtCNN/ArtCNN_{C4F32,C16F64,R16F96,R8F64}{_Chroma,_DS}: ArtCNN models for anime super-resolution and restoration.

With more to come.

Also check onnx models provided by the avs-mlrt community.

Usage

If an external model is not supported by the Python wrapper, you can use the generic vsmlrt.inference API to run these models (requires release v12.2 or later).

import vsmlrt
output = vsmlrt.inference(rgbs, "path/to/onnx", backend=vsmlrt.Backend.TRT(fp16=True))

The rife model requires auxiliary inputs and should be used from vsmlrt.RIFE or vsmlrt.RIFEMerge interface.