4. Compare 2 videos with SSIM, PSNR and VMAF metrics

The classic metrics PSNR (Peak Signal to Noise Ratio) and SSIM (Structural SIMilarity) are implemented in C for improved performance.

import pathlib
import subprocess
import tempfile

import matplotlib.pyplot as plt

from cutcutcodec.core.analysis.video.metric import compare
from cutcutcodec.core.io import read
from cutcutcodec.utils import get_project_root

4.1. Preparing videos to be compared

Here we’ll compare the original video with a noisy version.

ref_file = get_project_root().parent / "media" / "video" / "intro.webm"
dis_file = pathlib.Path(tempfile.gettempdir()) / "distorded.mp4"

converter = subprocess.run(["ffmpeg", "-y", "-i", ref_file, dis_file], check=True, capture_output=True)  # transcode the video

4.2. Simple comparison of 2 videos or images

• This function takes care of color spaces.

• Image comparisons are always made in yuv, not rgb.

• This function is optimized and parrallelized.

metrics = compare(ref_file, dis_file, psnr=True, ssim=True)

compare: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 294/294 [00:30<00:00,  9.68img/s]

plt.scatter(range(len(metrics["psnr"])), metrics["psnr"])
plt.xlabel("frames")
plt.ylabel("psnr (db)")
plt.show()

plt.scatter(range(len(metrics["ssim"])), metrics["ssim"])
plt.xlabel("frames")
plt.ylabel("ssim")
plt.show()

4.3. Compare frames manually for greater control

This makes it possible to specify the hyperparameters specific to each metric.

ref_frame = read(ref_file).out_select("video")[0].snapshot(5.0, (1080, 1920))
ref_frame_numpy = ref_frame.numpy(force=True)
ref_frame.requires_grad = True
dis_frame = read(dis_file).out_select("video")[0].snapshot(5.0, (1080, 1920))
dis_frame_numpy = dis_frame.numpy(force=True)
dis_frame.requires_grad = True

4.3.1. Compare peak signal to noise ratio (PSNR)

• This function has a C implementation (for cpu and no grad).

• This function is torch differentiable.

from cutcutcodec.core.analysis.video.metric import psnr

print(psnr(ref_frame_numpy, dis_frame_numpy))
print(psnr(ref_frame, dis_frame))

41.762505
tensor(41.7603, grad_fn=<ViewBackward0>)

4.3.2. Compare structural similarity index measure (SSIM)

• This function has a C implementation (for cpu and no grad).

• This function has a fft implementation (for stride = 1).

• This function is torch differentiable.

from cutcutcodec.core.analysis.video.metric import ssim

print(ssim(ref_frame_numpy, dis_frame_numpy))
print(ssim(ref_frame, dis_frame))

print(ssim(ref_frame_numpy, dis_frame_numpy, stride=2, sigma=1.2))

0.9862512
tensor(0.9863, grad_fn=<ViewBackward0>)
0.9870404

4.3.3. Compare Netflix VMAF

This function only allows efficient interfacing with the vmaf package. So you need to install vmaf.

from cutcutcodec.core.analysis.video.metric import vmaf

print(vmaf(ref_frame_numpy, dis_frame_numpy))

45.14047