4. Compare 2 videos with SSIM, PSNR and VMAF metrics

The classic metrics PSNR (Peak Signal to Noise Ratio) and SSIM (Structural SIMilarity) are implemented in C for improved performance.

import pathlib
import subprocess
import tempfile

import matplotlib.pyplot as plt

import cutcutcodec

4.1. Preparing videos to be compared

Here we’ll compare the original video with a noisy version.

from cutcutcodec.utils import get_project_root
ref_file = get_project_root().parent / "media" / "video" / "intro.webm"
dis_file = pathlib.Path(tempfile.gettempdir()) / "distorded.mp4"

converter = subprocess.run(["ffmpeg", "-y", "-i", ref_file, dis_file], check=True, capture_output=True)  # transcode the video

4.2. Simple comparison of 2 videos or images

• This function takes care of color spaces.

• Image comparisons are always made in yuv, not rgb.

• This function is optimized and parrallelized.

metrics = cutcutcodec.compare(ref_file, dis_file, psnr=True, ssim=True, vmaf=True)

compare: 100%|████████████████████████████████████████████████| 294/294 [00:57<00:00,  5.16img/s]

plt.scatter(range(len(metrics["psnr"])), metrics["psnr"])
plt.xlabel("frames")
plt.ylabel("psnr (db)")
plt.show()

plt.scatter(range(len(metrics["ssim"])), metrics["ssim"])
plt.xlabel("frames")
plt.ylabel("ssim")
plt.show()

plt.scatter(range(len(metrics["vmaf"])), metrics["vmaf"])
plt.xlabel("frames")
plt.ylabel("vmaf")
plt.show()

4.3. Compare frames manually for greater control

This makes it possible to specify the hyperparameters specific to each metric.

ref_frame = cutcutcodec.read(ref_file).out_select("video")[0].snapshot(5.0, (1080, 1920))
ref_frame_numpy = ref_frame.numpy(force=True)
ref_frame.requires_grad = True
dis_frame = cutcutcodec.read(dis_file).out_select("video")[0].snapshot(5.0, (1080, 1920))
dis_frame_numpy = dis_frame.numpy(force=True)
dis_frame.requires_grad = True

4.3.1. Compare peak signal to noise ratio (PSNR)

• This function has a C implementation (for cpu and no grad).

• This function is torch differentiable.

print(cutcutcodec.psnr(ref_frame_numpy, dis_frame_numpy))
print(cutcutcodec.psnr(ref_frame, dis_frame))

44.240646
tensor(44.2387, grad_fn=<ViewBackward0>)

4.3.2. Compare structural similarity index measure (SSIM)

• This function has a C implementation (for cpu and no grad).

• This function has a fft implementation (for stride = 1).

• This function is torch differentiable.

print(cutcutcodec.ssim(ref_frame_numpy, dis_frame_numpy))
print(cutcutcodec.ssim(ref_frame, dis_frame))

print(cutcutcodec.ssim(ref_frame_numpy, dis_frame_numpy, stride=2, sigma=1.2))

0.9906771
tensor(0.9907, grad_fn=<ViewBackward0>)
0.99125826

4.3.3. Compare Netflix Video Multi-Method Assessment Fusion (VMAF)

This function only allows efficient interfacing with the vmaf package. So you need to install vmaf.

print(cutcutcodec.vmaf(ref_frame_numpy, dis_frame_numpy))

41.257366