cutcutcodec.core.analysis.video.complexity.dct.temporal_dct¶

cutcutcodec.core.analysis.video.complexity.dct.temporal_dct(imgs: Tensor, threads: int = 0, patch: Integral = 32) → Tensor[source]¶

Compute the temporal dct complexity between 2 images.

The dct temporal complexity \(H_{\text{dct}} \in \mathbb{R}^+\) is defined as follow:

\[\begin{split}\begin{cases} H_{\text{dct}} = \frac{1}{n_{\text{blocs}}} \sum\limits_{m=1}^{n_{\text{blocs}}} \left| H_{m,t} - H_{m,t-1} \right| \\ H_{m,t} = \frac{1}{s^2} \sum\limits_{i=1}^s \sum\limits_{j=1}^s e^{\left(\frac{ij}{s^2}\right)^2-1} \left|\mathscr{D}_{m,t}(i,j)\right| \\ \mathscr{D}_{m,t}(i,j) = \begin{cases} 0 & \text{if } i + j = 2 \\ \mathscr{F}_{m,t}(i,j) & \text{otherwise} \\ \end{cases} \\ \end{cases}\end{split}\]

With \(\mathscr{F}_{m,t}(i,j)\) the DCT-II applied to the patch \(m\) of the image \(t\), calculated by the function compute_dct(). The patches cover the full image and are not overlapping.

Parameters¶

imgsarraylike: The Y[UV] images, of shape ([*batch], 2, height, width, [channels]). Only the Y component is used. It has to be in range [0, 1].
threads, patch:: Same as spatial_dct().

Returns¶

temporal_dctarraylike: The \(H_{dct} \in \mathbb{R}^+\) scalar for each couple of image (of shape batch).

Notes¶

It is inspired by the paper A NEW ENERGY FUNCTION FOR SEGMENTATION AND COMPRESSION.
The VCA tool offers an optimized version of a similar metric. The result is close to the h column of the .csv file generated with ffmpeg -i video.mp4 -f yuv4mpegpipe - | vca --y4m --input stdin --no-lowpass --complexity-csv result.csv.
This function can be called by cutcutcodec metric video.mp4 --temporal-dct -o result.json.

Examples¶

>>> import numpy as np
>>> from cutcutcodec.core.analysis.video.complexity import temporal_dct
>>> np.random.seed(0)
>>> imgs = np.random.random((2, 720, 1080, 3))  # It could also be a torch array list...
>>> temporal_dct(imgs).round(2)
array([0.03])
>>>