cutcutcodec.core.nn.loader

Implement some data-loader.

Classes

Dataset(root, selector, **kwargs)	Select files managing the probability.
ImageDataset(root, shape, *[, dataaug])	A specific dataset for managing images.

Details

class cutcutcodec.core.nn.loader.Dataset(root: str | bytes | Path, selector: Callable[[Path], bool], **kwargs)

Select files managing the probability.

Examples

>>> from cutcutcodec.core.nn.loader import Dataset
>>> from cutcutcodec.utils import get_project_root
>>> def selector(path):
...     return path.suffix == ".py"
...
>>> dataset = Dataset(get_project_root(), selector, max_size=128)
>>> len(dataset)
128
>>> dataset[0].relative_to(get_project_root())
PosixPath('__init__.py')
>>> dataset[1].relative_to(get_project_root())
PosixPath('__main__.py')
>>> dataset[2].relative_to(get_project_root())
PosixPath('utils.py')
>>> dataset[3].relative_to(get_project_root())
PosixPath('config/__init__.py')
>>> dataset[4].relative_to(get_project_root())
PosixPath('core/__init__.py')
>>> dataset[5].relative_to(get_project_root())
PosixPath('testing/__init__.py')
>>> dataset[6].relative_to(get_project_root())
PosixPath('config/config.py')
>>>

Initialise and create the class.

Parameters

rootpathlike

The root folder containing all the files of the dataset.

selectorcallable

Function that take a file pathlib.Path and return True to keep it or False to reject.

follow_symlinksbool, default=False

Follow the symbolink links if set to True.

max_sizeint, optional

The maximum number of files contained in the dataset.

decision_depthint, default=1

The thresold level befor to flatten the tree. If 0, all the file have the same proba to be drawn. If 1, the decision tree has only one root node If n, the decision tree has a maximum of n decks.

class cutcutcodec.core.nn.loader.ImageDataset(root: str | bytes | Path, shape: tuple[Integral, Integral] | list[Integral], *, dataaug: Callable[[FrameVideo], FrameVideo] | None = None, **kwargs)

A specific dataset for managing images.

Initialise and create the class.

Parameters

rootpathlike

Transmitted to Dataset initialisator.

shapeint and int

The pixel dimensions of the returned image. The image will be random reshaped and random cropped to reach this final shape. The convention adopted is the numpy convention (height, width).

dataaugcallable, optional

If provided, the function is called for each brut readed image before normalization.

**kwargsdict

Transmitted to Datset initialisator.