lightly.transforms

The lightly.transforms package transforms for various self-supervised learning methods.

It also contains some additional transforms that are not part of torchvisions transforms.

class lightly.transforms.add_grid_transform.AddGridTransform(num_rows: int, num_cols: int)

Implements the naive segmentation into a regular grid from DetCon. 0

Input to this transform:

Any datastructure containing one or several torchvision.tv_tensor.BoundingBoxes and/or torchvision.tv_tensor.Mask, such as tuples or arbitrarily nested dictionaries. For all supported data structures check 1. Masks should be of size (*, H, W) and BoundingBoxes can be of arbitrary shape.

Output of this transform:

Leaves any images in the data structure untouched, but overwrites any bounding boxes and masks by a regular grid. Bounding boxes will take shape (num_rows*num_cols, 4) and masks will be of shape (1, H, W) with integer values in the range [0, num_rows*num_cols-1].

Example

img = torch.randn((3, 16, 16))
bboxes = BoundingBoxes(
    torch.randn((1, 4)), format="XYXY", canvas_size=img.shape[-2:]
)
mask = Mask(torch.randn((16, 16)).to(torch.int64))
tr = AddGridTransform(num_rows=4, num_cols=4)
# the image will be untouched the bounding boxes will be a regular grid
img, bboxes, mask = tr(img, bboxes, mask)
# bboxes of shape (num_rows*num_cols, 4)
# mask with value range [0, num_rows*num_cols-1]

References

0

DetCon, 2021, https://arxiv.org/abs/2103.10957

1

torchvision Getting started with transforms v2, https://pytorch.org/vision/main/auto_examples/transforms/plot_transforms_getting_started.html

num_rows

number of grid rows in the segmentation

num_cols

number of grid columns in the segmentation

transform(inpt: Any, params: Dict[str, Any]) → Any

Method to override for custom transforms.

See sphx_glr_auto_examples_transforms_plot_custom_transforms.py

class lightly.transforms.byol_transform.BYOLTransform(view_1_transform: Optional[BYOLView1Transform] = None, view_2_transform: Optional[BYOLView2Transform] = None)

Implements the transformations for BYOL[0].

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length 2.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• Solarization

• ImageNet normalization

Note that SimCLR v1 and v2 use similar augmentations. In detail, BYOL has asymmetric gaussian blur and solarization. Furthermore, BYOL has weaker color jitter compared to SimCLR.

• [0]: Bootstrap Your Own Latent, 2020, https://arxiv.org/pdf/2006.07733.pdf

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of [tensor, tensor].

view_1_transform

The transform for the first view.

view_2_transform

The transform for the second view.

__init__(view_1_transform: Optional[BYOLView1Transform] = None, view_2_transform: Optional[BYOLView2Transform] = None)

class lightly.transforms.byol_transform.BYOLView1Transform(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.4, cj_contrast: float = 0.4, cj_sat: float = 0.2, cj_hue: float = 0.1, min_scale: float = 0.08, random_gray_scale: float = 0.2, gaussian_blur: float = 1.0, solarization_prob: float = 0.0, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Transforms an image into the first view for BYOL [0].

Used by BYOLTransform to create the first view of an image.

Input to this transform:

PIL Image. (Tensor inputs are supported when torchvision transforms v2 are available.)

Output of this transform:

Tensor.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• ImageNet normalization

• [0]: Bootstrap Your Own Latent, 2020, https://arxiv.org/pdf/2006.07733.pdf

__call__(image: Union[Tensor, Image]) → Tensor

Applies the transforms to the input image.

Parameters

image – The input image to apply the transforms to.

Returns

The transformed image.

__init__(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.4, cj_contrast: float = 0.4, cj_sat: float = 0.2, cj_hue: float = 0.1, min_scale: float = 0.08, random_gray_scale: float = 0.2, gaussian_blur: float = 1.0, solarization_prob: float = 0.0, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Initializes BYOLView1Transform.

Parameters

• input_size – Size of the input image in pixels.

• cj_prob – Probability that color jitter is applied.

• cj_strength – Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value. For datasets with small images, such as CIFAR, it is recommended to set cj_strength to 0.5.

• cj_bright – How much to jitter brightness.

• cj_contrast – How much to jitter contrast.

• cj_sat – How much to jitter saturation.

• cj_hue – How much to jitter hue.

• min_scale – Minimum size of the randomized crop relative to the input_size.

• random_gray_scale – Probability of conversion to grayscale.

• gaussian_blur – Probability of Gaussian blur.

• solarization_prob – Probability of solarization.

• sigmas – Tuple of min and max value from which the std of the gaussian kernel is sampled.

• vf_prob – Probability that vertical flip is applied.

• hf_prob – Probability that horizontal flip is applied.

• rr_prob – Probability that random rotation is applied.

• rr_degrees – Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

• normalize – Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.byol_transform.BYOLView2Transform(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.4, cj_contrast: float = 0.4, cj_sat: float = 0.2, cj_hue: float = 0.1, min_scale: float = 0.08, random_gray_scale: float = 0.2, gaussian_blur: float = 0.1, solarization_prob: float = 0.2, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Transforms an image into the second view for BYOL [0].

Used by BYOLTransform to create the second view of an image.

Input to this transform:

PIL Image. (Tensor inputs are supported when torchvision transforms v2 are available.)

Output of this transform:

Tensor.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• Solarization

• ImageNet normalization

• [0]: Bootstrap Your Own Latent, 2020, https://arxiv.org/pdf/2006.07733.pdf

__call__(image: Union[Tensor, Image]) → Tensor

Applies the transforms to the input image.

Parameters

image – The input image to apply the transforms to.

Returns

The transformed image.

__init__(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.4, cj_contrast: float = 0.4, cj_sat: float = 0.2, cj_hue: float = 0.1, min_scale: float = 0.08, random_gray_scale: float = 0.2, gaussian_blur: float = 0.1, solarization_prob: float = 0.2, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Initializes BYOLView2Transform.

Parameters

• input_size – Size of the input image in pixels.

• cj_prob – Probability that color jitter is applied.

• cj_strength – Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value. For datasets with small images, such as CIFAR, it is recommended to set cj_strength to 0.5.

• cj_bright – How much to jitter brightness.

• cj_contrast – How much to jitter contrast.

• cj_sat – How much to jitter saturation.

• cj_hue – How much to jitter hue.

• min_scale – Minimum size of the randomized crop relative to the input_size.

• random_gray_scale – Probability of conversion to grayscale.

• gaussian_blur – Probability of Gaussian blur.

• solarization_prob – Probability of solarization.

• sigmas – Tuple of min and max value from which the std of the gaussian kernel is sampled.

• vf_prob – Probability that vertical flip is applied.

• hf_prob – Probability that horizontal flip is applied.

• rr_prob – Probability that random rotation is applied.

• rr_degrees – Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

• normalize – Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.densecl_transform.DenseCLTransform(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.4, cj_contrast: float = 0.4, cj_sat: float = 0.4, cj_hue: float = 0.1, min_scale: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the transformations for DenseCL [0].

Identical to MoCoV2Transform.

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length 2.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• ImageNet normalization

• [0]: 2021, DenseCL: https://arxiv.org/abs/2011.09157

input_size

Size of the input image in pixels.

cj_prob

Probability that color jitter is applied.

cj_strength

Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value. For datasets with small images, such as CIFAR, it is recommended to set cj_strenght to 0.5.

cj_bright

How much to jitter brightness.

cj_contrast

How much to jitter contrast.

cj_sat

How much to jitter saturation.

cj_hue

How much to jitter hue.

min_scale

Minimum size of the randomized crop relative to the input_size.

random_gray_scale

Probability of conversion to grayscale.

gaussian_blur

Probability of Gaussian blur.

sigmas

Tuple of min and max value from which the std of the gaussian kernel is sampled.

vf_prob

Probability that vertical flip is applied.

hf_prob

Probability that horizontal flip is applied.

rr_prob

Probability that random rotation is applied.

rr_degrees

Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

normalize

Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.detcon_transform.DetConSTransform(grid_size: Optional[Tuple[int, int]] = None, mask_fn: Optional[Callable[[Union[Image, Tensor]], Union[Image, Tensor]]] = None, gaussian_blur_t1: float = 1.0, gaussian_blur_t2: float = 0.0, input_size: Union[Tuple[int, int], int] = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, min_scale: float = 0.08, random_gray_scale: float = 0.2, kernel_size: Tuple[float, float] = (23, 23), sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Union[float, Tuple[float, float]] = 0.0, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the transformations for DetConS [0], based on SimCLR.[1]

This transform creates two views of the input data, where the second view is different in that it does not apply Gaussian blurring.

Input to this transform:

Arbitrary data structure containing images and masks, that is compatible with torchvision transforms v2.[2]

Output of this transform:

A list of two views, where each view is a transformed version of the input.

Applies the following augmentations by default:

• RandomResizedCrop

• RandomHorizontalFlip

• RandomVerticalFlip

• RandomRotation

• ColorJitter

• RandomGrayscale

• GaussianBlur (only for the first view)

The segmentation mask can be provided in one of three ways:

• Passing grid_size segments the image into a regular grid.

• Passing mask_fn derives the mask from each image on the fly, for example with an unsupervised segmentation algorithm such as skimage.segmentation.felzenszwalb. The mask is generated once per image and shared by both views, so their region correspondence is preserved.

• Passing neither expects the caller to supply a pre-segmented mask alongside the image.

grid_size and mask_fn are mutually exclusive.

References

• [0] DetCon, 2021, https://arxiv.org/abs/2103.10957

• [1] SimCLR, 2020, https://arxiv.org/abs/2002.05709

• [2] torchvision Getting started with transforms v2, https://pytorch.org/vision/main/auto_examples/transforms/plot_transforms_getting_started.html

grid_size

Size of the grid segmentation as a tuple (num_rows, num_cols), or None if the segmentation mask is provided by the user or by mask_fn.

mask_fn

Optional callable that maps an image to a segmentation mask. It receives the image passed to the transform and returns the integer segmentation labels as a Tensor or PIL image, for example the output of an unsupervised segmenter such as skimage.segmentation.felzenszwalb wrapped as a tensor. The transform wraps the result into a torchvision.tv_tensors.Mask internally. It must be picklable to run in DataLoader worker processes, so use a module-level function or functools.partial rather than a lambda. Mutually exclusive with grid_size.

gaussian_blur_t1

Probability of applying Gaussian blur to the first view.

gaussian_blur_t2

Probability of applying Gaussian blur to the second view.

input_size

Size of the desired model input in pixels.

cj_prob

Probability that color jitter is applied.

cj_strength

Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value. For datasets with small images, such as CIFAR, it is recommended to set cj_strength to 0.5.

cj_bright

How much to jitter brightness.

cj_contrast

How much to jitter contrast.

cj_sat

How much to jitter saturation.

cj_hue

How much to jitter hue.

min_scale

Minimum size of the randomized crop relative to the input image.

random_gray_scale

Probability of conversion to grayscale.

kernel_size

Size of the Gaussian kernel for Gaussian blur.

sigmas

Tuple of min and max value from which the std of the gaussian kernel is sampled.

vf_prob

Probability that vertical flip is applied.

hf_prob

Probability that horizontal flip is applied.

rr_prob

Probability that random rotation is applied.

rr_degrees

Range of degrees to select from. If degrees is a number instead of sequence like (min, max), the range of degrees will be (-degrees, +degrees). The rotation is applied counter-clockwise.

normalize

Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

__call__(*args: Any) → List[Any]

Creates two views of the input, generating the mask via mask_fn if set.

When mask_fn is provided, the transform is called with the image only. The mask is generated once from that image and wrapped into a torchvision.tv_tensors.Mask, then passed through both view pipelines together with the image, so both views share the same segmentation. Otherwise the arguments are forwarded unchanged (image plus an optional pre-segmented or grid-placeholder mask).

Parameters

*args – Either a single image (when mask_fn is set) or an image together with a mask, compatible with torchvision transforms v2.

Returns

A list of two views, where each view is a transformed version of the input.

__init__(grid_size: Optional[Tuple[int, int]] = None, mask_fn: Optional[Callable[[Union[Image, Tensor]], Union[Image, Tensor]]] = None, gaussian_blur_t1: float = 1.0, gaussian_blur_t2: float = 0.0, input_size: Union[Tuple[int, int], int] = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, min_scale: float = 0.08, random_gray_scale: float = 0.2, kernel_size: Tuple[float, float] = (23, 23), sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Union[float, Tuple[float, float]] = 0.0, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]}) → None

class lightly.transforms.detcon_transform.DetConSViewTransform(input_size: Union[Tuple[int, int], int] = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, min_scale: float = 0.08, random_gray_scale: float = 0.2, gaussian_blur: float = 1.0, kernel_size: Tuple[float, float] = (23, 23), sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Union[float, Tuple[float, float]] = 0.0, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Transforms an image into a view for DetConS [0, 1].

Used by DetConSTransform to create the views of an image.

Input to this transform:

Arbitrary data structure containing images and masks compatible with torchvision transforms v2.

Output of this transform:

Transformed data structure of the same type as the input.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• ImageNet normalization

• [0]: DetCon, 2021, https://arxiv.org/abs/2103.10957

• [1]: SimCLR, 2020, https://arxiv.org/abs/2002.05709

__call__(*args: Any) → Any

Applies the transforms to arbitrary positional arguments containing arbitrary data structures of images, bounding boxes and masks.

Parameters

*args – Arbitrary positional arguments consisting of arbitrary data structures containing images, bounding boxes and masks.

Returns

The transformed input in the same data structure as the input.

__init__(input_size: Union[Tuple[int, int], int] = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, min_scale: float = 0.08, random_gray_scale: float = 0.2, gaussian_blur: float = 1.0, kernel_size: Tuple[float, float] = (23, 23), sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Union[float, Tuple[float, float]] = 0.0, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]}) → None

Initializes DetConSViewTransform.

Parameters

• input_size – Size of the desired model input in pixels.

• cj_prob – Probability that color jitter is applied.

• cj_strength – Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value. For datasets with small images, such as CIFAR, it is recommended to set cj_strength to 0.5.

• cj_bright – How much to jitter brightness.

• cj_contrast – How much to jitter contrast.

• cj_sat – How much to jitter saturation.

• cj_hue – How much to jitter hue.

• min_scale – Minimum size of the randomized crop relative to the input_size.

• random_gray_scale – Probability of conversion to grayscale.

• gaussian_blur – Probability of Gaussian blur.

• kernel_size – Size of the Gaussian kernel for Gaussian blur.

• sigmas – Tuple of min and max value from which the std of the gaussian kernel is sampled.

• vf_prob – Probability that vertical flip is applied.

• hf_prob – Probability that horizontal flip is applied.

• rr_prob – Probability that random rotation is applied.

• rr_degrees – Range of degrees to select from for random rotation. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

• normalize – Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.dino_transform.DINOTransform(global_crop_size: int = 224, global_crop_scale: Tuple[float, float] = (0.4, 1.0), local_crop_size: int = 96, local_crop_scale: Tuple[float, float] = (0.05, 0.4), n_local_views: int = 6, hf_prob: float = 0.5, vf_prob: float = 0, rr_prob: float = 0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, cj_prob: float = 0.8, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: Tuple[float, float, float] = (1.0, 0.1, 0.5), sigmas: Tuple[float, float] = (0.1, 2), solarization_prob: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the global and local view augmentations for DINO [0].

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length 2 * global + n_local_views. (8 by default)

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• Random solarization

• ImageNet normalization

This class generates two global and a user defined number of local views for each image in a batch. The code is adapted from [1].

• [0]: DINO, 2021, https://arxiv.org/abs/2104.14294

• [1]: https://github.com/facebookresearch/dino

global_crop_size

Crop size of the global views.

global_crop_scale

Tuple of min and max scales relative to global_crop_size.

local_crop_size

Crop size of the local views.

local_crop_scale

Tuple of min and max scales relative to local_crop_size.

n_local_views

Number of generated local views.

hf_prob

Probability that horizontal flip is applied.

vf_prob

Probability that vertical flip is applied.

rr_prob

Probability that random rotation is applied.

rr_degrees

Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

cj_prob

Probability that color jitter is applied.

cj_strength

Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

cj_bright

How much to jitter brightness.

cj_contrast

How much to jitter contrast.

cj_sat

How much to jitter saturation.

cj_hue

How much to jitter hue.

random_gray_scale

Probability of conversion to grayscale.

gaussian_blur

Tuple of probabilities to apply gaussian blur on the different views. The input is ordered as follows: (global_view_0, global_view_1, local_views)

sigmas

Tuple of min and max value from which the std of the gaussian kernel is sampled.

solarization

Probability to apply solarization on the second global view.

normalize

Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

__init__(global_crop_size: int = 224, global_crop_scale: Tuple[float, float] = (0.4, 1.0), local_crop_size: int = 96, local_crop_scale: Tuple[float, float] = (0.05, 0.4), n_local_views: int = 6, hf_prob: float = 0.5, vf_prob: float = 0, rr_prob: float = 0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, cj_prob: float = 0.8, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: Tuple[float, float, float] = (1.0, 0.1, 0.5), sigmas: Tuple[float, float] = (0.1, 2), solarization_prob: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

class lightly.transforms.dino_transform.DINOViewTransform(crop_size: int = 224, crop_scale: Tuple[float, float] = (0.4, 1.0), hf_prob: float = 0.5, vf_prob: float = 0, rr_prob: float = 0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, cj_prob: float = 0.8, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 1.0, sigmas: Tuple[float, float] = (0.1, 2), solarization_prob: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Transforms an image into a single view for DINO [0].

Used by DINOTransform to create the global and local views of an image.

Input to this transform:

PIL Image. (Tensor inputs are supported when torchvision transforms v2 are available.)

Output of this transform:

Tensor.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Random vertical flip

• Random rotation

• Color jitter

• Random gray scale

• Gaussian blur

• Random solarization

• ImageNet normalization

• [0]: DINO, 2021, https://arxiv.org/abs/2104.14294

__call__(image: Union[Tensor, Image]) → Tensor

Applies the transforms to the input image.

Parameters

image – The input image to apply the transforms to.

Returns

The transformed image.

__init__(crop_size: int = 224, crop_scale: Tuple[float, float] = (0.4, 1.0), hf_prob: float = 0.5, vf_prob: float = 0, rr_prob: float = 0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, cj_prob: float = 0.8, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 1.0, sigmas: Tuple[float, float] = (0.1, 2), solarization_prob: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Initializes DINOViewTransform.

Parameters

• crop_size – Size of the cropped image in pixels.

• crop_scale – Tuple of min and max scales relative to crop_size.

• hf_prob – Probability that horizontal flip is applied.

• vf_prob – Probability that vertical flip is applied.

• rr_prob – Probability that random rotation is applied.

• rr_degrees – Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

• cj_prob – Probability that color jitter is applied.

• cj_strength – Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

• cj_bright – How much to jitter brightness.

• cj_contrast – How much to jitter contrast.

• cj_sat – How much to jitter saturation.

• cj_hue – How much to jitter hue.

• random_gray_scale – Probability of conversion to grayscale.

• gaussian_blur – Probability of Gaussian blur.

• sigmas – Tuple of min and max value from which the std of the gaussian kernel is sampled.

• solarization_prob – Probability to apply solarization.

• normalize – Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.fast_siam_transform.FastSiamTransform(num_views: int = 4, input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.4, cj_contrast: float = 0.4, cj_sat: float = 0.4, cj_hue: float = 0.1, min_scale: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the transformations for FastSiam.

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length 4.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• ImageNet normalization

num_views

Number of views (num_views = K+1 where K is the number of target views).

input_size

Size of the input image in pixels.

cj_prob

Probability that color jitter is applied.

cj_strength

Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value. For datasets with small images, such as CIFAR, it is recommended to set cj_strength to 0.5.

cj_bright

How much to jitter brightness.

cj_contrast

How much to jitter contrast.

cj_sat

How much to jitter saturation.

cj_hue

How much to jitter hue.

min_scale

Minimum size of the randomized crop relative to the input_size.

random_gray_scale

Probability of conversion to grayscale.

gaussian_blur

Probability of Gaussian blur.

sigmas

Tuple of min and max value from which the std of the gaussian kernel is sampled.

vf_prob

Probability that vertical flip is applied.

hf_prob

Probability that horizontal flip is applied.

rr_prob

Probability that random rotation is applied.

rr_degrees

Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

normalize

Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.fda_transform.FDATransform(view_1_transform: Optional[FDAView1Transform] = None, view_2_transform: Optional[FDAView2Transform] = None)

Implements the transformations for FDA[0].

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length 2.

Applies the following augmentations by default:

• Random resized crop

• RFFT 2D transform

• Amplitude rescale transform

• Phase shift transform

• Random frequency mask transform

• Gaussian mixture mask

• IRFFT 2D transform

• Color jitter

• Random grayscale

• Gaussian blur

• Random solarization

• Random horizontal flip

• [0]: Disentangling the Effects of Data Augmentation and Format Transform in

Self-Supervised Learning of Image Representations, 2023, https://arxiv.org/pdf/2312.02205

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of [tensor, tensor].

view_1_transform

The transform for the first view.

view_2_transform

The transform for the second view.

__init__(view_1_transform: Optional[FDAView1Transform] = None, view_2_transform: Optional[FDAView2Transform] = None)

class lightly.transforms.fda_transform.FDAView1Transform(input_size: int = 224, min_scale: float = 0.08, cj_prob: float = 0.8, cj_contrast: float = 0.4, cj_bright: float = 0.4, cj_sat: float = 0.2, cj_hue: float = 0.1, cj_strength: float = 1.0, random_gray_scale: float = 0.2, gaussian_blur: float = 1.0, sigmas: Tuple[float, float] = (0.1, 2), ampl_rescale_range: Tuple[float, float] = (0.8, 1.75), ampl_rescale_prob: float = 0.2, phase_shift_range: Tuple[float, float] = (0.4, 0.7), phase_shift_prob: float = 0.2, rand_freq_mask_range: Tuple[float, float] = (0.01, 0.1), rand_freq_mask_prob: float = 0.5, gmm_num_gaussians: int = 20, gmm_std_range: Tuple[float, float] = (10, 15), gmm_prob: float = 0.2, solarization_prob: float = 0.0, vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Transforms an image into the first view for FDA [0].

Used by FDATransform to create the first view of an image.

Input to this transform:

PIL Image. (Tensor inputs are supported when torchvision transforms v2 are available.)

Output of this transform:

Tensor.

Applies the following augmentations by default:

• Random resized crop

• RFFT 2D transform

• Amplitude rescale transform

• Phase shift transform

• Random frequency mask transform

• Gaussian mixture mask

• IRFFT 2D transform

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• ImageNet normalization

• [0]: Disentangling the Effects of Data Augmentation and Format Transform in

  Self-Supervised Learning of Image Representations, 2023, https://arxiv.org/pdf/2312.02205

__call__(image: Union[Tensor, Image]) → Tensor

Applies the transforms to the input image.

Parameters

image – The input image to apply the transforms to.

Returns

The transformed image.

__init__(input_size: int = 224, min_scale: float = 0.08, cj_prob: float = 0.8, cj_contrast: float = 0.4, cj_bright: float = 0.4, cj_sat: float = 0.2, cj_hue: float = 0.1, cj_strength: float = 1.0, random_gray_scale: float = 0.2, gaussian_blur: float = 1.0, sigmas: Tuple[float, float] = (0.1, 2), ampl_rescale_range: Tuple[float, float] = (0.8, 1.75), ampl_rescale_prob: float = 0.2, phase_shift_range: Tuple[float, float] = (0.4, 0.7), phase_shift_prob: float = 0.2, rand_freq_mask_range: Tuple[float, float] = (0.01, 0.1), rand_freq_mask_prob: float = 0.5, gmm_num_gaussians: int = 20, gmm_std_range: Tuple[float, float] = (10, 15), gmm_prob: float = 0.2, solarization_prob: float = 0.0, vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Initializes FDAView1Transform.

Parameters

• input_size – Size of the input image in pixels.

• min_scale – Minimum size of the randomized crop relative to the input_size.

• cj_prob – Probability that color jitter is applied.

• cj_contrast – How much to jitter contrast.

• cj_bright – How much to jitter brightness.

• cj_sat – How much to jitter saturation.

• cj_hue – How much to jitter hue.

• cj_strength – Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

• random_gray_scale – Probability of conversion to grayscale.

• gaussian_blur – Probability of Gaussian blur.

• sigmas – Tuple of min and max value from which the std of the gaussian kernel is sampled.

• ampl_rescale_range – Range of the amplitude rescaling factor for frequency domain augmentation.

• ampl_rescale_prob – Probability of applying amplitude rescaling.

• phase_shift_range – Range of the phase shift for frequency domain augmentation.

• phase_shift_prob – Probability of applying phase shifting.

• rand_freq_mask_range – Range for the random frequency mask.

• rand_freq_mask_prob – Probability of applying random frequency masking.

• gmm_num_gaussians – Number of Gaussians in the Gaussian mixture mask.

• gmm_std_range – Range of the standard deviation for the Gaussian mixture mask in pixels.

• gmm_prob – Probability of applying the Gaussian mixture mask.

• solarization_prob – Probability of solarization.

• vf_prob – Probability that vertical flip is applied.

• hf_prob – Probability that horizontal flip is applied.

• rr_prob – Probability that random rotation is applied.

• rr_degrees – Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

• normalize – Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.fda_transform.FDAView2Transform(input_size: int = 224, min_scale: float = 0.08, cj_prob: float = 0.8, cj_contrast: float = 0.4, cj_bright: float = 0.4, cj_sat: float = 0.2, cj_hue: float = 0.1, cj_strength: float = 1.0, random_gray_scale: float = 0.2, gaussian_blur: float = 0.1, sigmas: Tuple[float, float] = (0.1, 2), ampl_rescale_range: Tuple[float, float] = (0.8, 1.75), ampl_rescale_prob: float = 0.2, phase_shift_range: Tuple[float, float] = (0.4, 0.7), phase_shift_prob: float = 0.2, rand_freq_mask_range: Tuple[float, float] = (0.01, 0.1), rand_freq_mask_prob: float = 0.5, gmm_num_gaussians: int = 20, gmm_std_range: Tuple[float, float] = (10, 15), gmm_prob: float = 0.0, solarization_prob: float = 0.0, vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Transforms an image into the second view for FDA [0].

Used by FDATransform to create the second view of an image.

Input to this transform:

PIL Image. (Tensor inputs are supported when torchvision transforms v2 are available.)

Output of this transform:

Tensor.

Applies the following augmentations by default:

• Random resized crop

• RFFT 2D transform

• Amplitude rescale transform

• Phase shift transform

• Random frequency mask transform

• IRFFT 2D transform

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• ImageNet normalization

• [0]: Disentangling the Effects of Data Augmentation and Format Transform in

  Self-Supervised Learning of Image Representations, 2023, https://arxiv.org/pdf/2312.02205

__call__(image: Union[Tensor, Image]) → Tensor

Applies the transforms to the input image.

Parameters

image – The input image to apply the transforms to.

Returns

The transformed image.

__init__(input_size: int = 224, min_scale: float = 0.08, cj_prob: float = 0.8, cj_contrast: float = 0.4, cj_bright: float = 0.4, cj_sat: float = 0.2, cj_hue: float = 0.1, cj_strength: float = 1.0, random_gray_scale: float = 0.2, gaussian_blur: float = 0.1, sigmas: Tuple[float, float] = (0.1, 2), ampl_rescale_range: Tuple[float, float] = (0.8, 1.75), ampl_rescale_prob: float = 0.2, phase_shift_range: Tuple[float, float] = (0.4, 0.7), phase_shift_prob: float = 0.2, rand_freq_mask_range: Tuple[float, float] = (0.01, 0.1), rand_freq_mask_prob: float = 0.5, gmm_num_gaussians: int = 20, gmm_std_range: Tuple[float, float] = (10, 15), gmm_prob: float = 0.0, solarization_prob: float = 0.0, vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Initializes FDAView2Transform.

Parameters

• input_size – Size of the input image in pixels.

• min_scale – Minimum size of the randomized crop relative to the input_size.

• cj_prob – Probability that color jitter is applied.

• cj_contrast – How much to jitter contrast.

• cj_bright – How much to jitter brightness.

• cj_sat – How much to jitter saturation.

• cj_hue – How much to jitter hue.

• cj_strength – Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

• random_gray_scale – Probability of conversion to grayscale.

• gaussian_blur – Probability of Gaussian blur.

• sigmas – Tuple of min and max value from which the std of the gaussian kernel is sampled.

• ampl_rescale_range – Range of the amplitude rescaling factor for frequency domain augmentation.

• ampl_rescale_prob – Probability of applying amplitude rescaling.

• phase_shift_range – Range of the phase shift for frequency domain augmentation.

• phase_shift_prob – Probability of applying phase shifting.

• rand_freq_mask_range – Range for the random frequency mask.

• rand_freq_mask_prob – Probability of applying random frequency masking.

• gmm_num_gaussians – Number of Gaussians in the Gaussian mixture mask.

• gmm_std_range – Range of the standard deviation for the Gaussian mixture mask in pixels.

• gmm_prob – Probability of applying the Gaussian mixture mask.

• solarization_prob – Probability of solarization.

• vf_prob – Probability that vertical flip is applied.

• hf_prob – Probability that horizontal flip is applied.

• rr_prob – Probability that random rotation is applied.

• rr_degrees – Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

• normalize – Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.gaussian_blur.GaussianBlur(*, prob: float = 0.5, sigmas: Tuple[float, float] = (0.2, 2))

Implementation of random Gaussian blur.

Utilizes the built-in ImageFilter method from PIL to apply a Gaussian blur to the input image with a certain probability. The blur is further randomized by sampling uniformly the values of the standard deviation of the Gaussian kernel.

prob

Probability with which the blur is applied.

sigmas

Tuple of min and max value from which the std of the gaussian kernel is sampled.

__call__(sample: Union[Tensor, Image]) → Union[Tensor, Image]

Blurs the image with a given probability.

Parameters

sample – PIL image to which blur will be applied.

Returns

Blurred image or original image.

class lightly.transforms.ibot_transform.IBOTTransform(global_crop_size: int = 224, global_crop_scale: Tuple[float, float] = (0.25, 1.0), local_crop_size: int = 96, local_crop_scale: Tuple[float, float] = (0.05, 0.25), n_local_views: int = 10, hf_prob: float = 0.5, cj_prob: float = 0.8, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: Tuple[float, float, float] = (1.0, 0.1, 0.5), sigmas: Tuple[float, float] = (0.1, 2), solarization_prob: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the global and local view augmentations for iBOT [0].

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length 2 * global + n_local_views. (10 by default)

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• Random solarization

• ImageNet normalization

This class generates two global and a user defined number of local views for each image in a batch. The code is adapted from [1].

• [0]: iBOT, 2021, https://arxiv.org/abs/2111.07832

• [1]: https://github.com/bytedance/ibot/blob/main/main_ibot.py#L574

global_crop_size

Crop size of the global views.

global_crop_scale

Tuple of min and max scales relative to global_crop_size.

local_crop_size

Crop size of the local views.

local_crop_scale

Tuple of min and max scales relative to local_crop_size.

n_local_views

Number of generated local views.

hf_prob

Probability that horizontal flip is applied.

cj_prob

Probability that color jitter is applied.

cj_strength

Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

cj_bright

How much to jitter brightness.

cj_contrast

How much to jitter contrast.

cj_sat

How much to jitter saturation.

cj_hue

How much to jitter hue.

random_gray_scale

Probability of conversion to grayscale.

gaussian_blur

Tuple of probabilities to apply gaussian blur on the different views. The input is ordered as follows: (global_view_0, global_view_1, local_views)

sigmas

Tuple of min and max value from which the std of the gaussian kernel is sampled.

solarization

Probability to apply solarization on the second global view.

normalize

Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

__init__(global_crop_size: int = 224, global_crop_scale: Tuple[float, float] = (0.25, 1.0), local_crop_size: int = 96, local_crop_scale: Tuple[float, float] = (0.05, 0.25), n_local_views: int = 10, hf_prob: float = 0.5, cj_prob: float = 0.8, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: Tuple[float, float, float] = (1.0, 0.1, 0.5), sigmas: Tuple[float, float] = (0.1, 2), solarization_prob: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

class lightly.transforms.ibot_transform.IBOTViewTransform(crop_size: int = 224, crop_scale: Tuple[float, float] = (0.4, 1.0), hf_prob: float = 0.5, cj_prob: float = 0.8, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 1.0, sigmas: Tuple[float, float] = (0.1, 2), solarization_prob: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Transforms an image into a single view for iBOT [0].

Used by IBOTTransform to create the global and local views of an image.

Input to this transform:

PIL Image. (Tensor inputs are supported when torchvision transforms v2 are available.)

Output of this transform:

Tensor.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• Random solarization

• ImageNet normalization

• [0]: iBOT, 2021, https://arxiv.org/abs/2111.07832

__call__(image: Union[Tensor, Image]) → Tensor

Applies the transforms to the input image.

Parameters

image – The input image to apply the transforms to.

Returns

The transformed image.

__init__(crop_size: int = 224, crop_scale: Tuple[float, float] = (0.4, 1.0), hf_prob: float = 0.5, cj_prob: float = 0.8, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 1.0, sigmas: Tuple[float, float] = (0.1, 2), solarization_prob: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Initializes IBOTViewTransform.

Parameters

• crop_size – Size of the cropped image in pixels.

• crop_scale – Tuple of min and max scales relative to crop_size.

• hf_prob – Probability that horizontal flip is applied.

• cj_prob – Probability that color jitter is applied.

• cj_strength – Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

• cj_bright – How much to jitter brightness.

• cj_contrast – How much to jitter contrast.

• cj_sat – How much to jitter saturation.

• cj_hue – How much to jitter hue.

• random_gray_scale – Probability of conversion to grayscale.

• gaussian_blur – Probability of Gaussian blur.

• sigmas – Tuple of min and max value from which the std of the gaussian kernel is sampled.

• solarization_prob – Probability to apply solarization.

• normalize – Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.image_grid_transform.ImageGridTransform(transforms: Sequence[Compose])

Transforms an image into multiple views and grids.

Used for VICRegL.

transforms

A sequence of (image_grid_transform, view_transform) tuples. The image_grid_transform creates a new view and grid from the image. The view_transform further augments the view. Every transform tuple is applied once to the image, creating len(transforms) views and grids.

__call__(image: Union[Tensor, Image]) → Union[List[Tensor], List[Image]]

Transforms an image into multiple views.

Every transform in self.transforms creates a new view.

Parameters

image – Image to be transformed into multiple views and grids.

Returns

List of views and grids tensors or PIL images. In the VICRegL implementation it has size:

[
    [3, global_crop_size, global_crop_size],
    [3, local_crop_size, local_crop_size],
    [global_grid_size, global_grid_size, 2],
    [local_grid_size, local_grid_size, 2],
]

class lightly.transforms.jigsaw.Jigsaw(n_grid: int = 3, img_size: int = 255, crop_size: int = 64, transform: ~torchvision.transforms.v2._container.Compose = ToTensor(Compose(       ToImage()       ToDtype(scale=True) )))

Implementation of Jigsaw image augmentation, inspired from PyContrast library.

Generates n_grid**2 random crops and returns a list.

This augmentation is instrumental to PIRL.

n_grid

Side length of the meshgrid, sqrt of the number of crops.

img_size

Size of image.

crop_size

Size of crops.

transform

Transformation to apply on each crop.

Examples

>>> from lightly.transforms import Jigsaw
>>>
>>> jigsaw_crop = Jigsaw(
...     n_grid=3, img_size=255, crop_size=64, transform=T.ToTensor()
... )
>>>
>>> # img is a PIL image
>>> crops = jigsaw_crops(img)

__call__(img: Image) → Tensor

Performs the Jigsaw augmentation :Parameters: img – PIL image to perform Jigsaw augmentation on.

Returns

Torch tensor with stacked crops.

class lightly.transforms.mae_transform.MAETransform(input_size: Union[int, Tuple[int, int]] = 224, min_scale: float = 0.2, normalize: Dict[str, List[float]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the view augmentation for MAE [0].

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length 1.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• [0]: Masked Autoencoder, 2021, https://arxiv.org/abs/2111.06377

input_size

Size of the input image in pixels.

min_scale

Minimum size of the randomized crop relative to the input_size.

normalize

Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

__call__(image: Union[Tensor, Image]) → List[Tensor]

Applies the transforms to the input image.

Parameters

image – The input image to apply the transforms to.

Returns

The transformed image.

class lightly.transforms.mmcr_transform.MMCRTransform(k: int = 8, input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.4, cj_contrast: float = 0.4, cj_sat: float = 0.2, cj_hue: float = 0.1, min_scale: float = 0.08, random_gray_scale: float = 0.2, gaussian_blur: float = 1.0, solarization_prob: float = 0.0, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the transformations for MMCR[0], which are based on BYOL[1].

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length k.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• Solarization

• ImageNet normalization

Please refer to the BYOL implementation for additional details.

• [0]: Efficient Coding of Natural Images using Maximum Manifold Capacity

  Representations, 2023, https://arxiv.org/pdf/2303.03307.pdf

• [1]: Bootstrap Your Own Latent, 2020, https://arxiv.org/pdf/2006.07733.pdf

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of tensors of length k.

k

Number of views.

transform

The transform to apply to each view.

class lightly.transforms.moco_transform.MoCoV1Transform(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.4, cj_contrast: float = 0.4, cj_sat: float = 0.4, cj_hue: float = 0.4, min_scale: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 0.0, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the transformations for MoCo v1.

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length 2.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• ImageNet normalization

input_size

Size of the input image in pixels.

cj_prob

Probability that color jitter is applied.

cj_strength

Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

cj_bright

How much to jitter brightness.

cj_contrast

How much to jitter contrast.

cj_sat

How much to jitter saturation.

cj_hue

How much to jitter hue.

min_scale

Minimum size of the randomized crop relative to the input_size.

random_gray_scale

Probability of conversion to grayscale.

gaussian_blur

Probability of Gaussian blur.

sigmas

Tuple of min and max value from which the std of the gaussian kernel is sampled.

vf_prob

Probability that vertical flip is applied.

hf_prob

Probability that horizontal flip is applied.

rr_prob

Probability that random rotation is applied.

rr_degrees

Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

normalize

Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.moco_transform.MoCoV2Transform(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.4, cj_contrast: float = 0.4, cj_sat: float = 0.4, cj_hue: float = 0.1, min_scale: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the transformations for MoCo v2 [0].

Similar to SimCLRTransform, but with different values for color jittering and minimum scale of the random resized crop.

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length 2.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• ImageNet normalization

• [0]: MoCo v2, 2020, https://arxiv.org/abs/2003.04297

input_size

Size of the input image in pixels.

cj_prob

Probability that color jitter is applied.

cj_strength

Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value. For datasets with small images, such as CIFAR, it is recommended to set cj_strenght to 0.5.

cj_bright

How much to jitter brightness.

cj_contrast

How much to jitter contrast.

cj_sat

How much to jitter saturation.

cj_hue

How much to jitter hue.

min_scale

Minimum size of the randomized crop relative to the input_size.

random_gray_scale

Probability of conversion to grayscale.

gaussian_blur

Probability of Gaussian blur.

sigmas

Tuple of min and max value from which the std of the gaussian kernel is sampled.

vf_prob

Probability that vertical flip is applied.

hf_prob

Probability that horizontal flip is applied.

rr_prob

Probability that random rotation is applied.

rr_degrees

Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

normalize

Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.msn_transform.MSNTransform(random_size: int = 224, focal_size: int = 96, random_views: int = 2, focal_views: int = 10, random_crop_scale: Tuple[float, float] = (0.3, 1.0), focal_crop_scale: Tuple[float, float] = (0.05, 0.3), cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), random_gray_scale: float = 0.2, hf_prob: float = 0.5, vf_prob: float = 0.0, normalize: Dict[str, List[float]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the transformations for MSN [0].

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length 2 * random_views + focal_views. (12 by default)

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• ImageNet normalization

Generates a set of random and focal views for each input image. The generated output is (views, target, filenames) where views is list with the following entries: [random_views_0, random_views_1, …, focal_views_0, focal_views_1, …].

• [0]: Masked Siamese Networks, 2022: https://arxiv.org/abs/2204.07141

random_size

Size of the random image views in pixels.

focal_size

Size of the focal image views in pixels.

random_views

Number of random views to generate.

focal_views

Number of focal views to generate.

random_crop_scale

Minimum and maximum size of the randomized crops for the relative to random_size.

focal_crop_scale

Minimum and maximum size of the randomized crops relative to focal_size.

cj_prob

Probability that color jittering is applied.

cj_strength

Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

cj_bright

How much to jitter brightness.

cj_contrast

How much to jitter contrast.

cj_sat

How much to jitter saturation.

cj_hue

How much to jitter hue.

gaussian_blur

Probability of Gaussian blur.

sigmas

Tuple of min and max value from which the std of the gaussian kernel is sampled.

random_gray_scale

Probability of conversion to grayscale.

hf_prob

Probability that horizontal flip is applied.

vf_prob

Probability that vertical flip is applied.

normalize

Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

__init__(random_size: int = 224, focal_size: int = 96, random_views: int = 2, focal_views: int = 10, random_crop_scale: Tuple[float, float] = (0.3, 1.0), focal_crop_scale: Tuple[float, float] = (0.05, 0.3), cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), random_gray_scale: float = 0.2, hf_prob: float = 0.5, vf_prob: float = 0.0, normalize: Dict[str, List[float]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

class lightly.transforms.msn_transform.MSNViewTransform(crop_size: int = 224, crop_scale: Tuple[float, float] = (0.3, 1.0), cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), random_gray_scale: float = 0.2, hf_prob: float = 0.5, vf_prob: float = 0.0, normalize: Dict[str, List[float]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Transforms an image into a single view for MSN [0].

Used by MSNTransform to create the random and focal views of an image.

Input to this transform:

PIL Image. (Tensor inputs are supported when torchvision transforms v2 are available.)

Output of this transform:

Tensor.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Random vertical flip

• Color jitter

• Random gray scale

• Gaussian blur

• ImageNet normalization

• [0]: Masked Siamese Networks, 2022: https://arxiv.org/abs/2204.07141

__call__(image: Union[Tensor, Image]) → Tensor

Applies the transforms to the input image.

Parameters

image – The input image to apply the transforms to.

Returns

The transformed image.

__init__(crop_size: int = 224, crop_scale: Tuple[float, float] = (0.3, 1.0), cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), random_gray_scale: float = 0.2, hf_prob: float = 0.5, vf_prob: float = 0.0, normalize: Dict[str, List[float]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Initializes MSNViewTransform.

Parameters

• crop_size – Size of the cropped image in pixels.

• crop_scale – Tuple of min and max scales relative to crop_size.

• cj_prob – Probability that color jitter is applied.

• cj_strength – Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

• cj_bright – How much to jitter brightness.

• cj_contrast – How much to jitter contrast.

• cj_sat – How much to jitter saturation.

• cj_hue – How much to jitter hue.

• gaussian_blur – Probability of Gaussian blur.

• sigmas – Tuple of min and max value from which the std of the gaussian kernel is sampled.

• random_gray_scale – Probability of conversion to grayscale.

• hf_prob – Probability that horizontal flip is applied.

• vf_prob – Probability that vertical flip is applied.

• normalize – Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.multi_crop_transform.MultiCropTranform(crop_sizes: Tuple[int, ...], crop_counts: Tuple[int, ...], crop_min_scales: Tuple[float, ...], crop_max_scales: Tuple[float, ...], transforms: Compose)

Implements the multi-crop transformations. Used by Swav.

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length crop_counts.

Applies the following augmentations by default:

• Random resized crop

• transforms passed by constructor

crop_sizes

Size of the input image in pixels for each crop category.

crop_counts

Number of crops for each crop category.

crop_min_scales

Min scales for each crop category.

crop_max_scales

Max_scales for each crop category.

transforms

Transforms which are applied to all crops.

class lightly.transforms.multi_view_transform.MultiViewTransform(transforms: Sequence[Compose])

Transforms an image into multiple views.

Parameters

transforms – A sequence of transforms. Every transform creates a new view.

__call__(image: Union[Tensor, Image]) → Union[List[Tensor], List[Image]]

Transforms an image into multiple views.

Every transform in self.transforms creates a new view.

Parameters

image – Image to be transformed into multiple views.

Returns

List of views.

class lightly.transforms.multi_view_transform_v2.MultiViewTransformV2(transforms: Sequence[Compose])

Transforms an image into multiple views and is compatible with transforms v2.

Parameters

transforms – A sequence of v2 transforms. Every transform creates a new view.

__call__(*args: Any) → List[Any]

Transforms a data structure containing images, bounding boxes and masks into a sequence of multiple views.

Every transform in self.transforms creates a new view.

Parameters

*args – Arbitrary positional arguments consisting of arbitrary data structures containing images, bounding boxes and masks.

Returns

A list of views, where each view is a transformed version of *args.

class lightly.transforms.pirl_transform.PIRLTransform(input_size: Union[int, Tuple[int, int]] = 64, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.4, cj_contrast: float = 0.4, cj_sat: float = 0.4, cj_hue: float = 0.4, min_scale: float = 0.08, random_gray_scale: float = 0.2, hf_prob: float = 0.5, n_grid: int = 3, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the transformations for PIRL [0]. The jigsaw augmentation is applied during the forward pass.

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length 2 (original, augmented).

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Jigsaw puzzle

• [0] PIRL, 2019: https://arxiv.org/abs/1912.01991

input_size

Size of the input image in pixels.

cj_prob

Probability that color jitter is applied.

cj_strength

Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

cj_bright

How much to jitter brightness.

cj_contrast

How much to jitter contrast.

cj_sat

How much to jitter saturation.

cj_hue

How much to jitter hue.

min_scale

Minimum size of the randomized crop relative to the input_size.

random_gray_scale

Probability of conversion to grayscale.

hf_prob

Probability that horizontal flip is applied.

n_grid

Sqrt of the number of grids in the jigsaw image.

normalize

Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.random_crop_and_flip_with_grid.Location(top: float, left: float, height: float, width: float, image_height: float, image_width: float, horizontal_flip: bool = False, vertical_flip: bool = False)

class lightly.transforms.random_crop_and_flip_with_grid.RandomHorizontalFlipWithLocation(p: float = 0.5)

See base class.

forward(img: Image, location: Location) → Tuple[Image, Location]

Horizontal flip image.

Horizontally flip the given image randomly with a given probability and return both the resulting image and the location.

Parameters

• img (PIL Image or Tensor) – Image to be flipped.

• location (Location) – Location object linked to the image

Returns

PIL Image or Tensor – Randomly flipped image Location: Location object with updated location.horizontal_flip parameter

class lightly.transforms.random_crop_and_flip_with_grid.RandomResizedCropAndFlip(grid_size: int = 7, crop_size: int = 224, crop_min_scale: float = 0.05, crop_max_scale: float = 0.2, hf_prob: float = 0.5, vf_prob: float = 0.5)

Randomly flip and crop an image.

A PyTorch module that applies random cropping, horizontal and vertical flipping to an image, and returns the transformed image and a grid tensor used to map the image back to the original image space in an NxN grid.

Parameters

• grid_size – The number of grid cells in the output grid tensor.

• crop_size – The size (in pixels) of the random crops.

• crop_min_scale – The minimum scale factor for random resized crops.

• crop_max_scale – The maximum scale factor for random resized crops.

• hf_prob – The probability of applying horizontal flipping to the image.

• vf_prob – The probability of applying vertical flipping to the image.

forward(img: Image) → Tuple[Image, Tensor]

Applies random cropping and horizontal flipping to an image, and returns the transformed image and a grid tensor used to map the image back to the original image space in an NxN grid.

Parameters

img – The input PIL image.

Returns

A tuple containing the transformed PIL image and the grid tensor.

location_to_NxN_grid(location: Location) → Tensor

Create grid from location object.

Create a grid tensor with grid_size rows and grid_size columns, where each cell represents a region of the original image. The grid is used to map the cropped and transformed image back to the original image space.

Parameters

location – An instance of the Location class, containing the location and size of the transformed image in the original image space.

Returns

A grid tensor of shape (grid_size, grid_size, 2), where the last dimension represents the (x, y) coordinate of the center of each cell in the original image space.

class lightly.transforms.random_crop_and_flip_with_grid.RandomResizedCropWithLocation(size: Union[int, Sequence[int]], scale: tuple[float, float] = (0.08, 1.0), ratio: tuple[float, float] = (0.75, 1.3333333333333333), interpolation: Union[InterpolationMode, int] = InterpolationMode.BILINEAR, antialias: Optional[bool] = True)

Do a random resized crop and return both the resulting image and the location. See base class.

forward(img: Image) → Tuple[Image, Location]

Apply a random resized crop and return the cropped image with its location.

Parameters

img (PIL Image or Tensor) – Image to be cropped.

Returns

PIL Image or Tensor – Randomly cropped image Location: Location object containing crop parameters

static get_params(img: Tensor, scale: list[float], ratio: list[float]) → tuple[int, int, int, int]

Get parameters for crop for a random sized crop.

Parameters

• img (PIL Image or Tensor) – Input image.

• scale (list) – range of scale of the origin size cropped

• ratio (list) – range of aspect ratio of the origin aspect ratio cropped

Returns

tuple – params (i, j, h, w) to be passed to crop for a random sized crop.

class lightly.transforms.random_crop_and_flip_with_grid.RandomVerticalFlipWithLocation(p: float = 0.5)

See base class.

forward(img: Image, location: Location) → Tuple[Image, Location]

Vertical flip image.

Vertically flip the given image randomly with a given probability and return both the resulting image and the location.

Parameters

• img (PIL Image or Tensor) – Image to be flipped.

• location (Location) – Location object linked to the image

Returns

PIL Image or Tensor – Randomly flipped image Location: Location object with updated location.vertical_flip parameter

class lightly.transforms.rotation.RandomRotate(prob: float = 0.5, angle: int = 90)

Implementation of random rotation.

Randomly rotates an input image by a fixed angle. By default, we rotate the image by 90 degrees with a probability of 50%.

This augmentation can be very useful for rotation invariant images such as in medical imaging or satellite imaginary.

prob

Probability with which image is rotated.

angle

Angle by which the image is rotated. We recommend multiples of 90 to prevent rasterization artifacts. If you pick numbers like 90, 180, 270 the tensor will be rotated without introducing any artifacts.

__call__(image: Union[Image, Tensor]) → Union[Image, Tensor]

Rotates the image with a given probability.

Parameters

image – PIL image or tensor which will be rotated.

Returns

Rotated image or original image.

class lightly.transforms.rotation.RandomRotateDegrees(prob: float, degrees: Union[float, Tuple[float, float]])

Random rotate image between two rotation angles with a random probability.

prob

Probability with which image is rotated.

degrees

Range of degrees to select from. If degrees is a number instead of a sequence like (min, max), the range of degrees will be (-degrees, +degrees). The image is rotated counter-clockwise with a random angle in the (min, max) range or in the (-degrees, +degrees) range.

__call__(image: Union[Image, Tensor]) → Union[Image, Tensor]

Rotates the images with a given probability.

Parameters

image – PIL image or tensor which will be rotated.

Returns

Rotated image or original image.

class lightly.transforms.simclr_transform.SimCLRTransform(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, min_scale: float = 0.08, random_gray_scale: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the transformations for SimCLR [0, 1].

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length 2.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• ImageNet normalization

Note that SimCLR v1 and v2 use the same data augmentations.

• [0]: SimCLR v1, 2020, https://arxiv.org/abs/2002.05709

• [1]: SimCLR v2, 2020, https://arxiv.org/abs/2006.10029

__init__(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, min_scale: float = 0.08, random_gray_scale: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Initializes SimCLRTransform.

Parameters

• input_size – Size of the input image in pixels.

• cj_prob – Probability that color jitter is applied.

• cj_strength – Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value. For datasets with small images, such as CIFAR, it is recommended to set cj_strength to 0.5.

• cj_bright – How much to jitter brightness.

• cj_contrast – How much to jitter contrast.

• cj_sat – How much to jitter saturation.

• cj_hue – How much to jitter hue.

• min_scale – Minimum size of the randomized crop relative to the input_size.

• random_gray_scale – Probability of conversion to grayscale.

• gaussian_blur – Probability of Gaussian blur.

• sigmas – Tuple of min and max value from which the std of the gaussian kernel is sampled.

• vf_prob – Probability that vertical flip is applied.

• hf_prob – Probability that horizontal flip is applied.

• rr_prob – Probability that random rotation is applied.

• rr_degrees – Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

• normalize – Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.simclr_transform.SimCLRViewTransform(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, min_scale: float = 0.08, random_gray_scale: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Transforms an image into a single view for SimCLR [0, 1].

Used by SimCLRTransform to create the views of an image.

Input to this transform:

PIL Image. (Tensor inputs are supported when torchvision transforms v2 are available.)

Output of this transform:

Tensor.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• ImageNet normalization

• [0]: SimCLR v1, 2020, https://arxiv.org/abs/2002.05709

• [1]: SimCLR v2, 2020, https://arxiv.org/abs/2006.10029

__call__(image: Union[Tensor, Image]) → Tensor

Applies the transforms to the input image.

Parameters

image – The input image to apply the transforms to.

Returns

The transformed image.

__init__(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, min_scale: float = 0.08, random_gray_scale: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Initializes SimCLRViewTransform.

Parameters

• input_size – Size of the input image in pixels.

• cj_prob – Probability that color jitter is applied.

• cj_strength – Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value. For datasets with small images, such as CIFAR, it is recommended to set cj_strength to 0.5.

• cj_bright – How much to jitter brightness.

• cj_contrast – How much to jitter contrast.

• cj_sat – How much to jitter saturation.

• cj_hue – How much to jitter hue.

• min_scale – Minimum size of the randomized crop relative to the input_size.

• random_gray_scale – Probability of conversion to grayscale.

• gaussian_blur – Probability of Gaussian blur.

• sigmas – Tuple of min and max value from which the std of the gaussian kernel is sampled.

• vf_prob – Probability that vertical flip is applied.

• hf_prob – Probability that horizontal flip is applied.

• rr_prob – Probability that random rotation is applied.

• rr_degrees – Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

• normalize – Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.simsiam_transform.SimSiamTransform(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.4, cj_contrast: float = 0.4, cj_sat: float = 0.4, cj_hue: float = 0.1, min_scale: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the transformations for SimSiam.

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length 2.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• ImageNet normalization

input_size

Size of the input image in pixels.

cj_prob

Probability that color jitter is applied.

cj_strength

Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value. For datasets with small images, such as CIFAR, it is recommended to set cj_strength to 0.5.

cj_bright

How much to jitter brightness.

cj_contrast

How much to jitter contrast.

cj_sat

How much to jitter saturation.

cj_hue

How much to jitter hue.

min_scale

Minimum size of the randomized crop relative to the input_size.

random_gray_scale

Probability of conversion to grayscale.

gaussian_blur

Probability of Gaussian blur.

sigmas

Tuple of min and max value from which the std of the gaussian kernel is sampled.

vf_prob

Probability that vertical flip is applied.

hf_prob

Probability that horizontal flip is applied.

rr_prob

Probability that random rotation is applied.

rr_degrees

Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

normalize

Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

__init__(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.4, cj_contrast: float = 0.4, cj_sat: float = 0.4, cj_hue: float = 0.1, min_scale: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

class lightly.transforms.simsiam_transform.SimSiamViewTransform(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.4, cj_contrast: float = 0.4, cj_sat: float = 0.4, cj_hue: float = 0.1, min_scale: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Transforms an image into a single view for SimSiam.

Used by SimSiamTransform to create the views of an image.

Input to this transform:

PIL Image. (Tensor inputs are supported when torchvision transforms v2 are available.)

Output of this transform:

Tensor.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Random vertical flip

• Random rotation

• Color jitter

• Random gray scale

• Gaussian blur

• ImageNet normalization

__call__(image: Union[Tensor, Image]) → Tensor

Applies the transforms to the input image.

Parameters

image – The input image to apply the transforms to.

Returns

The transformed image.

__init__(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.4, cj_contrast: float = 0.4, cj_sat: float = 0.4, cj_hue: float = 0.1, min_scale: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Initializes SimSiamViewTransform.

Parameters

• input_size – Size of the input image in pixels.

• cj_prob – Probability that color jitter is applied.

• cj_strength – Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value. For datasets with small images, such as CIFAR, it is recommended to set cj_strength to 0.5.

• cj_bright – How much to jitter brightness.

• cj_contrast – How much to jitter contrast.

• cj_sat – How much to jitter saturation.

• cj_hue – How much to jitter hue.

• min_scale – Minimum size of the randomized crop relative to the input_size.

• random_gray_scale – Probability of conversion to grayscale.

• gaussian_blur – Probability of Gaussian blur.

• sigmas – Tuple of min and max value from which the std of the gaussian kernel is sampled.

• vf_prob – Probability that vertical flip is applied.

• hf_prob – Probability that horizontal flip is applied.

• rr_prob – Probability that random rotation is applied.

• rr_degrees – Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

• normalize – Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.smog_transform.SMoGTransform(crop_sizes: Tuple[int, int] = (224, 96), crop_counts: Tuple[int, int] = (4, 4), crop_min_scales: Tuple[float, float] = (0.2, 0.05), crop_max_scales: Tuple[float, float] = (1.0, 0.2), gaussian_blur_probs: Tuple[float, float] = (0.5, 0.1), gaussian_blur_sigmas: Tuple[float, float] = (0.1, 2), solarize_probs: Tuple[float, float] = (0.0, 0.2), hf_prob: float = 0.5, cj_prob: float = 1.0, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the transformations for SMoG.

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length sum(crop_counts). (8 by default)

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• Random solarization

• ImageNet normalization

crop_sizes

Size of the input image in pixels for each crop category.

crop_counts

Number of crops for each crop category.

crop_min_scales

Min scales for each crop category.

crop_max_scales

Max_scales for each crop category.

gaussian_blur_probs

Probability of Gaussian blur for each crop category.

gaussian_blur_sigmas

Tuple of min and max value from which the std of the gaussian kernel is sampled.

solarize_probs

Probability of solarization for each crop category.

hf_prob

Probability that horizontal flip is applied.

cj_prob

Probability that color jitter is applied.

cj_strength

Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

cj_bright

How much to jitter brightness.

cj_contrast

How much to jitter contrast.

cj_sat

How much to jitter saturation.

cj_hue

How much to jitter hue.

random_gray_scale

Probability of conversion to grayscale.

normalize

Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

__init__(crop_sizes: Tuple[int, int] = (224, 96), crop_counts: Tuple[int, int] = (4, 4), crop_min_scales: Tuple[float, float] = (0.2, 0.05), crop_max_scales: Tuple[float, float] = (1.0, 0.2), gaussian_blur_probs: Tuple[float, float] = (0.5, 0.1), gaussian_blur_sigmas: Tuple[float, float] = (0.1, 2), solarize_probs: Tuple[float, float] = (0.0, 0.2), hf_prob: float = 0.5, cj_prob: float = 1.0, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

class lightly.transforms.smog_transform.SmoGViewTransform(crop_size: int = 224, crop_min_scale: float = 0.2, crop_max_scale: float = 1.0, gaussian_blur_prob: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), solarize_prob: float = 0.0, hf_prob: float = 0.5, cj_prob: float = 1.0, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Transforms an image into a view for SMoG.

Used by SMoGTransform to create the views of an image.

Input to this transform:

PIL Image. (Tensor inputs are supported when torchvision transforms v2 are available.)

Output of this transform:

Tensor.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• Random solarization

• ImageNet normalization

__call__(image: Union[Tensor, Image]) → Tensor

Applies the transforms to the input image.

Parameters

image – The input image to apply the transforms to.

Returns

The transformed image.

__init__(crop_size: int = 224, crop_min_scale: float = 0.2, crop_max_scale: float = 1.0, gaussian_blur_prob: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), solarize_prob: float = 0.0, hf_prob: float = 0.5, cj_prob: float = 1.0, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Initializes SmoGViewTransform.

Parameters

• crop_size – Size of the input image in pixels.

• crop_min_scale – Minimum size of the randomized crop relative to the input_size.

• crop_max_scale – Maximum size of the randomized crop relative to the input_size.

• gaussian_blur_prob – Probability of Gaussian blur.

• sigmas – Tuple of min and max value from which the std of the gaussian kernel is sampled.

• solarize_prob – Probability of solarization.

• hf_prob – Probability that horizontal flip is applied.

• cj_prob – Probability that color jitter is applied.

• cj_strength – Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

• cj_bright – How much to jitter brightness.

• cj_contrast – How much to jitter contrast.

• cj_sat – How much to jitter saturation.

• cj_hue – How much to jitter hue.

• random_gray_scale – Probability of conversion to grayscale.

• normalize – Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.solarize.RandomSolarization(prob: float = 0.5, threshold: int = 128)

Implementation of random image Solarization.

Utilizes the integrated image operation solarize from Pillow. Solarization inverts all pixel values above a threshold (default: 128).

probability

Probability to apply the transformation

threshold

Threshold for solarization.

__call__(sample: Image) → Image

Solarizes the given input image

Parameters

sample – PIL image to which solarize will be applied.

Returns

Solarized image or original image.

class lightly.transforms.swav_transform.SwaVTransform(crop_sizes: Tuple[int, int] = (224, 96), crop_counts: Tuple[int, int] = (2, 6), crop_min_scales: Tuple[float, float] = (0.14, 0.05), crop_max_scales: Tuple[float, float] = (1.0, 0.14), hf_prob: float = 0.5, vf_prob: float = 0.0, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the multi-crop transformations for SwaV.

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length sum(crop_counts). (8 by default)

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• ImageNet normalization

crop_sizes

Size of the input image in pixels for each crop category.

crop_counts

Number of crops for each crop category.

crop_min_scales

Min scales for each crop category.

crop_max_scales

Max_scales for each crop category.

hf_prob

Probability that horizontal flip is applied.

vf_prob

Probability that vertical flip is applied.

rr_prob

Probability that random rotation is applied.

rr_degrees

Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

cj_prob

Probability that color jitter is applied.

cj_strength

Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

cj_bright

How much to jitter brightness.

cj_contrast

How much to jitter contrast.

cj_sat

How much to jitter saturation.

cj_hue

How much to jitter hue.

random_gray_scale

Probability of conversion to grayscale.

gaussian_blur

Probability of Gaussian blur.

sigmas

Tuple of min and max value from which the std of the gaussian kernel is sampled.

normalize

Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

__init__(crop_sizes: Tuple[int, int] = (224, 96), crop_counts: Tuple[int, int] = (2, 6), crop_min_scales: Tuple[float, float] = (0.14, 0.05), crop_max_scales: Tuple[float, float] = (1.0, 0.14), hf_prob: float = 0.5, vf_prob: float = 0.0, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

class lightly.transforms.swav_transform.SwaVViewTransform(hf_prob: float = 0.5, vf_prob: float = 0.0, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Transforms an image into a view for SwaV.

Used by SwaVTransform to create the views of an image.

Input to this transform:

PIL Image. (Tensor inputs are supported when torchvision transforms v2 are available.)

Output of this transform:

Tensor.

Applies the following augmentations by default:

• Random horizontal flip

• Random vertical flip

• Random rotation

• Color jitter

• Random gray scale

• Gaussian blur

• ImageNet normalization

__call__(image: Union[Tensor, Image]) → Tensor

Applies the transforms to the input image.

Parameters

image – The input image to apply the transforms to.

Returns

The transformed image.

__init__(hf_prob: float = 0.5, vf_prob: float = 0.0, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, cj_prob: float = 0.8, cj_strength: float = 1.0, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.8, cj_hue: float = 0.2, random_gray_scale: float = 0.2, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Initializes SwaVViewTransform.

Parameters

• hf_prob – Probability that horizontal flip is applied.

• vf_prob – Probability that vertical flip is applied.

• rr_prob – Probability that random rotation is applied.

• rr_degrees – Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

• cj_prob – Probability that color jitter is applied.

• cj_strength – Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

• cj_bright – How much to jitter brightness.

• cj_contrast – How much to jitter contrast.

• cj_sat – How much to jitter saturation.

• cj_hue – How much to jitter hue.

• random_gray_scale – Probability of conversion to grayscale.

• gaussian_blur – Probability of Gaussian blur.

• sigmas – Tuple of min and max value from which the std of the gaussian kernel is sampled.

• normalize – Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.vicreg_transform.VICRegTransform(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, min_scale: float = 0.08, random_gray_scale: float = 0.2, solarize_prob: float = 0.1, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Implements the transformations for VICReg.

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length 2.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Random solarization

• Gaussian blur

• ImageNet normalization

Similar to SimCLR transform but with extra solarization.

input_size

Size of the input image in pixels.

cj_prob

Probability that color jitter is applied.

cj_strength

Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

cj_bright

How much to jitter brightness.

cj_contrast

How much to jitter contrast.

cj_sat

How much to jitter saturation.

cj_hue

How much to jitter hue.

min_scale

Minimum size of the randomized crop relative to the input_size.

random_gray_scale

Probability of conversion to grayscale.

solarize_prob

Probability of solarization.

gaussian_blur

Probability of Gaussian blur.

sigmas

Tuple of min and max value from which the std of the gaussian kernel is sampled.

vf_prob

Probability that vertical flip is applied.

hf_prob

Probability that horizontal flip is applied.

rr_prob

Probability that random rotation is applied.

rr_degrees

Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

normalize

Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

__init__(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, min_scale: float = 0.08, random_gray_scale: float = 0.2, solarize_prob: float = 0.1, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.1, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

class lightly.transforms.vicreg_transform.VICRegViewTransform(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, min_scale: float = 0.08, random_gray_scale: float = 0.2, solarize_prob: float = 0.1, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.2, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Transforms an image into a single view for VICReg.

Used by VICRegTransform to create the views of an image.

Input to this transform:

PIL Image. (Tensor inputs are supported when torchvision transforms v2 are available.)

Output of this transform:

Tensor.

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Random vertical flip

• Random rotation

• Color jitter

• Random gray scale

• Random solarization

• Gaussian blur

• ImageNet normalization

__call__(image: Union[Tensor, Image]) → Tensor

Applies the transforms to the input image.

Parameters

image – The input image to apply the transforms to.

Returns

The transformed image.

__init__(input_size: int = 224, cj_prob: float = 0.8, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, min_scale: float = 0.08, random_gray_scale: float = 0.2, solarize_prob: float = 0.1, gaussian_blur: float = 0.5, sigmas: Tuple[float, float] = (0.2, 2), vf_prob: float = 0.0, hf_prob: float = 0.5, rr_prob: float = 0.0, rr_degrees: Optional[Union[float, Tuple[float, float]]] = None, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Initializes VICRegViewTransform.

Parameters

• input_size – Size of the input image in pixels.

• cj_prob – Probability that color jitter is applied.

• cj_strength – Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

• cj_bright – How much to jitter brightness.

• cj_contrast – How much to jitter contrast.

• cj_sat – How much to jitter saturation.

• cj_hue – How much to jitter hue.

• min_scale – Minimum size of the randomized crop relative to the input_size.

• random_gray_scale – Probability of conversion to grayscale.

• solarize_prob – Probability of solarization.

• gaussian_blur – Probability of Gaussian blur.

• sigmas – Tuple of min and max value from which the std of the gaussian kernel is sampled.

• vf_prob – Probability that vertical flip is applied.

• hf_prob – Probability that horizontal flip is applied.

• rr_prob – Probability that random rotation is applied.

• rr_degrees – Range of degrees to select from for random rotation. If rr_degrees is None, images are rotated by 90 degrees. If rr_degrees is a (min, max) tuple, images are rotated by a random angle in [min, max]. If rr_degrees is a single number, images are rotated by a random angle in [-rr_degrees, +rr_degrees]. All rotations are counter-clockwise.

• normalize – Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.

class lightly.transforms.vicregl_transform.VICRegLTransform(global_crop_size: int = 224, local_crop_size: int = 96, n_global_views: int = 2, n_local_views: int = 6, global_crop_scale: Tuple[float, float] = (0.2, 1.0), local_crop_scale: Tuple[float, float] = (0.05, 0.2), global_grid_size: int = 7, local_grid_size: int = 3, global_gaussian_blur_prob: float = 0.5, local_gaussian_blur_prob: float = 0.1, global_gaussian_blur_sigmas: Tuple[float, float] = (0.1, 2), local_gaussian_blur_sigmas: Tuple[float, float] = (0.1, 2), global_solarize_prob: float = 0.0, local_solarize_prob: float = 0.2, hf_prob: float = 0.5, vf_prob: float = 0.0, cj_prob: float = 1.0, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Transforms images for VICRegL.

Input to this transform:

PIL Image or Tensor.

Output of this transform:

List of Tensor of length n_global_views + n_local_views. (8 by default)

Applies the following augmentations by default:

• Random resized crop

• Random horizontal flip

• Color jitter

• Random gray scale

• Gaussian blur

• Random solarization

• ImageNet normalization

• [0]: VICRegL, 2022, https://arxiv.org/abs/2210.01571

global_crop_size

Size of the input image in pixels for the global crop views.

local_crop_size

Size of the input image in pixels for the local crop views.

n_global_views

Number of global crop views to generate.

n_local_views

Number of local crop views to generate. For ResNet backbones it is recommended to set this to 0, see [0].

global_crop_scale

Min and max scales for the global crop views.

local_crop_scale

Min and max scales for the local crop views.

global_grid_size

Grid size for the global crop views.

local_grid_size

Grid size for the local crop views.

global_gaussian_blur_prob

Probability of Gaussian blur for the global crop views.

local_gaussian_blur_prob

Probability of Gaussian blur for the local crop views.

global_gaussian_blur_sigmas

Tuple of min and max value from which the std of the gaussian kernel is sampled. Applied to global crop views.

local_gaussian_blur_sigmas

Tuple of min and max value from which the std of the gaussian kernel is sampled. Applied to local crop views.

global_solarize_prob

Probability of solarization for the global crop views.

local_solarize_prob

Probability of solarization for the local crop views.

hf_prob

Probability that horizontal flip is applied.

cj_prob

Probability that color jitter is applied.

cj_strength

Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

cj_bright

How much to jitter brightness.

cj_contrast

How much to jitter contrast.

cj_sat

How much to jitter saturation.

cj_hue

How much to jitter hue.

random_gray_scale

Probability of conversion to grayscale.

normalize

Dictionary with mean and standard deviation for normalization.

__init__(global_crop_size: int = 224, local_crop_size: int = 96, n_global_views: int = 2, n_local_views: int = 6, global_crop_scale: Tuple[float, float] = (0.2, 1.0), local_crop_scale: Tuple[float, float] = (0.05, 0.2), global_grid_size: int = 7, local_grid_size: int = 3, global_gaussian_blur_prob: float = 0.5, local_gaussian_blur_prob: float = 0.1, global_gaussian_blur_sigmas: Tuple[float, float] = (0.1, 2), local_gaussian_blur_sigmas: Tuple[float, float] = (0.1, 2), global_solarize_prob: float = 0.0, local_solarize_prob: float = 0.2, hf_prob: float = 0.5, vf_prob: float = 0.0, cj_prob: float = 1.0, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

class lightly.transforms.vicregl_transform.VICRegLViewTransform(gaussian_blur_prob: float = 0.5, gaussian_blur_sigmas: Tuple[float, float] = (0.1, 2), solarize_prob: float = 0.0, cj_prob: float = 1.0, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Transforms an image into a single view for VICRegL [0].

Used by VICRegLTransform to create the global and local views of an image. The crop and flip augmentations are applied separately by VICRegLTransform; this transform applies the remaining photometric augmentations.

Input to this transform:

PIL Image. (Tensor inputs are supported when torchvision transforms v2 are available.)

Output of this transform:

Tensor.

Applies the following augmentations by default:

• Color jitter

• Random gray scale

• Gaussian blur

• Random solarization

• ImageNet normalization

• [0]: VICRegL, 2022, https://arxiv.org/abs/2210.01571

__call__(image: Union[Tensor, Image]) → Tensor

Applies the transforms to the input image.

Parameters

image – The input image to apply the transforms to.

Returns

The transformed image.

__init__(gaussian_blur_prob: float = 0.5, gaussian_blur_sigmas: Tuple[float, float] = (0.1, 2), solarize_prob: float = 0.0, cj_prob: float = 1.0, cj_strength: float = 0.5, cj_bright: float = 0.8, cj_contrast: float = 0.8, cj_sat: float = 0.4, cj_hue: float = 0.2, random_gray_scale: float = 0.2, normalize: Union[None, Dict[str, List[float]]] = {'mean': [0.485, 0.456, 0.406], 'std': [0.229, 0.224, 0.225]})

Initializes VICRegLViewTransform.

Parameters

• gaussian_blur_prob – Probability of Gaussian blur.

• gaussian_blur_sigmas – Tuple of min and max value from which the std of the gaussian kernel is sampled.

• solarize_prob – Probability of solarization.

• cj_prob – Probability that color jitter is applied.

• cj_strength – Strength of the color jitter. cj_bright, cj_contrast, cj_sat, and cj_hue are multiplied by this value.

• cj_bright – How much to jitter brightness.

• cj_contrast – How much to jitter contrast.

• cj_sat – How much to jitter saturation.

• cj_hue – How much to jitter hue.

• random_gray_scale – Probability of conversion to grayscale.

• normalize – Dictionary with ‘mean’ and ‘std’ for torchvision.transforms.Normalize.