Methods

LightlyTrain supports the following self-supervised learning methods:

• dino

  DINO is a popular self-supervised learning method that works well across various datasets, model architectures, and tasks.

• distillation (recommended)

  Distillation is a method that transfers knowledge from a pre-trained model, in particular a ViT-B/14 from DINOv2.

• densecl (experimental)

  DenseCL was developed for learning strong local image representations, particularly for detection and segmentation tasks.

• simclr

  SimCLR is a classic self-supervised learning method widely used as a baseline for model pre-training.