LightlySSL¶
Learn from images and video without labels
Pretrain vision backbones with 28 self-supervised methods, or train a latent world model that plans from raw frames. All in plain PyTorch.
$ pip install lightly # the example scripts live in the repository $ git clone https://github.com/lightly-ai/lightly.git # pretrain a ResNet-18 with SimCLR on CIFAR-10 $ python lightly/examples/pytorch/simclr.py # or with Lightning on several GPUs $ python lightly/examples/pytorch_lightning_distributed/simclr.py
What’s new¶
New in 1.6
Latent world models with LeWM
A world model learns how a scene responds to actions. LeWM learns it from raw frames and actions, then plans by imagining the outcome of each move in its own embedding space.
Each agent (orange) gets only a picture of its goal (white box). It learned from random play: no rewards, no labels, no map of the room.
New in 1.6
CAPI
Masked image modeling that predicts the cluster assignments of masked patches instead of their pixels.
New in 1.5.26
Pixio, FroSSL and DetConS
Three more methods: a masked autoencoder variant, Frobenius norm minimization, and object-level contrastive learning.
New in 1.5.25
LeJEPA
Invariance plus SIGReg: no negative samples, momentum encoders or stop-gradients.
Start here¶
Install LightlySSL and learn the main concepts.
Step-by-step notebooks, from data loading to fine-tuning.
Reference implementations in PyTorch and Lightning.
Losses, heads, transforms and data utilities.
All methods¶
More from Lightly¶
Pretrain and fine-tune vision foundation models with a few lines of code.
Label, curate and inspect your datasets.
Join the community and our weekly paper sessions.
Star the repo, file issues and contribute.