LightlyTrain with YOLOv12

In this notebook we will demonstrate how you can use LightlyTrain to pretrain a YOLOv12 model by the original authors. To this end, we will first use the raw images (no labels) from the PASCAL/VOC dataset for pretraining and then we’ll fine-tune later on the super small labeled COCO8 dataset.

Open In Colab

Important: When running on Google Colab make sure to select a GPU runtime for faster processing. You can do this by going to Runtime > Change runtime type and selecting a GPU hardware accelerator.

Installation

You can install lightly_train directly from PyPI using pip.

!pip install lightly_train

Important: LightlyTrain is officially supported on

  • Linux: CPU or CUDA

  • MacOS: CPU only

  • Windows (experimental): CPU or CUDA

We are planning to support MPS for MacOS.

Check the installation instructions for more details on installation.

Please install YOLOv12 directly from GitHub through:

!pip install git+https://github.com/sunsmarterjie/yolov12

Note: YOLOv12 is a custom fork of a specific version of the ultralytics package. For this reason, YOLOv12 is not fully integrated with LightlyTrain and has to be installed manually.

In case you are facing a version mismatch issue using CUDA and FlashAttention:

FlashAttention is not available on this device. Using scaled_dot_product_attention instead.

you can fix it by running the following commands:

!pip install flash-attn --no-build-isolation

You can verify the results by:

import flash_attn

print("FlashAttention version:", flash_attn.__version__)

and a successful installation will give you:

FlashAttention version: <some-version>

See this GitHub issue for more information.

Pretraining with LightlyTrain on VOC

We can directly use Ultralytics’ check_det_dataset function to download the VOC dataset.

from ultralytics.data.utils import check_det_dataset

dataset = check_det_dataset("VOC.yaml")

Ultralytics always uses a fixed directory to save your datasets and you can fetch the location through their settings module:

from ultralytics import settings

settings["datasets_dir"]

Our dataset directory is now ready under the path from above and will have the following structure:

<some-path>/datasets/VOC
├── images
│   ├── test2007
│   ├── train2007
│   ├── train2012
│   ├── val2007
│   └── val2012
└── labels
    ├── test2007
    ├── train2007
    ├── train2012
    ├── val2007
    └── val2012

We will not use the labels for pretraining, so you could safely delete them:

rm -rf <some-path>/datasets/VOC/labels

For the pretraining we then only need to point to the images directory and pass the model specification. Note that the config file is named yolov12.yaml instead of yolo12.yaml, as it would be in the official Ultralytics releases.

import lightly_train

lightly_train.pretrain(
    out="out/my_experiment",  # Output directory.
    data=f"{settings['datasets_dir']}/VOC/images/train2012",  # Directory with images, no labels!
    model="ultralytics/yolov12s.yaml",  # Pass the YOLO model.
    epochs=10,  # Number of epochs to train
    batch_size=32,  # Batch size
    overwrite=True,
)

Fine-tuning the pretrained model on COCO8

Fine-tuning the previously pretrained model works in the same way as for any other Ultralytics model. Simply specify the path to the pretrained weights and give the "coc8.yaml" dataset to the .train method.

from ultralytics import YOLO

# Load the exported model.
model = YOLO("out/my_experiment/exported_models/exported_last.pt")

# Fine-tune with ultralytics.
model.train(data="coco8.yaml", epochs=10)