{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {},
   "source": [
    "# LightlyTrain with YOLOv12"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1",
   "metadata": {},
   "source": [
    "In this notebook we will demonstrate how you can use [LightlyTrain](https://docs.lightly.ai/train/stable/index.html) to pretrain a [YOLOv12 model by the original authors](https://github.com/sunsmarterjie/yolov12). To this end, we will first use the raw images (**no labels**) from the [PASCAL/VOC dataset](http://host.robots.ox.ac.uk/pascal/VOC/) for pretraining and then we'll fine-tune later on the super small labeled [COCO8 dataset](https://docs.ultralytics.com/datasets/detect/coco8/).\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lightly-ai/lightly-train/blob/main/examples/notebooks/yolov12.ipynb)\n",
    "\n",
    "> **Important**: When running on Google Colab make sure to select a GPU runtime for faster processing. You can do this by going to `Runtime` > `Change runtime type` and selecting a GPU hardware accelerator."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2",
   "metadata": {},
   "source": [
    "## Installation\n",
    "\n",
    "You can install `lightly_train` directly from PyPI using pip."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install lightly_train"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4",
   "metadata": {},
   "source": [
    "> **Important**: LightlyTrain is officially supported on\n",
    "> - Linux: CPU or CUDA\n",
    "> - MacOS: CPU only\n",
    "> - Windows (experimental): CPU or CUDA\n",
    ">\n",
    "> We are planning to support MPS for MacOS.\n",
    ">\n",
    "> Check the installation instructions for more details on installation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5",
   "metadata": {},
   "source": [
    "Please install YOLOv12 directly from GitHub through:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install git+https://github.com/sunsmarterjie/yolov12"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7",
   "metadata": {},
   "source": [
    "> **Note:**\n",
    "> YOLOv12 is a custom fork of a specific version of the `ultralytics` package. For this reason, YOLOv12 is not fully integrated with LightlyTrain and has to be installed manually."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8",
   "metadata": {},
   "source": [
    "In case you are facing a version mismatch issue using CUDA and FlashAttention:\n",
    "\n",
    "```bash\n",
    "FlashAttention is not available on this device. Using scaled_dot_product_attention instead.\n",
    "```\n",
    "\n",
    "you can fix it by running the following commands:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install flash-attn --no-build-isolation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10",
   "metadata": {},
   "source": [
    "You can verify the results by:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11",
   "metadata": {},
   "outputs": [],
   "source": [
    "import flash_attn\n",
    "\n",
    "print(\"FlashAttention version:\", flash_attn.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12",
   "metadata": {},
   "source": [
    "and a successful installation will give you:\n",
    "\n",
    "```bash\n",
    "FlashAttention version: <some-version>\n",
    "```\n",
    "\n",
    "See this [GitHub issue](https://github.com/sunsmarterjie/yolov12/issues/66) for more information."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13",
   "metadata": {},
   "source": [
    "## Pretraining with Lightly**Train** on VOC\n",
    "We can directly use Ultralytics' `check_det_dataset` function to download the VOC dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14",
   "metadata": {},
   "outputs": [],
   "source": [
    "from ultralytics.data.utils import check_det_dataset\n",
    "\n",
    "dataset = check_det_dataset(\"VOC.yaml\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15",
   "metadata": {},
   "source": [
    "Ultralytics always uses a fixed directory to save your datasets and you can fetch the location through their `settings` module:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "16",
   "metadata": {},
   "outputs": [],
   "source": [
    "from ultralytics import settings\n",
    "\n",
    "settings[\"datasets_dir\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17",
   "metadata": {},
   "source": [
    "Our dataset directory is now ready under the path from above and will have the following structure:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18",
   "metadata": {},
   "source": [
    "```bash\n",
    "<some-path>/datasets/VOC\n",
    "├── images\n",
    "│   ├── test2007\n",
    "│   ├── train2007\n",
    "│   ├── train2012\n",
    "│   ├── val2007\n",
    "│   └── val2012\n",
    "└── labels\n",
    "    ├── test2007\n",
    "    ├── train2007\n",
    "    ├── train2012\n",
    "    ├── val2007\n",
    "    └── val2012\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19",
   "metadata": {},
   "source": [
    "We will not use the labels for pretraining, so you could safely delete them:\n",
    "```bash\n",
    "rm -rf <some-path>/datasets/VOC/labels\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20",
   "metadata": {},
   "source": [
    "For the pretraining we then only need to point to the images directory and pass the model specification. Note that the config file is named `yolov12.yaml` instead of `yolo12.yaml`, as it would be in the official Ultralytics releases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21",
   "metadata": {},
   "outputs": [],
   "source": [
    "import lightly_train\n",
    "\n",
    "lightly_train.pretrain(\n",
    "    out=\"out/my_experiment\",  # Output directory.\n",
    "    data=f\"{settings['datasets_dir']}/VOC/images/train2012\",  # Directory with images, no labels!\n",
    "    model=\"ultralytics/yolov12s.yaml\",  # Pass the YOLO model.\n",
    "    epochs=10,  # Number of epochs to train\n",
    "    batch_size=32,  # Batch size\n",
    "    overwrite=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22",
   "metadata": {},
   "source": [
    "## Fine-tuning the pretrained model on COCO8\n",
    "Fine-tuning the previously pretrained model works in the same way as for any other Ultralytics model. Simply specify the path to the pretrained weights and give the `\"coc8.yaml\"` dataset to the `.train` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23",
   "metadata": {},
   "outputs": [],
   "source": [
    "from ultralytics import YOLO\n",
    "\n",
    "# Load the exported model.\n",
    "model = YOLO(\"out/my_experiment/exported_models/exported_last.pt\")\n",
    "\n",
    "# Fine-tune with ultralytics.\n",
    "model.train(data=\"coco8.yaml\", epochs=10)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}