{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {},
   "source": [
    "# Depth Estimation with Depth Anything V3\n",
    "\n",
    "[Depth Anything V3](https://huggingface.co/depth-anything) is a monocular depth model built on the [DINOv2](https://github.com/facebookresearch/dinov2) foundation backbone. LightlyTrain's version is a faithful implementation of the official code.\n",
    "\n",
    "This notebook shows both **relative** depth (up-to-scale) and **metric** depth (in meters).\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lightly-ai/lightly-train/blob/main/examples/notebooks/depth_estimation.ipynb)\n",
    "\n",
    "> **Important**: When running on Google Colab make sure to select a GPU runtime for faster processing. You can do this by going to `Runtime` > `Change runtime type` and selecting a GPU hardware accelerator."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1",
   "metadata": {},
   "source": [
    "## Installation\n",
    "\n",
    "LightlyTrain can be installed directly via `pip`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install lightly-train"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3",
   "metadata": {},
   "source": [
    "> **Important**: LightlyTrain is officially supported on\n",
    "> - Linux: CPU or CUDA\n",
    "> - MacOS: CPU only\n",
    "> - Windows (experimental): CPU or CUDA\n",
    ">\n",
    "> We are planning to support MPS for MacOS.\n",
    ">\n",
    "> Check the [installation instructions](https://docs.lightly.ai/train/stable/installation.html) for more details on installation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4",
   "metadata": {},
   "source": [
    "## Prediction using LightlyTrain's model weights\n",
    "\n",
    "### Download an example image\n",
    "\n",
    "Download an example image for inference with the following command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5",
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget -O image.jpg http://images.cocodataset.org/val2017/000000039769.jpg"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6",
   "metadata": {},
   "source": [
    "## Relative depth estimation\n",
    "\n",
    "Relative depth is predicted *up to scale*: values are consistent within an image (larger means farther) but have no absolute unit."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7",
   "metadata": {},
   "source": [
    "### Load the model weights\n",
    "\n",
    "Then load the model with LightlyTrain's `load_model` function. This will automatically download the model weights and load the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import lightly_train\n",
    "\n",
    "model = lightly_train.load_model(\"dinov2/dav3-relative-large\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9",
   "metadata": {},
   "source": [
    "### Predict relative depth map\n",
    "\n",
    "Call `predict` on the image to obtain the relative depth map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10",
   "metadata": {},
   "outputs": [],
   "source": [
    "depth = model.predict(\n",
    "    \"image.jpg\"\n",
    ")  # Depth map as a tensor of shape (H, W). Larger values mean farther from the camera."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11",
   "metadata": {},
   "source": [
    "### Visualize the results\n",
    "\n",
    "Finally, we visualize the input image alongside the predicted depth map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from PIL import Image\n",
    "\n",
    "image = Image.open(\"image.jpg\")\n",
    "\n",
    "fig, axes = plt.subplots(1, 2, figsize=(14, 6))\n",
    "axes[0].imshow(image)\n",
    "axes[0].set_title(\"Input\")\n",
    "axes[0].axis(\"off\")\n",
    "depth_vis = axes[1].imshow(depth.cpu(), cmap=\"Spectral_r\")\n",
    "axes[1].set_title(\"Relative depth (larger = farther)\")\n",
    "axes[1].axis(\"off\")\n",
    "fig.colorbar(depth_vis, ax=axes[1], fraction=0.046, pad=0.04)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13",
   "metadata": {},
   "source": [
    "## Metric depth estimation\n",
    "\n",
    "The metric model predicts depth in **meters** and needs the camera **intrinsics** as a `(3, 3)` matrix `[[fx, 0, cx], [0, fy, cy], [0, 0, 1]]`. If you don't know them, approximate from an assumed field of view - the metric scale is only as accurate as that guess."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14",
   "metadata": {},
   "source": [
    "### Load the model weights\n",
    "\n",
    "Then load the model with LightlyTrain's `load_model` function. This will automatically download the model weights and load the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15",
   "metadata": {},
   "outputs": [],
   "source": [
    "metric_model = lightly_train.load_model(\"dinov2/dav3-metric-large\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16",
   "metadata": {},
   "source": [
    "### Predict metric depth map\n",
    "\n",
    "Pass the intrinsics to `predict` to obtain depth in meters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17",
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "import torch\n",
    "\n",
    "# Approximate intrinsics from an assumed 60° horizontal field of view.\n",
    "width, height = image.size\n",
    "focal_px = (width / 2) / math.tan(math.radians(60.0) / 2)\n",
    "intrinsics = torch.tensor(\n",
    "    [\n",
    "        [focal_px, 0.0, width / 2],\n",
    "        [0.0, focal_px, height / 2],\n",
    "        [0.0, 0.0, 1.0],\n",
    "    ]\n",
    ")\n",
    "\n",
    "depth_m = metric_model.predict(\"image.jpg\", intrinsics=intrinsics)\n",
    "print(f\"depth range: {depth_m.min():.2f} m – {depth_m.max():.2f} m\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18",
   "metadata": {},
   "source": [
    "### Visualize the results\n",
    "\n",
    "Finally, we visualize the input image alongside the predicted metric depth map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(1, 2, figsize=(14, 6))\n",
    "axes[0].imshow(image)\n",
    "axes[0].set_title(\"Input\")\n",
    "axes[0].axis(\"off\")\n",
    "depth_vis = axes[1].imshow(depth_m.cpu(), cmap=\"Spectral_r\")\n",
    "axes[1].set_title(\"Metric depth [m]\")\n",
    "axes[1].axis(\"off\")\n",
    "fig.colorbar(depth_vis, ax=axes[1], fraction=0.046, pad=0.04, label=\"meters\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20",
   "metadata": {},
   "source": [
    "## Next Steps\n",
    "\n",
    "* [Depth Estimation Documentation](https://docs.lightly.ai/train/stable/depth_estimation.html): If you want to learn more about depth estimation with LightlyTrain."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}