Logo

Getting Started

  • Main Concepts
    • Self-Supervised Learning
  • Installation
    • Supported Python versions
    • Installing Lightly
    • Dependencies
    • Next Steps
  • Command-line tool
    • Check the installation of lightly
    • Crop Images using Labels or Predictions
    • Training and Embedding in a Go – Magic
    • Train a model using the CLI
    • Create embeddings using the CLI
    • Download data using the CLI
    • Breakdown of lightly-magic
  • Self-supervised learning
    • How Lightly Works
      • Data and Transformations
      • Model, Loss and Training
      • Embeddings
    • Lightly in Three Lines
    • What’s next?

Advanced

  • Advanced Concepts in Self-Supervised Learning
    • Augmentations
      • Transforms
      • Custom Transforms
      • Previewing Augmentations
    • Models
    • Losses
      • Memory Bank
    • Obtaining Good Embeddings
      • Monitoring Embedding Quality
  • Distributed Training
    • Distributed Training Benchmarks
      • Observations
  • Benchmarks
    • ImageNet
    • ImageNette
    • CIFAR-10
    • Imagenet100
    • Next Steps

Tutorials

  • Python Package
    • Tutorial 1: Structure Your Input
      • Supported File Types
        • Images
        • Videos
      • Image Folder Datasets
        • Flat Directory Containing Images
        • Directory with Subdirectories Containing Images
      • Video Folder Datasets
      • Embedding Files
        • Advanced usage of Embeddings
      • Next Steps
    • Tutorial 2: Train MoCo on CIFAR-10
      • Imports
      • Configuration
      • Setup data augmentations and loaders
      • Create the MoCo Lightning Module
      • Create the Classifier Lightning Module
      • Train the MoCo model
      • Next Steps
    • Tutorial 3: Train SimCLR on Clothing
      • Imports
      • Configuration
      • Setup data augmentations and loaders
      • Create the SimCLR Model
      • Visualize Nearest Neighbors
      • Color Invariance
      • Next Steps
    • Tutorial 4: Train SimSiam on Satellite Images
      • Imports
      • Configuration
      • Setup data augmentations and loaders
      • Create the SimSiam model
      • Train SimSiam
      • Scatter Plot and Nearest Neighbors
      • Next Steps
    • Tutorial 5: Custom Augmentations
      • Imports
      • Configuration
      • Setup custom data augmentations
      • Setup dataset and dataloader
      • Create the MoCo model
      • Train MoCo with custom augmentations
      • Evaluate the results
    • Tutorial 6: Pre-train a Detectron2 Backbone with Lightly
      • Introduction
      • Prerequisites:
      • Imports
      • Configuration
      • Initialize the Detectron2 Model
      • Setup data augmentations and loaders
      • Self-supervised pre-training
      • Storing the checkpoint
      • Finetuning with Detectron2
      • Next Steps
  • Platform
    • Tutorial 1: Curate Pizza Images
      • What you will learn
      • Requirements
      • Upload the data
      • Filter the dataset using metadata
      • Download the curated dataset
      • Training a model using the curated data
    • Tutorial 2: Diversify the Sunflowers Dataset
      • What you will learn
      • Requirements
      • Create a Selection
      • Download selected samples
    • Tutorial 3: Active learning for classification
      • What you will learn
      • Define your dataset
      • Creation of the dataset on the Lightly Platform with embeddings
      • Active learning
    • Tutorial 4: Active Learning using Detectron2 on Comma10k
      • Requirements
      • Upload dataset to Lightly
      • Inference on unlabeled data
      • Create our Detectron2 model
      • Get Model Predictions
      • Query Samples for Labeling
      • Next Steps
    • Tutorial 5: Custom Metadata and Rebalancing
      • What you will learn
      • Requirements
      • Custom Metadata
      • Configuration
      • Rebalancing
    • Tutorial 6: Find false negatives of object detection
      • What you will learn
      • Requirements
      • Steps
    • Tutorial 7: Active Learning with Nvidia TLT
    • Tutorial 8: Integration with LabelStudio for Active Learning
    • Tutorial 9: Embedded COVID mask detection
    • Tutorial 10: Export to LabelStudio
      • What you will learn
      • Requirements
      • Launch LabelStudio
      • Export from Lightly in the LabelStudio format
      • Import the tasks into LabelStudio
      • Start labeling

Examples

  • Models
    • Barlow Twins
    • BYOL
    • DCL & DCLW
    • DINO
    • FastSiam
    • MAE
    • MSN
    • MoCo
    • NNCLR
    • PMSN
    • SimCLR
    • SimMIM
    • SimSiam
    • SMoG
    • SwaV
    • SwaV Queue
    • TiCo
    • VICReg
    • VICRegL

Python API

  • lightly
  • lightly.api
  • lightly.cli
    • .lightly_cli
    • .train_cli
    • .embed_cli
    • .download_cli
    • .version_cli
    • .crop_cli
    • .config.config.yaml
      • Overwrites
      • Additional Arguments
      • Default Settings
  • .core
  • lightly.data
    • .dataset
    • .multi_view_collate
    • .collate:
  • lightly.loss
  • lightly.models
    • .resnet
    • .zoo
    • .nn_memory_bank
    • .heads
  • lightly.transforms
  • lightly.utils
    • .io
    • .embeddings_2d
    • .benchmarking
    • .debug
    • .dist
    • .reordering
    • .version_compare

On-Premise

  • Docker Archive
    • Setup
      • Analytics
      • Licensing
      • Download the Docker Image
      • Sanity Check
      • Update Lightly Docker
    • First Steps
      • Storage Access
      • Specify Relevant Files
      • Embedding a Dataset and Selecting from it
      • Train a Self-Supervised Model
      • Accessing Lightly Input Parameters
      • Selecting from Embeddings File
      • Manually Inspecting the Embeddings
      • Selecting from Video Files
      • Removing Exact Duplicates
      • Upload Sampled Dataset To Lightly Platform
      • Reporting
        • Live View of Docker Status
        • Docker Output
        • Evaluation of the Selection Proces
    • Advanced
      • Meta Information
        • Mask Samples
        • Use Pre-Selected Samples
        • Custom Weak Labels
      • Datapool
        • How It Works
        • Usage
      • Pretagging
        • How It Works
        • Usage
      • Add Predictions to a Datasource
        • Predictions Folder Structure
        • Prediction Tasks
        • Prediction Schema
        • Prediction Files
        • Prediction Files for Videos
        • Prediction Format
        • Prediction Singletons
        • Creating the predictions folder
      • Add Metadata to a Datasource
        • Metadata Folder Structure
        • Metadata Schema
        • Metadata Files
        • Metadata Format
        • Next Steps
      • Active Learning
        • Prerequisites
        • Selection
        • Active Learning with Custom Scores (not recommended as of March 2022)
      • Sequence Selection
        • How It Works
        • Usage
      • Object Level
        • Prerequisites
        • Predictions
        • Selection on Object Level
        • Lightly Pretagging
        • Padding
        • Object Crops Dataset
        • Analyzing the Crop Dataset
        • Multiple Object Level Runs
    • Integration
      • Using the Docker with a Cloud Bucket as Remote Datasource
        • Introduction
        • Advantages
        • Requirements
        • Download the Lightly Docker
        • Run the Lightly Docker with the datasource
        • View the progress of the Lightly Docker
        • Use your selected dataset
        • Process new data in your bucket using a datapool
        • Network traffic
      • Trigger a Docker Job from from the Platform or code
        • Introduction
        • Advantages
        • Download the Lightly Docker
        • Register the Lightly Docker as a Worker
        • Create a Dataset and Trigger a Job
        • View the progress of the Lightly Docker
        • Use your selected dataset
        • Process new data in your bucket using a datapool
      • Load data directly from S3 buckets using s3fs-fuse
        • What is s3fs-fuse?
        • Get an AWS Bucket and Credentials
        • Install s3fs-fuse
        • Configure S3 Credentials
        • Use S3 Storage with Lightly Docker
        • Use Caching
        • Common Issues
      • Data Pre-processing Pipeline on AWS with Dagster
        • Introduction
        • Dagster
        • Setting up the EC2 Instance
        • Setting up the S3 Bucket
        • Integration
    • Configuration
      • List of Parameters
      • Choosing the Right Parameters
      • Increase I/O Performance
    • Examples
      • Extract Diverse Video Frames
        • Using ffmpeg
        • Using Lightly Docker
      • ImageNet
      • Combining Cityscapes with Kitti
    • Known Issues and FAQ
      • Docker is slow when working with long videos
      • Docker Crashes when running with GPUs
      • Shared Memory Error when running Lightly Docker
      • Docker crashes because of too many open files
      • Permission denied for input created with sudo
      • Error when using S3 fuse and mounting to docker
      • Token printed to shared stdout or logs
    • Hardware recommendations
      • Finding the compute speed bottleneck
      • Updating the machine
lightly
  • Docker Archive
  • Advanced
  • View page source

Advanced

Warning

The Docker Archive documentation is deprecated

The old workflow described in these docs will not be supported with new Lightly Worker versions above 2.6. Please switch to our new documentation page instead.

Here you learn more advanced usage patterns of Lightly Docker.

  • Meta Information
    • Mask Samples
    • Use Pre-Selected Samples
    • Custom Weak Labels
  • Datapool
    • How It Works
    • Usage
  • Pretagging
    • How It Works
    • Usage
  • Add Predictions to a Datasource
    • Predictions Folder Structure
    • Prediction Tasks
    • Prediction Schema
    • Prediction Files
    • Prediction Files for Videos
    • Prediction Format
    • Prediction Singletons
    • Creating the predictions folder
  • Add Metadata to a Datasource
    • Metadata Folder Structure
    • Metadata Schema
    • Metadata Files
    • Metadata Format
    • Next Steps
  • Active Learning
    • Prerequisites
    • Selection
    • Active Learning with Custom Scores (not recommended as of March 2022)
  • Sequence Selection
    • How It Works
    • Usage
  • Object Level
    • Prerequisites
    • Predictions
    • Selection on Object Level
    • Lightly Pretagging
    • Padding
    • Object Crops Dataset
    • Analyzing the Crop Dataset
    • Multiple Object Level Runs
Next Previous

© Copyright 2020, Lightly AG

Built with Sphinx using a theme provided by Read the Docs.