Active Learning

Lightly makes use of active learning scores to select the samples which will yield the biggest improvements of your machine learning model. The scores are calculated on-the-fly based on model predictions and provide the selection algorithm with feedback about the uncertainty of the model for the given sample.

Learn more about the concept of active learning scores: Scorers.

Note

Note that the active learning features require a minimum Lightly Worker of version 2.2. You can check your installed version of the Lightly Worker by running the Sanity Check.

Prerequisites

In order to do active learning with Lightly, you will need the following things:

Note

The dataset does not need to be new! For example, an initial selection without active learning can be used to train a model. The predictions from this model can then be used to improve your dataset by adding new images to it through active learning.

Selection

Once you have everything set up as described above, you can do an active learning iteration by specifying the following three things in your Lightly docker config:

  • method

  • active_learning.task_name

  • active_learning.score_name

Here’s an example of how to configure an active learning run:

Trigger the Job

To trigger a new job you can click on the schedule run button on the dataset overview as shown in the screenshot below:

../../_images/schedule-compute-run.png

After clicking on the button you will see a wizard to configure the parameters for the job.

../../_images/schedule-compute-run-config.png

In this example we have to set the active_learning.task_name parameter in the docker config. Additionally, we set the method to coral which simultaneously considers the diversity and the active learning scores of the samples. All other settings are default values. The resulting docker config should look like this:

Docker Config
{
  enable_corruptness_check: true,
  remove_exact_duplicates: true,
  enable_training: false,
  pretagging: false,
  pretagging_debug: false,
  method: 'coral',
  stopping_condition: {
    n_samples: 0.1,
    min_distance: -1
  },
  scorer: 'object-frequency',
  scorer_config: {
    frequency_penalty: 0.25,
    min_score: 0.9
  },
  active_learning: {
    task_name: 'my-classification-task',
    score_name: 'uncertainty_margin'
  }
}

The Lightly config remains unchanged.

After the worker has finished its job you can see the selected images with their active learning score in the web-app.