Command-line tool

The Lightly framework provides you with a command-line interface (CLI) to train self-supervised models and create embeddings without having to write a single line of code.

You can also have a look at this video to get an overview of how to work with the CLI.

Check the installation of lightly

To see if the lightly command-line tool was installed correctly, you can run the following command which will print the installed lightly version:

lightly-version

If lightly was installed correctly, you should see something like this:

lightly version 1.1.4

Train a model using the CLI

Training a model using default parameters can be done with just one command. Let’s assume you have a folder of cat images named cat and want to train a model on it. You can use the following command to train a model and save the checkpoint:

# train a model using default parameters
lightly-train input_dir=cat

# train a model for 5 epochs
lightly-train input_dir=cat trainer.max_epochs=5

For a full list of supported arguments run

lightly-train --help

You can get an overview of the various CLI parameters you can set in Default Settings.

Create embeddings using the CLI

Once you have a trained model checkpoint, you can create an embedding of a dataset.

# use pre-trained models provided by Lighly
lightly-embed input_dir=cat

# use custom checkpoint
lightly-embed input_dir=cat checkpoint=mycheckpoint.ckpt

The embeddings.csv file should look like the following:

embeddings_example.csv

filenames

embedding_0

embedding_1

embedding_2

embedding_3

labels

101053-1.jpg

-51.535

-2.325

-21.750

78.265

0

101101-1.jpg

-67.958

-2.800

-28.861

103.812

0

101146-1.jpg

-59.831

-2.719

-25.413

90.945

0

Upload data using the CLI

In this example we will upload a dataset to the Lightly Platform. First, make sure you have an account on Lightly. A free account is sufficient. Log in to the app and create a new dataset. You will get a token and dataset_id which can be used to upload your dataset. Alternatively, you can create a new dataset directly with the token by providing the new_dataset_name instead of the dataset_id.

# upload only the dataset
lightly-upload input_dir=cat token=your_token dataset_id=your_dataset_id

# you can upload the dataset together with the embeddings
lightly-upload input_dir=cat embeddings=your_embedding.csv \
               token=your_token dataset_id=your_dataset_id

# create a new dataset and upload to it
lightly-upload input_dir=cat token=your_token new_dataset_name=your_dataset_name

Note

To obtain your token and dataset_id check: Authentication API Token and Dataset Identifier.

Upload embeddings using the CLI

You can upload embeddings directly to the Lightly Platform using the CLI. Again, you can use the dataset_id and new_dataset_name interchangeably.

# upload only the embeddings
lightly-upload embeddings=your_embedding.csv token=your_token \
               dataset_id=your_dataset_id

# you can upload the dataset together with the embeddings
lightly-upload input_dir=cat embeddings=your_embedding.csv \
               token=your_token new_dataset_name=your_dataset_name

Download data using the CLI

You can download a dataset with a given tag from the Lightly Platform using the following CLI command. The CLI provides you with three options:

  • Download the list of filenames for a given tag in the dataset.

  • Download the images for a given tag in the dataset.

  • Copy the images for a given tag from an input directory to a target directory.

The last option allows you to very quickly extract only the images in a given tag without the need to download them explicitly.

# download a list of files
lightly-download tag_name=my_tag_name dataset_id=your_dataset_id token=your_token

# download the images and store them in an output directory
lightly-download tag_name=my_tag_name dataset_id=your_dataset_id token=your_token \
                 output_dir=path/to/output/dir

# copy images from an input directory to an output directory
lightly-download tag_name=my_tag_name dataset_id=your_dataset_id token=your_token \
                 input_dir=path/to/input/dir output_dir=path/to/output/dir