Command-line tool¶

The Lightly framework provides you with a command-line interface (CLI) to train self-supervised models and create embeddings without having to write a single line of code.

You can also have a look at this video to get an overview of how to work with the CLI.

Check the installation of lightly¶

To see if the lightly command-line tool was installed correctly, you can run the following command which will print the installed lightly version:

lightly-version

If lightly was installed correctly, you should see something like this:

lightly version 1.1.4

Train a model using the CLI¶

Training a model using default parameters can be done with just one command. Let’s assume you have a folder of cat images named cat and want to train a model on it. You can use the following command to train a model and save the checkpoint:

# train a model using default parameters
lightly-train input_dir=cat

# train a model for 5 epochs
lightly-train input_dir=cat trainer.max_epochs=5

For a full list of supported arguments run

lightly-train --help

You can get an overview of the various CLI parameters you can set in Default Settings.

Create embeddings using the CLI¶

Once you have a trained model checkpoint, you can create an embedding of a dataset.

# use pre-trained models provided by Lighly
lightly-embed input_dir=cat

# use custom checkpoint
lightly-embed input_dir=cat checkpoint=mycheckpoint.ckpt

The embeddings.csv file should look like the following:

embeddings_example.csv¶
filenames	embedding_0	embedding_1	embedding_2	embedding_3
101053-1.jpg	-51.535	-2.325	-21.750	78.265
101101-1.jpg	-67.958	-2.800	-28.861	103.812
101146-1.jpg	-59.831	-2.719	-25.413	90.945

Upload data using the CLI¶

In this example we will upload a dataset to the Lightly Platform. First, make sure you have an account on Lightly. A free account is sufficient. Log in to the app and create a new dataset. You will get a token and dataset_id which can be used to upload your dataset. Alternatively, you can create a new dataset directly with the token by providing the new_dataset_name instead of the dataset_id.

# upload only the dataset
lightly-upload input_dir=cat token=your_token dataset_id=your_dataset_id

# you can upload the dataset together with the embeddings
lightly-upload input_dir=cat embeddings=your_embedding.csv \
               token=your_token dataset_id=your_dataset_id

# create a new dataset and upload to it
lightly-upload input_dir=cat token=your_token new_dataset_name=your_dataset_name

Note

To obtain your token and dataset_id check: Authentication API Token and Dataset Identifier.

Upload embeddings using the CLI¶

You can upload embeddings directly to the Lightly Platform using the CLI. Again, you can use the dataset_id and new_dataset_name interchangeably.

# upload only the embeddings
lightly-upload embeddings=your_embedding.csv token=your_token \
               dataset_id=your_dataset_id

# you can upload the dataset together with the embeddings
lightly-upload input_dir=cat embeddings=your_embedding.csv \
               token=your_token new_dataset_name=your_dataset_name

Download data using the CLI¶

You can download a dataset with a given tag from the Lightly Platform using the following CLI command. The CLI provides you with three options:

Download the list of filenames for a given tag in the dataset.
Download the images for a given tag in the dataset.
Copy the images for a given tag from an input directory to a target directory.

The last option allows you to very quickly extract only the images in a given tag without the need to download them explicitly.

# download a list of files
lightly-download tag_name=my_tag_name dataset_id=your_dataset_id token=your_token

# download the images and store them in an output directory
lightly-download tag_name=my_tag_name dataset_id=your_dataset_id token=your_token \
                 output_dir=path/to/output/dir

# copy images from an input directory to an output directory
lightly-download tag_name=my_tag_name dataset_id=your_dataset_id token=your_token \
                 input_dir=path/to/input/dir output_dir=path/to/output/dir