LightlyEdge C++ SDK
|
In this guide, we introduce the "adaptive diversity" strategy.
A disadvantage of Diversity Selection is that we have to specify a threshold to control the sensitivity of the selection. With adaptive diversity, instead, we specify a target ratio of selected images. The strategy then optimizes two goals - select diverse images, and keep the ratio of selected images close to the target.
For example, if we have a dataset of 1000 images and want to select the 20% most diverse images, we set target_ratio
to 0.2
.
We showcase the strategy on the same images as in the previous guides. Note however that this strategy is designed for input datasets with thousands of images, like e.g. a camera stream.
Code for this tutorial is provided in examples/05_adaptive_diversity_selection
directory. Before starting this tutorial, copy the model file to examples/lightly_model.tar
and verify that your project layout is as follows:
See below the content of the main.cpp file. We will first run the example, and explain it right after.
Build and run:
The output should be similar to the following, the distances might slightly differ on your machine architecture:
With the chosen parameters, the output is identical to the output of Diversity Selection example.
lightly_model_14.tar
. You might need to adjust the thresholds in this tutorial if your model version differs.There are several steps needed to set up adaptive diversity selection. Note that in a real application the exceptions should be handled in a try-catch block. For clarity, we annotate the return types.
Image loading is identical to the previous guides. What differs is how LightlyEdge is the set up. Let's start with the initialization code at the beginning of the main
function:
LightlyEdge is first initialized from a TAR archive. Recall that we can register any number of selection strategies with LightlyEdge. We register a single adaptive diversity strategy with lightly_edge_sdk::LightlyEdge::register_adaptive_diversity_strategy
.
The function accepts two arguments: target_ratio
and optional buffer_max_length
.
The target_ratio
is a float between 0 and 1 indicating what proportion of images should be selected. Because the strategy simultaneously tries to select diverse images, the ratio of actual selected images will fluctuate. Over long datasets of thousands of images the actual selected ratio should converge to the target ratio.
The optional buffer_max_length
argument is set to 3000
by default. The strategy internally keeps track of information from buffer_max_length
previously observed images. It determines whether the next image should be selected based on this buffer and the current selection ratio.
Note that for initialisation, the first image is always marked as selected, and the second one as not selected.
We set the target ratio to 70% and keep the second argument default. The images are then processed in the loop in the main
function analogously as in the previous guide:
The code first embeds the image and calls lightly_edge_sdk::LightlyEdge::should_select
which returns lightly_edge_sdk::SelectInfo
. The structure contains the decision of each registered strategy whether a frame should be selected.
We load the decision for the single strategy we registered into adaptive_diversity_select_info
. LightlyEdge does not by default remember the images that are observed, we have to manually insert the embeddings into the embedding database by calling lightly_edge_sdk::LightlyEdge::insert_into_embedding_database
.
Finally, we print whether the image was selected and its distance to the closest embedding in the database.
Set target_ratio
to the proportion of data you want to select. Note that the actual selection ratio might fluctuate, however over stretches of thousands of images it should converge to the specified value.
You can trade off between selecting diverse images versus accurately matching the target ratio by tweaking the buffer_max_length
parameter. Setting it to a large value will give the algorithm large memory, and therefore prioritise diversity. Smaller values will on the other hand more dynamically adapt to current data distribution. We however recommend to set the value to at least ~100 to capture at least some statistics of the observed data.
In the next section you will learn how to run Object Detection with LightlyEdge.