Outputs

Dataset

The datasets created by the Lightly Worker can be analyzed interactively with the tools provided in the Lightly Platform. These tools include interactive histograms, scatter, and correlation plots.

For more information, head to The Lightly Platform.

Artifacts

Each scheduled run creates several artifacts upon execution. These artifacts are uploaded to the Lightly Platform and can be accessed with the help of the Python client.

Here’s a list of the artifacts generated by the Lightly Worker:

Download the Log File After a Crash

In case the run failed you can download the log.txt file which contains detailed information to debug the error. It's also possible to retrieve the logs using the Python api as outlined here.

640 — You can download the `log.txt` file directly from the UI by clicking on the failed run.

Download All Artifacts

The following code shows how to schedule a Lightly Worker run, wait until it has finished, and then download all artifacts to a local directory named my_run/artifacts.

scheduled_run_id = client.schedule_compute_worker_run(
    selection_config={
        "n_samples": 50,
        "strategies": [
            {
                "input": {
                    "type": "EMBEDDINGS"
                },
                "strategy": {
                    "type": "DIVERSITY"
                }
            }
        ]
    }
)

# wait until the run is finished
for run_info in client.compute_worker_run_info_generator(scheduled_run_id=scheduled_run_id):
    pass

# download all artifacts to "my_run/artifacts"
run = client.get_compute_worker_run_from_scheduled_run(scheduled_run_id=scheduled_run_id)
client.download_compute_worker_run_artifacts(run=run, output_dir="my_run/artifacts")

# you can also only download specific artifacts as outlined in the api docs
# https://docs.lightly.ai/self-supervised-learning/lightly.api.html
# e.g. to only get the report.json file
client.download_compute_worker_run_report_json(run=run, output_path="my_run/artifacts/report.json")

Download Artifacts by Dataset

It’s also possible to get the artifacts by only knowing the dataset id.

from lightly.api import ApiWorkflowClient

# Create the Lightly client to connect to the API.
client = ApiWorkflowClient(token="MY_LIGHTLY_TOKEN", dataset_id="MY_DATASET_ID")

# get all runs for a given dataset sorted from old to new
runs = client.get_compute_worker_runs(dataset_id=client.dataset_id)
run = runs[-1] # get the latest run

# download all artifacts to "my_run/artifacts"
client.download_compute_worker_run_artifacts(run=run, output_dir="my_run/artifacts")

📘
The above examples show how to download all artifacts at once. It’s also possible to download each artifact on its own. To see how, please refer to the individual artifacts on the page Report above.

Direct Access Via Cloud Provider

You can also inspect your cloud bucket or local directory manually to fetch some of the artifacts described above. Check out the cloud storage - lightly path structure document to see where this is possible and what path pattern to expect.