Migrating to Worker v2.11

Breaking Changes

Lightly Worker version 2.11 requires Lightly-SSL version 1.5.0

The complete changelog can be found here.

Breaking changes

• use_datapool is disabled by default to safeguard your dataset from accidental datapool usage. To enable datapool, set "worker": { "use_datapool": True}. It is safe to enable this also for datasets not using datapool. See our docs for more information.
• When using S3 datasources that have a retention policy enabled, the worker errors if it has not been configured how it should handle expiring files. You can either SKIP the file or ABORT the run to safeguard from failing runs. The time difference which triggers the aforementioned handling can also be configured. See our docs for more information.
• input_dir and shared_dir mount points previously used to pass in checkpoints or relevant filenames have been removed. Please use the workflow outlined in Local Storage.

Potentially breaking

• The worker now errors when all predictions are missing or erroneous. If only some are invalid, the worker skips and ignores these and continues the selection.
• The internal user within the Lightly Worker was renamed from boris to lightly_worker to reduce confusion when using local storage and dealing with permissions. In the unlikely case of you specifically giving permission to the boris user for mounted directories, please update the user to lightly_worker.

Recommended

• We recommend upgrading the Nvidia Display Drivers to version 535.154.05 to reduce memory consumption. This is made possible by the updated pytorch version within the Lightly Worker 2.11. This update does not affect the CUDA version or Lightly Worker speed.