LightlyEdge Python SDK

LightlyEdge Python SDK#

Welcome!#

LightlyEdge Python SDK is a Python library for smart data selection not only on edge devices.

Primary use cases are in automotive, robotics, video analytics or visual inspection industries. LightlyEdge can identify data with the most signal when only a small subset of what is observed can be processed. A typical example is data selection for machine learning with a constrained annotation budget. LightlyEdge identifies the data directly at the point of collection, saving data transfer and storage costs.

Features#

LightlyEdge allows you to pass in camera frames one-by-one. For each frame, LightlyEdge:

  • Enriches the frame with information obtained by running ML models.

  • Decides whether the frame should be selected based on your configuration.

Supported features are:

  • Similarity Search: Select frames matching a text description or a known image.

  • Diversity Selection: Select frames which are visually different from previously selected frames.

  • Adaptive Diversity Selection: Select frames based on expected ratio of the frames to select.

  • Object Detection: Select frames containing objects from a predefined set of classes. Our current detector was trained on automotive classes.

LightlyEdge architecture overview

LightlyEdge architecture overview.#

Note

We are glad to have you try LightlyEdge! It is under active development, your feedback is welcome.

Why LightlyEdge#

LightlyEdge is tailored for resource-limited environments:

  • Our custom models have low footprint and can be used on hardware where large open-source models would use too much memory or be too slow.

  • Having speed in mind, LightlyEdge processes images in real-time. It is built in Rust with C++ and Python bindings.

  • For ease of integration, LightlyEdge supports multiple platforms and handles GPU inference calls for you.

  • We provide models tailored for automotive industry trained in-house on highly curated data. For other domains, please reach out to us.

Requirements#

  • Operating System: Linux / Windows / MacOS

  • Architecture: x86 / ARM CPU

  • GPU: Not required. Support for NVIDIA GPUs.

  • Disk space:

    • CPU version: 20MB SDK + 450MB (small models) / 900MB (large models)

    • GPU version: 700MB SDK + 450MB (small models) / 900MB (large models)

  • Memory: 500MB (small models) / 1GB (large models)

  • Internet connection: Not used.

  • Python 3.8 or higher

We provide ready-made packages for these configurations:

  • Ubuntu 20 or higher, x86 64bit CPU

  • MacOS 14 or higher, ARM 64bit CPU

  • Windows 11 or higher, x86 64bit CPU

Reference Benchmarks#

We use Jetson Orin™ Nano 8GB with Jetpack 6.0 installed as a reference device for benchmarks. The two most complex tasks are processing for text and similarity search, as well as object detection, they can be selectively turned on or off. We assume both to be on for the overall benchmark.

Task

CUDA

Tensor RT

Processing for text and similarity search

85 ms/image

64 ms/image

Object detection

83 ms/image

40 ms/image

All features

168 ms/image

104 ms/image

Benchmark details:

  • Input image of size 960x540

  • Ubuntu 22 LTS

  • CUDA 12.2

  • cuDNN 8.9

  • Tensor RT 8.6

  • Python 3.10, Rust 1.81

  • Power mode: 15W

  • Inference throughput (INT8): 40 TOPS

Getting Started#

If you are new to LightlyEdge please follow our Getting started guide.

API Reference#

For interface documentation refer to the API reference. A good starting point is lightly_edge_sdk.LightlyEdge.