Skip to content

Lightly Query Language

This document explains how to write queries in the LightlyStudio query editor.

A query helps you find images that match certain rules. For example, you can search for:

  • large images
  • images with a specific file name
  • tagged images
  • images with specific metadata values
  • images with a certain annotation, such as a classification, bounding box, or segmentation mask
  • a logical combination of the above, using AND/OR/NOT operators.

Query examples

The query language is quite simple and resembles WHERE clauses in SQL. We recommend learning it from examples.

# Images that are at least 640 pixels wide or 400 pixels tall
width >= 640 OR height >= 400

# Images which do not have the tag "reviewed"
NOT "reviewed" IN tags

# Images with a "car" segmentation mask that is less than 100 pixels wide
segmentation_mask(class_name = "car" AND width < 100)

Supported image fields

These fields can be used directly in a query:

  • width
  • height
  • file_name
  • file_path_abs
  • created_at

Example queries:

height >= 400
width >= 640
file_name = "cat.jpg"
file_path_abs != "/datasets/archive/bad.jpg"
created_at >= "2025-01-01T00:00:00Z"

Metadata filtering

Use metadata["path"] to compare sample metadata. Quote the path with double or single quotes. Use dots for nested fields and zero-based array indices in brackets:

metadata["camera"] = "front"
metadata["reviewed"] = true
metadata["quality_score"] >= 0.8
metadata["odometry.acc[1]"] < -0.2
width >= 640 AND metadata["quality_score"] >= 0.8

Numbers support =, !=, <, <=, >, and >=. Strings and booleans support only = and !=. Boolean literals are case-insensitive, so true and TRUE mean the same thing.

Metadata comparisons work on image and video samples. They are not supported inside annotation match functions. Metadata IN (...) and comparisons with null are not yet supported.

Missing values and negation

A metadata comparison does not match a missing path, null, or a value of another type. This also applies to !=:

# Matches string camera values other than "front"; excludes missing, null, and other types
metadata["camera"] != "front"

# Matches everything except the string camera value "front"; includes missing, null, and other types
NOT metadata["camera"] = "front"

NOT negates the entire comparison, including whether a compatible metadata value exists.

The editor suggests a metadata["path"] snippet in sample expressions and true/false after metadata = or !=. Hover over metadata for syntax and comparison help. Enter a path from your dataset; completion does not list dataset metadata keys.

Tag filtering

Filtering for tags can be done using "foo" IN tags, which matches each image that has the tag foo:

"car" IN tags

You can combine this with other criteria:

"car" IN tags OR (height >= 400 AND width >= 640)

Other examples:

"training" IN tags
NOT "rejected" IN tags
"training" IN tags OR "validation" IN tags

Annotation match functions

An annotation is a classification, bounding box, or segmentation mask attached to an image.

The query language supports three annotation functions:

  • classification(...)
  • object_detection(...)
  • segmentation_mask(...)

Each function contains another query inside the parentheses and matches an image when it has at least one annotation of that type that satisfies the nested query.

Classification queries

Use classification(...) when you want to match image-level classifications. Example queries:

classification(class_name = "cat")
classification(class_name != "background")

Object detection queries

Use object_detection(...) when you want to match bounding boxes. The following fields are supported:

  • class_name
  • x
  • y
  • width
  • height

Example queries:

object_detection(x >= 10 AND y < 200)
object_detection(class_name = "cat" AND width >= 50 AND height >= 40)
object_detection(class_name = "cat" OR class_name = "dog")
object_detection(class_name != "background")

Segmentation mask queries

Use segmentation_mask(...) when you want to match segmentation annotations.

It is part of the query grammar, uses the same boolean operators as object_detection(...), and supports these fields:

  • class_name
  • x
  • y
  • width
  • height

Example queries:

segmentation_mask(class_name = "cat")
segmentation_mask(x != 0)
segmentation_mask(width > 80)
segmentation_mask(class_name = "cat" AND width >= 50 AND height >= 40)

Combining top-level and annotation expressions

You can combine image properties with annotation queries:

height > 400 AND object_detection(class_name = "cat")
width >= 640 AND classification(class_name = "approved")
"reviewed" IN tags AND segmentation_mask(class_name = "road")

Sample evaluation filtering

Use sample_evaluation(...) to filter samples by metrics from a named evaluation run:

sample_evaluation(name = "eval-tagged-samples" AND "tp" >= 1 AND "fn" >= 5 AND "fp" = 0)
sample_evaluation(name = "eval-tagged-samples" AND ("tp" > 0 OR "fn" > 0) AND NOT "fp" > 2)
sample_evaluation(name = "eval-tagged-samples" AND "custom-score" < -0.5)

Start with name = "run-name", then optionally add AND followed by metric criteria. A name-only query matches all samples evaluated by that run. Metric criteria support parentheses, NOT, AND, and OR (in that precedence order), and numeric comparisons with =, !=, <, <=, >, and >=.

All metric names require single or double quotes, including the built-in keywords "tp", "fp", "fn", "disagreement", and "miou". Custom metric names are also quoted, such as "custom_score". Metric thresholds must be finite numbers, including decimals.

Missing metrics do not match comparisons, including !=. An inner NOT can match a missing metric, but still requires the sample to belong to the selected run. An outer NOT sample_evaluation(...) also matches samples outside that run.

Combine evaluation functions with other sample conditions or separate evaluation functions using AND, OR, and NOT. Evaluation functions cannot be nested inside annotation functions.

Complex examples

# Large reviewed sample with a matching object detection
height > 400 AND width >= 640
AND "reviewed" IN tags
AND object_detection(class_name = "cat" AND width > 80 AND height > 80)

# Nested query with grouped sample filters and segmentation constraints
(file_path_abs != "/datasets/archive/bad.jpg" AND created_at >= "2025-01-01T00:00:00Z")
AND ("training" IN tags OR "validation" IN tags)
AND segmentation_mask(
    (class_name = "car" OR class_name = "truck")
    AND width >= 100
    AND height >= 60
    AND NOT (x < 10 OR y < 10)
)

# Nested query combining multiple annotation functions
"reviewed" IN tags
AND classification(class_name = "urban-scene")
AND (
    object_detection(class_name = "person" AND height >= 120)
    OR segmentation_mask(class_name = "road" AND width > 300)
)

Grammar Notes

Operator precedence

Operators are evaluated in this order:

  1. NOT
  2. AND
  3. OR

Use parentheses whenever you want to make grouping explicit.

Strings and comments

Strings can use double quotes or single quotes:

file_name = "frame-0001.jpg"
file_name = 'frame-0001.jpg'

Comments start with # and continue to the end of the line:

height >= 400 # keep only large images

Keyword casing

The boolean and membership operators are case-insensitive. For example, AND, OR, NOT, and IN can also be written in lowercase.

Miscellaneous

  • created_at must be a valid datetime string that JavaScript Date can parse.
  • x, y are valid inside object_detection(...) and segmentation_mask(...), but not as top-level image fields.