Lightly Query Language¶
This document explains how to write queries in the LightlyStudio query editor.
A query helps you find images that match certain rules. For example, you can search for:
- large images
- images with a specific file name
- tagged images
- images with specific metadata values
- images with a certain annotation, such as a classification, bounding box, or segmentation mask
- a logical combination of the above, using AND/OR/NOT operators.
Query examples¶
The query language is quite simple and resembles WHERE clauses in SQL. We recommend learning it from examples.
# Images that are at least 640 pixels wide or 400 pixels tall
width >= 640 OR height >= 400
# Images which do not have the tag "reviewed"
NOT "reviewed" IN tags
# Images with a "car" segmentation mask that is less than 100 pixels wide
segmentation_mask(class_name = "car" AND width < 100)
Supported image fields¶
These fields can be used directly in a query:
widthheightfile_namefile_path_abscreated_at
Example queries:
height >= 400
width >= 640
file_name = "cat.jpg"
file_path_abs != "/datasets/archive/bad.jpg"
created_at >= "2025-01-01T00:00:00Z"
Metadata filtering¶
Use metadata["path"] to compare sample metadata. Quote the path with double or single quotes.
Use dots for nested fields and zero-based array indices in brackets:
metadata["camera"] = "front"
metadata["reviewed"] = true
metadata["quality_score"] >= 0.8
metadata["odometry.acc[1]"] < -0.2
width >= 640 AND metadata["quality_score"] >= 0.8
Numbers support =, !=, <, <=, >, and >=. Strings and booleans support only = and !=. Boolean literals are case-insensitive, so true and TRUE mean the same thing.
Metadata comparisons work on image and video samples. They are not supported inside annotation
match functions. Metadata IN (...) and comparisons with null are not yet supported.
Missing values and negation¶
A metadata comparison does not match a missing path, null, or a value of another type.
This also applies to !=:
# Matches string camera values other than "front"; excludes missing, null, and other types
metadata["camera"] != "front"
# Matches everything except the string camera value "front"; includes missing, null, and other types
NOT metadata["camera"] = "front"
NOT negates the entire comparison, including whether a compatible metadata value exists.
The editor suggests a metadata["path"] snippet in sample expressions and true/false after
metadata = or !=. Hover over metadata for syntax and comparison help. Enter a path from
your dataset; completion does not list dataset metadata keys.
Tag filtering¶
Filtering for tags can be done using "foo" IN tags, which matches each image that has the tag foo:
"car" IN tags
You can combine this with other criteria:
"car" IN tags OR (height >= 400 AND width >= 640)
Other examples:
"training" IN tags
NOT "rejected" IN tags
"training" IN tags OR "validation" IN tags
Annotation match functions¶
An annotation is a classification, bounding box, or segmentation mask attached to an image.
The query language supports three annotation functions:
classification(...)object_detection(...)segmentation_mask(...)
Each function contains another query inside the parentheses and matches an image when it has at least one annotation of that type that satisfies the nested query.
Classification queries¶
Use classification(...) when you want to match image-level classifications. Example queries:
classification(class_name = "cat")
classification(class_name != "background")
Object detection queries¶
Use object_detection(...) when you want to match bounding boxes. The following fields are supported:
class_namexywidthheight
Example queries:
object_detection(x >= 10 AND y < 200)
object_detection(class_name = "cat" AND width >= 50 AND height >= 40)
object_detection(class_name = "cat" OR class_name = "dog")
object_detection(class_name != "background")
Segmentation mask queries¶
Use segmentation_mask(...) when you want to match segmentation annotations.
It is part of the query grammar, uses the same boolean operators as object_detection(...), and supports these fields:
class_namexywidthheight
Example queries:
segmentation_mask(class_name = "cat")
segmentation_mask(x != 0)
segmentation_mask(width > 80)
segmentation_mask(class_name = "cat" AND width >= 50 AND height >= 40)
Combining top-level and annotation expressions¶
You can combine image properties with annotation queries:
height > 400 AND object_detection(class_name = "cat")
width >= 640 AND classification(class_name = "approved")
"reviewed" IN tags AND segmentation_mask(class_name = "road")
Sample evaluation filtering¶
Use sample_evaluation(...) to filter samples by metrics from a named evaluation run:
sample_evaluation(name = "eval-tagged-samples" AND "tp" >= 1 AND "fn" >= 5 AND "fp" = 0)
sample_evaluation(name = "eval-tagged-samples" AND ("tp" > 0 OR "fn" > 0) AND NOT "fp" > 2)
sample_evaluation(name = "eval-tagged-samples" AND "custom-score" < -0.5)
Start with name = "run-name", then optionally add AND followed by metric criteria.
A name-only query matches all samples evaluated by that run. Metric criteria support
parentheses, NOT, AND, and OR (in that precedence order), and numeric comparisons
with =, !=, <, <=, >, and >=.
All metric names require single or double quotes, including the built-in keywords
"tp", "fp", "fn", "disagreement", and "miou". Custom metric names are also
quoted, such as "custom_score".
Metric thresholds must be finite numbers, including decimals.
Missing metrics do not match comparisons, including !=. An inner NOT can match a
missing metric, but still requires the sample to belong to the selected run. An outer
NOT sample_evaluation(...) also matches samples outside that run.
Combine evaluation functions with other sample conditions or separate evaluation functions
using AND, OR, and NOT. Evaluation functions cannot be nested inside annotation functions.
Complex examples¶
# Large reviewed sample with a matching object detection
height > 400 AND width >= 640
AND "reviewed" IN tags
AND object_detection(class_name = "cat" AND width > 80 AND height > 80)
# Nested query with grouped sample filters and segmentation constraints
(file_path_abs != "/datasets/archive/bad.jpg" AND created_at >= "2025-01-01T00:00:00Z")
AND ("training" IN tags OR "validation" IN tags)
AND segmentation_mask(
(class_name = "car" OR class_name = "truck")
AND width >= 100
AND height >= 60
AND NOT (x < 10 OR y < 10)
)
# Nested query combining multiple annotation functions
"reviewed" IN tags
AND classification(class_name = "urban-scene")
AND (
object_detection(class_name = "person" AND height >= 120)
OR segmentation_mask(class_name = "road" AND width > 300)
)
Grammar Notes¶
Operator precedence¶
Operators are evaluated in this order:
NOTANDOR
Use parentheses whenever you want to make grouping explicit.
Strings and comments¶
Strings can use double quotes or single quotes:
file_name = "frame-0001.jpg"
file_name = 'frame-0001.jpg'
Comments start with # and continue to the end of the line:
height >= 400 # keep only large images
Keyword casing¶
The boolean and membership operators are case-insensitive. For example, AND, OR, NOT, and IN can also be written in lowercase.
Miscellaneous¶
created_atmust be a valid datetime string that JavaScriptDatecan parse.x,yare valid insideobject_detection(...)andsegmentation_mask(...), but not as top-level image fields.