lightly.api

The lightly.api module provides access to the Lightly web-app.

.api_workflow_client

class lightly.api.api_workflow_client.ApiWorkflowClient(token: str, dataset_id: str = None, embedding_id: str = None)

Provides a uniform interface to communicate with the api

The APIWorkflowClient is used to communicaate with the Lightly API. The client can run also more complex workflows which include multiple API calls at once.

The client can be used in combination with the active learning agent.

Args:

token:

the token of the user, provided in webapp

dataset_id:

the id of the dataset, provided in webapp. If it is not set, but used by a workflow, the last modfied dataset is taken by default.

embedding_id:

the id of the embedding to use. If it is not set, but used by a workflow, the newest embedding is taken by default

create_dataset(dataset_name: str)

Creates a dataset on the webplatform

If a dataset with that name already exists, instead the dataset_id is set. Args:

dataset_name:

The name of the dataset to be created.

create_new_dataset_with_unique_name(dataset_basename: str)

Creates a new dataset on the web platform

If a dataset with the specified name already exists, a counter is added to the name to be able to still create it. Args:

dataset_basename:

The name of the dataset to be created.

property dataset_id

The current dataset_id.

If the dataset_id is set, it is returned. If it is not set, then the dataset_id of the last modified dataset is selected.

delete_dataset_by_id(dataset_id: str)

Deletes a dataset on the web platform

Args:

dataset_id:

The id of the dataset to be deleted.

download_dataset(output_dir: str, tag_name: str = 'initial-tag', verbose: bool = True)

Downloads images from the web-app and stores them in output_dir.

Args:

output_dir:

Where to store the downloaded images.

tag_name:

Name of the tag which should be downloaded.

verbose:

Whether or not to show the progress bar.

Raises:

ValueError if the specified tag does not exist on the dataset. RuntimeError if the connection to the server failed.

property filenames_on_server

The list of the filenames in the dataset.

index_custom_metadata_by_filename(filenames: List[str], custom_metadata: Dict)

Creates an index to lookup custom metadata by filename.

Args:

filenames:

List of filenames.

custom_metadata:

Dictionary of custom metadata, see upload_custom_metadata for the required format.

Returns:

A dictionary containing custom metdata indexed by filename.

sampling(sampler_config: lightly.active_learning.config.sampler_config.SamplerConfig, preselected_tag_id: str = None, query_tag_id: str = None) → lightly.openapi_generated.swagger_client.models.tag_data.TagData

Performs a sampling given the arguments.

Args:

sampler_config:

The configuration of the sampler.

al_scores:

The active learning scores for the sampler.

preselected_tag_id:

The tag defining the already chosen samples (e.g. already labelled ones), default: None.

query_tag_id:

The tag defining where to sample from, default: None resolves to the initial-tag.

Returns:

The newly created tag of the sampling.

Raises:

ApiException, ValueError, RuntimeError

set_dataset_id_by_name(dataset_name: str)

Sets the dataset id given the name of the dataset

Args:

dataset_name:

The name of the dataset for which the dataset_id should be set as attribute

Raises: ValueError

set_embedding_id_by_name(embedding_name: str = None)

Sets the embedding id of the client by embedding name.

Args:

embedding_name:

Name under which the embedding was uploaded.

Raises:

ValueError if the embedding does not exist.

upload_custom_metadata(custom_metadata: Dict, verbose: bool = False)

Uploads custom metadata to the Lightly platform.

The custom metadata is expected in a format similar to the COCO annotations: Under the key “images” there should be a list of dictionaries, each with a file_name and id. Under the key “metadata” the custom metadata is stored as a list of dictionaries, each with a image_id to match it to the image.

Example:

>>> custom_metadata = {
>>>     "images": [
>>>         {
>>>             "file_name": "image0.jpg",
>>>             "id": 0,
>>>         },
>>>         {
>>>             "file_name": "image1.jpg",
>>>             "id": 1,
>>>         }
>>>     ],
>>>     "metadata": [
>>>         {
>>>             "image_id": 0,
>>>             "number_of_people": 3,
>>>             "weather": {
>>>                 "scenario": "cloudy",
>>>                 "temperature": 20.3
>>>             }
>>>         },
>>>         {
>>>             "image_id": 1,
>>>             "number_of_people": 1,
>>>             "weather": {
>>>                 "scenario": "rainy",
>>>                 "temperature": 15.0
>>>             }
>>>         }
>>>     ]
>>> }

Args:

custom_metadata:

Custom metadata as described above.

verbose:

If True displays a progress bar during the upload.

upload_dataset(input: Union[str, lightly.data.dataset.LightlyDataset], max_workers: int = 8, mode: str = 'thumbnails', verbose: bool = True, custom_metadata: Optional[Dict] = None)

Uploads a dataset to to the Lightly cloud solution.

Args:

input:

one of the following:

• the path to the dataset, e.g. “path/to/dataset”

• the dataset in form of a LightlyDataset

max_workers:

Maximum number of workers uploading images in parallel.

max_requests:

Maximum number of requests a single worker can do before he has to wait for the others.

mode:

One of [full, thumbnails, metadata]. Whether to upload thumbnails, full images, or metadata only.

Raises:

ValueError if dataset is too large or input has the wrong type RuntimeError if the connection to the server failed.

upload_embeddings(path_to_embeddings_csv: str, name: str)

Uploads embeddings to the server.

First checks that the specified embedding name is not on ther server. If it is, the upload is aborted. Then creates a new csv with the embeddings in the order specified on the server. Next it uploads it to the server. The received embedding_id is saved as a property of self.

Args:

path_to_embeddings_csv:

The path to the .csv containing the embeddings, e.g. “path/to/embeddings.csv”

name:

The name of the embedding. If an embedding with such a name already exists on the server, the upload is aborted.

upload_file_with_signed_url(file: io.IOBase, signed_write_url: str, max_backoff: int = 32, max_retries: int = 5) → requests.models.Response

Uploads a file to a url via a put request.

Args:

file:

The file to upload.

signed_write_url:

The url to upload the file to. As no authorization is used, the url must be a signed write url.

max_backoff:

Maximal backoff before retrying.

max_retries:

Maximum number of retries before timing out.

Returns:

The response of the put request, usually a 200 for the success case.

verify_custom_metadata_format(custom_metadata: Dict)

Verifies that the custom metadata is in the correct format.

Args:

custom_metadata:

Dictionary of custom metadata, see upload_custom_metadata for the required format.

Raises:

KeyError if “images” or “metadata” aren’t a key of custom_metadata.