Import empty/negative labels in a project

Python SDK
1

Hi,
Using the Python SDK, I would like to import empty labels to all images inside a project (negative images). To achieve this, I followed the following guide: Import a labeled dataset (images)


queued_data_rows = project.export_queued_data_rows()
ground_truth_list = LabelList()

for datarow in queued_data_rows:
  annotations_list = []
  folder = datarow['externalId'].split("/")[0]
  id = datarow['externalId'].split("/")[1]
  if folder == "positive_image_set":
    for image in annotations['images']:
      if (image['file_name']==id):
        for annotation in annotations['annotations']:
          if annotation['image_id'] == image['id']:
            bbox = annotation['bbox']
            id = annotation['category_id'] - 1
            class_name = ontology_from_project.tools[id].name
            annotations_list.append(ObjectAnnotation(
                name = class_name,
                value = Rectangle(start = Point(x = bbox[0], y = bbox[1]), end = Point(x = bbox[2]+bbox[0], y = bbox[3]+bbox[1])),
            ))
  image = ImageData(uid = datarow['id'])
  ground_truth_list.append(Label(data = image, annotations = annotations_list))

In this example, if an image is from the “negative_image_set”, the annotations_list would be added as an empty list in the ground_truth_list (Label(data = image, annotation = []). This works fine up to this point.
However, in the next steps, the ground truth list is converted to NDJSON format with the following function:

ground_truth_ndjson = list(NDJsonConverter.serialize(ground_truth_list))

When this is applied, all the Labels that had their annotations = [] are removed from the converted list. Thus, when importing this NDJSON, the negative images do not receive their empty labels and are stuck in the “To Label” state in the workflow.

Is there a way to add empty labels to images through Python or to push some specific data rows to the “Done” state?

Empty label ex: 'annotations': {'objects': [], 'classifications': [], 'relationships': []}

2

Hi @millx ,

Importing ground truth is fundamentally to have some sort of annotation/classification unsure what would be the use case to import empty labels? If you could provide more context here.

Many thanks,
PT
Labelbox Support

3

Hi @ptancre ,

Thanks for the feedback.

Negative or empty images are important when training a model to reduce the amount of false positives. In my case, I need to minimize false bounding box detection by increasing the weight of negative images in the dataset.

Empty labels are btw already supported by the tool when labeling manually. You just need to hit “Skip” to submit images without any label. Now, I’m looking for a way to automatize it to avoid hitting “Skip” thousands of times…

4

Ok, found a workaround. The images without labels will not get processed to the “DONE” workflow status with above script, but you can force them into this state with the following command:

project_v2.move_data_rows_to_task_queue(data_row_list, None)   
# None = 'DONE' queue

The only difference with this process versus using the “Skip” button is that the Label object under the data_rows will not be create in this case.

“Skip” button behavior on data rows:
data_row['projects'][<PROJECT_ID>]['labels'] list element [0] created with empty annotation.

move_data_rows_to_task_queue->DONE behavior on data rows:
data_row['projects'][<PROJECT_ID>]['labels'] stays a empty list [].

In both cases, the negative images are included in the COCO JSON export. So we are good with option #2.

5

After, there are more glitches coming with this option. If the label element is not created, the tool is not able to display the image in the label editor and there are some issues with the export as well.
I need a way to create this label automatically:
data_row[‘projects’][<PROJECT_ID>][‘labels’] list element [0] with empty annotation.

6

Since this is currently not possible, would be it more suitable to create a classification named ‘skipped’ or ‘empty’ and then filter then for downstream processing?

1 Like
7

We use the classification label of “NONE” to achieve this on my project.