Submitting an Image with no Objects to Label

Using Labelbox
1

Hi all,

I’m working on a task to annotate polygons on objects in images. Some images do not have any objects in them and thus have no labels to annotate. It would be nice to count these images as “labelled”, but I can’t submit them if I don’t have at least one label, and using “Skip” won’t include them with the resulting labelled images as far as I understand. Is there a way to get around this problem?

Thanks in advance

2

Hi Faiz, thanks for your post!

Skipped assets are actually considered as labeled assets in Labelbox.

While Skipped assets are presently differentiated from Submitted assets in the Overview tab of a project, you may notice that in the Data Rows tab the key filters on the left-hand side are Labeled and Unlabeled – and the skipped data rows will land in the Submitted section.

As we work towards a data row-centric paradigm in Labelbox, the Overview tab will eventually display counts that reflect these statuses just described in the Data Rows tab, so this should be even clearer in the near future.

Perhaps most importantly, though, is how skipped assets appear as a label in an export. Below I have copied an example of a skipped label from an export I just conducted:

  {
    "ID": "cl7ajtryj002108xne42y4fgd",
    "DataRow ID": "cl6uw4x9p0b7d08070yts8mc8",
    "Labeled Data": "https://raw.githubusercontent.com/Labelbox/labelbox-python/develop/examples/assets/2560px-Kitano_Street_Kobe01s5s4110.jpg",
    "Label": { "objects": [], "classifications": [], "relationships": [] },
    "Created By": "eemerson@labelbox.com",
    "Project Name": "polyline",
    "Created At": "2022-08-26T14:08:18.000Z",
    "Updated At": "2022-08-26T14:08:18.000Z",
    "Seconds to Label": 10.003,
    "External ID": null,
    "Global Key": null,
    "Agreement": -1,
    "Benchmark Agreement": -1,
    "Benchmark ID": null,
    "Dataset Name": "image_annotation_import_demo_dataset",
    "Reviews": [],
    "View Label": "https://editor.labelbox.com?project=cl78cuq7k1bmb07y74te104eb&label=cl7ajtryj002108xne42y4fgd",
    "Has Open Issues": 0,
    "Skipped": true
  }

Please note that the "Skipped" field contains a value of true and that the "Label" field includes only empty lists. This JSON entry for a skipped label will always be included in the export amidst all other submitted labels.

Hope this helps!

3

Hi Zeke, thanks for your response!

I’ve just tested this out myself and it works as you say - that’s what I needed, so thank you! The overview page made me think that the skipped images weren’t being included with the labelled ones. Thanks for clarifying!

4

I ran into a similar situation. There is value in adding background images without any features present to help the model reduce false positives! You may want to differentiate them from skipped labels. I ended up making a “no defects present” (I am looking for defects) attribute. That way I can exclude skipped images which may have issues and include background images which have technically been labeled.