Export only labeled frames in video labeling (semantic segmentation)

Using Labelbox
1

Hi,

So instead of downloading the videos and trying to find every clean frame to take out and then upload for labeling, would we be able to upload video files and then labelers can go frame by frame and label frames that would fit our criteria?

Then we only export frames that have been labeled to use for model training?

We would be doing semantic segmentation.

2

No. Labelbox editor renders video natively (i.e. it does not preprocess them into frames).
You will have to export the labels, use the frame numbers corresponding to labels, extract frames outside Labelbox and import images.

3

We are releasing exports v2 later this month. It includes new format and will make working with video a bit easier (including the SDK methods).

E.g. one API call to fetch label for an array of data row IDs

"labels": [
   {
       "label_id": "",
       "label_kind" : "video",
       "version" : "1.0",
       "annotations": {
           "objects": [], "relationships": [], # Not included for now
           "classifications": [], # global classifications
           "frames": {
               [frameIndex] : {
                   "objects": [
                       {[featureName]:{
                           "annotation_kind" : "video_bounding_box",
                           "name": "cat-bbox",
                           "bounding_box": {
                               "top": 186,
                               "left": 192,
                               "height": 300,
                               "width": 519
                           }
                           "key_frame": True,
                       }
         
                   }
                   ],
                   "classifications": [],
                   "relationships": [] # Not included for now.
               }
           },
           "segments": {
               [featureName] :   [[frameIndex1], [frameIndex2]]
           }
 
       }
   }
]
1 Like
4

Just thought of it… it could be useful to have a service provided by Labelbox to fetch frame from the video or export frame URL in the export itself. @msokoloff wdyt?

1 Like
5

thank you for the detailed reply @manu and pitching the new feature to your team members.

I can’t wait for the new export version as it make things much easier

6

Hi,

So another feature that I realized that could be added (and if there is a way around it please let me know).

I turned a video into sequence of images however having more than one person in the within the image can mess up the model and it will take a very long time to crop them out. It would be great if labelers could label the best person in each image and during export it would export only the selected area plus custom amount of padding around the image.

i hope that makes sense and if there is a way around this that I have missed please let me know.