Importing annotations; S3 integration

Python SDK
1

Hi labelbox community and the labelbox dev team,

I need some assistance with getting the annotation uploading to work;

I already attempted following the notebook here:

no matter if I format my annotation objects as Python annotation or as a NDJSON payload, and no matter if i use the MALPredictionImport or LabelImport, i get the same error from the uploading job:

Import failed.

from AnnotationImport._fetch_remote_ndjson**(self, url)**

From the docs, I understood that it should be possible to upload .png masks as (instance) annotations; why would the uploader be fetching a remote ndjson file at all?

The annotation objects looks like this:

mask_data_1 = lb_types.MaskData(url=f"")

Python annotation

mask_1 = lb_types.ObjectAnnotation(
name = “Tooth_normal”, # must match your ontology feature"s name
value=lb_types.Mask(mask=mask_data_1, color=(255, 255, 255))
)

mask_annotation_ndjson = {
“name”: “Tooth_normal”,
“classifications”: ,
“mask”: {“instanceURI”: f"",
“colorRGB”: (255, 255, 255)}
}

As mentioned, attempted both ways - python sdk and ndjson importing.
The integration with s3 works, i had no issues uploading raw images - I am now aiming to upload existing ground truth annotations for each individual image.

Please help me out with understanding what im doing wrong :slight_smile:
Thanks,

2

the maskData url in both examples is correctly formatted and pointing to the correct object (a .png mask), i had to remove it because ‘new users can only post two links in a post’;

3

I confirmed that there is nothing wrong with the workflow as I am able to upload annotations from local file system, but fetching the annotation .png mask from S3 fails with the ndjson fetching error like described before.

Raw image uploading from S3, however, works as intended.

Uploading a mask from google storage, like demonstrated in the collab notebook, seems to work.

I would really appreciate an update on this topic :slight_smile:

1 Like
4

I have the same issue, except unable the upload with Python types, failing with the less than helpful error Import failed . @dejan.dukic could you post a more complete example of your payload?

5

Hi @nw26 !

def get_masks(data_row_id, basepath):
    
    # get the mask names based on the dataframe column image_id
    annotations = df[df['image_id'] == data_row_id].to_dict('records')
    # get the mask paths
    mask_names = [an['mask_id'] for an in annotations]
    mask_paths = [os.path.join(basepath, f'masks/{mn}.png') for mn in mask_names]
    mask_paths = [os.path.normpath(mp) for mp in mask_paths]
    # load the masks
    masks = [cv2.imread(mp, -1) for mp in mask_paths]
    # return the mask id and the mask as a dictionary

    return dict(zip(mask_names, masks))

def create_labelbox_masks(mask_dict, class_color_dict):
        
        labelbox_masks = []
        
        for mask_id, mask in mask_dict.items():
            
            mask_data = lb_types.MaskData.from_2D_arr(mask)
            # create a labelbox mask object
            cls = df[df['mask_id'] == mask_id].class_name.values[0]

            labelbox_mask = lb_types.ObjectAnnotation(
                name=name_mapping[cls],
                value=lb_types.Mask(mask=mask_data, 
                                    color=class_color_dict[cls]),
            )
            
            labelbox_masks.append(labelbox_mask)
            
        return labelbox_masks

That is the function for preparing the labelbox mask objects; I used the functions like:

for dr in tqdm(data_rows[1:]):

    global_key = dr.global_key

    # get the masks for the data row
    mask_dict = get_masks(global_key, basepath)
    # create labelbox masks
    labelbox_masks = create_labelbox_masks(mask_dict, class_color_dict)

    labels = list()

    labels.append(
        lb_types.Label(data=lb_types.ImageData(global_key=global_key),
                    annotations=labelbox_masks))
    
    try:
        # Upload label for this data row in project
        upload_job = lb.LabelImport.create_from_objects(
            client = client, 
            project_id = ps.uid, 
            name="label_import_job"+str(uuid.uuid4()),  
            labels=labels)
    except:
        print(f"Errors: {upload_job.errors}", )
        print(f"Status of uploads: {upload_job.statuses}")

The label object ends up being something like:

[Label(uid=None, data=ImageData(im_bytes=None,file_path=None,url=None,arr=None), annotations=[ObjectAnnotation(confidence=None, name='normal', feature_schema_id=None, extra={}, value=Mask(extra={}, mask=MaskData(im_bytes=None,file_path=None,url=None,arr=...), color=(0, 255, 0)), classifications=[]), ObjectAnnotation(confidence=None, name='normal', feature_schema_id=None, extra={}, value=Mask(extra={}, mask=MaskData(im_bytes=None,file_path=None,url=None,arr=...), color=(0, 255, 0)), classifications=[]), ObjectAnnotation(confidence=None, name='normal', feature_schema_id=None, extra={}, value=Mask(extra={}, mask=MaskData(im_bytes=None,file_path=None,url=None,arr=...), color=(0, 255, 0)), classifications=[])], extra={})]

Hope that helps.
You should be able to unpack the upload job errors to try and understand what the ‘import failed’ error is about.

One ‘gotcha’ I had is that the color argument specified in the ObjectAnnotation object, is what filters the mask object;
e.g. if your mask is [1, 1, 1], and your color [2, 2, 2], the mask will get recognized as empty.
So, the ‘color’ argument is not what determines the actual color in the labelbox UI (this is done via the ontology), it is more of a filter for your mask object.
I didn’t understand this straight away and I don’t really see it well documented anywhere.

I did update the labelbox python package but I didn’t yet try to import masks from s3;
if that ends up still being impossible, it will be potentially a major annoyance but i’m sure we can work around it - @labelbox team.