Sorry for the delayed response. I’ll provide a detailed explanation.
Project ID
I could share the project IDs for both annotation projects, but they are not linked to an account associated with the email I am writing from.
Benchmark annotation project_id: cm8h82a0l0arr07vlgnnf8r1v
Upload annotation project_id: cm8h9cbaq0nqg07zmflpxashg
Workflow
I have annotations stored locally on my computer, and I want to upload them as pre-labeling/MAL into an existing project that already contains unlabeled data rows.
Objective
I’m trying to achieve the following:
- For each frame of a video, I collect all the corresponding grayscale masks and merge them into a single RGB mask.
- I then write this composite mask to a byte buffer and use it as the mask associated with that frame.
Code for Loading Video Annotation Masks
The _write function processes all annotations for a given video. It generates a Label from a list of VideoMaskAnnotation objects by creating a composite mask for each frame using the combine_masks function.
THIS IS PSEUDO CODE JUST TO GIVE AN IDEA OF WHAT I AM DOING
import labelbox.types as lb_types
import io
import numpy as np
from PIL import Image
instance_colors = {
"solid" : (255,0,0)
}
def is_empty_mask(mask:np.ndarray):
return np.max(mask) == 0
def is_gray_scale(mask:np.ndarray):
return len(mask.shape) == 2
def validate_mask(mask:np.ndarray):
if is_empty_mask(mask) and is_gray_scale(mask):
raise ValueError("Mask not valid")
def combine_binary_masks(mask_data_rows, instance_value_mapping):
"""
mask_paths: list of file paths for the instance masks (for one frame).
instance_value_mapping: dict mapping each mask filename (or an identifier) to a unique int value.
"""
# Assume all masks have the same shape; load the first mask to get shape
mask_paths = mask_data_rows["file_uri"].to_list()
keys = mask_data_rows["file_name"].to_list()
sample_mask = np.array(Image.open(mask_paths[0]))
height, width = sample_mask.shape
# Initialize an empty RGB composite mask
composite = np.zeros((height, width, 3), dtype=np.uint8)
for path, key in zip(mask_paths, keys):
# Use a key that is consistent (for example, the file basename) to get the unique value
color = instance_value_mapping.get(key, 0)
arr_img = np.array(Image.open(path))
validate_mask(arr_img)
# Set composite pixels to the unique value where the mask is present
indices = arr_img > 0
composite[indices] = color
return Image.fromarray(composite)
def _write(self, data_rows:pd.DataFrame):
instances_mask = list()
# Create unique Mask instance value one for each annotation class
global_key = "SomeGlobalKey"
for file_name in data_rows["file_name"].unique().tolist():
annotation_name = file_name
color_rgb = instance_colors[file_name]
instances_mask.append(
lb_types.MaskInstance(
color_rgb=color_rgb,
name=annotation_name
)
)
# For each frame Compose multiple binary masks in a single RGB one
frames_mask = list()
for _, f_group in data_rows:
frame_no = f_group["frame"].to_list()[0]
pil_image = combine_binary_masks(
mask_data_rows=f_group,
instance_value_mapping=instance_colors
)
# Save the PIL image to this buffer
buffer = io.BytesIO()
pil_image.save(buffer, format="PNG")
# Get the byte data
img_bytes_io = buffer.getvalue()
frames_mask.append(
lb_types.MaskFrame(
index=int(frame_no) + 1,
im_bytes=img_bytes_io # Instead of bytes you could also pass an instance URI : instance_uri=url
)
)
annotations = list()
annotations.append(lb_types.VideoMaskAnnotation(
frames= frames_mask,
instances= instances_mask
))
return lb_types.Label(
data= {"global_key": global_key},
annotations = annotations,
)
I get those warning errors and I do not know why since actually the type of the buffer is “bytes”
~/miniconda3/envs/general/lib/python3.10/site-packages/pydantic/main.py:426: UserWarning: Pydantic serializer warnings:
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...LnWtPAPAAAAAElFTkSuQmCC'` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...npr8gAAAABJRU5ErkJggg=='` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...G4gLoMAAAAASUVORK5CYII='` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...Pw6QQAAAABJRU5ErkJggg=='` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...WQK6wAAAABJRU5ErkJggg=='` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...8ifwgkAAAAASUVORK5CYII='` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...7T5uDcAAAAASUVORK5CYII='` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...jEYQsUAAAAASUVORK5CYII='` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...cAM0WoSAAAAAElFTkSuQmCC'` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...oT9QAAAAABJRU5ErkJggg=='` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...fQYPaAsAAAAAElFTkSuQmCC'` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...AW+5GeGAAAAAElFTkSuQmCC'` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...5DkEo4AAAAASUVORK5CYII='` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...F7o3wAAAABJRU5ErkJggg=='` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...Zc5JeMAAAAASUVORK5CYII='` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...kXztgAAAABJRU5ErkJggg=='` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...+W2e+8AAAAASUVORK5CYII='` - serialized value may not be as expected
Expected `bytes` but got `str` with value `'iVBORw0KGgoAAAANSUhEUgAA...rj97X+sAAAAAElFTkSuQmCC'` - serialized value may not be as expected
