I’m trying to transfer some annotations from one dataset in a project to a different dataset in the same project. I have the same ontology in both projects. Here is
import labelbox as lb
from labelbox.schema.conflict_resolution_strategy import ConflictResolutionStrategy
import labelbox.data.annotation_types as lb_types
from labelbox import MALPredictionImport
import uuid
API_KEY = "..."
def export(project):
export_task = project.export_v2()
export_task.wait_till_done()
return export_task.result
def build_external_to_internal_key_dict(exported_data) -> dict[str, str]:
d = {}
for row in exported_data:
internal_id = row["data_row"]["id"]
external_id = row["data_row"]["external_id"]
d[external_id] = internal_id
return d
def convert_bbox(bbox):
left = int(bbox["left"])
top = int(bbox["top"])
width = int(bbox["width"])
height = int(bbox["height"])
return lb_types.Rectangle(
start=lb_types.Point(
x=left,
y=top,
),
end=lb_types.Point(x=left + width, y=top + height),
)
def build_matching_dictionary(source_project, destination_project):
source_export = export(source_project)
dest_export = export(destination_project)
external_to_dest = build_external_to_internal_key_dict(dest_export)
labels = []
for row in source_export:
external_id = row["data_row"]["external_id"]
if external_id not in external_to_dest:
continue
matching_dest_key = external_to_dest[external_id]
json_annotations = list(row["projects"].values())[0]["labels"][0][
"annotations"
]["objects"]
annotations = [
lb_types.ObjectAnnotation(
name="wire", value=convert_bbox(a["bounding_box"])
)
for a in json_annotations
]
labels.append(
lb_types.Label(
data={"global_key": matching_dest_key}, annotations=annotations
)
)
return labels
def main():
client = lb.Client(api_key=API_KEY)
# Get source and destination projects
source_project = client.get_project("cmfz7kd0p23ei07zj5caj5my2")
destination_project = client.get_project("cmgq9svmk18i0071g59o544kk")
labels = build_matching_dictionary(source_project, destination_project)
upload_job = MALPredictionImport.create_from_objects(
client=client,
project_id="cmgq9svmk18i0071g59o544kk",
name="mal_job" + str(uuid.uuid4()),
predictions=labels,
)
upload_job.wait_till_done(sleep_time_seconds=1, show_progress=True)
if __name__ == "__main__":
main()
When I run this script, I get a tqdm-style progress bar that, within a few seconds, gets to 96%, and then stalls. I’ve left it hanging for quite some time, and it doesn’t seem to be making any progress. Is there a problem with how I’m using the MALPredictionImport tool?
I’ve also tried with LabelImport.create_from_objects and find the same behavior. Here’s a taskid for a running job: mal_job1ef4bd4f-5523-4838-83af-13ae28c68a84
I’d be happy to try but I don’t understand the send_to_annotate_from_catalog arguments. It looks like it takes a list of global keys; how can I send the bounding boxes? Since I’m copying the annotations to different images, the global keys in the source project don’t match exactly the global keys i the destination project.
Unfortunately this resulted in all the data, including images, being copied from my source project to the destination project, which isn’t want I’m trying to do. I instead want to copy the annotations that we’ve made on image1.png in dataset 1 to the image named image1.png on dataset 2 (i.e., an image with the same name but a different id and contents). The context here is that we labeled some images on filtered images but we’re now interested in labeling on unfiltered images, which means certain changes on how the labeling is done, but we still want to start with the original labels to speed up the process.
I see that the project cmgq9svmk18i0071g59o544kk is now deleted. In the original script, what part was stalled? Did you obtain the same results with export() instead of export_v2() (now deprecated).
Sorry, yes, because I had added the images from the original source project I cleaned up the old project and started a new one. The new project ID is mal_jobb93cc9a4-5392-40e5-ab33-a6c723a80474 and an example running import job is mal_jobb93cc9a4-5392-40e5-ab33-a6c723a80474.
This doesn’t stall on the export stage, and I can successfully export and parse the results. It’s only at the MALPredictionImport.create_from_objects stage. Just to test it though I’ve switched to export and am getting the same results.
The symptom is that the script runs fine up to MALPredictionImport.create_from_objects". I get the returned upload_job object, and call upload_job.wait_till_done(sleep_time_seconds=1, show_progress=True). I then get a progress bar, which stalls at 96%:
I think I’ve figured it out. The issue was that I was setting global_key instead of uid in the label creation process. At least, fixing that error changed the behavior.