Hello,
I was wondering if it’s possible to use the Python SDK to automatically create an Issue on a Label (e.g., the ! pin in the labeling interface, see attached image). For example, if I download the Labels for several samples and find a problem with them, is there a way to automatically create Issues that describe the problem on the corresponding Labels in Labelbox?
Thank you in advance for your advice!
Hi @rszeto ,
Thank you for posting! While it is not currently possible to create issues with the SDK, we may consider adding the functionality in the future.
Best,
Paul N.
Got it, thanks! This would be a huge boon for automatic verification that Labelbox doesn’t support natively.
Thanks! @rszeto, in your use case, what kind of projects would you create issues with? Would you need to place the issue precisely in the data row?
Hi Paul,
In my particular case, I would be interested in placing an issue anywhere on an annotator’s image labels (i.e., each person who labeled an image might see a different automatic issue). Long term, I might annotate videos as well.
For more context, I am looking at annotating human keypoints. As shown in the screenshot above, our top-level labels are person instances. Each keypoint has a “keypoint type” attribute, e.g., nose, left shoulder, right ankle, etc. However, there’s no way I found to enforce uniqueness of keypoint types per person, so Person 1 could be labeled with two left elbows, for instance.
In the above example, I would process the labels to find the duplicate keypoint types per person, then automatically create issues on the incorrect labels and send back for rework. If multiple people labeled the same image with different keypoint type issues, they would see issues reflecting their own situation. Location isn’t as important for this example since it’s unclear where the issue marker would go anyways.
Let me know if you have any other suggestions or questions regarding this use case. Thanks!
Thank you for the additional details!
In terms of monitoring, you could use webhook (see webhooks or specialised articles in the Community) to create the list of annotations in a newly created label, then use project.move_data_rows_to_task_queue() (see move-data-rows-to-a-workflow-task) when required.
