Hi,
the slices are great feature but a great add-on would be to allow slices to be shared with other team members. So if I create a slice to go through the data I can share it with other team members so they can view those data as well.
maybe allow notes on some images to be made so other team members can review them and address them.
Hi @lefty. We are glad to hear that you find Slices useful.
Slices are currently visible for all the admin users in your organization (not for labelers currently). Just to understand your needs, do you need to share a Slice with non-admin users? Or do you want to share a particular Slice (maybe via URL) to others?
For the second note around allowing notes on some images for others to review, we are introducing a new feature called global issues soon that allows team members to comment and collaborate on data rows for labeling issues or model errors. We will announce once they are available.
Hi,
No I would only want to share them with admins and reviewers.
Just to give you more background to paint a better picture. We had too many very detailed labels for a small areas that it was throwing our model off. We decided to split our model into 2 models and:
but a list of images were “throwing off” the models so we were filtering through the images to find all of the different variants of these images and then see which object / model to group these images with.
So when I created a slice I wanted to save it and have it show up in the other admins plan so there is a lot less back and forth. This way we both can go through the list of filtered images and leave notes if needed on images or even that particular slice (for example, this slice wouldn’t work because of xyz reasons and I think we should change or add abc to the slice).
this way all parties can try out different slices, share and review each other slices all within labelbox to better coordinate and plan things.
The other thing is filtering slices based on the schemas. Right now if we related different objects together it has a huge effect on all labeled data. So it really limits the ability of the admins to play with the data. As projects grow schemas change and schemas also are setup to assist labelers during their job.
in short: also allow admins to relate objects in different schemas and even rename objects so they can filter data better. this is causing another big issue.
P.S. sorry about the long post, just wanted to be detailed and hopefully get my message across better.
Hi @lefty! Thanks for the context! I think there are ways to accomplish the workflow you are describing here with our current product features.
To collect specific images that “throw off” the model and put them in a Slice. You can do that via assigning metadata to those images. For example, you can tag them as “problematic images for model XYZ” and use Catalog’s filters to create a Slice based on that metadata tag.
To filter within a Slice based on schema: Once you navigate to Slice, you can still add more Catalog filters on that Slice to get more refined results. From here, we support adding filters such as project (each project should have its unique ontology and schema), or adding filters based on annotations on that asset (e.g. you can filter for images containing “cat” object labels).
Hope this helps and let me know if you have more questions!
(P.S. Once we release global issues, it might be a better place to collaborate on a model issue like this in the future.)
Hey @lefty, out of curiosity, how are you determining which images are throwing off the models?
Thanks!
Hi,
@mvoisin, the 2nd option:
is it in Model, by looking at the labels and the predictions?
we then wanted to go through the rest of the images to see what other different scenarios could cause an issue if we combined the objects differently.
@kyang thank you for taking the time to respond. For the 2nd option, i wouldn’t be able to combine different schema into one search though, correct?
for example our situation could be something like this:
If I could say rename this object from all different schema to left hand - index finger and it would show me all of those labeled images and then it would make things easier. Right now we can do that with code but we lose the capability of doing it on the platform via schema. Plus doing it on the platform it would help the development group to come to an agreement on the naming because what works for our development team doesn’t work with the labelers.
I hope that makes sense and i’m sorry if we can already do this but somehow I keep missing it.
by the way over the years I have come to the conclusion that all projects break down to 3 groups of people working together for all ML projects.
People that understand the problem and machine learning but don’t have the technical understanding of doing it
the engineers
the labelers
Labelbox is alright at filtering through the raw data and good at filtering through labeled data but there is still this massive gap between the none technical people and the engineers on communicating.
For example, it’s easier to teach a doctor basics of machine learning than vice versa but there is nowhere, were these 2 group of people can easily remotely collaborate. This is where labelbox can really shine.
If there is another platform that I should be using with labelbox to help do this please let me know but I have personally yet to find any.
@lefty Thanks for the feedback. We are working on major upgrade for collaboration. From issue management to role based access controls across all three products. We want to make the most common E2E data-centric ML workflows seamless on the platform.