In the Workforce Boost express checkout, it appears I can only specify the “Qty” of labellers, i.e. the number of 40hrs units.
Can I also specify the number of labellers I get to distribute each of those 40hr units? I think this would be a good feature to get labeller diversity. I still want to use “Benchmark” instead of “Consensus”, because I only need one label per data row.
Thanks!
Hi @user0,
Thank you for the feedback! It was forwarded internally.
[edited]
A quantity of 1 is for 40 hours for 1 labeller.
Best regards,
Paul N
Labelbox Support
Thanks Paul, very helpful!
We would like to administer some “qualifying questions” to each labeller. Will it be possible to message each labeller separately, or only as a group? Or would you suggest another way to do this?
My two cents would be this: grade the quality of the labor results and be indifferent as to the quality the labor input. Unless what you are producing is quality labeling labor, then you would be pointing at the moon but looking at the finger to care about individual quality results per laborer.
With that said, you can review the Performance by each label-er inside an Annotate Project. You will see Throughput, Efficiency, and Quality broken down into several views that should allow you to understand performance an an individual label-er level. If the results are unsatisfactory, and/or Performance metrics need improvement you could review the quality of your labeling instructions as well as the overall quality of your data architecture, taxonomy, and labeling project design.
Would be more than happy to offer some detailed advice about your projects unique needs if you want. I can DM you a link to my calendar. Can’t promise I will have any better advice than shared here, but I’m a friendly sort that is always super interested in helping out the labelbox community.
@user0 Apologies but I shared incorrect details with you.
The quantity works like this:
Qty 1: 1 labeler for 40 hours
Qty 2: 2 labelers for 80 hours (2 x 40)
Qty 3: 3 labelers for 120 (3 x 40)
Qty X: X labelers for X x 40
Best regards,
Paul N
Labelbox Support
Hi Paul, thank you for clarifying your response.
It is crucial to our project and budget to have many labellers but less than 40hrs per labeller – our using Labelbox was contingent upon this possibility. We paid for Boost express labellers based on your original answer, but now your updated answer tells us it is impossible.
The incorrect info has caused us to pay for a service that is not as described. Could we please discuss how to remedy the situation – happy to continue the conversation in the Help Portal support ticket I originally created for this question.
Hi @user0,
Yes, please create a ticket with Labelbox Support and attach this thread to share it with our billing team.
Apologies for the inconvenience.
Best regards,
Paul N
Labelbox Support
