Cannot share "In review" items

[j1wu]

We have about 1000 text files that contain pre-annotated entities. These entities are hints for annotators and we want them to expand the annotations to wider text spans. Using the Python SDK, we have successfully imported pre-annotated text files but we are having two problems.

1. The system did not import pre-annotated text files into the “to label” category. Instead, they were imported into the “done” category. We manually moved them to the “in review” category.
2. After sharing the project with a collaborator, the collaborator sees 0 items under the “in review” category. This happens even if the “Unhide from labelers” is checked. (see Image 1)

My questions are

1. How can we directly import pre-annotated text as the “to label” category?
2. If a dataset is annotated by more than 1 annotator, how could Labelbox keep multiple versions of the annotations and calculate the consensus rate?

[Zeke]

Hi @j1wu, thanks for the detailed post.

1. Instead of importing your annotations as ground truth using LabelImport.create_from_objects, you should import your annotations as pre-labels (model-assisted labeling) using MALPredictionImport.create_from_objects. Examples for every data type are available in our documentation – here’s the page for importing image annotations in which you can see the difference between these two approaches in step 6.

2. I recommend checking out our documentation on Consensus to learn more and start using consensus.