We have about 1000 text files that contain pre-annotated entities. These entities are hints for annotators and we want them to expand the annotations to wider text spans. Using the Python SDK, we have successfully imported pre-annotated text files but we are having two problems.
- The system did not import pre-annotated text files into the “to label” category. Instead, they were imported into the “done” category. We manually moved them to the “in review” category.
- After sharing the project with a collaborator, the collaborator sees 0 items under the “in review” category. This happens even if the “Unhide from labelers” is checked. (see Image 1)
My questions are
- How can we directly import pre-annotated text as the “to label” category?
- If a dataset is annotated by more than 1 annotator, how could Labelbox keep multiple versions of the annotations and calculate the consensus rate?