Tasks (legacy)

Tasks are batches of images that annotators or users can work on independently.
Jan 26, 2020

How to use tasks for quality control

Tasks help automate the workflow of annotators, giving them a batch of images that they can work on independently whilst other users may be browsing the dataset.

To generate a task, you can select images in dataset management. You may select them in batches using shift+click, and then manually assign them as a task to a user. Annotators may also generate tasks to suit their available time. A user may be able to set the number of images in a generated task via the dataset's Settings tab.

Once completed, a tasks' status moves from "Assigned" to "Pending Review". This will notify users and admins that a new task is availabe for review. As soon as a user opens the task for review, it can no longer be edited and their status becomes "In Review". The user may accept or reject images as they see fit, leaving comments to provide feedback to the annotator. Each comment left on a rejected image counts as a -1 point for the annotator's quality score. Each object instance in an approved image counts as a +1.

Once complete, approved images from a task are marked as "Done", rejected ones stay in the task for the assigned annotator to correct, and any discarded images are archived.