How to Label 1M Data Points/Week




  • Given a snippet of text and some possible continuations for it, the researchers at OpenAI wanted humans to pick the continuation that best fit a given prompt, such as “which continuation has the most positive sentiment” or “which continuation is the most descriptive”. (View Highlight)
  • on some computer vision projects, one feature that determines labeler quality is based on localization loss from a deep learning model. (View Highlight)
  • This is in line with a general principle we use in designing our operational systems:* leverage people we have highest trust in*. (View Highlight)
  • Every distributed system needs to be tested and robustly monitored, and a human-based distributed system like ours is no different. (View Highlight)