Inter-annotator agreement
How often independent human labellers give the same answer on the same item, usually reported as Cohen's or Fleiss' kappa, which correct for agreement by chance.
What people get wrong
It caps what any model can score. If two experts agree only 70% of the time, the task definition is ambiguous, and no amount of modelling will push measured accuracy past that ceiling.
A definition is not an understanding. This is taught properly in Where ground truth comes from, part of Data quality & labelling.
Learn this properlyTerms from the same topic