UoR at SemEval-2021 task 12: on crowd annotations: learning with disagreements to optimise crowd truth

Osei-Brefo, E., Markchom, T. and Liang, H. (2021) UoR at SemEval-2021 task 12: on crowd annotations: learning with disagreements to optimise crowd truth. In: SemEval-2021, 5-6 August 2021, Bangkok.

Abstract/Summary

Crowd sourcing has been ubiquitously used for annotating enormous collections of data. However, the major obstacles of using crowd-sourced labels are noise and errors from non-expert annotations. In this work, two approaches dealing with the noise and errors in crowd-sourced labels are proposed. The first approach uses Sharpness-Aware Minimization (SAM), an optimization technique robust to noisy labels. The other approach leverages a neural network layer called crowd layer specifically designed to learn from crowd-sourced annotations. According to the results, the proposed approaches can improve the performance of Wide Residual Network model and Multi-layer Perception model applied on two crowd-sourced datasets in image processing domain.

Item Type	Conference or Workshop Item (Paper)
URI	https://reading-clone.eprints-hosting.org/id/eprint/97212
Refereed	Yes
Divisions	Science > School of Mathematical, Physical and Computational Sciences > Department of Computer Science
Download/View statistics	View download statistics for this item