Improving Textual Ekman-Based Emotion Mapping on Social Media and RoBERTa Models

Abstract

Recognizing emotions from social media short text continues to be a difficult problem due to the presence of serious class imbalance and semantic confusion in the fine-grained emotion corpus. Although the GoEmotions dataset is realistic, it involves 28 different emotion labels in an imbalanced manner, with significant class underrepresentation. Imbalance in class distribution negatively impacts the performance of multi-label classification models in terms of generalization capability and minority class detection.

In this paper, the effects of emotion label representations and adaptive decision criteria are investigated on multi-label emotion classification. More precisely, the number of emotion labels is reduced from 28 to seven basic emotions according to Ekman’s theory to mitigate the problem of class imbalance and decrease the complexity of the classification problem. Moreover, a label-based threshold adjustment technique is used, where different optimal threshold values are determined.

In the experiments, we utilized a RoBERTa model that implements partial freezing of layers and make a comparison with a hybrid BERT–BiLSTM–Attention model within both fine-grained and Ekman-based configurations. The Binary Cross-Entropy loss function is used for training the model, while early stopping is implemented to avoid overfitting. In terms of results, our method provides an average Micro-F1 score of 0.6501 and Macro-F1 score of 0.5951 for the GoEmotions test dataset.

These results indicate that the reduction of the label set space positively affects the stability and accuracy of classification. Moreover, threshold tuning for individual labels allows balancing precision and recall and, consequently, achieves better results for minority classes. Therefore, label engineering and adaptive decisions prove to be more effective than complex models in improving emotion classification accuracy.