Performance Evaluation of the BERT Model in Sentiment Analysis of DANA Application User Reviews

Abstract

The rapid growth of digital wallets in Indonesia generates a large volume of user reviews on platforms such as the Google Play Store that cannot be efficiently analyzed manually. This study aims to evaluate the performance of the BERT (Bidirectional Encoder Representations from Transformers) model in sentiment classification tasks on a dataset of DANA application user reviews collected from the Google Play Store. The BERT model is fine-tuned using labeled Indonesian-language data with three sentiment classes: positive, negative, and neutral. Specialized preprocessing strategies are applied to handle the characteristics of informal text, abbreviations, and code-switching phenomena prevalent in Indonesian user reviews. Evaluation is conducted using accuracy, precision, recall, and F1-score metrics. Experimental results indicate that the fine-tuned IndoBERT model achieves an accuracy of 91.24% with a weighted F1-score of 0.91 on a test dataset of 6,106 samples. The Negative class achieves the highest performance with an F1-score of 0.95, followed by the Positive class (0.88) and Neutral class (0.84). This study provides empirical evidence of the effectiveness of the IndoBERT Transformer architecture for sentiment analysis in the Indonesian-language fintech domain and can serve as a reference for developing deep learning-based NLP systems in similar contexts.