Real-Time Web-Based Indonesian Sign Language (BISINDO) Translator System Using CNN-LSTM Deep Learning and Text-to-Speech
DOI:
https://doi.org/10.35335/jict.v17i1.320Keywords:
BISINDO, CNN-LSTM, Deep Learning, Real-Time Web, Text-to-SpeechAbstract
Indonesian Sign Language (BISINDO) is the primary communication medium for the deaf community, yet low public understanding often causes communication barriers. Previous sign language recognition studies mostly operated offline, lacked real-time web integration, and only produced text output. This study designs and develops a real-time web-based BISINDO translator system using a hybrid Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) method, integrated with a Text-to-Speech (TTS) feature. The dataset consisted of primary video data from 3 subjects, covering 11 category classes with 1,000 frames per class in grayscale format (100x89 pixels). The hybrid CNN-LSTM model was integrated into a React.js and Node.js web application (NzSignify). Testing results demonstrate that the model achieved 96% static accuracy based on Confusion Matrix evaluation. In real-time functional testing, an 80% Confidence Threshold effectively filtered incorrect gestures, enabling accurate translation of valid sign gestures into text and voice output.
References
World Health Organization. (2025). Deafness and hearing loss. WHO. https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss
Aljabar, A. (2020). BISINDO (Bahasa Isyarat Indonesia) Sign Language Recognition Using CNN and LSTM. Journal of Information Systems, 5(5), 282-287.
Altairika, E., & Sari, W. P. (2023). Pengembangan Deteksi Realtime Bahasa Isyarat Indonesia Menggunakan CNN dan LSTM. Jurnal Teknologi Informatika Dan Komputer, 9(1), 1-13. https://doi.org/10.37012/jtik.v9i1.1272
Fadillah, R. Z., Irawan, A., & Susanty, M. (2022). Model Penerjemah Bahasa Isyarat Indonesia (BISINDO) Menggunakan Pendekatan Transfer Learning. 15(1), 1-9.
Isnaniah, S., Agustina, T., Islahuddin, & Annisa, F. (2023). Perbandingan Pemahaman Bahasa Isyarat Indonesia dan SIBI dalam Pembelajaran Siswa Tuli. Jurnal Pendidikan Luar Biasa.
Khan, S., Rahmani, H., Shah, S. A. A., & Bennamoun, M. (2021). A Survey of the Recent Architectures of Deep Convolutional Neural Networks. Artificial Intelligence Review, 53(8), 5455-5516. https://doi.org/10.1007/s10462-020-09825-6
Alzubi, J., Nayyar, A., & Kumar, A. (2024). Recurrent Neural Networks: A Comprehensive Review of Architectures, Variants, and Applications. Information, 15(9), 517. https://doi.org/10.3390/info15090517
Myagila, K., & Nyambo, D. G. (2025). Efficient spatio-temporal modeling for sign language recognition using CNN and RNN architectures. Frontiers in Artificial Intelligence. https://doi.org/10.3389/frai.2025.1630743
Tan, X., Wang, T., & Chen, J. (2023). NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality. IEEE Transactions on Audio, Speech, and Language Processing.
Kumari, D., & Anand, R. S. (2024). Isolated Video-Based Sign Language Recognition Using a Hybrid CNN-LSTM Framework Based on Attention Mechanism.
Russell, S., & Norvig, P. (2021). Artificial Intelligence: A Modern Approach. Pearson.
Axza, F., Sofi, F., & Qoiriah, A. (2023). Analisis Perbandingan Framework Front-End Javascript React dan Vue Pada Pengembangan Website. 05, 157-164.
Ramadhani, A., Iriadi, N., & Hidayat, R. (2025). Implementasi Teknologi Rest API Dengan Node Js Untuk Aplikasi Rekomendasi Destinasi Wisata. 4(1), 22-29.
Deleviar, M. A., et al. (2025). Speech-to-Video BISINDO Website Using LSTM. Jurnal Teknologi Informatika.
Hochreiter, S., & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735-1780.
Jurnal ICT : Information and Communication Technologies is licensed under a 