Page 84 - 2024F

UEC Int’l Mini-Conference No.53 77

Deep Learning Based Japanese Sign Language Translation

Rattapoom KEDTIWERASAK , Hiroki TAKAHASHI
∗
Department of Informatics
The University of Electro-Communications
Tokyo, Japan

Keywords: Sign Language Translation (SLT), Transformer, Self-Supervised Video Pretraining for
Sign Language Translation (SSVP-SLT).

Abstract
Sign Language is the native language for the deaf and hard-of-hearing people. For supporting
communication, Sign Language Translation (SLT) aims to convert sign language videos to spoken
languages, serving as a crucial bridge for better accessibility and communication. However, Japanese
Sign Language (JSL) translation challenges on the scarcity of parallel corpus. To address this issue,
we utilize Self-Supervised Video Pretraining for Sign Language Translation (SSVP-SLT) as a pre-
trained large-scale American Sign Language (ASL) model with fine-tuning on small JSL dataset.
By transferring knowledge, we aim to overcome the data scarcity problem in JSL translation. Our
SLT approach uses a Transformer model, which has shown effectiveness in capturing spatio-temporal
dependencies in sign language videos. The proposed method addresses the lack of JSL datasets and
demonstrates cross-lingual transfer learning in SLT for improving communication technologies for
the Deaf community in Japan.

∗
The author is supported by (SESS) MEXT Scholarship.