DESIGNING TRANSLATION TOOL: BETWEEN SIGN LANGUAGE TO SPOKEN TEXT ON KINECT TIME SERIES DATA USING DYNAMIC TIME WARPING
Zico Pratama Putera, Mila Desi Anasanti, Bagus Priambodo
Abstract
The gesture is one of the most natural and expressive methods for the hearing impaired. Most researchers, however, focus on either static gestures, postures or a small group of dynamic gestures due to the complexity of dynamic gestures. We propose the Kinect Translation Tool to recognize the user's gesture. As a result, the Kinect Translation Tool can be used for bilateral communication with the deaf community. Since real-time detection of a large number of dynamic gestures is taken into account, some efficient algorithms and models are required. The dynamic time warping algorithm is used here to detect and translate the gesture. Kinect Sign Language should translate sign language into written and spoken words. Conversely, people can reply directly with their spoken word, which is converted into literal text together with the animated 3D sign language gestures. The user study, which included several prototypes of the user interface, was carried out with the observation of ten participants who had to gesture and spell the phrases in American Sign Language (ASL). The speech recognition tests for simple phrases have therefore shown good results. The system also recognized the participant's gesture very well during the test. The study suggested that a natural user interface with Microsoft Kinect could be interpreted as a sign language translator for the hearing impaired.
Keywords
Human-Computer Interaction: Natural User Interface; Speech Recognition; 3D Animation; Sign Language
DOI:
http://dx.doi.org/10.22441/sinergi.2018.2.004
Refbacks
- There are currently no refbacks.
SINERGI
Published by:
Fakultas Teknik Universitas Mercu Buana
Jl. Raya Meruya Selatan, Kembangan, Jakarta 11650
Tlp./Fax: +62215871335
p-ISSN: 1410-2331
e-ISSN: 2460-1217
Journal URL: http://publikasi.mercubuana.ac.id/index.php/sinergi
Journal DOI: 10.22441/sinergi
Journal by SINERGI is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License
View My Stats
The Journal is Indexed and Journal List Title by:
POSKOBET
POSKOBET
POSTOTO787
POSTOTO787
EMAS787
EMAS787
SUNDA787
SUNDA787
https://www.thedecliningwinter.com
ASIABET777
ASIABET777
https://mega888slots.com
https://www.thecarecommunity.com
https://mega888slots.com
diamond murah
voucher game
langkah 4d
toke88
gdtoto
mideatototokeslot88
langkah4d
langkah4d
langkah4d