Investigasi Pengaruh Skema Stride dan Step Training untuk Deteksi Jari Pada Region-based Fully Convolutional Network (R-FCN) dalam Teknologi Augmented Reality

Hashfi Fadhillah, Suryo Adhi Wibowo, Rita Purnamasari




Combining the real world with the virtual world and then modeling it in 3D is an effort carried on Augmented Reality (AR) technology. Using fingers for computer operations on multi-devices makes the system more interactive. Marker-based AR is one type of AR that uses markers in its detection. This study designed the AR system by detecting fingertips as markers. This system is designed using the Region-based Deep Fully Convolutional Network (R-FCN) deep learning method. This method develops detection results obtained from the Fully Connected Network (FCN). Detection results will be integrated with a computer pointer for basic operations. This study uses a predetermined step scheme to get the best IoU parameters, precision and accuracy. The scheme in this study uses a step scheme, namely: 25K, 50K and 75K step. High precision creates centroid point changes that are not too far away. High accuracy can improve AR performance under conditions of rapid movement and improper finger conditions. The system design uses a dataset in the form of an index finger image with a configuration of 10,800 training data and 3,600 test data. The model will be tested on each scheme using video at different distances, locations and times. This study produced the best results on the 25K step scheme with IoU of 69%, precision of 5.56 and accuracy of 96%.

Keyword: Augmented Reality, Region-based Convolutional Network, Fully Convolutional Network, Pointer, Step training



Menggabungkan dunia nyata dengan dunia virtual lalu memodelkannya bentuk 3D merupakan upaya yang diusung pada teknologi Augmented Reality (AR). Menggunakan jari untuk operasi komputer pada multi-device membuat sistem yang lebih interaktif. Marker-based AR merupakan salah satu jenis AR yang menggunakan marker dalam deteksinya. Penelitian ini merancang sistem AR dengan mendeteksi ujung jari sebagai marker. Sistem ini dirancang menggunakan metode deep learning Region-based Fully Convolutional Network (R-FCN). Metode ini mengembangkan hasil deteksi yang didapat dari Fully Connected Network (FCN). Hasil deteksi akan diintegrasikan dengan pointer komputer untuk operasi dasar. Penelitian ini menggunakan skema step training yang telah ditentukan untuk mendapatkan parameter IoU, presisi dan akurasi yang terbaik. Skema pada penelitian ini menggunakan skema step yaitu: 25K, 50K dan 75K step. Presisi tinggi menciptakan perubahan titik centroid yang tidak terlalu jauh. Akurasi  yang tinggi dapat meningkatkan kinerja AR dalam kondisi pergerakan yang cepat dan kondisi jari yang tidak tepat. Perancangan sistem menggunakan dataset berupa citra jari telunjuk dengan konfigurasi 10.800 data latih dan 3.600 data uji. Model akan diuji pada tiap skema dilakukan menggunakan video pada jarak, lokasi dan waktu yang berbeda. Penelitian ini menghasilkan hasil terbaik pada skema step 25K dengan IoU sebesar 69%, presisi sebesar 5,56 dan akurasi sebesar 96%.

Kata kunci: Augmented Reality, Region-based Convolutional Network, Fully Convolutional Network, Pointer, Step training



Augmented Reality; Region-based Convolutional Network; Fully Convolutional Network; Pointer; Step training

Full Text:



Negrotti M. 2010. Virtual Reality. The Reality of the Artificial Edisi pertama Springer-Verlag Berlin Heidelberg. Jerman.

Aniket P. 2015. Augmented Reality. Intenational Journal of Research in Advance Engineering (IJRAE).

Lim C., Choi J., Park J. I. 2015. Interactive Augmented Reality System Using Projector-Camera System and Smartphone. EEE International Symposium On Consumer Electronics (ISCE).

Madni M. S., Rathod R. N. 2016. Color Segementation for Sixth Sense Device. Bonfring International Journal of Research in Communication Engineering.

Gupta J., Bankar A., Warankar M., Shelke A. 2014. Augmented Reality by using Hand Gesture. International Journal of Engineering Reesearch & Technology (IJERT).

Choi J., Seo B.-K., Park J.-I. 2009. Robust Hand Detection for Augmented Reality Interface.

Lambrecht J., Walzel H., Kruger J. 2013. Robust finger gesture recognition on handheld devices for spatial programming of industrial robots. Proceedings IEEE International Workshop on Robot and Human Interactive Communication.

Khan S., Rahmani H., Shah S. A. A., Bennamoun M. 2018. A Guide to Convolutional Neural Networks for Computer Vision. Synthesis Lectures on Computer Vision.

Wibowo S. A., Lee H., Kim E. K., Kim S.. 2017. Convolutional shallow features for performance

improvement of histogram of oriented gradients in visual object tracking. Mathematical Problems in


Darmadi R. 2018. Mengenal Convolutional Layer Dan Pooling Layer. Diambil pada tanggal 6 November 2019 dari

Wibowo S. A., Lee H., Kim E. K., Kim S.. 2018. Collaborative learning based on convolutional features and correlation filter for visual tracking. International Journal of Control, Automation and Systems.

Wibowo S. A., Lee H., Kim E. K., Kim S.. 2017. Visual tracking based on complementary learners with distractor handling. Mathematical Problems in Engineering.

Darmadi R.. 2018. Mengenal Convolutional Layer Dan Pooling Layer diambil pada tanggal 7 November 2019 dari

Dai J., Li Y., He K., Sun J. 2016. R-FCN : Object Detection via Region-based Fully Convolutional Network. Advances In Neural Information Systems 29 (NIPS 2016).

Ren S., He K., Girshick R., Sun J. 2017. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans. Pattern Anal. Mach.



  • There are currently no refbacks.

Copyright (c) 2020 Jurnal Ilmiah FIFO

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Jurnal Ilmiah FIFO
Fakultas Ilmu Komputer
Universitas Mercu Buana
Jl. Raya Meruya Selatan, Kembangan, Jakarta 11650
Tlp./Fax: +62215871335
p-ISSN: 2085-4315
e-ISSN: 2502-8332

e-mail:[email protected]

  Creative Commons Licence 
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.


width= width=

Image result for jurnal sinta