Deteksi Gestur Lengan Dinamis pada Lingkungan Virtual Tiga Dimensi Koleksi Warisan Budaya
Abstract
Virtual reality technology can be used to support museum exhibitions. Implementation could be in various platforms. There are many implementation options, for example in smartphones, tablet, and desktop computers. Most objects of museum collections are very fragile. Minimizing the direct touch on a collection object is one of the benefits of this technology. This study aims to prepare gestures suitable for the exploration of virtual objects of cultural heritage collection. Five sets of gestures have been prepared, namely lifting, picking, holding, sweeping from both directions, left and right. Dynamic arm gestures are recorded using the forearm sensor. The recorded data contains coordinates of gestures in form of x, y, z, raw, pitch, and yaw. Gaussian mixture models are used in selecting features to produce good accuracy in the classification process.Two functions are used, namely probability density function and cumulative distribution function for the feature selection process. In this study, two experiments were used to train the gesture model. The accuracy of the two experiments is shown in the form of a confusion matrix. Each of the confusion matrices show excellent results of 99.8% for SVM and k-NN. Furthermore, modeling results are tested using new data. The testing shows 89.25% result for SVM classifier and 90.09% for k-NN. Four other dynamic arm gestures have a very satisfactory rate of 100% for the two classifiers. The five gestures can be used in the development of virtual reality applications.
References
S. Sumpeno, A. Zaini, M. Muhtadin, S.M.S. Nugroho, E.M. Yuniarno, dan I.K.E. Purnama, “Ragam Teknologi Informasi untuk Revitalisasi Museum,” Seminar Nasional Otomasi Industri dan Teknologi Informasi, 2015, Vol. 1, hal. 1–6.
S. Sumpeno dan A.G. Sooai, “3D Stereoskopik untuk Peninggalan Sejarah,” AEK FDI, Vol. 1, No. 1, hal. 33–37, Dec. 2016.
A.G. Sooai, S. Sumpeno, dan M.H. Purnomo, “User Perception on 3D Stereoscopic Cultural Heritage Ancient Collection,” Proceedings of the The 2nd International Conference in HCI and UX on Indonesia 2016, 2016, hal. 112–119.
A.G. Sooai, A. Nugroho, M.N.A. Azam, S. Sumpeno, dan M.H. Purnomo, “Virtual artifact: Enhancing Museum Exhibit Using 3D Virtual Reality,” 2017 TRON Symposium (TRONSHOW), 2017, hal. 1–5.
S.L. Kim, H.J. Suk, J.H. Kang, J.M. Jung, T.H. Laine, dan J. Westlin, “Using Unity 3D to Facilitate Mobile Augmented Reality Game Development,” 2014 IEEE World Forum on Internet of Things (WFIoT), 2014, hal. 21–26.
Z. Tůma, J. Tůma, R. Knoflíček, P. Blecha, dan F. Bradáč, “The Process Simulation Using by Virtual Reality,” Procedia Eng., 2014, Vol. 69, hal. 1015–1020.
G. Kontogianni dan A. Georgopoulos, “A Realistic Gamification Attempt for the Ancient Agora of Athens,” 2015 Digital Heritage, 2015, Vol. 1, hal. 377–380.
L. Chittaro dan F. Buttussi, “Assessing Knowledge Retention of an Immersive Serious Game vs. a Traditional Education Method in Aviation Safety,” IEEE Trans. Vis. Comput. Graph., Vol. 21, No. 4, hal. 529–538, Apr. 2015.
E. Redondo, D. Fonseca, A. Sánchez, dan I. Navarro, “New Strategies Using Handheld Augmented Reality and Mobile Learning-teaching Methodologies, in Architecture and Building Engineering Degrees,” Procedia Comput. Sci., Vol. 25, hal. 52–61, Jan. 2013.
I.G.A. Dharmayasa, S. Sumpeno, I.K.E. Purnama, dan A.G. Sooai, “Exploration of Prayer Tools in 3D Virtual Museum Using Leap Motion for Hand Motion Sensor,” 2017 TRON Symposium (TRONSHOW), 2017, hal. 1–8.
A.G. Sooai, K. Yoshimoto, H. Takahashi, S. Sumpeno, dan M.H. Purnomo, “Dynamic Hand Gesture Recognition on 3D Virtual Cultural Heritage Ancient Collection Objects Using k-Nearest Neighbor,” Eng. Lett., Vol. 26, No. 3, hal. 356–363, 2018.
F. Bruno, S. Bruno, G. De Sensi, M.L. Luchi, S. Mancuso, dan M. Muzzupappa, “From 3D Reconstruction to Virtual Reality: A Complete Methodology for Digital Archaeological Exhibition,” J. Cult. Herit., Vol. 11, No. 1, hal. 42–49, 2010.
M. Nabil dan F. Saleh, “3D Reconstruction from Images for Museum Artefacts: A Comparative Study,” 2014 International Conference on Virtual Systems Multimedia (VSMM), 2014, hal. 257–260.
D. Rojas, B. Kapralos, A. Hogue, K. Collins, L. Nacke, S. Cristancho, C. Conati, dan A. Dubrowski, “The Effect of Sound on Visual Fidelity Perception in Stereoscopic 3-D,” IEEE Trans. Cybern., Vol. 43, No. 6, hal. 1572–1583, Dec. 2013.
M. Hayashi, S. Bachelder, M. Nakajima, dan A. Iguchi, “A New Virtual Museum Equipped with Automatic Video Content Generator,” 2014 International Conference on Cyberworlds (CW), 2014, hal. 377–383.
F. Gabellone, I. Ferrari, M.T. Giannotta, dan A. Dell’Aglio, “From Museum to Original Site: A 3D Environment for Virtual Visits to Finds Re-Contextualized in Their Original Setting,” Digital Heritage International Congress (DigitalHeritage) 2013, 2013, Vol. 2, hal. 215–222.
A.-H.G. Abulrub, K. Budabuss, P. Mayer, dan M.A. Williams, “The 3D Immersive Virtual Reality Technology Use for Spatial Planning and Public Acceptance,” Procedia - Soc. Behav. Sci., Vol. 75, hal. 328–337, Apr. 2013.
F. Caputo, A. Greco, E. D’Amato, I. Notaro, dan S. Spada, “On the Use of Virtual Reality for a Human-Centered Workplace Design,” Procedia Struct. Integr., Vol. 8, hal. 297–308, Jan. 2018.
H. Zhuang, M. Yang, Z. Cui, dan Q. Zheng, “A Method for Static Hand Gesture Recognition Based on Non-Negative Matrix Factorization and Compressive Sensing,” IAENG Int. J. Comput. Sci., Vol. 44, No. 1, hal. 52–59, Feb. 2017.
K. Barczewska dan A. Drozd, “Comparison of Methods for Hand Gesture Recognition Based on Dynamic Time Warping Algorithm,” 2013 Federated Conference on Computer Science and Information Systems, 2013, hal. 207–210.
W. Lu, Z. Tong, dan J. Chu, “Dynamic Hand Gesture Recognition with Leap Motion Controller,” IEEE Signal Process. Lett., Vol. 23, No. 9, hal. 1188–1192, Sep. 2016.
R. Srivastava dan P. Sinha, “Hand Movements and Gestures Characterization Using Quaternion Dynamic Time Warping Technique,” IEEE Sens. J., Vol. 16, No. 5, hal. 1333–1341, Mar. 2016.
Khamid, A. D. Wibawa, dan S. Sumpeno, “Gesture Recognition for Indonesian Sign Language Systems (ISLS) Using Multimodal Sensor Leap Motion and Myo Armband Controllers Based-on Naive Bayes Classifier,” 2017 International Conference on Soft Computing, Intelligent System and Information Technology (ICSIIT), 2017, hal. 1–6.
D. Wu, L. Pigou, P.-J. Kindermans, N.D.-H. Le, L. Shao, J. Dambre, dan J.-M. Odobez, “Deep Dynamic Neural Networks for Multimodal Gesture Segmentation and Recognition,” IEEE Trans. Pattern Anal. Mach. Intell., Vol. 38, No. 8, hal. 1583–1597, Agt. 2016.
P. Lorrentz, “Classification of Incomplete Data by Observation,” EngineeringLetters, Vol. 18, No. 4, hal. 1–10, 2010.
A.N. Rumaksari, S. Sumpeno, dan A.D. Wibawa, “Background Subtraction Using Spatial Mixture of Gaussian Model with Dynamic Shadow Filtering,” 2017 International Seminar on Intelligent Technology and Its Applications (ISITIA), 2017, hal. 296–301.
M.S. Annas, A. Rizal, dan R.D. Atmaja, “Pengenalan Individu Berdasarkan Gait Menggunakan Sensor Giroskop,” J. Nas. Tek. Elektro Dan Teknol. Inf. JNTETI, Vol. 6, No. 2, hal. 210-214, 2017.
D. Huynh, D. Tran, and W. Ma, “Contextual Analysis for the Representation of Words,” IAENG Int. J. Comput. Sci., Vol. 41, No. 2, hal. 1–5, 2014.
P. Zhang, X. Zhang, dan W. Wang, “The General Lp-Dual Mixed Brightness Integrals,” IAENG Int. J. Appl. Math., Vol. 47, No. 2, hal. 1–5, 2016.
A. Handayani, A. Jamal, dan A.A. Septiandri, “Evaluasi Tiga Jenis Algoritme Berbasis Pembelajaran Mesin untuk Klasifikasi Jenis Tumor Payudara,” J. Nas. Tek. Elektro Dan Teknol. Inf. JNTETI, Vol. 6, No. 4, hal. 394-403, 2017.
© Jurnal Nasional Teknik Elektro dan Teknologi Informasi, under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
1.png)
