Abstract
Hand Gesture Recognition (HGR) is a form of perceptual computing that allows artificial systems to capture and interpret human gestures. HGR has applications in human-machine interaction, virtual reality, augmented reality, and human behavior analysis. The human hand can assume a near-infinite number of poses and orientations to form myriad gestures, thus increasing the difficulty of the HGR task.
The hand skeleton of connected joints effectively describes the hand’s geometric shape and thus contains richer semantic gesture information while eliminating noise from individual differences in physical hand characteristics. The efficacy and computational efficiency of skeleton-based HGR frameworks can be significantly enhanced by transforming the hand gesture recognition task into an image classification task, taking advantage of the success of ML algorithms/architectures in the image classification domain.
This research thesis document presents a skeleton-based HGR framework that implements temporal information condensation via data-level fusion to encode spatiotemporal gesture information into RGB images. The spatiotemporal images are processed through an end-to-end ensemble multi-stream CNN architecture that generates the class probabilities.
The framework was extensively evaluated on the CNR, FPHA, LMDHG, SHREC2017, and DHG1428 HGR benchmark datasets. The framework’s performance was competitive with the SOTA for the evaluation datasets, within -4.10% and +6.86% of reported classification accuracies. The experimental results confidently demonstrated the viability of the temporal information condensation and data-level fusion techniques within the HGR domain.
School
School of Sciences and Engineering
Department
Robotics, Control & Smart Systems Program
Degree Name
MS in Robotics, Control and Smart Systems
Graduation Date
Winter 1-31-2023
Submission Date
12-20-2022
First Advisor
Dr. Maki Habib
Second Advisor
Dr. Mohamed Moustafa
Committee Member 1
Dr. Amr Goneid
Committee Member 2
Dr. Hazem Abbas
Committee Member 3
Dr. Mohamed El-Morsi
Extent
139 p.
Document Type
Master's Thesis
Institutional Review Board (IRB) Approval
Not necessary for this item
Recommended Citation
APA Citation
Yusuf, O.
(2023).Skeleton-Based Hand Gesture Recognition Using Data-Level Fusion [Master's Thesis, the American University in Cairo]. AUC Knowledge Fountain.
https://fount.aucegypt.edu/etds/1986
MLA Citation
Yusuf, Oluwaleke. Skeleton-Based Hand Gesture Recognition Using Data-Level Fusion. 2023. American University in Cairo, Master's Thesis. AUC Knowledge Fountain.
https://fount.aucegypt.edu/etds/1986
Included in
Artificial Intelligence and Robotics Commons, Data Science Commons, Graphics and Human Computer Interfaces Commons, Robotics Commons