Page 75 - 2024S

68 UEC Int’l Mini-Conference No.52

TouchlessGesture: Hand Gesture Recognition
Model Based on Landmarks

Ulises ARROYO 1,2,* , Gibran BENITEZ , Hiroki TAKAHASHI , and Jesus OLIVARES 2
1
1
l
1 Department of Informatics, The University of Electro-Communications, Tokyo, Japan
2 Instituto Politécnico Nacional, Mexico City, Mexico
* a2495001@gl.cc.uec.ac.jp

Fig. 1. Overview of the proposed approach for hand gesture recognition.

INTRODUCTION TouchlessGesture Methodology
The current study presents the development of a contactless Hand This methodology was inspired by Efficient Hand Gesture
Gesture Recognition model based on landmarks. The main goal is Recognition for Human-Robot Interaction [3].
to create a model that can be used for user interface manipulation,
with a particular focus on reducing the risk of contagion in high- 1. Video Sequence Input (Fig. 1a): Processes a sequence of
traffic areas such as information kiosks and medical environments. video images.
2. Landmark Detection (Fig. 1b): Utilizing the MediaPipe pose
Using the IPN Hand dataset [2], which includes 13 specific detector, hand landmarks are located in each video frame.
gestures designed to emulate computer mouse functions, the 3. Feature Representation (Fig. 1c): The detected landmarks are
model incorporates landmarks obtained using state-of-the-art hand processed into a feature representation.
pose estimation models like MediaPipe and NSRM [1]. 4. Gesture Recognition (Fig. 1d): The feature representation is
used to perform hand gesture recognition via a densely
connected network.
5. Predicted Gesture (Fig. 1e): The system generates the
prediction of the recognized gesture, allowing user interface
manipulation based on the detected gesture.

Fig. 2. Examples of the 4 gesture classes selected from the IPN
dataset [2]. Fig. 3. Depiction of proposed feature representations. a) Raw
Landmarks Representation: raw landmarks directly taken from the
hand joints detector MediaPipe. b) DistTime: computed by
calculating distances from each of the points in the present frame to
Contributions of theTouchlessGesture each of the points in the past frame, in the figure simplified for only 2
of the landmarks.
Landmark Integration: Use of MediaPipe and NSRM landmarks
to improve real-time hand gesture recognition accuracy. REFERENCES
1. Inference Time Optimization: Techniques to reduce inference [1] Yifei Chen, undefined., et al. "Nonparametric Structure Regularization Machine for 2D Hand
time for quick responses in high-traffic areas. Pose Estimation," in CoRR, vol. abs/2001.08869, 2020.
[2] Gibran Benitez-Garcia, undefined., et al. "IPN Hand: A Video Dataset and Benchmark for Real-
2. Accuracy Improvement: Enhancing gesture recognition Time Continuous Hand Gesture Recognition," in CoRR, vol. abs/2005.02134, 2020.
precision for more effective contactless interactions. [3] M. Peral, A. Sanfeliu, A. Garrell. "Efficient Hand Gesture Recognition for Human-Robot
Interaction," in IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 10272-10279, 2022.

CONCLUSIONS TouchlessGesture enhances safety and efficiency in contactless interaction. It aims to develop a hand gesture
recognition model with high accuracy and rapid response time. This represents a significant step towards adopting contactless interaction
technologies, promoting safer and more efficient user interfaces in high-traffic environments.