Page 74 - 2024S

UEC Int’l Mini-Conference No.52 67

TouchlessGesture: Hand Gesture Recognition Model Based on

Landmarks

2
2
Ulises ARROYO ∗1,2 , Gibran BENITEZ , Hiroki TAKAHASHI , and Jesus OLIVARES 1
1 Department of Informatics,
The University of Electro-Communications, Tokyo, Japan
2 Instituto Polit´ecnico Nacional, Mexico City, Mexico

Keywords: Hand Gesture Recognition (HGR), Landmarks, User Interface Manipulation, Contagion
Risk Reduction, MediaPipe Landmarks.

Abstract
The current study presents the development of a contactless Hand Gesture Recognition model based
on landmarks. The main goal is to create a model that can be used for user interface manipulation,
with a particular focus on reducing the risk of contagion in high-traffic areas such as information
kiosks and medical environments. Using the IPN Hand dataset, which includes 13 specific gestures
designed to emulate computer mouse functions, the project implements a robust methodology inspired
by Efficient Hand Gesture Recognition for Human-Robot Interaction. The model processes a sequence
of video images, utilizes the MediaPipe pose detector to locate hand landmarks in each video frame,
and processes these landmarks into a feature representation. This feature representation is then used
to perform hand gesture recognition via a densely connected network. Finally, the system generates
the prediction of the recognized gesture, allowing user interface manipulation based on the detected
gesture.

∗
The author is supported by JASSO Scholarship.