CNN-LSTM Proposal for Colombian Sign Language Greetings Classification: Preliminary Results

Ivette Rojas, Katherin Sierra, Angelica Maria Rojas Tocora, Luis Calderon, Natalia Buitrago, Alvaro D. Orjuela-Canon

Research output: Chapter in Book/InformConference contribution

Abstract

Colombian sign language (CSL) have shown different advances to automatize the translation and communication between deaf or disabling hearing people and speakers. However, these current efforts are oriented on classification of static signs, mainly. The present proposal implements the use a convolutional neural network (CNN) that also involves the use an architecture of the long short-term memory (LSTM) one. This allowed to include the time dependent information for dynamic signs like a sequence data. For this, a dataset was created with greetings for the CSL through the recording videos of signs. The methodology used the MediaPipe tool for feature extraction, which was used to represent relevant points from fingers articulation for frames of the videos. CNN-LSTM was applied with the joint points information for the classification. Results show that 71.50 % (+/-4.33%) for five greetings classification, employing a crossvalidation and early stopping criteria for the training of models.

Original languageEnglish (US)
Title of host publication2024 3rd International Congress of Biomedical Engineering and Bioengineering, CIIBBI 2024
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9798331532352
DOIs
StatePublished - 2024
Event3rd International Congress of Biomedical Engineering and Bioengineering, CIIBBI 2024 - Cali, Colombia
Duration: Nov 6 2024Nov 8 2024

Publication series

Name2024 3rd International Congress of Biomedical Engineering and Bioengineering, CIIBBI 2024

Conference

Conference3rd International Congress of Biomedical Engineering and Bioengineering, CIIBBI 2024
Country/TerritoryColombia
CityCali
Period11/6/2411/8/24

All Science Journal Classification (ASJC) codes

  • Orthopedics and Sports Medicine
  • Bioengineering
  • Biomedical Engineering
  • Media Technology

Cite this