CNN-LSTM Proposal for Colombian Sign Language Greetings Classification: Preliminary Results

Ivette Rojas, Katherin Sierra, Angelica Maria Rojas Tocora, Luis Calderon, Natalia Buitrago, Alvaro D. Orjuela-Canon

Producción científica: Capítulo en Libro/InformeContribución a la conferencia

Resumen

Colombian sign language (CSL) have shown different advances to automatize the translation and communication between deaf or disabling hearing people and speakers. However, these current efforts are oriented on classification of static signs, mainly. The present proposal implements the use a convolutional neural network (CNN) that also involves the use an architecture of the long short-term memory (LSTM) one. This allowed to include the time dependent information for dynamic signs like a sequence data. For this, a dataset was created with greetings for the CSL through the recording videos of signs. The methodology used the MediaPipe tool for feature extraction, which was used to represent relevant points from fingers articulation for frames of the videos. CNN-LSTM was applied with the joint points information for the classification. Results show that 71.50 % (+/-4.33%) for five greetings classification, employing a crossvalidation and early stopping criteria for the training of models.

Idioma originalInglés estadounidense
Título de la publicación alojada2024 3rd International Congress of Biomedical Engineering and Bioengineering, CIIBBI 2024
EditorialInstitute of Electrical and Electronics Engineers Inc.
ISBN (versión digital)9798331532352
DOI
EstadoPublicada - 2024
Evento3rd International Congress of Biomedical Engineering and Bioengineering, CIIBBI 2024 - Cali, Colombia
Duración: nov. 6 2024nov. 8 2024

Serie de la publicación

Nombre2024 3rd International Congress of Biomedical Engineering and Bioengineering, CIIBBI 2024

Conferencia

Conferencia3rd International Congress of Biomedical Engineering and Bioengineering, CIIBBI 2024
País/TerritorioColombia
CiudadCali
Período11/6/2411/8/24

Áreas temáticas de ASJC Scopus

  • Ortopedia y medicina del deporte
  • Bioingeniería
  • Ingeniería biomédica
  • Tecnología de medios

Citar esto