Bobby's Blog

Current Research

My current research focuses on Sign Language Translation (SLT). This project aims to perform cross-modal translation tasks using neural networks, specifically translating sign language video frames into spoken language. The goal of my research is to fuse raw information from sign language video frames with keypoint data, and to jointly train the combined features with spoken language. Additionally, we use pretrained models to facilitate the sign language translation generation tasks.

Key aspects of my research include:

• Computer Vision: Utilizing convolutional neural networks (CNNs) to process and extract features from video frames.
• Sequence Processing: Employing recurrent neural networks (RNNs), such as LSTM or GRU, to manage time-series data in sign language.
• Cross-Modal Learning: Mapping visual data and textual information to a common feature space for effective translation.
• Translation Techniques: Applying sequence-to-sequence (seq2seq) models, enhanced with attention mechanisms, to translate visual signs into coherent spoken language.
• Pretrained Models: Leveraging existing models to improve the efficiency and quality of translation tasks.

My research primarily involves the application of machine learning and deep learning algorithms, particularly in the fields of image recognition and natural language processing (NLP).