Department of Computer Science

Permanent URI for this communityhttps://repository.cuilahore.edu.pk/handle/123456789/16

Browse

Author: search.filters.author.Dr. Allah Bux SarganoSubject: search.filters.subject.fa19

Search Results

Now showing 1 - 2 of 2
  • No Thumbnail Available
    Item
    Activity Recognition for Assisted Living based on Multimodal Features using Deep Learning
    (Library Information Services, COMSATS University Islamabad, Lahore Campus, 2021) Hafsa Yaseen; FA19-RCS-022; LHR TP 7599; Dr. Allah Bux Sargano
    Human activity recognition (HAR) is a prominent field in computer vision and signal processing that analyzes the information obtained from numerous sensors, including vision sensors and wearable sensors. The purpose of HAR is to recognize actions from a sequence of observations on the activities of individuals and environment events. It provides a broad variety of applications, including ambient assisted living, robotic technology, intelligent surveillance, human-computer interaction, smart home, transportation, and smart healthcare. As assisted living indicates the technological services that help impaired people and senior citizens to spend independent life. Therefore, HAR, which facilitates proactive gestures and interactions with their surroundings, has become a significant precondition for assisted living applications. For all that, tremendous efforts have been made to reliably capture human action and behavior by manipulating single modality data, but the combined analysis of multimodal data has received less attention. Different modalities usually contain complementary information that must be combined for better learning of action recognition for Ambient Assisted Living. In this research, a novel framework called “Activity Recognition for Assisted Living based on Multimodal Features using Deep learning” is proposed to leverage intra-modality discriminative features as well as inter-modality connection in visual and inertial data using deep neural networks. Two separate unimodal, i.e., visual and inertial models, are proposed to learn action recognition classifiers for these modalities effectively. These models automatically acquire high-quality discriminative action-related images and inertial features. Finally, these heterogeneous models are combined into an end-to-end approach via decision-level fusion. The comprehensive experiments are conducted using the publicly accessible benchmark C-MHAD dataset. The outcomes showed that the proposed methodology surpassed existing methods in action recognition by a significant margin, with an F1-score of 89%
  • No Thumbnail Available
    Item
    Attention-based Multimodal Sentiment Analysis and Emotion Recognition using Deep Neural Networks
    (Library Information Services, COMSATS University Islamabad, Lahore Campus, 2021) Ajwa Aslam; FA19-RCS-009; LHR TP 7284; Dr. Allah Bux Sargano
    In recent years, multimodal sentiment analysis and emotion recognition has become a hot research topic among researchers. Instead of manually scrutinizing the sentiments and emotions, there is a need for an automatic process that would easily be applied to numerous tasks, such as product analysis, social media monitoring, election prediction, etc. Though great efforts have been devoted to gauging the people’s sentiments and emotions by manipulating the single modality data like text, or images or audio, but less attention is paid to the joint analysis of multimodal data in social media. Different modalities usually contain complementary information that must be combined for better learning of sentiments and emotions. In this research work, a novel framework: Attention-based Multimodal Sentiment Analysis and Emotion Recognition (AMSAER) using deep neural networks is proposed to exploit intra-modality discriminative features, as well as inter-modality correlation in visual, audio, and text modalities. Three separate unimodal i.e., visual, acoustic and semantic attention models are proposed to effectively learn sentiment and emotion classifiers for these modalities. These models automatically acquire the high-quality discriminative sentiment and emotion-related semantic words, image-regions, and raw audio features. Subsequently, deep hierarchical multimodal based on intermediate fusion is proposed to learn an internal- correlation between these modality features hierarchically. This model first learns the correlation among bimodal features and then among the trimodal features for the joint classification of sentiments and emotions. Finally, all four distinct models are fused into a holistic framework through decision level fusion for multimodal sentiment analysis and emotion recognition. Extensive experiments have been conducted on publically available benchmark IEMOCAP corpus for both sentiment and emotion prediction. The results demonstrated that our proposed architecture outperformed existing techniques with a solid margin in both sentiment and emotion classification, achieving 84% and 92% accuracy, respectively