國立高雄大學圖資館 |

Language: English

Back

Towards Multimodal and Context-Aware Emotion Perception.

Record Type:	Electronic resources : Monograph/item
Title/Author:	Towards Multimodal and Context-Aware Emotion Perception.
Author:	Mittal, Trisha.
Published:	Ann Arbor : ProQuest Dissertations & Theses, 2023
Description:	193 p.
Notes:	Source: Dissertations Abstracts International, Volume: 85-04, Section: A.
Notes:	Advisor: Manocha, Dinesh.
Contained By:	Dissertations Abstracts International85-04A.
Subject:	Computer science.
Online resource:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30488143
ISBN:	9798380578936

Towards Multimodal and Context-Aware Emotion Perception.
Mittal, Trisha.

Towards Multimodal and Context-Aware Emotion Perception. - Ann Arbor : ProQuest Dissertations & Theses, 2023 - 193 p.

Source: Dissertations Abstracts International, Volume: 85-04, Section: A.

Thesis (Ph.D.)--University of Maryland, College Park, 2023.

This item must not be sold to any third party vendors.

Human emotion perception is a part of affective computing, a branch of computing that studies and develops systems and devices that can recognize, interpret, process, and simulate human affects. Research in human emotion perception, however, has been mostly restricted to psychology-based literature which explores the theoretical aspects of emotion perception, but does not touch upon its practical applications. For instance, human emotion perception plays a pivotal role in an extensive array of sophisticated intelligent systems, encompassing domains such as behavior prediction, social robotics, medicine, surveillance, and entertainment. In order to deploy emotion perception in these applications, extensive research in psychology has demonstrated that humans not only perceive emotions and behavior through diverse human modalities but also glean insights from situational and contextual cues.This dissertation not only enhances the capabilities of existing human emotion perception systems but also forges novel connections between emotion perception and multimedia analysis, social media analysis, and multimedia forensics. Specifically, this work introduces two innovative algorithms that revolutionize the construction of human emotion perception models. These algorithms are then applied to detect falsified multimedia, understand human behavior and psychology on social media networks, and extract the intricate array of emotions evoked by movies.In the first part of this dissertation, we delve into two unique approaches to advance emotion perception models. The first approach capitalizes on the power of multiple modalities to perceive human emotion. The second approach leverages the contextual information, such as the background scene, diverse modalities of the human subject, and intricate socio-dynamic inter-agent interactions. These elements converge to predict perceived emotions with better accuracy, culminating in the development of context-aware human emotion perception models.In the second part of this thesis, we forge connections between emotion perception and three prominent domains of artificial intelligence applications. These domains include video manipulations and deepfake detection, multimedia content analysis, and user behavior analysis on social media platforms. Drawing inspiration from emotion perception, we conceptualize enriched solutions that push the conventional boundaries and redefine the possibilities within these domains.All experiments in this dissertation have been conducted on all state-of-the-art emotion perception datasets, including IEMOCAP, CMU-MOSEI, EMOTIC, SENDv1, MovieGraphs, LIRIS-ACCEDE, DF-TIMIT, DFDC, Intentonomy, MDID, and MET-Meme. In fact, we propose three additional datasets to this list, namely GroupWalk, VideoSham and IntentGram. In addition to providing quantitative results to validate our claims, we conduct user evaluations where applicable, serving as a compelling testament to the remarkable outcomes of our experiments.

ISBN: 9798380578936Subjects--Topical Terms:

199325
Computer science.
Subjects--Index Terms:

Affective analysis

Towards Multimodal and Context-Aware Emotion Perception.
LDR:04290nmm a2200421 4500 001 655766
005 20240414211925.5
006 m o d
007 cr#unu||||||||
008 240620s2023 ||||||||||||||||| ||eng d
020 $a 9798380578936
035 $a (MiAaPQ)AAI30488143
035 $a AAI30488143
040 $a MiAaPQ $c MiAaPQ
100 1 $a Mittal, Trisha. $3 966840
245 1 0 $a Towards Multimodal and Context-Aware Emotion Perception.
260 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2023
300 $a 193 p.
500 $a Source: Dissertations Abstracts International, Volume: 85-04, Section: A.
500 $a Advisor: Manocha, Dinesh.
502 $a Thesis (Ph.D.)--University of Maryland, College Park, 2023.
506 $a This item must not be sold to any third party vendors.
520 $a Human emotion perception is a part of affective computing, a branch of computing that studies and develops systems and devices that can recognize, interpret, process, and simulate human affects. Research in human emotion perception, however, has been mostly restricted to psychology-based literature which explores the theoretical aspects of emotion perception, but does not touch upon its practical applications. For instance, human emotion perception plays a pivotal role in an extensive array of sophisticated intelligent systems, encompassing domains such as behavior prediction, social robotics, medicine, surveillance, and entertainment. In order to deploy emotion perception in these applications, extensive research in psychology has demonstrated that humans not only perceive emotions and behavior through diverse human modalities but also glean insights from situational and contextual cues.This dissertation not only enhances the capabilities of existing human emotion perception systems but also forges novel connections between emotion perception and multimedia analysis, social media analysis, and multimedia forensics. Specifically, this work introduces two innovative algorithms that revolutionize the construction of human emotion perception models. These algorithms are then applied to detect falsified multimedia, understand human behavior and psychology on social media networks, and extract the intricate array of emotions evoked by movies.In the first part of this dissertation, we delve into two unique approaches to advance emotion perception models. The first approach capitalizes on the power of multiple modalities to perceive human emotion. The second approach leverages the contextual information, such as the background scene, diverse modalities of the human subject, and intricate socio-dynamic inter-agent interactions. These elements converge to predict perceived emotions with better accuracy, culminating in the development of context-aware human emotion perception models.In the second part of this thesis, we forge connections between emotion perception and three prominent domains of artificial intelligence applications. These domains include video manipulations and deepfake detection, multimedia content analysis, and user behavior analysis on social media platforms. Drawing inspiration from emotion perception, we conceptualize enriched solutions that push the conventional boundaries and redefine the possibilities within these domains.All experiments in this dissertation have been conducted on all state-of-the-art emotion perception datasets, including IEMOCAP, CMU-MOSEI, EMOTIC, SENDv1, MovieGraphs, LIRIS-ACCEDE, DF-TIMIT, DFDC, Intentonomy, MDID, and MET-Meme. In fact, we propose three additional datasets to this list, namely GroupWalk, VideoSham and IntentGram. In addition to providing quantitative results to validate our claims, we conduct user evaluations where applicable, serving as a compelling testament to the remarkable outcomes of our experiments.
590 $a School code: 0117.
650 4 $a Computer science. $3 199325
650 4 $a Information technology. $3 184390
650 4 $a Web studies. $3 708690
653 $a Affective analysis
653 $a Context-aware emotion
653 $a Emotion perception
653 $a Multimodal emotion
653 $a Social media
653 $a Video manipulation
690 $a 0984
690 $a 0800
690 $a 0489
690 $a 0646
710 2 $a University of Maryland, College Park. $b Computer Science. $3 660362
773 0 $t Dissertations Abstracts International $g 85-04A.
790 $a 0117
791 $a Ph.D.
792 $a 2023
793 $a English
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30488143