國立高雄大學圖資館 |

Language: English

Back

Identifying, Evaluating and Applying Importance Maps for Speech.

Record Type:	Electronic resources : Monograph/item
Title/Author:	Identifying, Evaluating and Applying Importance Maps for Speech.
Author:	Trinh, Viet Anh.
Published:	Ann Arbor : ProQuest Dissertations & Theses, 2022
Description:	119 p.
Notes:	Source: Dissertations Abstracts International, Volume: 83-06, Section: B.
Notes:	Advisor: Mandel, Michael I.
Contained By:	Dissertations Abstracts International83-06B.
Subject:	Computer science.
Online resource:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28861594
ISBN:	9798759951452

Identifying, Evaluating and Applying Importance Maps for Speech.
Trinh, Viet Anh.

Identifying, Evaluating and Applying Importance Maps for Speech. - Ann Arbor : ProQuest Dissertations & Theses, 2022 - 119 p.

Source: Dissertations Abstracts International, Volume: 83-06, Section: B.

Thesis (Ph.D.)--City University of New York, 2022.

This item must not be sold to any third party vendors.

Like many machine learning systems, speech models often perform well when employed on data in the same domain as their training data. However, when the inference is on out-of-domain data, performance suffers. With a fast-growing number of applications of speech models in healthcare, education, automotive, automation, etc., it is essential to ensure that speech models can generalize to out-of-domain data, especially to noisy environments in real-world scenarios. In contrast, human listeners are quite robust to noisy environments. Thus, a thorough understanding of the differences between human listeners and speech models is urgently required to enhance speech model performance in noise. These differences exist presumably because the speech model does not use the same information as humans for recognizing the speech. A possible solution is encouraging the speech model to attend to the same time-frequency regions as human listeners. In this way, speech model generalization in noise may be improved. We define those time-frequency regions that humans or machines focus on to recognize the speech as importance maps (IMs). In this research, first, we investigate how to identify speech importance maps. Second, we compare human and machine importance maps to understand how they differ and how the speech model can learn from humans to improve its performance in noise. Third, we develop a structured saliency benchmark (SSBM), a metric for evaluating IMs. Finally, we propose a new application of IMs as data augmentation for speech models, enhancing their performance and enabling them to better generalize to out-of-domain noise.Overall, our work demonstrates that we can improve speech models and achieve out-of-domain generalization to different noise environments with importance maps. In the future, we will expand our work with large-scale speech models and deploy different methods to identify IMs and use them to augment the speech data, such as those based on human responses. We can also extend the technique to computer vision tasks, such as image recognition by predicting importance maps for images and use IMs to enhance model performance to out-of-domain data.

ISBN: 9798759951452Subjects--Topical Terms:

199325
Computer science.
Subjects--Index Terms:

Data augmentation

Identifying, Evaluating and Applying Importance Maps for Speech.
LDR:03357nmm a2200361 4500 001 616501
005 20220513114354.5
008 220920s2022 ||||||||||||||||| ||eng d
020 $a 9798759951452
035 $a (MiAaPQ)AAI28861594
035 $a AAI28861594
040 $a MiAaPQ $c MiAaPQ
100 1 $a Trinh, Viet Anh. $3 915880
245 1 0 $a Identifying, Evaluating and Applying Importance Maps for Speech.
260 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2022
300 $a 119 p.
500 $a Source: Dissertations Abstracts International, Volume: 83-06, Section: B.
500 $a Advisor: Mandel, Michael I.
502 $a Thesis (Ph.D.)--City University of New York, 2022.
506 $a This item must not be sold to any third party vendors.
520 $a Like many machine learning systems, speech models often perform well when employed on data in the same domain as their training data. However, when the inference is on out-of-domain data, performance suffers. With a fast-growing number of applications of speech models in healthcare, education, automotive, automation, etc., it is essential to ensure that speech models can generalize to out-of-domain data, especially to noisy environments in real-world scenarios. In contrast, human listeners are quite robust to noisy environments. Thus, a thorough understanding of the differences between human listeners and speech models is urgently required to enhance speech model performance in noise. These differences exist presumably because the speech model does not use the same information as humans for recognizing the speech. A possible solution is encouraging the speech model to attend to the same time-frequency regions as human listeners. In this way, speech model generalization in noise may be improved. We define those time-frequency regions that humans or machines focus on to recognize the speech as importance maps (IMs). In this research, first, we investigate how to identify speech importance maps. Second, we compare human and machine importance maps to understand how they differ and how the speech model can learn from humans to improve its performance in noise. Third, we develop a structured saliency benchmark (SSBM), a metric for evaluating IMs. Finally, we propose a new application of IMs as data augmentation for speech models, enhancing their performance and enabling them to better generalize to out-of-domain noise.Overall, our work demonstrates that we can improve speech models and achieve out-of-domain generalization to different noise environments with importance maps. In the future, we will expand our work with large-scale speech models and deploy different methods to identify IMs and use them to augment the speech data, such as those based on human responses. We can also extend the technique to computer vision tasks, such as image recognition by predicting importance maps for images and use IMs to enhance model performance to out-of-domain data.
590 $a School code: 0046.
650 4 $a Computer science. $3 199325
653 $a Data augmentation
653 $a Explainable artificial intelligence
653 $a Importance maps
653 $a Noise robustness
653 $a Speech perception
653 $a Speech recognition
690 $a 0984
710 2 $a City University of New York. $b Computer Science. $3 492891
773 0 $t Dissertations Abstracts International $g 83-06B.
790 $a 0046
791 $a Ph.D.
792 $a 2022
793 $a English
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28861594