國立高雄大學圖資館 |

Language: English

Back

Recurrent Neural Network Architectur...

Ger, Stephanie.

Recurrent Neural Network Architectures for Sequence Classification and Explanation.

Record Type:	Electronic resources : Monograph/item
Title/Author:	Recurrent Neural Network Architectures for Sequence Classification and Explanation.
Author:	Ger, Stephanie.
Published:	Ann Arbor : ProQuest Dissertations & Theses, 2020
Description:	95 p.
Notes:	Source: Dissertations Abstracts International, Volume: 82-06, Section: B.
Notes:	Advisor: Klabjan, Diego.
Contained By:	Dissertations Abstracts International82-06B.
Subject:	Applied mathematics.
Online resource:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28156279
ISBN:	9798698575030

Recurrent Neural Network Architectures for Sequence Classification and Explanation.
Ger, Stephanie.

Recurrent Neural Network Architectures for Sequence Classification and Explanation. - Ann Arbor : ProQuest Dissertations & Theses, 2020 - 95 p.

Source: Dissertations Abstracts International, Volume: 82-06, Section: B.

Thesis (Ph.D.)--Northwestern University, 2020.

This item must not be sold to any third party vendors.

This thesis focuses on applications of recurrent neural networks (RNNs) for three aspects of sequential classification. In the first chapter, a novel method to generate synthetic minority data generation to improve imbalanced classification is discussed. Generative Adversarial Networks (GANs) have been used in many different applications to generate realistic synthetic data. We introduce a novel GAN with Autoencoder (GAN-AE) architecture to generate synthetic samples for variable length, multi-feature sequence datasets. This model consists of a GAN architecture with an additional autoencoder component, where RNNs are used for each component of the model in order to generate synthetic data to improve classification accuracy for a highly imbalanced medical device dataset. In addition to the medical device dataset, we also evaluate the GAN-AE performance on two additional datasets and demonstrate the application of GAN-AE to a sequence-to-sequence task where both synthetic sequence inputs and sequence outputs must be generated. To evaluate the quality of the synthetic data, we train encoder-decoder models both with and without the synthetic data and compare the classification model performance. We show that a model trained with GAN-AE generated synthetic data outperforms models trained with synthetic data generated both with standard oversampling techniques such as SMOTE and Autoencoders as well as with state of the art GAN-based models.Next, we discuss the applications of RNNs to partially ordered sequential data. Many models such as Long Short Term Memory (LSTMs), Gated Recurrent Units (GRUs) and transformers have been developed to classify time series data with the assumption that events in a sequence are ordered. On the other hand, fewer models have been developed for set based inputs, where order does not matter. There are several use cases where data at the uptake point is ordered but due to deficiencies in the processes some batches of data arrive unordered, resulting in partially ordered sequences. We introduce a novel transformer based model for such prediction tasks, and benchmark against extensions of existing order invariant models. We also discuss how transition probabilities between events in a sequence can be used to improve model performance. We show that the transformer based equal time model outperforms extensions of existing set models on three datasets. Lastly, we consider how to understand which features are important to sequential classification tasks. While many methods such as Locally Interpretable Model-agnostic Explanation (LIME), Integrated Gradients and Layerwise Relevance Propagation (LRP) have been developed to explain how recurrent neural networks make predictions, the explanations generated by each method often times vary dramatically. There is no consensus about which explainability methods most accurately and robustly determine features important for model prediction. We consider a classification task on a sequence of events and apply both gradient and attention based explanation models to compute explanations on the event type level. We implement a hierarchical attention model to compute explanations with respect to event type directly and show that attention based models return a higher similarity score between explanations for models initialized with different random seeds. However, there are still significant differences in explanations between model runs. We develop an optimization based model to find a low-loss, high-accuracy path between trained weights to understand how model explanations morph between different local minima. We use this low-loss path to provide insight as to why explanations vary on two sentiment datasets.

ISBN: 9798698575030Subjects--Topical Terms:

377601
Applied mathematics.
Subjects--Index Terms:

Explainability

Recurrent Neural Network Architectures for Sequence Classification and Explanation.
LDR:04851nmm a2200349 4500 001 594647
005 20210521101715.5
008 210917s2020 ||||||||||||||||| ||eng d
020 $a 9798698575030
035 $a (MiAaPQ)AAI28156279
035 $a AAI28156279
040 $a MiAaPQ $c MiAaPQ
100 1 $a Ger, Stephanie. $3 886703
245 1 0 $a Recurrent Neural Network Architectures for Sequence Classification and Explanation.
260 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2020
300 $a 95 p.
500 $a Source: Dissertations Abstracts International, Volume: 82-06, Section: B.
500 $a Advisor: Klabjan, Diego.
502 $a Thesis (Ph.D.)--Northwestern University, 2020.
506 $a This item must not be sold to any third party vendors.
520 $a This thesis focuses on applications of recurrent neural networks (RNNs) for three aspects of sequential classification. In the first chapter, a novel method to generate synthetic minority data generation to improve imbalanced classification is discussed. Generative Adversarial Networks (GANs) have been used in many different applications to generate realistic synthetic data. We introduce a novel GAN with Autoencoder (GAN-AE) architecture to generate synthetic samples for variable length, multi-feature sequence datasets. This model consists of a GAN architecture with an additional autoencoder component, where RNNs are used for each component of the model in order to generate synthetic data to improve classification accuracy for a highly imbalanced medical device dataset. In addition to the medical device dataset, we also evaluate the GAN-AE performance on two additional datasets and demonstrate the application of GAN-AE to a sequence-to-sequence task where both synthetic sequence inputs and sequence outputs must be generated. To evaluate the quality of the synthetic data, we train encoder-decoder models both with and without the synthetic data and compare the classification model performance. We show that a model trained with GAN-AE generated synthetic data outperforms models trained with synthetic data generated both with standard oversampling techniques such as SMOTE and Autoencoders as well as with state of the art GAN-based models.Next, we discuss the applications of RNNs to partially ordered sequential data. Many models such as Long Short Term Memory (LSTMs), Gated Recurrent Units (GRUs) and transformers have been developed to classify time series data with the assumption that events in a sequence are ordered. On the other hand, fewer models have been developed for set based inputs, where order does not matter. There are several use cases where data at the uptake point is ordered but due to deficiencies in the processes some batches of data arrive unordered, resulting in partially ordered sequences. We introduce a novel transformer based model for such prediction tasks, and benchmark against extensions of existing order invariant models. We also discuss how transition probabilities between events in a sequence can be used to improve model performance. We show that the transformer based equal time model outperforms extensions of existing set models on three datasets. Lastly, we consider how to understand which features are important to sequential classification tasks. While many methods such as Locally Interpretable Model-agnostic Explanation (LIME), Integrated Gradients and Layerwise Relevance Propagation (LRP) have been developed to explain how recurrent neural networks make predictions, the explanations generated by each method often times vary dramatically. There is no consensus about which explainability methods most accurately and robustly determine features important for model prediction. We consider a classification task on a sequence of events and apply both gradient and attention based explanation models to compute explanations on the event type level. We implement a hierarchical attention model to compute explanations with respect to event type directly and show that attention based models return a higher similarity score between explanations for models initialized with different random seeds. However, there are still significant differences in explanations between model runs. We develop an optimization based model to find a low-loss, high-accuracy path between trained weights to understand how model explanations morph between different local minima. We use this low-loss path to provide insight as to why explanations vary on two sentiment datasets.
590 $a School code: 0163.
650 4 $a Applied mathematics. $3 377601
650 4 $a Artificial intelligence. $3 194058
653 $a Explainability
653 $a Machine Learning
653 $a Recurrent Neural Networks
653 $a Low-loss path
690 $a 0364
690 $a 0800
710 2 $a Northwestern University. $b Engineering Sciences and Applied Mathematics. $3 886704
773 0 $t Dissertations Abstracts International $g 82-06B.
790 $a 0163
791 $a Ph.D.
792 $a 2020
793 $a English
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28156279