國立高雄大學圖資館 |

Language: English

Back

Training Models to Ignore Dataset Bias.

Clark, Christopher.

Training Models to Ignore Dataset Bias.

Record Type:	Electronic resources : Monograph/item
Title/Author:	Training Models to Ignore Dataset Bias.
Author:	Clark, Christopher.
Published:	Ann Arbor : ProQuest Dissertations & Theses, 2020
Description:	106 p.
Notes:	Source: Dissertations Abstracts International, Volume: 82-05, Section: B.
Notes:	Advisor: Zettlemoyer, Luke.
Contained By:	Dissertations Abstracts International82-05B.
Subject:	Artificial intelligence.
Online resource:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28092295
ISBN:	9798684669552

Training Models to Ignore Dataset Bias.
Clark, Christopher.

Training Models to Ignore Dataset Bias. - Ann Arbor : ProQuest Dissertations & Theses, 2020 - 106 p.

Source: Dissertations Abstracts International, Volume: 82-05, Section: B.

Thesis (Ph.D.)--University of Washington, 2020.

This item must not be sold to any third party vendors.

Modern machine learning algorithms have been able to achieve impressive results on complex tasks such as language comprehension or image understanding. However, recent work has cautioned that this success is often partially due to exploiting incidental correlations that were introduced during dataset creation, and are not fundamental to the target task. For example, sentence entailment datasets can have spurious word-class correlations if nearly all contradiction sentences contain the word ``not'', and image recognition datasets can have tell-tale object-background correlations if dogs are always indoors. Models that exploit these incidental correlation, which we call dataset bias, can be brittle and perform poorly on out-of-domain examples. In this thesis, we present several methods of solving this issue by preventing models from using dataset bias.A key challenge for this task is determining which predictive patterns in the training data are bias. This thesis proposes several solutions, ranging from methods that exploit domain expertise when such knowledge is available, to more broadly applicable domain-general solutions. Solving this task also requires preventing complex neural models from exploiting these biased patterns, even though they are often easy to learn and effective on the training data. We present ensembling and data augmentation based methods to handle this difficulty. In all cases, we evaluate our models by showing improved performance on out-of-domain datasets that were built to penalize biased models.Our first focus is on question answering, motivated by the observation biases can lead to poor performance when applying a model to multiple paragraphs. To solve this task, we propose a modified training scheme that exposes the model to additional paragraphs that do not answer the question. We then consider the case where expert knowledge of the bias can be used to construct a \extit{bias-only} model that captures biased methods. In this case, we can build an unbiased model by ensembling it with the bias-only model in order to disincentive it from learning bias. Finally, we generalize this approach by proposing a method to automatically construct the bias-only model when no such expert knowledge is available. Overall, this thesis shows that it is possible to train unbiased models on biased datasets, and proposes some fundamental answers to question about how bias can be detected and avoided.

ISBN: 9798684669552Subjects--Topical Terms:

194058
Artificial intelligence.
Subjects--Index Terms:

Computer vision

Training Models to Ignore Dataset Bias.
LDR:03552nmm a2200361 4500 001 594619
005 20210521101709.5
008 210917s2020 ||||||||||||||||| ||eng d
020 $a 9798684669552
035 $a (MiAaPQ)AAI28092295
035 $a AAI28092295
040 $a MiAaPQ $c MiAaPQ
100 1 $a Clark, Christopher. $3 528384
245 1 0 $a Training Models to Ignore Dataset Bias.
260 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2020
300 $a 106 p.
500 $a Source: Dissertations Abstracts International, Volume: 82-05, Section: B.
500 $a Advisor: Zettlemoyer, Luke.
502 $a Thesis (Ph.D.)--University of Washington, 2020.
506 $a This item must not be sold to any third party vendors.
520 $a Modern machine learning algorithms have been able to achieve impressive results on complex tasks such as language comprehension or image understanding. However, recent work has cautioned that this success is often partially due to exploiting incidental correlations that were introduced during dataset creation, and are not fundamental to the target task. For example, sentence entailment datasets can have spurious word-class correlations if nearly all contradiction sentences contain the word ``not'', and image recognition datasets can have tell-tale object-background correlations if dogs are always indoors. Models that exploit these incidental correlation, which we call dataset bias, can be brittle and perform poorly on out-of-domain examples. In this thesis, we present several methods of solving this issue by preventing models from using dataset bias.A key challenge for this task is determining which predictive patterns in the training data are bias. This thesis proposes several solutions, ranging from methods that exploit domain expertise when such knowledge is available, to more broadly applicable domain-general solutions. Solving this task also requires preventing complex neural models from exploiting these biased patterns, even though they are often easy to learn and effective on the training data. We present ensembling and data augmentation based methods to handle this difficulty. In all cases, we evaluate our models by showing improved performance on out-of-domain datasets that were built to penalize biased models.Our first focus is on question answering, motivated by the observation biases can lead to poor performance when applying a model to multiple paragraphs. To solve this task, we propose a modified training scheme that exposes the model to additional paragraphs that do not answer the question. We then consider the case where expert knowledge of the bias can be used to construct a \extit{bias-only} model that captures biased methods. In this case, we can build an unbiased model by ensembling it with the bias-only model in order to disincentive it from learning bias. Finally, we generalize this approach by proposing a method to automatically construct the bias-only model when no such expert knowledge is available. Overall, this thesis shows that it is possible to train unbiased models on biased datasets, and proposes some fundamental answers to question about how bias can be detected and avoided.
590 $a School code: 0250.
650 4 $a Artificial intelligence. $3 194058
650 4 $a Information science. $3 190425
650 4 $a Information technology. $3 184390
653 $a Computer vision
653 $a Dataset bias
653 $a Machine learning
653 $a Natural language processing
690 $a 0800
690 $a 0723
690 $a 0489
710 2 $a University of Washington. $b Computer Science and Engineering. $3 886663
773 0 $t Dissertations Abstracts International $g 82-05B.
790 $a 0250
791 $a Ph.D.
792 $a 2020
793 $a English
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28092295