國立高雄大學圖資館 |

Applications of Random Matrix Theory in Statistics and Machine Learning.

Record Type:	Electronic resources : Monograph/item
Title/Author:	Applications of Random Matrix Theory in Statistics and Machine Learning.
Author:	Sheng, Yue.
Published:	Ann Arbor : ProQuest Dissertations & Theses, 2021
Description:	238 p.
Notes:	Source: Dissertations Abstracts International, Volume: 82-11, Section: B.
Notes:	Advisor: Dobriban, Edgar;Pemantle, Robin.
Contained By:	Dissertations Abstracts International82-11B.
Subject:	Applied mathematics.
Online resource:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28413122
ISBN:	9798738617904

Applications of Random Matrix Theory in Statistics and Machine Learning.
Sheng, Yue.

Applications of Random Matrix Theory in Statistics and Machine Learning. - Ann Arbor : ProQuest Dissertations & Theses, 2021 - 238 p.

Source: Dissertations Abstracts International, Volume: 82-11, Section: B.

Thesis (Ph.D.)--University of Pennsylvania, 2021.

This item must not be sold to any third party vendors.

We live in an age of big data. Analyzing modern data sets can be very difficult because they usually present the following features: massive, high-dimensional, and heterogeneous. How to deal with these new features often plays a key role in modern statistical and machine learning research. This dissertation uses random matrix theory (RMT), a powerful mathematical tool, to study several important problems where the data is massive, high-dimensional, and sometimes heterogeneous.The first chapter briefly introduces some basics of random matrix theory (RMT). We also cover some classical applications of RMT to statistics and machine learning.The second chapter is about distributed linear regression, where we consider the ordinary least squares (OLS) estimators. Distributed statistical learning problems arise commonly when dealing with large datasets. In this setup, datasets are partitioned over machines, which compute locally and communicate short messages. Communication is often the bottleneck. We study one-step and iterative weighted parameter averaging in statistical linear models under data parallelism. We do linear regression on each machine, send the results to a central server, and take a weighted average of the parameters. Optionally, we iterate, sending back the weighted average and doing local ridge regressions centered at it. How does this work compare to doing linear regression on the full data? Here we study the performance loss in estimation and test error, and confidence interval length in high dimensions, where the number of parameters is comparable to the training data size. We find the performance loss in one-step weighted averaging, and also give results for iterative averaging. We also find that different problems are affected differently by the distributed framework.The third chapter studies a fundamental and highly important problem in this area: How to do ridge regression in a distributed computing environment? Ridge regression is an extremely popular method for supervised learning and has several optimality properties, thus it is important to study. We study one-shot methods that construct weighted combinations of ridge regression estimators computed on each machine. By analyzing the mean squared error in a high dimensional random-effects model where each predictor has a small effect, we discover several new phenomena. We also propose a new Weighted ONe-shot DistributEd Ridge regression (WONDER) algorithm. We test WONDER in simulation studies and using the Million Song Dataset as an example. There it can save at least 100x in computation time, while nearly preserving test accuracy.The fourth chapter is trying to solve another possible issue with modern data sets, that is heterogeneity. Dimensionality reduction via PCA and factor analysis is an important tool of data analysis. A critical step is selecting the number of components. However, existing methods (such as the scree plot, likelihood ratio, parallel analysis, etc) do not have statistical guarantees in the increasingly common setting where the data are heterogeneous. There each noise entry can have a different distribution. To address this problem, we propose the Signflip Parallel Analysis (Signflip PA) method: it compares data singular values to those of “empirical null” data generated by flipping the sign of each entry randomly with probability one-half. We show that Signflip PA consistently selects factors above the noise level in high-dimensional signal-plus-noise models (including spiked models and factor models) under heterogeneous settings. Here the classical parallel analysis is no longer effective. To do this, we propose to leverage recent breakthroughs in random matrix theory, such as dimension-free operator norm bounds and large deviations for the top eigenvalues of nonhomogeneous matrices. We also illustrate that Signflip PA performs well in numerical simulations and on empirical data examples.

ISBN: 9798738617904Subjects--Topical Terms:

377601
Applied mathematics.
Subjects--Index Terms:

Massive high-dimensional heterogeneous data sets

Applications of Random Matrix Theory in Statistics and Machine Learning.
LDR:05345nmm a2200421 4500 001 616407
005 20220513114329.5
008 220920s2021 ||||||||||||||||| ||eng d
020 $a 9798738617904
035 $a (MiAaPQ)AAI28413122
035 $a AAI28413122
040 $a MiAaPQ $c MiAaPQ
100 1 $a Sheng, Yue. $3 915611
245 1 0 $a Applications of Random Matrix Theory in Statistics and Machine Learning.
260 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2021
300 $a 238 p.
500 $a Source: Dissertations Abstracts International, Volume: 82-11, Section: B.
500 $a Advisor: Dobriban, Edgar;Pemantle, Robin.
502 $a Thesis (Ph.D.)--University of Pennsylvania, 2021.
506 $a This item must not be sold to any third party vendors.
520 $a We live in an age of big data. Analyzing modern data sets can be very difficult because they usually present the following features: massive, high-dimensional, and heterogeneous. How to deal with these new features often plays a key role in modern statistical and machine learning research. This dissertation uses random matrix theory (RMT), a powerful mathematical tool, to study several important problems where the data is massive, high-dimensional, and sometimes heterogeneous.The first chapter briefly introduces some basics of random matrix theory (RMT). We also cover some classical applications of RMT to statistics and machine learning.The second chapter is about distributed linear regression, where we consider the ordinary least squares (OLS) estimators. Distributed statistical learning problems arise commonly when dealing with large datasets. In this setup, datasets are partitioned over machines, which compute locally and communicate short messages. Communication is often the bottleneck. We study one-step and iterative weighted parameter averaging in statistical linear models under data parallelism. We do linear regression on each machine, send the results to a central server, and take a weighted average of the parameters. Optionally, we iterate, sending back the weighted average and doing local ridge regressions centered at it. How does this work compare to doing linear regression on the full data? Here we study the performance loss in estimation and test error, and confidence interval length in high dimensions, where the number of parameters is comparable to the training data size. We find the performance loss in one-step weighted averaging, and also give results for iterative averaging. We also find that different problems are affected differently by the distributed framework.The third chapter studies a fundamental and highly important problem in this area: How to do ridge regression in a distributed computing environment? Ridge regression is an extremely popular method for supervised learning and has several optimality properties, thus it is important to study. We study one-shot methods that construct weighted combinations of ridge regression estimators computed on each machine. By analyzing the mean squared error in a high dimensional random-effects model where each predictor has a small effect, we discover several new phenomena. We also propose a new Weighted ONe-shot DistributEd Ridge regression (WONDER) algorithm. We test WONDER in simulation studies and using the Million Song Dataset as an example. There it can save at least 100x in computation time, while nearly preserving test accuracy.The fourth chapter is trying to solve another possible issue with modern data sets, that is heterogeneity. Dimensionality reduction via PCA and factor analysis is an important tool of data analysis. A critical step is selecting the number of components. However, existing methods (such as the scree plot, likelihood ratio, parallel analysis, etc) do not have statistical guarantees in the increasingly common setting where the data are heterogeneous. There each noise entry can have a different distribution. To address this problem, we propose the Signflip Parallel Analysis (Signflip PA) method: it compares data singular values to those of “empirical null” data generated by flipping the sign of each entry randomly with probability one-half. We show that Signflip PA consistently selects factors above the noise level in high-dimensional signal-plus-noise models (including spiked models and factor models) under heterogeneous settings. Here the classical parallel analysis is no longer effective. To do this, we propose to leverage recent breakthroughs in random matrix theory, such as dimension-free operator norm bounds and large deviations for the top eigenvalues of nonhomogeneous matrices. We also illustrate that Signflip PA performs well in numerical simulations and on empirical data examples.
590 $a School code: 0175.
650 4 $a Applied mathematics. $3 377601
650 4 $a Statistics. $3 182057
650 4 $a Artificial intelligence. $3 194058
653 $a Massive high-dimensional heterogeneous data sets
653 $a Radom matrix theory
653 $a Ordinary least squares estimators
653 $a Distributed linear regression
653 $a Ridge regression
653 $a Distributed computing
653 $a Dimensionality reduction
653 $a Factor analysis
653 $a One-shot methods
690 $a 0364
690 $a 0463
690 $a 0800
710 2 $a University of Pennsylvania. $b Applied Mathematics and Computational Science. $3 660319
773 0 $t Dissertations Abstracts International $g 82-11B.
790 $a 0175
791 $a Ph.D.
792 $a 2021
793 $a English
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28413122