國立高雄大學圖資館 |

Data Validation and Selection for Modern Machine Learning /

Record Type:	Electronic resources : Monograph/item
Title/Author:	Data Validation and Selection for Modern Machine Learning /Zifan Liu.
Author:	Liu, Zifan,
Description:	1 electronic resource (207 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-11, Section: B.
Notes:	Advisors: Koutris, Paraschos; Rekatsinas, Theodoros Committee members: Papailiopoulos, Dimitris; Patel, Jignesh; Venkataraman, Shivaram.
Contained By:	Dissertations Abstracts International85-11B.
Subject:	Computer science.
Online resource:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31241587
ISBN:	9798382345765

Data Validation and Selection for Modern Machine Learning /
Liu, Zifan,

Data Validation and Selection for Modern Machine Learning /Zifan Liu. - 1 electronic resource (207 pages)

Source: Dissertations Abstracts International, Volume: 85-11, Section: B.

Machine learning (ML) has revolutionized a wide range of fields with its capacity to learn from data and make informed decisions. Recognizing the critical role of well-curated data in the advancement of modern ML, the data-centric ML community emphasizes the importance of careful data preparation and strategic selection to ensure both data quality and their relevance to target tasks. However, the prevailing approach for data validation and selection employed in practice remains largely manual or ad-hoc, while automated methods often fall short in either effectiveness or efficiency, thus limiting their practical application. In this dissertation, we revisit the current methodologies for automated data validation and selection, aiming to propose new techniques with improved performance and practicality. In the first part of this dissertation, we study the verification and discovery of denial constraints, a general formalism that can express a wide range of quality rules for tabular data. Verification entails detecting whether a given denial constraint holds on a specific dataset, while discovery focuses on the automated mining of valid constraints. The current state-of-the-art methods for denial constraint verification and discovery are inefficient on large-scale datasets due to their quadratic complexity relative to the dataset size. In addition, existing works on denial constraint discovery rely on a time-consuming blocking phase of building intermediate data structures, further limiting their practicality. To address the limitations of prior works, we make a dual contribution. First, we introduce a novel verification algorithm that demonstrates near-linear complexity relative to dataset size by connecting denial constraint verification to orthogonal range search, showing a theoretical improvement over prior works. Second, we present an anytime algorithm for denial constraint discovery by combining our verification algorithm with lattice searches, eliminating the need for the blocking structure-building phase in existing solutions. Our verification algorithm achieves up to 84xfaster compared to state-of-the-art approaches. In addition, our discovery algorithm is able to start providing valid constraints within the initial 10 minutes of execution, while existing methods are blocked for over 48 hours.In the second part, we focus on defending against data corruption in ML pipelines. Data corruption is an impediment to modern ML applications as they can severely bias the learned model and also lead to invalid inferences. Data corruption in practice can be highly diverse, ranging from random noise, systematic errors to adversarial attacks, which are often beyond the scope of standard error detection methods. We present a simple framework to safeguard against data corruption during both training and deployment of ML models over tabular data. In the training stage, our framework identifies and removes corrupted data points from the training data to avoid obtaining a biased model. In the deployment stage, our framework flags, in an online manner, corrupted query points to a trained ML model that due to noise will result in incorrect predictions. To detect corrupted data, we develop a self-supervised deep learning model for mixed-type tabular data. To minimize the burden of deployment, learning the model does not require any human-labeled data. Our framework is designed as a plugin that can increase the robustness of any ML pipeline. We show that our framework consistently safeguards against corrupted data during both training and deployment of various models ranging from SVMs to neural networks, beating a diverse array of competing methods that span from data quality validation models to robust outlier-detection models. In addition, to promote the understanding of the worst-case effects that data corruption can have on learning performance, we present an information-theoretic analysis of robust mean estimation under coordinate-level corruption. Our analysis shows that leveraging the dependencies between features is the key to accurate mean estimation for corrupted data.In the last part, our attention turns to task-specific data selection. The goal is to select training data for specific tasks from a massive and heterogeneous pool of candidate data, guided by a small set of representative examples from the target task. We highlight two critical properties for the selected data: distribution alignment and diversity, which are not adequately satisfied by previous works. Aligning the distribution of the training data with query data expected during service time ensures that the model is customized for the intended usage. On the other hand, sufficient diversity allows the model to learn more knowledge and avoid overfitting. We present a framework that formulates task-specific data selection as an optimization problem based on optimal transport, a notion that captures the distance between two distributions. We add a regularization term to the optimal transport formulation to provide a smooth tradeoff between distribution alignment and diversity. In addition, we incorporate kernel density estimation into the regularizer to reduce the negative effects of near-duplicates in the candidate pool. Finally. we connect our optimization problem to nearest neighbor search and design efficient algorithms to compute the optimal solution based on approximate nearest neighbor search techniques. Our approach achieves an improvement of up to 5 points in F1 scores for targeted instruction tuning compared to the state-of-the-art method and demonstrates robustness against near-duplicates.

English

ISBN: 9798382345765Subjects--Topical Terms:

199325
Computer science.
Subjects--Index Terms:

Data management

Data Validation and Selection for Modern Machine Learning /
LDR:07255nmm a22004573i 4500 001 676312
005 20250420214340.5
006 m o d
007 cr|nu||||||||
008 250901s2024 miu||||||m |||||||eng d
020 $a 9798382345765
035 $a (MiAaPQD)AAI31241587
035 $a AAI31241587
040 $a MiAaPQD $b eng $c MiAaPQD $e rda
100 1 $a Liu, Zifan, $e author. $3 990078
245 1 0 $a Data Validation and Selection for Modern Machine Learning / $c Zifan Liu.
264 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2024
300 $a 1 electronic resource (207 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-11, Section: B.
500 $a Advisors: Koutris, Paraschos; Rekatsinas, Theodoros Committee members: Papailiopoulos, Dimitris; Patel, Jignesh; Venkataraman, Shivaram.
502 $b Ph.D. $c The University of Wisconsin - Madison $d 2024.
520 $a Machine learning (ML) has revolutionized a wide range of fields with its capacity to learn from data and make informed decisions. Recognizing the critical role of well-curated data in the advancement of modern ML, the data-centric ML community emphasizes the importance of careful data preparation and strategic selection to ensure both data quality and their relevance to target tasks. However, the prevailing approach for data validation and selection employed in practice remains largely manual or ad-hoc, while automated methods often fall short in either effectiveness or efficiency, thus limiting their practical application. In this dissertation, we revisit the current methodologies for automated data validation and selection, aiming to propose new techniques with improved performance and practicality. In the first part of this dissertation, we study the verification and discovery of denial constraints, a general formalism that can express a wide range of quality rules for tabular data. Verification entails detecting whether a given denial constraint holds on a specific dataset, while discovery focuses on the automated mining of valid constraints. The current state-of-the-art methods for denial constraint verification and discovery are inefficient on large-scale datasets due to their quadratic complexity relative to the dataset size. In addition, existing works on denial constraint discovery rely on a time-consuming blocking phase of building intermediate data structures, further limiting their practicality. To address the limitations of prior works, we make a dual contribution. First, we introduce a novel verification algorithm that demonstrates near-linear complexity relative to dataset size by connecting denial constraint verification to orthogonal range search, showing a theoretical improvement over prior works. Second, we present an anytime algorithm for denial constraint discovery by combining our verification algorithm with lattice searches, eliminating the need for the blocking structure-building phase in existing solutions. Our verification algorithm achieves up to 84xfaster compared to state-of-the-art approaches. In addition, our discovery algorithm is able to start providing valid constraints within the initial 10 minutes of execution, while existing methods are blocked for over 48 hours.In the second part, we focus on defending against data corruption in ML pipelines. Data corruption is an impediment to modern ML applications as they can severely bias the learned model and also lead to invalid inferences. Data corruption in practice can be highly diverse, ranging from random noise, systematic errors to adversarial attacks, which are often beyond the scope of standard error detection methods. We present a simple framework to safeguard against data corruption during both training and deployment of ML models over tabular data. In the training stage, our framework identifies and removes corrupted data points from the training data to avoid obtaining a biased model. In the deployment stage, our framework flags, in an online manner, corrupted query points to a trained ML model that due to noise will result in incorrect predictions. To detect corrupted data, we develop a self-supervised deep learning model for mixed-type tabular data. To minimize the burden of deployment, learning the model does not require any human-labeled data. Our framework is designed as a plugin that can increase the robustness of any ML pipeline. We show that our framework consistently safeguards against corrupted data during both training and deployment of various models ranging from SVMs to neural networks, beating a diverse array of competing methods that span from data quality validation models to robust outlier-detection models. In addition, to promote the understanding of the worst-case effects that data corruption can have on learning performance, we present an information-theoretic analysis of robust mean estimation under coordinate-level corruption. Our analysis shows that leveraging the dependencies between features is the key to accurate mean estimation for corrupted data.In the last part, our attention turns to task-specific data selection. The goal is to select training data for specific tasks from a massive and heterogeneous pool of candidate data, guided by a small set of representative examples from the target task. We highlight two critical properties for the selected data: distribution alignment and diversity, which are not adequately satisfied by previous works. Aligning the distribution of the training data with query data expected during service time ensures that the model is customized for the intended usage. On the other hand, sufficient diversity allows the model to learn more knowledge and avoid overfitting. We present a framework that formulates task-specific data selection as an optimization problem based on optimal transport, a notion that captures the distance between two distributions. We add a regularization term to the optimal transport formulation to provide a smooth tradeoff between distribution alignment and diversity. In addition, we incorporate kernel density estimation into the regularizer to reduce the negative effects of near-duplicates in the candidate pool. Finally. we connect our optimization problem to nearest neighbor search and design efficient algorithms to compute the optimal solution based on approximate nearest neighbor search techniques. Our approach achieves an improvement of up to 5 points in F1 scores for targeted instruction tuning compared to the state-of-the-art method and demonstrates robustness against near-duplicates.
546 $a English
590 $a School code: 0262
650 4 $a Computer science. $3 199325
650 4 $a Computer engineering. $3 212944
650 4 $a Information technology. $3 184390
653 $a Data management
653 $a Data selection
653 $a Data validation
653 $a Denial constraints
653 $a Machine learning
690 $a 0984
690 $a 0489
690 $a 0464
690 $a 0800
710 2 $a The University of Wisconsin - Madison. $b Computer Sciences. $3 708692
720 1 $a Koutris, Paraschos $e degree supervisor.
720 1 $a Rekatsinas, Theodoros $e degree supervisor.
773 0 $t Dissertations Abstracts International $g 85-11B.
790 $a 0262
791 $a Ph.D.
792 $a 2024
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31241587