國立高雄大學圖資館 |

Reinforcement learning and stochastic optimization :a unified framework for sequential decisions /

Record Type:	Language materials, printed : Monograph/item
Title/Author:	Reinforcement learning and stochastic optimization :Warren B Powell.
Reminder of title:	a unified framework for sequential decisions /
Author:	Powell, Warren B.,
Description:	1 online resource
Subject:	Reinforcement learning.
Online resource:	http://dx.doi.org/10.1002/9781119815068
Online resource:	https://www.oreilly.com/library/view/-/9781119815037/
Online resource:	https://ieeexplore.ieee.org/book/9820922
Online resource:	https://www.oreilly.com/covers/urn:orm:book:9781119815037/
ISBN:	9781119815068

Reinforcement learning and stochastic optimization :a unified framework for sequential decisions /
Powell, Warren B.,1955-

Reinforcement learning and stochastic optimization :a unified framework for sequential decisions /Warren B Powell. - First Edition. - 1 online resource

Includes bibliographical references and index.

Preface xxv -- Acknowledgments xxxi -- Part I - Introduction 1 -- 1 Sequential Decision Problems 3 -- 1.1 The Audience 7 -- 1.2 The Communities of Sequential Decision Problems 8 -- 1.3 Our Universal Modeling Framework 10 -- 1.4 Designing Policies for Sequential Decision Problems 15 -- 1.5 Learning 20 -- 1.6 Themes 21 -- 1.7 Our Modeling Approach 27 -- 1.8 How to Read this Book 27 -- 1.9 Bibliographic Notes 33 -- Exercises 34 -- Bibliography 38 -- 2 Canonical Problems and Applications 39 -- 2.1 Canonical Problems 39 -- 2.2 A Universal Modeling Framework for Sequential Decision Problems 64 -- 2.3 Applications 69 -- 2.4 Bibliographic Notes 85 -- Exercises 90 -- Bibliography 93 -- 3 Online Learning 101 -- 3.1 Machine Learning for Sequential Decisions 102 -- 3.2 Adaptive Learning Using Exponential Smoothing 110 -- 3.3 Lookup Tables with Frequentist Updating 111 -- 3.4 Lookup Tables with Bayesian Updating 112 -- 3.5 Computing Bias and Variance* 118 -- 3.6 Lookup Tables and Aggregation* 121 -- 3.7 Linear Parametric Models 131 -- 3.8 Recursive Least Squares for Linear Models 136 -- 3.9 Nonlinear Parametric Models 140 -- 3.10 Nonparametric Models* 149 -- 3.11 Nonstationary Learning* 159 -- 3.12 The Curse of Dimensionality 162 -- 3.13 Designing Approximation Architectures in Adaptive Learning 165 -- 3.14 Why Does It Work?** 166 -- 3.15 Bibliographic Notes 174 -- Exercises 176 -- Bibliography 180 -- 4 Introduction to Stochastic Search 183 -- 4.1 Illustrations of the Basic Stochastic Optimization Problem 185 -- 4.2 Deterministic Methods 188 -- 4.3 Sampled Models 193 -- 4.4 Adaptive Learning Algorithms 202 -- 4.5 Closing Remarks 210 -- 4.6 Bibliographic Notes 210 -- Exercises 212 -- Bibliography 218 -- Part II - Stochastic Search 221 -- 5 Derivative-Based Stochastic Search 223 -- 5.1 Some Sample Applications 225 -- 5.2 Modeling Uncertainty 228 -- 5.3 Stochastic Gradient Methods 231 -- 5.4 Styles of Gradients 237 -- 5.5 Parameter Optimization for Neural Networks* 242 -- 5.6 Stochastic Gradient Algorithm as a Sequential Decision Problem 247 -- 5.7 Empirical Issues 248 -- 5.8 Transient Problems* 249 -- 5.9 Theoretical Performance* 250 -- 5.10 Why Does it Work? 250 -- 5.11 Bibliographic Notes 263 -- Exercises 264 -- Bibliography 270 -- 6 Stepsize Policies 273 -- 6.1 Deterministic Stepsize Policies 276 -- 6.2 Adaptive Stepsize Policies 282 -- 6.3 Optimal Stepsize Policies* 289 -- 6.4 Optimal Step sizes for Approximate Value Iteration* 297 -- 6.5 Convergence 300 -- 6.6 Guidelines for Choosing Stepsize Policies 301 -- 6.7 Why Does it Work* 303 -- 6.8 Bibliographic Notes 306 -- Exercises 307 -- Bibliography 314 -- 7 Derivative-Free Stochastic Search 317 -- 7.1 Overview of Derivative-free Stochastic Search 319 -- 7.2 Modeling Derivative-free Stochastic Search 325 -- 7.3 Designing Policies 330 -- 7.4 Policy Function Approximations 333 -- 7.5 Cost Function Approximations 335 -- 7.6 VFA-based Policies 338 -- 7.7 Direct Lookahead Policies 348 -- 7.8 The Knowledge Gradient (Continued)* 362 -- 7.9 Learning in Batches 380 -- 7.10 Simulation Optimization* 382 -- 7.11 Evaluating Policies 385 -- 7.12 Designing Policies 394 -- 7.13 Extensions* 398 -- 7.14 Bibliographic Notes 409 -- Exercises 412 -- Bibliography 424 -- Part III - State-dependent Problems 429 -- 8 State-dependent Problems 431 -- 8.1 Graph Problems 433 -- 8.2 Inventory Problems 439 -- 8.3 Complex Resource Allocation Problems 446 -- 8.4 State-dependent Learning Problems 456 -- 8.5 A Sequence of Problem Classes 460 -- 8.6 Bibliographic Notes 461 -- Exercises 462 -- Bibliography 466 -- 9 Modeling Sequential Decision Problems 467 -- 9.1 A Simple Modeling Illustration 471 -- 9.2 Notational Style 476 -- 9.3 Modeling Time 478 -- 9.4 The States of Our System 481 -- 9.5 Modeling Decisions 500 -- 9.6 The Exogenous Information Process 506 -- 9.7 The Transition Function 515 -- 9.8 The Objective Function 518 -- 9.9 Illustration: An Energy Storage Model 523 -- 9.10 Base Models and Lookahead Models 528 -- 9.11 A Classification of Problems* 529 -- 9.12 Policy Evaluation* 532 -- 9.13 Advanced Probabilistic Modeling Concepts** 534 -- 9.14 Looking Forward 540 -- 9.15 Bibliographic Notes 542 -- Exercises 544 -- Bibliography 557 -- 10 Uncertainty Modeling 559 -- 10.1 Sources of Uncertainty 560 -- 10.2 A Modeling Case Study: The COVID Pandemic 575 -- 10.3 Stochastic Modeling 575 -- 10.4 Monte Carlo Simulation 581 -- 10.5 Case Study: Modeling Electricity Prices 589 -- 10.6 Sampling vs. Sampled Models 595 -- 10.7 Closing Notes 597 -- 10.8 Bibliographic Notes 597 -- Exercises 598 -- Bibliography 601 -- 11 Designing Policies 603 -- 11.1 From Optimization to Machine Learning to Sequential Decision Problems 605 -- 11.2 The Classes of Policies 606 -- 11.3 Policy Function Approximations 610 -- 11.4 Cost Function Approximations 613 -- 11.5 Value Function Approximations 614 -- 11.6 Direct Lookahead Approximations 616 -- 11.7 Hybrid Strategies 620 -- 11.8 Randomized Policies 626 -- 11.9 Illustration: An Energy Storage Model Revisited 627 -- 11.10 Choosing the Policy Class 631 -- 11.11 Policy Evaluation 641 -- 11.12 Parameter Tuning 642 -- 11.13 Bibliographic Notes 646 -- Exercises 646 -- Bibliography 651 -- Part IV - Policy Search 653 -- 12 Policy Function Approximations and Policy Search 655 -- 12.1 Policy Search as a Sequential Decision Problem 657 -- 12.2 Classes of Policy Function Approximations 658 -- 12.3 Problem Characteristics 665 -- 12.4 Flavors of Policy Search 666 -- 12.5 Policy Search with Numerical Derivatives 669 -- 12.6 Derivative-Free Methods for Policy Search 670 -- 12.7 Exact Derivatives for Continuous Sequential Problems* 677 -- 12.8 Exact Derivatives for Discrete Dynamic Programs** 680 -- 12.9 Supervised Learning 686 -- 12.10 Why Does it Work? 687 -- 12.11 Bibliographic Notes 690 -- Exercises 691 -- Bibliography 698 -- 13 Cost Function Approximations 701 -- 13.1 General Formulation for Parametric CFA 703 -- 13.2 Objective-Modified CFAs 704 -- 13.3 Constraint-Modified CFAs 714 -- 13.4 Bibliographic Notes 725 -- Exercises 726 -- Bibliography 729 -- Part V - Lookahead Policies 731 -- 14 Exact Dynamic Programming 737 -- 14.1 Discrete Dynamic Programming 738 -- 14.2 The Optimality Equations 740 -- 14.3 Finite Horizon Problems 747 -- 14.4 Continuous Problems with Exact Solutions 750 -- 14.5 Infinite Horizon Problems* 755 -- 14.6 Value Iteration for Infinite Horizon Problems* 757 -- 14.7 Policy Iteration for Infinite Horizon Problems* 762 -- 14.8 Hybrid Value-Policy Iteration* 764 -- 14.9 Average Reward Dynamic Programming* 765 -- 14.10 The Linear Programming Method for Dynamic Programs** 766 -- 14.11 Linear Quadratic Regulation 767 -- 14.12 Why Does it Work?** 770 -- 14.13 Bibliographic Notes 783 -- Exercises 783 -- Bibliography 793 -- 15 Backward Approximate Dynamic Programming 795 -- 15.1 Backward Approximate Dynamic Programming for Finite Horizon Problems 797 -- 15.2 Fitted Value Iteration for Infinite Horizon Problems 804 -- 15.3 Value Function Approximation Strategies 805 -- 15.4 Computational Observations 810 -- 15.5 Bibliographic Notes 816 -- Exercises 816 -- Bibliography 821 -- 16 Forward ADP I: The Value of a Policy 823 -- 16.1 Sampling the Value of a Policy 824 -- 16.2 Stochastic Approximation Methods 835 -- 16.3 Bellman's Equation Using a Linear Model* 837 -- 16.4 Analysis of TD(0), LSTD, and LSPE Using a Single State* 842 -- 16.5 Gradient-based Methods for Approximate Value Iteration* 845 -- 16.6 Value Function Approximations Based on Bayesian Learning* 852 -- 16.7 Learning Algorithms and Atepsizes 855 -- 16.8 Bibliographic Notes 860 -- Exercises 862 -- Bibliography 864 -- 17 Forward ADP II: Policy Optimization 867 -- 17.1 Overview of Algorithmic Strategies 869 -- 17.2 Approximate Value Iteration and Q-Learning Using Lookup Tables 871 -- 17.3 Styles of Learning 881 -- 17.4 Approximate Value Iteration Using Linear Models 886 -- 17.5 On-policy vs. off-policy learning and the exploration-exploitation problem 888 -- 17.6 Applications 894 -- 17.7 Approximate Policy Iteration 900 -- 17.8 The Actor-Critic Paradigm 907 -- 17.9 Statistical Bias in the Max Operator* 909 -- 17.10 The Linear Programming Method Using Linear Models* 912 -- 17.11 Finite Horizon Approximations for Steady-State Applications 915 -- 17.12 Bibliographic Notes 917 -- Exercises 918 -- Bibliography 924 -- 18 Forward ADP III: Convex Resource Allocation Problems 927 -- 18.1 Resource Allocation Problems 930 -- 18.2 Values Versus Marginal Values 937 -- 18.3 Piecewise Linear Approximations for Scalar Functions 938 -- 18.4 Regression Methods 941 -- 18.5 Separable Piecewise Linear Approximations 944 -- 18.6 Benders Decomposition for Nonseparable Approximations** 946 -- 18.7 Linear Approximations for High-Dimensional Applications 956 -- 18.8 Resource Allocation with Exogenous Information State 958 -- 18.9 Closing Notes 959 -- 18.10 Bibliographic Notes 960 -- Exercises 962 -- Bibliography 967 -- 19 Direct Lookahead Policies 971 -- 19.1 Optimal Policies Using Lookahead Models 974 -- 19.2 Creating an Approximate Lookahead Model 978 -- 19.3 Modified Objectives in Lookahead Models 985 -- 19.4 Evaluating DLA Policies 992 -- 19.5 Why Use a DLA? 997 -- 19.6 Deterministic Lookaheads 999 -- 19.7 A Tour of Stochastic Lookahead Policies 1005 -- 19.8 Monte Carlo Tree Search for Discrete Decisions 1009 -- 19.9 Two-Stage Stochastic Programming for Vector Decisions* 1018 -- 19.10 Observations on DLA Policies 1024 -- 19.11 Bibliographic Notes 1025 -- Exercises 1027 -- Bibliography 1031 -- Part VI - Multiagent Systems 1033 -- 20 Multiagent Modeling and Learning 1035 -- 20.1 Overview of Multiagent Systems 1036 -- 20.2 A Learning Problem - Flu Mitigation 1044 -- 20.3 The POMDP Perspective* 1059 -- 20.4 The Two-Agent Newsvendor Problem 1062 -- 20.5 Multipl ...

"The first step in sequential decision problems is to understand what decisions are being made. It is surprising how often it is that people faced with complex problems, which spans scientists in a lab to people trying to solve major health problems, are not able to identify the decisions they face. We then want to find a method for making decisions. There are at least 45 words in the English language that are equivalent to "method for making a decision," but the one we have settled on is policy. The term policy is very familiar to fields such as Markov decision processes and reinforcement learning, but with a much narrower interpretation than we will use. Other fields do not use the term at all. Designing effective policies will be the focus of most of this book. Even more subtle is identifying the different sources of uncertainty. It can be hard enough trying to identify potential decisions, but thinking about all the random events that might affect whatever it is that you are managing, whether it is reducing disease, managing inventories, or making investments, can seem like a hopeless challenge"--

ISBN: 9781119815068

Standard No.: 10.1002/9781119815068doi

Source: 9781119815037O'Reilly Media

LCCN: 2021060405Subjects--Topical Terms:

349131
Reinforcement learning.
Index Terms--Genre/Form:

214472
Electronic books.

LC Class. No.: Q325.6 / .P69 2022

Dewey Class. No.: 006.3/1

Reinforcement learning and stochastic optimization :a unified framework for sequential decisions /
LDR:13248nam a2200565 a 4500 001 616699
003 OCoLC
005 20221006041404.0
006 m o d
007 cr |||||||||||
008 221006r20222021nju ob 001 0 eng
010 $a 2021060405
019 $a 1304399865 $a 1304466541
020 $a 9781119815068 $q electronic book
020 $a 1119815061 $q electronic book
020 $a 9781119815044 $q electronic book
020 $a 1119815045 $q electronic book
020 $a 9781119815051 $q electronic publication
020 $a 1119815053 $q electronic publication
020 $z 9781119815037 $q Hardback
020 $z 1119815037 $q Hardback
024 7 $a 10.1002/9781119815068 $2 doi
029 1 $a AU@ $b 000071509765
035 $a (OCoLC)1304342551 $z (OCoLC)1304399865 $z (OCoLC)1304466541
035 $a on1304342551
037 $a 9781119815037 $b O'Reilly Media
037 $a 9820922 $b IEEE
040 $a DLC $b eng $e rda $c DLC $d YDX $d DG1 $d NOC $d OCLCO $d YDX $d OCLCF $d ORMDA $d IEEEE
042 $a pcc
049 $a NUKM
050 0 0 $a Q325.6 $b .P69 2022
082 0 0 $a 006.3/1 $2 23/eng20220412
100 1 $a Powell, Warren B., $d 1955- $3 265406
132 1 2 $a cam 2200553 i 4500
245 1 0 $a Reinforcement learning and stochastic optimization : $b a unified framework for sequential decisions / $c Warren B Powell.
250 $a First Edition.
264 1 $a Hoboken, New Jersey : $b John Wiley & Sons, Inc, $c [2022]
300 $a 1 online resource
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
504 $a Includes bibliographical references and index.
505 0 $a Preface xxv -- Acknowledgments xxxi -- Part I - Introduction 1 -- 1 Sequential Decision Problems 3 -- 1.1 The Audience 7 -- 1.2 The Communities of Sequential Decision Problems 8 -- 1.3 Our Universal Modeling Framework 10 -- 1.4 Designing Policies for Sequential Decision Problems 15 -- 1.5 Learning 20 -- 1.6 Themes 21 -- 1.7 Our Modeling Approach 27 -- 1.8 How to Read this Book 27 -- 1.9 Bibliographic Notes 33 -- Exercises 34 -- Bibliography 38 -- 2 Canonical Problems and Applications 39 -- 2.1 Canonical Problems 39 -- 2.2 A Universal Modeling Framework for Sequential Decision Problems 64 -- 2.3 Applications 69 -- 2.4 Bibliographic Notes 85 -- Exercises 90 -- Bibliography 93 -- 3 Online Learning 101 -- 3.1 Machine Learning for Sequential Decisions 102 -- 3.2 Adaptive Learning Using Exponential Smoothing 110 -- 3.3 Lookup Tables with Frequentist Updating 111 -- 3.4 Lookup Tables with Bayesian Updating 112 -- 3.5 Computing Bias and Variance* 118 -- 3.6 Lookup Tables and Aggregation* 121 -- 3.7 Linear Parametric Models 131 -- 3.8 Recursive Least Squares for Linear Models 136 -- 3.9 Nonlinear Parametric Models 140 -- 3.10 Nonparametric Models* 149 -- 3.11 Nonstationary Learning* 159 -- 3.12 The Curse of Dimensionality 162 -- 3.13 Designing Approximation Architectures in Adaptive Learning 165 -- 3.14 Why Does It Work?** 166 -- 3.15 Bibliographic Notes 174 -- Exercises 176 -- Bibliography 180 -- 4 Introduction to Stochastic Search 183 -- 4.1 Illustrations of the Basic Stochastic Optimization Problem 185 -- 4.2 Deterministic Methods 188 -- 4.3 Sampled Models 193 -- 4.4 Adaptive Learning Algorithms 202 -- 4.5 Closing Remarks 210 -- 4.6 Bibliographic Notes 210 -- Exercises 212 -- Bibliography 218 -- Part II - Stochastic Search 221 -- 5 Derivative-Based Stochastic Search 223 -- 5.1 Some Sample Applications 225 -- 5.2 Modeling Uncertainty 228 -- 5.3 Stochastic Gradient Methods 231 -- 5.4 Styles of Gradients 237 -- 5.5 Parameter Optimization for Neural Networks* 242 -- 5.6 Stochastic Gradient Algorithm as a Sequential Decision Problem 247 -- 5.7 Empirical Issues 248 -- 5.8 Transient Problems* 249 -- 5.9 Theoretical Performance* 250 -- 5.10 Why Does it Work? 250 -- 5.11 Bibliographic Notes 263 -- Exercises 264 -- Bibliography 270 -- 6 Stepsize Policies 273 -- 6.1 Deterministic Stepsize Policies 276 -- 6.2 Adaptive Stepsize Policies 282 -- 6.3 Optimal Stepsize Policies* 289 -- 6.4 Optimal Step sizes for Approximate Value Iteration* 297 -- 6.5 Convergence 300 -- 6.6 Guidelines for Choosing Stepsize Policies 301 -- 6.7 Why Does it Work* 303 -- 6.8 Bibliographic Notes 306 -- Exercises 307 -- Bibliography 314 -- 7 Derivative-Free Stochastic Search 317 -- 7.1 Overview of Derivative-free Stochastic Search 319 -- 7.2 Modeling Derivative-free Stochastic Search 325 -- 7.3 Designing Policies 330 -- 7.4 Policy Function Approximations 333 -- 7.5 Cost Function Approximations 335 -- 7.6 VFA-based Policies 338 -- 7.7 Direct Lookahead Policies 348 -- 7.8 The Knowledge Gradient (Continued)* 362 -- 7.9 Learning in Batches 380 -- 7.10 Simulation Optimization* 382 -- 7.11 Evaluating Policies 385 -- 7.12 Designing Policies 394 -- 7.13 Extensions* 398 -- 7.14 Bibliographic Notes 409 -- Exercises 412 -- Bibliography 424 -- Part III - State-dependent Problems 429 -- 8 State-dependent Problems 431 -- 8.1 Graph Problems 433 -- 8.2 Inventory Problems 439 -- 8.3 Complex Resource Allocation Problems 446 -- 8.4 State-dependent Learning Problems 456 -- 8.5 A Sequence of Problem Classes 460 -- 8.6 Bibliographic Notes 461 -- Exercises 462 -- Bibliography 466 -- 9 Modeling Sequential Decision Problems 467 -- 9.1 A Simple Modeling Illustration 471 -- 9.2 Notational Style 476 -- 9.3 Modeling Time 478 -- 9.4 The States of Our System 481 -- 9.5 Modeling Decisions 500 -- 9.6 The Exogenous Information Process 506 -- 9.7 The Transition Function 515 -- 9.8 The Objective Function 518 -- 9.9 Illustration: An Energy Storage Model 523 -- 9.10 Base Models and Lookahead Models 528 -- 9.11 A Classification of Problems* 529 -- 9.12 Policy Evaluation* 532 -- 9.13 Advanced Probabilistic Modeling Concepts** 534 -- 9.14 Looking Forward 540 -- 9.15 Bibliographic Notes 542 -- Exercises 544 -- Bibliography 557 -- 10 Uncertainty Modeling 559 -- 10.1 Sources of Uncertainty 560 -- 10.2 A Modeling Case Study: The COVID Pandemic 575 -- 10.3 Stochastic Modeling 575 -- 10.4 Monte Carlo Simulation 581 -- 10.5 Case Study: Modeling Electricity Prices 589 -- 10.6 Sampling vs. Sampled Models 595 -- 10.7 Closing Notes 597 -- 10.8 Bibliographic Notes 597 -- Exercises 598 -- Bibliography 601 -- 11 Designing Policies 603 -- 11.1 From Optimization to Machine Learning to Sequential Decision Problems 605 -- 11.2 The Classes of Policies 606 -- 11.3 Policy Function Approximations 610 -- 11.4 Cost Function Approximations 613 -- 11.5 Value Function Approximations 614 -- 11.6 Direct Lookahead Approximations 616 -- 11.7 Hybrid Strategies 620 -- 11.8 Randomized Policies 626 -- 11.9 Illustration: An Energy Storage Model Revisited 627 -- 11.10 Choosing the Policy Class 631 -- 11.11 Policy Evaluation 641 -- 11.12 Parameter Tuning 642 -- 11.13 Bibliographic Notes 646 -- Exercises 646 -- Bibliography 651 -- Part IV - Policy Search 653 -- 12 Policy Function Approximations and Policy Search 655 -- 12.1 Policy Search as a Sequential Decision Problem 657 -- 12.2 Classes of Policy Function Approximations 658 -- 12.3 Problem Characteristics 665 -- 12.4 Flavors of Policy Search 666 -- 12.5 Policy Search with Numerical Derivatives 669 -- 12.6 Derivative-Free Methods for Policy Search 670 -- 12.7 Exact Derivatives for Continuous Sequential Problems* 677 -- 12.8 Exact Derivatives for Discrete Dynamic Programs** 680 -- 12.9 Supervised Learning 686 -- 12.10 Why Does it Work? 687 -- 12.11 Bibliographic Notes 690 -- Exercises 691 -- Bibliography 698 -- 13 Cost Function Approximations 701 -- 13.1 General Formulation for Parametric CFA 703 -- 13.2 Objective-Modified CFAs 704 -- 13.3 Constraint-Modified CFAs 714 -- 13.4 Bibliographic Notes 725 -- Exercises 726 -- Bibliography 729 -- Part V - Lookahead Policies 731 -- 14 Exact Dynamic Programming 737 -- 14.1 Discrete Dynamic Programming 738 -- 14.2 The Optimality Equations 740 -- 14.3 Finite Horizon Problems 747 -- 14.4 Continuous Problems with Exact Solutions 750 -- 14.5 Infinite Horizon Problems* 755 -- 14.6 Value Iteration for Infinite Horizon Problems* 757 -- 14.7 Policy Iteration for Infinite Horizon Problems* 762 -- 14.8 Hybrid Value-Policy Iteration* 764 -- 14.9 Average Reward Dynamic Programming* 765 -- 14.10 The Linear Programming Method for Dynamic Programs** 766 -- 14.11 Linear Quadratic Regulation 767 -- 14.12 Why Does it Work?** 770 -- 14.13 Bibliographic Notes 783 -- Exercises 783 -- Bibliography 793 -- 15 Backward Approximate Dynamic Programming 795 -- 15.1 Backward Approximate Dynamic Programming for Finite Horizon Problems 797 -- 15.2 Fitted Value Iteration for Infinite Horizon Problems 804 -- 15.3 Value Function Approximation Strategies 805 -- 15.4 Computational Observations 810 -- 15.5 Bibliographic Notes 816 -- Exercises 816 -- Bibliography 821 -- 16 Forward ADP I: The Value of a Policy 823 -- 16.1 Sampling the Value of a Policy 824 -- 16.2 Stochastic Approximation Methods 835 -- 16.3 Bellman's Equation Using a Linear Model* 837 -- 16.4 Analysis of TD(0), LSTD, and LSPE Using a Single State* 842 -- 16.5 Gradient-based Methods for Approximate Value Iteration* 845 -- 16.6 Value Function Approximations Based on Bayesian Learning* 852 -- 16.7 Learning Algorithms and Atepsizes 855 -- 16.8 Bibliographic Notes 860 -- Exercises 862 -- Bibliography 864 -- 17 Forward ADP II: Policy Optimization 867 -- 17.1 Overview of Algorithmic Strategies 869 -- 17.2 Approximate Value Iteration and Q-Learning Using Lookup Tables 871 -- 17.3 Styles of Learning 881 -- 17.4 Approximate Value Iteration Using Linear Models 886 -- 17.5 On-policy vs. off-policy learning and the exploration-exploitation problem 888 -- 17.6 Applications 894 -- 17.7 Approximate Policy Iteration 900 -- 17.8 The Actor-Critic Paradigm 907 -- 17.9 Statistical Bias in the Max Operator* 909 -- 17.10 The Linear Programming Method Using Linear Models* 912 -- 17.11 Finite Horizon Approximations for Steady-State Applications 915 -- 17.12 Bibliographic Notes 917 -- Exercises 918 -- Bibliography 924 -- 18 Forward ADP III: Convex Resource Allocation Problems 927 -- 18.1 Resource Allocation Problems 930 -- 18.2 Values Versus Marginal Values 937 -- 18.3 Piecewise Linear Approximations for Scalar Functions 938 -- 18.4 Regression Methods 941 -- 18.5 Separable Piecewise Linear Approximations 944 -- 18.6 Benders Decomposition for Nonseparable Approximations** 946 -- 18.7 Linear Approximations for High-Dimensional Applications 956 -- 18.8 Resource Allocation with Exogenous Information State 958 -- 18.9 Closing Notes 959 -- 18.10 Bibliographic Notes 960 -- Exercises 962 -- Bibliography 967 -- 19 Direct Lookahead Policies 971 -- 19.1 Optimal Policies Using Lookahead Models 974 -- 19.2 Creating an Approximate Lookahead Model 978 -- 19.3 Modified Objectives in Lookahead Models 985 -- 19.4 Evaluating DLA Policies 992 -- 19.5 Why Use a DLA? 997 -- 19.6 Deterministic Lookaheads 999 -- 19.7 A Tour of Stochastic Lookahead Policies 1005 -- 19.8 Monte Carlo Tree Search for Discrete Decisions 1009 -- 19.9 Two-Stage Stochastic Programming for Vector Decisions* 1018 -- 19.10 Observations on DLA Policies 1024 -- 19.11 Bibliographic Notes 1025 -- Exercises 1027 -- Bibliography 1031 -- Part VI - Multiagent Systems 1033 -- 20 Multiagent Modeling and Learning 1035 -- 20.1 Overview of Multiagent Systems 1036 -- 20.2 A Learning Problem - Flu Mitigation 1044 -- 20.3 The POMDP Perspective* 1059 -- 20.4 The Two-Agent Newsvendor Problem 1062 -- 20.5 Multipl ...
520 $a "The first step in sequential decision problems is to understand what decisions are being made. It is surprising how often it is that people faced with complex problems, which spans scientists in a lab to people trying to solve major health problems, are not able to identify the decisions they face. We then want to find a method for making decisions. There are at least 45 words in the English language that are equivalent to "method for making a decision," but the one we have settled on is policy. The term policy is very familiar to fields such as Markov decision processes and reinforcement learning, but with a much narrower interpretation than we will use. Other fields do not use the term at all. Designing effective policies will be the focus of most of this book. Even more subtle is identifying the different sources of uncertainty. It can be hard enough trying to identify potential decisions, but thinking about all the random events that might affect whatever it is that you are managing, whether it is reducing disease, managing inventories, or making investments, can seem like a hopeless challenge"-- $c Provided by publisher.
588 $a Description based upon online resource; title from PDF title page (viewed April 27, 2022).
650 0 $a Reinforcement learning. $3 349131
650 0 $a Decision making $x Statistical methods. $3 208497
650 0 $a Mathematical optimization. $3 183292
650 0 $a Stochastic analysis. $3 183332
650 6 $a Apprentissage par renforcement (Intelligence artificielle) $3 916275
650 6 $a Prise de dècision $x Mèthodes statistiques. $3 916375
650 6 $a Optimisation mathèmatique. $3 916376
650 6 $a Analyse stochastique. $3 916277
655 4 $a Electronic books. $2 local. $3 214472
776 0 8 $i Print version: $a Powell, Warren B., 1955- $t Reinforcement learning and stochastic optimization $b First Edition. $d Hoboken, New Jersey : John Wiley & Sons, Inc, [2022] $z 9781119815037 $w (DLC) 2021060404
856 4 0 $3 John Wiley $u http://dx.doi.org/10.1002/9781119815068
856 4 0 $3 O'Reilly Media $u https://www.oreilly.com/library/view/-/9781119815037/
856 4 0 $3 IEEE Xplore $u https://ieeexplore.ieee.org/book/9820922
856 4 $3 Image $u https://www.oreilly.com/covers/urn:orm:book:9781119815037/
938 $a YBP Library Services $b YANK $n 302795454
994 $a C0 $b TWNUK