Elements of Applied Stochastic Processes

Transcription

1 Elements of Applied Stochastic Processes Third Edition U. NARAYAN BHAT Southern Methodist University GREGORY K. MILLER Stephen E Austin State University,WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION

2 Contents Preface ix 1 STOCHASTIC PROCESSES: Description and Definition Introduction Description and Definition Probability Distributions The Markov Process The Renewal Process The Stationary Process A Plan for the Remaining Chapters 14 References 14 Exercises 16 Elementary Review Exercises 18 Advanced Review Exercises 22 2 MARKOV CHAINS Introduction The n-step Transition Probability Matrix Classification of States ' A Canonical Representation of the Transition Probability Matrix Classification of States in Practice Finite Markov Chains with Transient States 45

3 vl References 54 Exercises 54 3 IRREDUCIBLE MARKOV CHAINS WITH ERGODIC STATES Transient Behavior Limiting Behavior First Passage and Related Results 86 References 96 Exercises 97 4 BRANCHING PROCESSES AND OTHER SPECIAL TOPICS Branching Processes Markov Chains of Order Higher than Lumpable Markov Chains Reversed Markov Chains 123 References 125 Exercises ; ' STATISTICAL INFERENCE FOR MARKOV CHAINS Estimation of the Elements in a Transition Probability Matrix Hypothesis Testing Issues for Markov Chains Inference From Partially Observable Markov Chains Statistical Inference for Branching Processes Additional Comments 146 References 147 Exercises APPLIED MARKOV CHAINS Queueing Models Inventory Systems Storage Models Industrial Mobility of Labor Educational Advancement Human Resource Management Term Structure Income Determination under Uncertainty A Markov Decision Process 185 References SIMPLE MARKOV PROCESSES ' Examples Markov Processes: General Properties The Poisson Process The Pure Birth Process 209

4 Vii 7.5 The Pure Death Process Birth and Death Processes Limiting Distributions Markovian Networks Additional Examples 231 References 235 Exercises STATISTICAL INFERENCE FOR SIMPLE MARKOV PROCESSES Estimation of Parameters Hypothesis Testing for Simple Markov Processes Statistical Inference for Queues Additional Examples 263 References 266 Exercises APPLIED MARKOV PROCESSES Queueing Models jl The Machine Interference Problem Queueing Networks Flexible Manufacturing Systems Inventory Systems Reliability Models Markovian Combat Models Stochastic Models for Social Networks Recovery, Relapse, and Death Due to Disease 320 References RENEWAL PROCESSES Introduction Renewal Processes when Time is Discrete Renewal Processes when Time is Continuous Alternating Renewal Processes Markov Renewal Processes (Semi-Markov Processes) Renewal Reward Processes Statistical Inference for Renewal Processes Additional Examples 353 References 357 Exercises STATIONARY PROCESSES AND TIME SERIES ANALYSIS Definition Some Examples Ergodic Theorems 378

5 Viii 11.4 Covariance Stationary Processes in the Frequency Domain Time Series Analysis: Introduction Stochastic Models for Time Series The Autoregressive Process The Moving Average Process A Mixed Autoregressive Moving Average Process Autoregressive Integrated Moving Average Processes Time Series Analysis in the Time Domain Spectral Analysis of Time Series Data 413 References 417 Exercises SIMULATION AND MARKOV CHAIN MONTE CARLO Introduction Simulation Markov Chain Monte Carlo 426 References y 430 Answers to Selected Exercises 433 Appendix 443 Author Index 451 Subject Index 455