Preface xiii 1 A Signal Theoretic Introduction to Random Processes 1 1.1 Introduction 1 1.2 Motivation 2 1.3 Book Overview 8 2 Background: Mathematics 11 2.1 Introduction 11 2.2 Set Theory 11 2.3 Function Theory 13 2.4 Measure Theory 18 2.5 Measurable Functions 24 2.6 Lebesgue Integration 28 2.7 Convergence 37 2.8 Lebesgue-Stieltjes Measure 39 2.9 Lebesgue-Stieltjes Integration 50 2.10 Miscellaneous Results 61 2.11 Problems 62 3 Background: Signal Theory 71 3.1 Introduction 71 3.2 Signal Orthogonality 71 3.3 Theory for Dirichlet Points 75 3.4 Dirac Delta 78 3.5 Fourier Theory 79 3.6 Signal Power 82 3.7 The Power Spectral Density 84 3.8 The Autocorrelation Function 91 3.9 Power Spectral Density-Autocorrelation Function 95 3.10 Results for the Infinite Interval 96 3.11 Convergence of Fourier Coefficients 103 3.12 Cramer's Representation and Transform 106 3.13 Problems 125 4 Background: Probability and Random Variable Theory 153 4.1 Introduction 153 4.2 Basic Concepts: Experiments-Probability Theory 153 4.3 The Random Variable 160 4.4 Discrete and Continuous Random Variables 162 4.5 Standard Random Variables 165 4.6 Functions of a Random Variable 165 4.7 Expectation 166 4.8 Generation of Data Consistent with Defined PDF 172 4.9 Vector Random Variables 173 4.10 Pairs of Random Variables 175 4.11 Covariance and Correlation 186 4.12 Sums of Random Variables 191 4.13 Jointly Gaussian Random Variables 193 4.14 Stirling's Formula and Approximations to Binomial 194 4.15 Problems 199 5 Introduction to Random Processes 219 5.1 Random Processes 219 5.2 Definition of a Random Process 219 5.3 Examples of Random Processes 221 5.4 Experiments and Experimental Outcomes 225 5.5 Prototypical Experiments 228 5.6 Random Variables Defined by a Random Process 232 5.7 Classification of Random Processes 233 5.8 Classification: One-Dimensional RPs 236 5.9 Sums of Random Processes 239 5.10 Problems 239 6 Prototypical Random Processes 243 6.1 Introduction 243 6.2 Bernoulli Random Processes 243 6.3 Poisson Random Processes 246 6.4 Clustered Random Processes 255 6.5 Signalling Random Processes 257 6.6 Jitter 262 6.7 White Noise 265 6.8 1/f Noise 272 6.9 Birth-Death Random Processes 275 6.10 Orthogonal Increment Random Processes 278 6.11 Linear Filtering of Random Processes 282 6.12 Summary of Random Processes 283 6.13 Problems 285 7 Characterizing Random Processes 289 7.1 Introduction 289 7.2 Time Evolution of PMF or PDF 291 7.3 First-, Second-, and Higher-Order Characterization 292 7.4 Autocorrelation and Power Spectral Density 297 7.5 Correlation 308 7.6 Notes on Average Power and Average Energy 310 7.7 Classification: Stationarity vs Non-Stationarity 316 7.8 Cramer's Representation 323 7.9 State Space Characterization of Random Processes 335 7.10 Time Series Characterization 347 7.11 Problems 347 8 PMF and PDF Evolution 369 8.1 Introduction 369 8.2 Probability Mass/Density Function Estimation 370 8.3 Non/Semi-parametric PDF Estimation 372 8.4 PMF/PDF Evolution: Signal Plus Noise 378 8.5 PMF Evolution of a Random Walk 381 8.6 PDF Evolution: Brownian Motion 384 8.7 PDF Evolution: Signalling Random Process 388 8.8 PDF Evolution: Generalized Shot Noise 390 8.9 PDF Evolution: Switching in a CMOS Inverter 396 8.10 PDF Evolution: General Case 400 8.11 Problems 405 9 The Autocorrelation Function 417 9.1 Introduction 417 9.2 Notation and Definitions 417 9.3 Basic Results and Independence Information 419 9.4 Sinusoid with Random Amplitude and Phase 421 9.5 Random Telegraph Signal 423 9.6 Generalized Shot Noise 424 9.7 Signalling Random Process-Fixed Pulse Case 434 9.8 Generalized Signalling Random Process 441 9.9 Autocorrelation: Jittered Random Processes 453 9.10 Random Walk 456 9.11 Problems 457 10 Power Spectral Density Theory 481 10.1 Introduction 481 10.2 Power Spectral Density Theory 481 10.3 Power Spectral Density of a Periodic Pulse Train 485 10.4 PSD of a Signalling Random Process 487 10.5 Digital to Analogue Conversion 501 10.6 PSD of Shot Noise Random Processes 505 10.7 White Noise 509 10.8 1/f Noise 510 10.9 PSD of a Jittered Binary Random Process 513 10.10 PSD of a Jittered Pulse Train 517 10.11 Problems 525 11 Order Statistics 553 11.1 Introduction 553 11.2 Ordered Random Variable Theory 557 11.3 Identical RVs With Uniform Distribution 574 11.4 Uniform Distribution and Infinite Interval 584 11.5 Problems 590 12 Poisson Point Random Processes 621 12.1 Introduction 621 12.2 Characterizing Poisson Random Processes 623 12.3 PMF: Number of Points in a Subset of an Interval 625 12.4 Results From Order Statistics 630 12.5 Alternative Characterization for Infinite Interval 634 12.6 Modelling with Unordered or Ordered Times 636 12.7 Zero Crossing Times of Random Telegraph Signal 638 12.8 Point Processes: The General Case 639 12.9 Problems 639 13 Birth-Death Random Processes 649 13.1 Introduction 649 13.2 Defining and Characterizing Birth-Death Processes 649 13.3 Constant Birth Rate, Zero Death Rate Process 656 13.4 State Dependent Birth Rate - Zero Death Rate 662 13.5 Constant Death Rate, Zero Birth Rate, Process 665 13.6 Constant Birth and Constant Death Rate Process 667 13.7 Problems 669 14 The First Passage Time 677 14.1 Introduction 677 14.2 First Passage Time 677 14.3 Approaches: Establishing the First Passage Time 681 14.4 Maximum Level and the First Passage Time 685 14.5 Solutions for the First Passage Time PDF 690 14.6 Problems 695 Reference Material 709 References 717 Index 721