Introduction to Probability Theory and Stochastic Processes

A unique approach to stochastic processes that connects the mathematical formulation of random processes to their use in applications This book presents an innovative approach to teaching probability theory and stochastic processes based on the binary expansion of the unit interval. Departing from standard pedagogy, it uses the binary expansion of the unit interval to explicitly construct an infinite sequence of independent random variables (of any given distribution) on a single probability space. This construction then provides the framework to understand the mathematical formulation of probability theory for its use in applications. Features include: The theory is presented first for countable sample spaces (Chapters 1-3) and then for uncountable sample spaces (Chapters 4-18) Coverage of the explicit construction of i.i.d. random variables on a single probability space to explain why it is the distribution function rather than the functional form of random variables that matters when it comes to modeling random phenomena Explicit construction of continuous random variables to facilitate the "digestion" of random variables, i.e., how they are used in contrast to how they are defined Explicit construction of continuous random variables to facilitate the two views of expectation: as integration over the underlying probability space (abstract view) or as integration using the density function (usual view) A discussion of the connections between Bernoulli, geometric, and Poisson processes Incorporation of the Johnson-Nyquist noise model and an explanation of why (and when) it is valid to use a delta function to model its autocovariance Comprehensive, astute, and practical, Introduction to Probability Theory and Stochastic Processes is a clear presentation of essential topics for those studying communications, control, machine learning, digital signal processing, computer networks, pattern recognition, image processing, and coding theory.

1 Coin Tossing 1 2 Countable Sample Spaces 61 3 Conditional Probability in Countable Sample Spaces 105 4 Uncountable Sample Spaces 151 5 Continuous Random Variables 213 6 Expectation 245 7 Modeling Random Phenomena 267 8 Functions of One Random Variables and Transforms 321 9 Functions of Two Random Variables 365 10 Two Functions of Two Random Variables 431 11 Conditional Probability for Continuous Random Variables 473 12 Random Vectors 549 13 Bernoulli, Geometric, and Poisson Processes 587 14 Brownian Motions and White Noise 645 15 Stationary Random Processes 703 16 Convergence of Random Variables 777 17 Statistics 839 18 Kalman Filter 905 Further Reading 933 Table of Common Distributions 935 References 941 Index 946