PREFACE

1 PROBABILITY BASICS
1.1 What Is Probability?
1.2 Experiments, Outcomes, and Events
1.3 Venn Diagrams
1.4 Random Variables
1.5 Basic Probability Rules
1.6 Probability Formalized
1.7 Simple Theorems
1.8 Compound Experiments
1.9 Independence
1.10 Example: Can S Communicate With D?
1.10.1 List All Outcomes
1.10.2 Probability of a Union
1.10.3 Probability of the Complement
1.11 Example: Now Can S Communicate With D?
1.11.1 A Big Table
1.11.2 Break Into Pieces
1.11.3 Probability of the Complement
1.12 Computational Procedures
Summary
Problems

2 CONDITIONAL PROBABILITY
2.1 Definitions of Conditional Probability
2.2 Law of Total Probability and Bayes Theorem
2.3 Example: Urn Models
2.4 Example: A Binary Channel
2.5 Example: Drug Testing
2.6 Example: A Diamond Network
Summary
Problems

3 A LITTLE COMBINATORICS
3.1 Basics of Counting
3.2 Notes on Computation
3.3 Combinations and the Binomial Coefficients
3.4 The Binomial Theorem
3.5 Multinomial Coefficient and Theorem
3.6 The Birthday Paradox and Message Authentication
3.7 Hypergeometric Probabilities and Card Games
Summary
Problems

4 DISCRETE PROBABILITIES AND RANDOM VARIABLES
4.1 Discrete Random Variable and Probability Mass Functions
4.2 Cumulative Distribution Functions
4.3 Expected Values
4.4 Moment Generating Functions
4.5 Several Important Discrete PMFs
4.5.1 Uniform PMF
4.5.2 Geometric PMF
4.5.3 The Poisson Distribution
4.6 Gambling and Financial Decision Making
Summary
Problems

5 MULTIPLE DISCRETE RANDOM VARIABLES
5.1 Multiple Random Variables and PMFs
5.2 Independence
5.3 Moments and Expected Values
5.3.1 Expected Values for Two Random Variables
5.3.2 Moments for Two Random Variables
5.4 Example: Two Discrete Random Variables
5.4.1 Marginal PMFs and Expected Values
5.4.2 Independence
5.4.3 Joint CDF
5.4.4 Transformations With One Output
5.4.5 Transformations With Several Outputs
5.4.6 Discussion
5.5 Sums of Independent Random Variables
5.6 Sample Probabilities, Mean, and Variance
5.7 Histograms
5.8 Entropy and Data Compression
5.8.1 Entropy and Information Theory
5.8.2 Variable Length Coding
5.8.3 Encoding Binary Sequences
5.8.4 Maximum Entropy
Summary
Problems
6 BINOMIAL PROBABILITIES
6.1 Basics of the Binomial Distribution
6.2 Computing Binomial Probabilities
6.3 Moments of the Binomial Distribution
6.4 Sums of Independent Binomial Random Variables
6.5 Distributions Related to the Binomial
6.5.1 Connections Between Binomial and Hypergeometric Probabilities
6.5.2 Multinomial Probabilities
6.5.3 The Negative Binomial Distribution
6.5.4 The Poisson Distribution
6.6 Parameter Estimation for Binomial and Multinomial Distributions
6.7 Alohanet
6.8 Error Control Codes
6.8.1 Repetition-by-Three Code
6.8.2 General Linear Block Codes
6.8.3 Conclusions
Summary
Problems

7 A CONTINUOUS RANDOM VARIABLE
7.1 A Continuous Random Variable and Its Density, Distribution Function, and Expected Values
7.2 Example Calculations for a Single Random Variable
7.3 Selected Continuous Distributions
7.3.1 The Uniform Distribution
7.3.2 The Exponential Distribution
7.4 Conditional Probabilities for a Continuous Random Variable
7.5 Discrete PMFs and Delta Functions
7.6 Quantization
7.7 A Final Word
Summary
Problems

8 MULTIPLE CONTINUOUS RANDOM VARIABLES
8.1 Joint Densities and Distribution Functions
8.2 Expected Values and Moments
8.3 Independence
8.4 Conditional Probabilities for Multiple Random Variables
8.5 Extended Example: Two Continuous Random Variables
8.6 Sums of Independent Random Variables
8.7 Random Sums
8.8 General Transformations and the Jacobian
8.9 Parameter Estimation for the Exponential Distribution
8.10 Comparison of Discrete and Continuous Distributions
Summary
Problems

9 THE GAUSSIAN AND RELATED DISTRIBUTIONS
9.1 The Gaussian Distribution and Density
9.2 Quantile Function
9.3 Moments of the Gaussian Distribution
9.4 The Central Limit Theorem
9.5 Related Distributions
9.5.1 The Laplace Distribution
9.5.2 The Rayleigh Distribution
9.5.3 The Chi-Squared and F Distributions
9.6 Multiple Gaussian Random Variables
9.6.1 Independent Gaussian Random Variables
9.6.2 Transformation to Polar Coordinates
9.6.3 Two Correlated Gaussian Random Variables
9.7 Example: Digital Communications Using QAM
9.7.1 Background
9.7.2 Discrete Time Model
9.7.3 Monte Carlo Exercise
9.7.4 QAM Recap
Summary
Problems

10 ELEMENTS OF STATISTICS
10.1 A Simple Election Poll
10.2 Estimating the Mean and Variance
10.3 Recursive Calculation of the Sample Mean
10.4 Exponential Weighting
10.5 Order Statistics and Robust Estimates
10.6 Estimating the Distribution Function
10.7 PMF and Density Estimates
10.8 Confidence Intervals
10.9 Significance Tests and p-Values
10.10 Introduction to Estimation Theory
10.11 Minimum Mean Squared Error Estimation
10.12 Bayesian Estimation
Summary
Problems

11 GAUSSIAN RANDOM VECTORS AND LINEAR REGRESSION
11.1 Gaussian Random Vectors
11.2 Linear Operations on Gaussian Random Vectors
11.3 Linear Regression
11.3.1 Linear Regression in Detail
11.3.2 Statistics of the Linear Regression Estimates
11.3.3 Computational Issues
11.3.4 Linear Regression Examples
11.3.5 Extensions of Linear Regression
Summary
Problems

12 HYPOTHESIS TESTING
12.1 Hypothesis Testing: Basic Principles
12.2 Example: Radar Detection
12.3 Hypothesis Tests and Likelihood Ratios
12.4 MAP Tests
Summary
Problems

13 RANDOM SIGNALS AND NOISE
13.1 Introduction to Random Signals
13.2 A Simple Random Process
13.3 Fourier Transforms
13.4 WSS Random Processes
13.5 WSS Signals and Linear Filters
13.6 Noise
13.6.1 Probabilistic Properties of Noise
13.6.2 Spectral Properties of Noise
13.7 Example: Amplitude Modulation
13.8 Example: Discrete Time Wiener Filter
13.9 The Sampling Theorem for WSS Random Processes
13.9.1 Discussion
13.9.2 Example: Figure 13.4
13.9.3 Proof of the Random Sampling Theorem
Summary
Problems

14 SELECTED RANDOM PROCESSES
14.1 The Lightbulb Process
14.2 The Poisson Process
14.3 Markov Chains
14.4 Kalman Filter
14.4.1 The Optimal Filter and Example
14.4.2 QR Method Applied to the Kalman Filter
Summary
Problems
A COMPUTATION EXAMPLES
A.1 Matlab
A.2 Python
A.3 R
B ACRONYMS
C PROBABILITY TABLES
C.1 Tables of Gaussian Probabilities
D BIBLIOGRAPHY
INDEX