Probability Density Function

Definition (Probability Density Function): For a continuous r.v. $X$ with CDF $F$ , the probability density function (PDF) of $X$ is the derivative $f$ of the $F$

P (a \leq X \leq b) = \int_{a}^{b} f_{X} (x) d x = F (b) - F (a)

$P(a\leq X \leq b) = \int_a^b f_X(x)dx = F(b) - F(a)$

P (X \in [x, x + δ]) \approx f_{X} (x) \cdot δ

$P(X\in [x,x+\delta]) \approx f_X(x) \cdot \delta$

Relation between PDF & PMF: The PDF is the analogous to the PMF in many ways. But, the PDF $f(x)$ is not a probability.

Relation between PDF & CDF: Let $X$ be a continuous r.v. with PDF $f$ . Then the CDF of $X$ is given by

F (x) = \int_{- \infty}^{x} f (x) d t

$F(x) = \int_{-\infty}^x f(x) dt$

CDF of Logistic Distribution

F (x) = \frac{e^{x}}{1 + e^{x}}, x \in R

$F(x) = \frac{e^x}{1+e^x}, x\in R$

CDF of Rayleigh Distribution

F (x) = 1 - e^{- x^{2} / 2}, x > 0

$F(x) = 1 - e^{-x^2/2}, x>0$

Theorem (Expectation of Continuous R.V.)

E (X) = \int_{- \infty}^{\infty} x f (x) d x

$E(X) = \int_{-\infty}^{\infty} x f(x) dx$

Theorem (Expectation via Survial Function) Let $G$ be the survial function of $X$ , Then

E (X) = \int_{0}^{\infty} G (x) d x

$E(X) = \int_0^{\infty} G(x) dx$

Theorem (LOTUS: Continuous)

E (g (X)) = \int_{- \infty}^{\infty} g (x) f (x) d x

$E(g(X)) = \int_{-\infty}^{\infty} g(x) f(x) dx$

Uniform Distribution

Definition (Uniform Distribution) Distribution on the interval $(a,b)$ , and its PDF is

f (x) = {\begin{cases} \frac{1}{b - a} & i f a < x < b \\ 0 & o t h e r w i s e \end{cases}

$f(x) = \left \{ \begin{array}{ll} \frac{1}{b-a} & if\ a<x<b \\ 0 & otherwise \end{array} \right.$

We denote this by $U\sim Unif(a,b)$

Example: suppose $X_1,X_2,…,X_n$ are i.i.d $Unif(0,1)$ random variable and let $Y = min(X_1,X_2,…,X_n)$ be their minimum. Find $E(Y)$

If $0<x<1$ , we have $P(X_i \leq x) = x$ , it follows that

\begin{aligned} P (Y > x) & = P (m i n (X_{1}, . . ., X_{n}) > x) \\ = P (X_{1} > x, . . ., X_{n} > x) \\ = \prod_{i = 1}^{n} P (X_{i} > x) = (1 - x)^{n} \end{aligned}

$\begin{align} P(Y>x)&= P(min (X_1,...,X_n)>x) \\ &= P(X_1>x,...,X_n>x)\\ &= \prod_{i=1}^n P(X_i > x) = (1-x)^n \end{align}$

From above, we use the survial function to calculate the expectation

E (Y) = \int_{0}^{\infty} P (Y > x) d x = \int_{0}^{1} (1 - x)^{n} d x = \int_{0}^{1} x^{n} d x = \frac{1}{n + 1}

$E(Y) = \int_0^{\infty} P(Y>x) dx = \int_0^1(1-x)^ndx = \int_0^1 x^n dx = \frac{1}{n+1}$

Normal

Definition (Standard Normal Distribution) A c.r.v. $Z$ is said to have the standard Normal Distribution if its $PDF$ $\varphi$ is given by:

φ (z) = \frac{1}{\sqrt{(} 2 π)} e^{- z^{2} / 2}, - \infty < z < \infty

$\varphi(z) = \frac{1}{\sqrt(2\pi)} e^{-z^2/2},-\infty < z < \infty$

We write this as $Z\sim N(0,1)$ , and $Z$ has mean 0 and variance 1, the CDF $\phi$ is

ϕ (z) = \int_{- \infty}^{z} φ (t) d t = \int_{- \infty}^{z} \frac{1}{\sqrt{2 π}} e^{- t^{2} / 2} d t

$\phi(z) = \int_{-\infty}^z \varphi(t) dt = \int_{-\infty}^z \frac{1}{\sqrt{2\pi}}e^{-t^2/2}dt$

Definition (Normal Distribution) If $Z\sim N(0,1)$ then

X = μ + σ Z

$X = \mu + \sigma Z$

is said to have the Normal Distribution with mean $\mu$ and variance $\sigma^2$ , denote this by $X\sim N(\mu,\sigma^2)$

Theorem (Normal CDF and PDF) Let $X\sim N(\mu, \sigma^2)$ ,
CDF of $X$ is

F (x) = ϕ (\frac{x - μ}{σ})

$F(x) = \phi(\frac{x-\mu}{\sigma})$

PDF of $X$ is

f (x) = φ (\frac{x - μ}{σ}) \frac{1}{σ}

$f(x) = \varphi (\frac{x-\mu}{\sigma})\frac{1}{\sigma}$

Exponential

Definition (Exponential Distribution)

f (x) = λ e^{- λ x}, x > 0

$f(x) = \lambda e ^{-\lambda x}, x>0$

we denote this by $X\sim Expo(\lambda)$ . The corresponding CDF is

F (x) = 1 - e^{- λ x}, x > 0

$F(x) = 1-e^{-\lambda x},x>0$

Theorem (Memoryless Property)

P (X \geq s + t | X \geq s) = P (X \geq t)

$P(X\geq s+t | X\geq s) = P(X\geq t)$

If $X$ is a positive continuous random variable with memoryless property, then $X$ has an Exponential distribution
Geometric Distribution is also Memoryless
Exponential distribution as the “continuous counterpart” of the Geometric distribution

Exponential \& Geometric via $\delta$ - Steps
We devide a unit of time into n pieces, each of size $\delta = \frac{1}{n}$ , and the trial occurs every $\delta$ time period and success with probability $\lambda \delta$ . Denote $Y$ as the number of trials until first success, $\hat{Y}$ as the time until first success under $Y$ .

Y \sim F S (λ δ)

$Y\sim FS(\lambda \delta)$

Thus we have

F (\hat{Y}) = E (Y) \cdot δ = \frac{1}{λ δ} \cdot δ = \frac{1}{λ}

$F(\hat{Y}) = E(Y)\cdot \delta = \frac{1}{\lambda\delta}\cdot \delta=\frac{1}{\lambda}$

And

\begin{matrix} (Y > t) & = P {a l l t r i a l s u p t o t i m e t h a s b e e n f a i l u r e s} \\ = P {a t l e a s t \frac{t}{δ} f a i l u r e s} \\ = (1 - p)^{\frac{t}{δ}} = (1 - λ δ)^{\frac{t}{δ}} \\ = {[(1 - λ δ)^{\frac{1}{λ δ}}]}^{λ t} \overset{δ \to 0}{\to} e^{- λ t} \end{matrix}

$\begin{array} P(Y>t) &= P\{ all\ trials\ up\ to\ time\ t\ has\ been\ failures \} \\ &=P\{ at\ least\ \frac{t}{\delta}\ failures\}\\ &=(1-p)^{\frac{t}{\delta}}=(1-\lambda \delta)^{\frac{t}{\delta}}\\ &=\left[ (1-\lambda\delta)^{\frac{1}{\lambda\delta}} \right]^{\lambda t} \xrightarrow{\delta \rightarrow 0} e^{-\lambda t} \end{array}$

Theorem property of Exponential Given $X_1\sim Expo(\lambda_1)$ , $X_2\sim Expo(\lambda_2)$ , $X_1 \bot X_2$ ( $X_1$ and $X_2$ are independent), then

P (X_{1} < X_{2}) = \frac{λ_{1}}{λ_{1} + λ_{2}}

$P(X_1<X_2) = \frac{\lambda_1}{\lambda_1 + \lambda_2}$

It can be solved by $\delta$ -step

Some physical phenomenon follow the Exponential distribution like the radioactive decay.

Stochastic-Process (Continuous Random Variable)

2018-07-25

Stochastic-Process (Continuous Random Variable)

Probability Density Function

Uniform Distribution

Normal

Exponential

Stochastic-Process (Continuous Random Variable)

Probability Density Function

Uniform Distribution

Normal

Exponential

thanks~