Probability Density Function

Definition (Probability Density Function): For a continuous r.v. $X$ with CDF $F$, the probability density function (PDF) of $X$ is the derivative $f$ of the $F$

$$P(a\leq X \leq b) = \int_a^b f_X(x)dx = F(b) - F(a)$$ $$P(X\in [x,x+\delta]) \approx f_X(x) \cdot \delta$$

Relation between PDF & PMF: The PDF is the analogous to the PMF in many ways. But, the PDF $f(x)$ is not a probability.

Relation between PDF & CDF: Let $X$ be a continuous r.v. with PDF $f$. Then the CDF of $X$ is given by

$$F(x) = \int_{-\infty}^x f(x) dt$$

CDF of Logistic Distribution

$$F(x) = \frac{e^x}{1+e^x}, x\in R$$

CDF of Rayleigh Distribution

$$F(x) = 1 - e^{-x^2/2}, x>0$$

Theorem (Expectation of Continuous R.V.)

$$E(X) = \int_{-\infty}^{\infty} x f(x) dx$$

Theorem (Expectation via Survial Function) Let $G$ be the survial function of $X$, Then

$$E(X) = \int_0^{\infty} G(x) dx$$

Theorem (LOTUS: Continuous)

$$E(g(X)) = \int_{-\infty}^{\infty} g(x) f(x) dx$$

---

Uniform Distribution

Definition (Uniform Distribution) Distribution on the interval$(a,b)$, and its PDF is

$$f(x) = \left \{ \begin{array}{ll} \frac{1}{b-a} & if\ a<x<b \\ 0 & otherwise \end{array} \right.$$

We denote this by $U\sim Unif(a,b)$

Example: suppose $X_1,X_2,…,X_n$ are i.i.d $Unif(0,1)$ random variable and let $Y = min(X_1,X_2,…,X_n)$ be their minimum. Find $E(Y)$

If $0<x<1$, we have $P(X_i \leq x) = x$, it follows that

$$\begin{align} P(Y>x)&= P(min (X_1,...,X_n)>x) \\ &= P(X_1>x,...,X_n>x)\\ &= \prod_{i=1}^n P(X_i > x) = (1-x)^n \end{align}$$

From above, we use the survial function to calculate the expectation

$$E(Y) = \int_0^{\infty} P(Y>x) dx = \int_0^1(1-x)^ndx = \int_0^1 x^n dx = \frac{1}{n+1}$$

---

Normal

Definition (Standard Normal Distribution) A c.r.v. $Z$ is said to have the standard Normal Distribution if its $PDF$ $\varphi$ is given by:

$$\varphi(z) = \frac{1}{\sqrt(2\pi)} e^{-z^2/2},-\infty < z < \infty$$

We write this as $Z\sim N(0,1)$, and $Z$ has mean 0 and variance 1, the CDF $\phi$ is

$$\phi(z) = \int_{-\infty}^z \varphi(t) dt = \int_{-\infty}^z \frac{1}{\sqrt{2\pi}}e^{-t^2/2}dt$$

Definition (Normal Distribution) If $Z\sim N(0,1)$ then

$$X = \mu + \sigma Z$$

is said to have the Normal Distribution with mean $\mu$ and variance $\sigma^2$, denote this by $X\sim N(\mu,\sigma^2)$

Theorem (Normal CDF and PDF) Let $X\sim N(\mu, \sigma^2)$,
CDF of $X$ is

$$F(x) = \phi(\frac{x-\mu}{\sigma})$$

PDF of $X$ is

$$f(x) = \varphi (\frac{x-\mu}{\sigma})\frac{1}{\sigma}$$

---

Exponential

Definition (Exponential Distribution)

$$f(x) = \lambda e ^{-\lambda x}, x>0$$

we denote this by $X\sim Expo(\lambda)$. The corresponding CDF is

$$F(x) = 1-e^{-\lambda x},x>0$$

Theorem (Memoryless Property)

$$P(X\geq s+t | X\geq s) = P(X\geq t)$$

• If $X$ is a positive continuous random variable with memoryless property, then $X$ has an Exponential distribution
• Geometric Distribution is also Memoryless
• Exponential distribution as the “continuous counterpart” of the Geometric distribution

Exponential \& Geometric via $\delta$- Steps
We devide a unit of time into n pieces, each of size $\delta = \frac{1}{n}$, and the trial occurs every $\delta$ time period and success with probability $\lambda \delta$. Denote $Y$ as the number of trials until first success, $\hat{Y}$ as the time until first success under $Y$.

$$Y\sim FS(\lambda \delta)$$

Thus we have

$$F(\hat{Y}) = E(Y)\cdot \delta = \frac{1}{\lambda\delta}\cdot \delta=\frac{1}{\lambda}$$

And

$$\begin{array} P(Y>t) &= P\{ all\ trials\ up\ to\ time\ t\ has\ been\ failures \} \\ &=P\{ at\ least\ \frac{t}{\delta}\ failures\}\\ &=(1-p)^{\frac{t}{\delta}}=(1-\lambda \delta)^{\frac{t}{\delta}}\\ &=\left[ (1-\lambda\delta)^{\frac{1}{\lambda\delta}} \right]^{\lambda t} \xrightarrow{\delta \rightarrow 0} e^{-\lambda t} \end{array}$$

Theorem property of Exponential Given $X_1\sim Expo(\lambda_1)$, $X_2\sim Expo(\lambda_2)$, $X_1 \bot X_2$ ($X_1$ and $X_2$ are independent), then

$$P(X_1<X_2) = \frac{\lambda_1}{\lambda_1 + \lambda_2}$$

It can be solved by $\delta$-step

Some physical phenomenon follow the Exponential distribution like the radioactive decay.