There are some notation occationals

Definition (Discrete Random Variable) A variable $X$ is discrete if there is a finite list of value $a_1,a_2,…,a_n$ that $P(X=a_j) = 1$, $P(X=x)>0$ is the support of $X$

Definition (Probability Mass Function) The probability mass function (PMF) of a discrete r.v. $X$ is the function $p_X$ given by $p_X(x) = P(X=x)$.

Bernoulli & Binomial

Definition (Bernoulli Distribution) shortly, $P(X=1)=p$ and $P(X=0) = 1 - p$, and write as $X\sim Bern(p)$

Definition (Indicator Random Variable) The indicator random variable of an event $A$ is the r.v. equals 1 if $A$ occurs and 0 otherwise, We denote the indicator of $A$ by $I_A$ or $I(A)$. Note $I_A \sim Bern(p)$ with p = P(A)

Theorem (Binomial PMF) Binomial Distribution is the repeatation of Bernoulli Distribution. If $X\sim Bin(n, p)$ then the PMF of X is

$$P(X=k) = \left( \begin{array}{c} n \\ k \end{array} \right) p^k (1-p)^{n-k}$$

Hypergeometric

urn Model A box is fiiled with $w$ white and $b$ black balls, then drawing n balls

• With replacement: $Bin(n,w/(w+b))$ for the number of white balls
• Without replacement : Hypergeometric distribution $HGeom(w,b,n)$

Theorem (Hypergeometric PMF) If $X \sim HGeom(w,b,n)$, then the PMF of $X$ is

$$P(X=k) = \frac {\left ( \begin{array}{c} w \\ k \end{array} \right ) \left( \begin{array}{c} b \\ n-k \end{array} \right)} {\left( \begin{array}{c} w+b \\ n \end{array} \right)}$$

Zipf Distribution If $X\sim Zipf(\alpha > 0)$, then PMF of $X$ is:

$$P(X=k) = \frac{\frac{1}{k^{\alpha + 1}}} {\sum_{j=1}^{\infty}(\frac{1}{j})^{\alpha + 1}}$$

• Zipf Distribution can measure the Word Frequency

Cumulative Distribution Functions

Definition (Cumulative Distribution Function) The cumulative distribution function(CDF) os an r.v. $X$ is the function $F_X$ given by $F_X(x) = P(X\leq x)$

Theorem (Valid CDFs) CDF has the following properties

• Increasing: If $x_1 < x_2$, then $F(x_1) < F(x_2)$
• Right-Continuous: $F(a) = lim_{x\rightarrow a^+} F(x)$
• Convergence to $0$ and $1$: $lim_{x\rightarrow - \infty} F(x) = 0$ and $lim_{x \rightarrow \infty} F(x) = 1$

Functions of Random Variable:

Definition (Function of an r.v.) An experiment with sample space S, an r.v. $X$, and a function $g$, also the $g(X)$ is the variable that maps $s$ to $g(X(s))$, for all $s\in S$

Theorem (PMF of $g(X)$) for all y in the support of $g(X)$

$$P(g(X) = y) = \sum_{x:g(x)=y} P(X=x)$$

The function of r.v. map the sample space into real number, which is easy for us calculate in mathematic.

Independence of R.V.s

Definition (Independence of two R.V.s) Random variables $X$ and $Y$ are said to be independent

$$P(X\leq x,Y\leq y) = P(X\leq x)P(Y\leq y)$$

for all $x,y\in R$,
In the discrete case, equivalent to :

$$P(X=x,Y=y) = P(X=x)P(Y=y)$$

Definition (Independence of many R.V.s) Random variables $X_1,…,X_n$ are independent if

$$P(X_1 \leq x_1,\dotsb , X_n \leq x_n) = P(X_1 \leq x_1) \dotsb P(X_n \leq x_n)$$

for all $x_1,\dotsb,x_n \in R$

Definition (i.i.d) We call some r.v. that are independent and have the same distribution independent and identicallly distributed or i.i.d for short

• Independent: r.v.s provide no information about each others
• Identically distributed: r.v.s have the same PMF

Theorem If $X\sim Bin(n,p)$ , $Y \sim Bin(m,p)$, and $X$ is independent of $Y$, then $X+Y \sim Bin(n+m,p)$

Definition (Conditional Independence of two R.V.s)

$$P(X\leq x, Y \leq y| Z= z) = P(X\leq x |Z =z) P(Y\leq y|Z=z)$$

w