Statistics II Lesson 0. Review of basic concepts

Statistics IILesson 0. Review of basic concepts

Year 2009/10

Lesson 0. Review of basic concepts

Contents

I Random variables and probability distributions

I The normal distribution

I Random samples, statistics and sampling distributions

I The distribution of the sample mean

I The central limit theorem

Lesson 0. Review of basic concepts

References in the bibliography

I Meyer, P. “Probabilidad y aplicaciones estadısticas”(1992)I Chapters 4, 9, 12 y 13

I Newbold, P. “Estadıstica para los negocios y la economıa”(1997)I Chapters 4, 5 y 6

Random variables

I Random experiment: a process with different possible outcomes, andwhere the specific outcome for a given instance is uncertain

I Random variable: variable that takes (numerical) values associatedto the outcomes of a random experiment

I Discrete random variable: one that can take a finite or countablenumber of different values

I Continuous random variable: one that can take an infiniteuncountable number of values (for example, values in an interval ofthe real line)

I Notation: we will represent the random variables using capitalletters, X , and their values using lowercase letters, x1.

Examples:

I The number of unemployed persons at a given time in a given region

I The profits of a firm in a given period

I The changes in the stock price of a given company during a week

Probability distributions

I We wish to associate probabilities to the values of a random variable

I We define functions (probability, distribution, density functions)associated to the probabilities of values or sets of values of a randomvariable

I These functions are used to classify the random variables

Discrete random variables

I For a discrete random variable X taking values {x1, x2, . . .}, wedefine its probability function or mass function as

pi = P[X = xi ], for i = 1, 2, . . .

I The cumulative probability function is defined as

F (x0) = P[X ≤ x0] =∑

i :xi≤x0

pi .


Properties

I 0 ≤ P[X = xi ] ≤ 1

I F (∞) =∑

i P[X = xi ] = 1

I F (y) ≤ F (x), ∀y ≤ x

I P[X > x ] = 1− P[X ≤ x ] = 1− F (x)

I E[X ] =∑

i xipi

I Var[X ] =∑

i (xi − E[X ])2pi =∑

i x2i pi − E[X ]2


Exercise 0.1A discrete random variable X takes values with the probabilities indicatedin the table

Value 0 1 2 3Probability 0.1 0.3 0.2 0.4

I Compute its expected value and its variance

I Compute the expected value of the variable defined asY = max(2,X )

Results

E[X ] = 0× 0,1 + 1× 0,3 + 2× 0,2 + 3× 0,4 = 1,9

Var[X ] =P

i (xi − E [X ])2pi = 1,09

E[Y ] = 2× (0,1 + 0,3 + 0,2) + 3× 0,4 = 2,4


Continuous random variables

I For a continuous random variable X , its distribution function isdefined as

F (x) = P[X ≤ x ]

I For a continuous random variable (without mass points) it holdsthat P(X = x) = 0

I Instead of the probability function, we define the density function as

f (x) =dF (x)

dx= F ′(x)


Properties

I F (−∞) = 0

I F (∞) = 1

I F (y) ≤ F (x), ∀y ≤ x

I f (x) ≥ 0 ∀x ∈ RI P(a ≤ X ≤ b) =

∫ b

af (x)dx ∀a, b ∈ R

I F (x) = P(X ≤ x) =∫ x

−∞ f (u)du

I∫∞−∞ f (x)dx = 1

I E[X ] =∫

xf (x)dx

I Var[X ] =∫

x2f (x)dx − E[X ]2

The normal distribution

Description

I The best known and more widely used continuous distribution

I Related to many other distributions (sums, sample mean)

Definition

I A continuous random variable X follows a normal distribution withparameters µ and σ, X ∼ N (µ, σ), if its density is of the form

f (x) = 1σ√

2πexp

{− 1

2σ2 (x − µ)2}

I In this case, E[X ] = µ and Var[X ] = σ2


Linear transformations

I A linear combination of a finite number of independent normalrandom variables follows a normal distribution

I E[∑

i aiXi ] =∑

i aiE[Xi ] and Var[∑

i aiXi ] =∑

i a2i Var[Xi ]

I If X ∼ N (µ, σ), then Y = aX + b ∼ N (aµ+ b, aσ)

Standardizing

I If X ∼ N (µ, σ), it holds that

Z =X − µσ

∼ N (0, 1)

I The distribution N (0, 1) is known as the standard normaldistribution

I It suffices to have (a table of) values for the standard normaldistribution


Exercise 0.2

I For a normal distribution with mean 2,5 and variance 4, obtain theprobability that it takes a value larger than 4

I Consider two independent normal variables X1 and X2 with the sameparameters as above. Compute the probability that X1 − X2 is largerthan 1.

Results

P(X > 4) = P

„X − 2,5√

4>

4− 2,5√4

«= P(Z > 0,75) = 0,227

Var[X1 − X2] = 2Var[X1] = 8, ∼ N(0,√

8)

P(X1 − X2 > 1) = P

„X1 − X2√

8>

1√8

«= P(Z > 0,354) = 0,362

Random samples

Definitions

I Population: the complete set of all information about the value of interest

I Sample: a subset of values from the population

I Inference: the process of obtaining information regarding unknownpopulation values from the values in a sample

Motivation

I We wish to obtain reliable information on the whole population from thestudy of a reduced subset of data (sample) with limited cost

Simple random sample

I Each member of the population has the same probability of belonging to asimple random sample

I Different members of the sample are selected independently

I The selection of one individual does not affect the probability ofselecting another

Sampling distributions

Definitions

I Statistic: a function of the information in the sample (its mean, variance,etc.)

I A statistic is a random variable, as its value depends on the selectedsample

I Sampling distribution: the probability distribution of an statistic over allsamples of a given size. It changes with the sample size

Population mean

I The population mean is a very relevant parameter in many practicalsituations

I Statistic: for a simple random sample X1, . . . ,Xn, we will obtaininformation for the population mean from the sample mean,

X =1

n

nXi=1

Xi

The distribution of the sample mean

Properties

I The expected value of the sample mean is the population mean

E[X ] = E

"1

n

nXi=1

Xi

#= E[X ]

I The variance of the sample mean is

Var[X ] = Var

"1

n

nXi=1

Xi

#=

1

nVar[X ]

I The value of this variance decreases as n increasesI We could reduce the error by increasing the sample size

The distribution of the sample mean

Distribution

I We are interested in the form of the distribution of the sample mean

I In many cases additional information to its mean and variance isrequired (probabilities)

I If the variable X follows a normal distribution N (µ, σ), for a simplerandom sample of size n,

X − µσ/√

n∼ N(0, 1)

I If X does not follow a normal distribution, there exist approximate resultsbased on the central limit theorem

I If the variables Xi have mean µ and standard deviation σ (bothfinite), and if n is large enough, it (approximately) holds that

X − µσ/√

n∼ N (0, 1)

The central limit theorem

Exercise 0.3You have a simple random sample of 100 values from a distribution withmean 25 and standard deviation 20Compute the probability that the sample mean is between 22 and 28

Results

X − µσ/√

n∼ N(0, 1)

P(22 ≤ X ≤ 28) = P

„22− 25

20/√

100≤ X − µσ/√

n≤ 28− 25

20/√

100

«= P(−1,5 ≤ Z ≤ 1,5) = 0,866


Approximations for proportions

I We are interested in the proportion of population members that satisfy acertain property

I If Xi represents if the property of interest is satisfied (or not) by agiven member of a simple random sample of size n, and theprobability of satisfaction is p, then

I Xi follows a Bernoulli distributionI the number of cases in the sample X =

Pi Xi follows a binomial

distribution with parameters n and p

I If n is large, we could use the central limit theorem to approximatethe distribution of p = X/n, the proportion in the sample (and anestimator for p)

I In this case, the variables Xi have mean p and standard deviationp(1− p). From the central limit theorem we have (approximately) that

X/n − ppp(1− p)/n

∼ N (0, 1)


Exercise 0.4A candidate for a local election has commisioned a poll on a sample of 36voters. If the proportion of persons willing to vote for her in the populationwould be 36 %,

I compute the probability that the proportion in the sample is larger than38 %

I How does this result change if the sample size is increased to 100 persons?

Results

p − ppp(1− p)/n

∼ N(0, 1)

P(p ≥ 0,38) = P

p − pp

p(1− p)/n≥ 0,38− 0,36p

0,36(1− 0,36)/36

!= P(Z ≥ 0,25) = 0,401

P(p ≥ 0,38) = P

Z ≥ 0,38− 0,36p

0,36(1− 0,36)/100

!= 0,338

Statistics II Lesson 0. Review of basic concepts

Documents

Transcript of Statistics II Lesson 0. Review of basic concepts