BAYESIAN BETA REGRESSION MODELS JOINT MEAN AND ...

BAYESIAN BETA REGRESSION MODELS JOINT

MEAN AND PRECISION MODELING

Edilberto Cepeda Cuervo

[email protected]

Universidad Nacional de Colombia

Edilberto Cepeda Cuervo . ( Universidad Nacional de Colombia, )Bayesian Beta Regression 1 / 29

Outline organization

1 Summary

2 Introduction

3 The model

4 Bayesian methodology

5 A simulated studies

6 Application

7 Nonlinear beta regression models

8 Beta Regression: Joint mean and variance modeling.

9 Bivariate beta regression models.

10 Double generalized spatial econometric models

11 References

ENDEdilberto Cepeda Cuervo . ( Universidad Nacional de Colombia, )Bayesian Beta Regression 2 / 29

1 Summary

2 Introduction

3 The model

6 Application

11 References

1 Summary

2 Introduction

3 The model

6 Application

11 References

1 Summary

2 Introduction

3 The model

6 Application

11 References

1 Summary

2 Introduction

3 The model

6 Application

11 References

1 Summary

2 Introduction

3 The model

6 Application

11 References

1 Summary

2 Introduction

3 The model

6 Application

11 References

1 Summary

2 Introduction

3 The model

6 Application

11 References

1 Summary

2 Introduction

3 The model

6 Application

11 References

1 Summary

2 Introduction

3 The model

6 Application

11 References

1 Summary

2 Introduction

3 The model

6 Application

11 References

1 Summary

2 Introduction

3 The model

6 Application

11 References

Summary

Bayesian beta regression models (BBRM)

Beta regression models, with joint modeling of the mean and precision parameters, were

proposed by Cepeda (2001) and Cepeda and Gamerman (2005).

Methodology

The Bayesian methodology to fit (BBRM) was proposed by Cepeda (2001) and Cepeda

and Gamenrman (2005), in the framework of DGLM.

Bivariate beta regression models

The Bivariate beta regression models, with joint modeling of the mean and dispersion

parameters, were proposed by Cepeda and Achcar (2011)

Double generalized spatial econometric models

Double generalized spatial econometric were proposed by (Cepeda, Urdinola and

Rodriguez (2011).

Summary

Methodology

Rodriguez (2011).

Summary

Methodology

Rodriguez (2011).

Summary

Methodology

Rodriguez (2011).

Introduction

Beta density function

f (y |p, q) =Γ(p + q)

Γ(p)Γ(q)yp−1(1− y)q−1

I(0,1)(y) (1)

where p > 0, q > 0 and Γ(.) denote the gamma function. The mean and variance

are given by

p + q(2)

σ2 =p q

(p + q)2(p + q + 1)(3)

Applications: 1) Financial sciences 2) Social sciences

Introduction

f (y |p, q) =Γ(p + q)

Γ(p)Γ(q)yp−1(1− y)q−1

I(0,1)(y) (1)

where p > 0, q > 0 and Γ(.) denote the gamma function. The mean and variance

are given by

p + q(2)

σ2 =p q

(p + q)2(p + q + 1)(3)

Applications: 1) Financial sciences 2) Social sciences

Introduction

Doing φ = p + q we can see that p = µφ, q = φ(1− µ) and σ2 = µ(1−µ)φ+1

f (y |α, β) =Γ(φ)

Γ(µφ)Γ((1− µ)φ)yµφ−1(1− y)(1−µφ)−1

I(0,1)(y) (4)

In this case, φ can be interpreted as a precision parameter in the sense that, for

fixed values of µ, larger values of φ correspond to smaller values of the variance of

Y . This reparametrizacion that is presented in Ferrari and Cribari-Neto (2004),

had already appeared in the literature, for example in Jorgensen (1997) or in

Cepeda (2001).

Introduction

f (y |α, β) =Γ(φ)

Γ(µφ)Γ((1− µ)φ)yµφ−1(1− y)(1−µφ)−1

I(0,1)(y) (4)

Cepeda (2001).

Introduction

f (y |α, β) =Γ(φ)

Γ(µφ)Γ((1− µ)φ)yµφ−1(1− y)(1−µφ)−1

I(0,1)(y) (4)

Cepeda (2001).

Beta regression model: Join mean and dispersion modeling

: http://www.bdigital.unal.edu.co/5947/

The beta regression model, as it was proposed by Cepeda (2001) and Cepeda and

Gamerman (2005)

logit(µ) = xti β (5)

log(φ) = zti γ, φ = p + q

Four years later, Ferrari and Cribari-Neto (2004) proposed is µ = p/(p + q) and

φ = p + q, assuming that g(µi ) = xtiβ and that φ is constant.

Joint beta regression models proposed by Cepeda(2001), was later studied by

Smithson and Verkuilen (2006) and then by Simas et al. (2010). Nonlinear beta

regression was proposed by Cepeda and Achcar (2010).Edilberto Cepeda Cuervo . ( Universidad Nacional de Colombia, )Bayesian Beta Regression 6 / 29

Gamerman (2005)

Joint mean and dispersion beta regression models

Bayesian methodology

Independent normal prior distributions are assumed:

β ∼ N(b,B)

γ ∼ N(g,G)

Posterior distribution is given by π(β,γ| data) ∝ L(β,γ)p(β,γ)

Conditional posterior distribution

π(β|γ, data)

π(γ|β, data)

β ∼ N(b,B)

γ ∼ N(g,G)

π(β|γ, data)

π(γ|β, data)

β ∼ N(b,B)

γ ∼ N(g,G)

π(β|γ, data)

π(γ|β, data)

β ∼ N(b,B)

γ ∼ N(g,G)

π(β|γ, data)

π(γ|β, data)

β ∼ N(b,B)

γ ∼ N(g,G)

π(β|γ, data)

π(γ|β, data)

β ∼ N(b,B)

γ ∼ N(g,G)

π(β|γ, data)

π(γ|β, data)

β ∼ N(b,B)

γ ∼ N(g,G)

π(β|γ, data)

π(γ|β, data)

Bayesian methodology (mean parameters)

yi = x′

iβ(c) + h

′[h−1(x′

iβ(c))][yi − h

−1(x′

iβ(c))], for i = 1, 2, ..., n, (6)

q1(β|β, γ) = N(b∗ ,B∗), (7)

b∗ = B∗(B−1b+ X′

Σ−1Y )

β∗ = (B−1 + X

Σ−1X)−1

Bayesian methodology (mean parameters)

yi = x′

iβ(c) + h

′[h−1(x′

iβ(c))][yi − h

−1(x′

iβ(c))], for i = 1, 2, ..., n, (6)

q1(β|β, γ) = N(b∗ ,B∗), (7)

b∗ = B∗(B−1b+ X′

Σ−1Y )

β∗ = (B−1 + X

Σ−1X)−1

Bayesian methodology (dispersion parameters)

yi = z′

iγ(c) + g

′[g−1(z′

iγ(c))][ti − h

−1(z′

iγ(c))], for i = 1, 2, ..., n, (8)

q2(γ|γ, β) = N(g∗ ,G∗), (9)

g∗ = G∗(G−1g + Z′

Ψ−1Y ),

G∗ = (G−1 + Z′

Ψ−1Z )−1.

Bayesian methodology (dispersion parameters)

yi = z′

iγ(c) + g

′[g−1(z′

iγ(c))][ti − h

−1(z′

iγ(c))], for i = 1, 2, ..., n, (8)

q2(γ|γ, β) = N(g∗ ,G∗), (9)

g∗ = G∗(G−1g + Z′

Ψ−1Y ),

G∗ = (G−1 + Z′

Ψ−1Z )−1.

A simulated study

go to eq. 9

Two independent explanatory variables X ∼ U(0, 20) and Z ∼ (0, 20).

Y ∼ B(µ, φ) with logit(µ) = β0 + β1x and log(φ) = γ0 + γ1x .

Mean model dispersion model

Parameters β0 β1 γ0 γ1

t.v. 0.75 -0.055 0.15 -0.04

B.e. 0.437 -0.047 0.017 -0.046

s.d. 0.410 0.036 0.303 0.025

Table: Posterior parameter estimates

A simulated study

go to eq. 9

Two independent explanatory variables X ∼ U(0, 20) and Z ∼ (0, 20).

Y ∼ B(µ, φ) with logit(µ) = β0 + β1x and log(φ) = γ0 + γ1x .

Mean model dispersion model

Parameters β0 β1 γ0 γ1

t.v. 0.75 -0.055 0.15 -0.04

B.e. 0.437 -0.047 0.017 -0.046

s.d. 0.410 0.036 0.303 0.025

Table: Posterior parameter estimates

Simulate studies

Join mean and dispersion regression

6000 7000 8000 9000 10000−1

−1 0 1 20

6000 7000 8000 9000 10000−0.2

−0.1

−0.2 −0.1 0 0.1 0.20

Figure: Chains and histograms of the mean parameters

Simulate studies

Join mean and dispersion regression

6000 7000 8000 9000 10000−2

−2 −1 0 10

6000 7000 8000 9000 10000−0.15

−0.1

−0.05

−0.15 −0.1 −0.05 0 0.050

Figure: Chains and histograms of the dispersion parameters

Application studies

Join mean and dispersion beta regression

The data consists of 38 households. The interest variable is the proportion

of the income spent on food, and the explanatory variables are the level of

income INC and the number of persons in the household NUM . The model:

logit(µ) = β0 + β1INC + β2NUM , log(φ) = γ0 + γ1INC + γ2NUM .

Mean model Dispersion model

Parameters β0 β1 β2 γ0 γ1 γ2

θ -0.7404 -0.0093 0.0978 4.6529 0.0048 -0.3593

s.d. 0.2074 0.0031 0.0411 0.7840 0.0123 0.2113

θ -0.7582 -0.0091 0.1021 4.8428 .... -0.3500

s.d. 0.2091 0.0030 0.0386 0.4936 ..... 0.1298

Table: Estimates of the parameters of the dispersion modelsEdilberto Cepeda Cuervo . ( Universidad Nacional de Colombia, )Bayesian Beta Regression 13 / 29

Application studies

Join mean and dispersion beta regression

Model with constant dispersion parameter

Parameters β0 β1 β2 φ

Classical Estimates -0.62255 -0.01230 0.11846 35.60975

d.s 0.22385 0.00304 0.03534 8.07960

Bayesian Estimates -0.6237 -0.0124 0.1190 32.8666

d.s 0.2357 0.0033 0.0379 7.1815

Table: Bayesian posterior estimates (BIC=-82.2636)

The BIC valu for the last model are given by BIC = −118.2793,

BIC = −118.2793.

Nonlinear beta regression models: Joint mean and

dispersion modeling

http://www.tandfonline.com/doi/abs/10.1080/03610910903480784

The nonlinear beta regression model was proposed by Cepeda and Achcar (2010)

µi =β0

1 + β1exp(β2ti)(10)

φ = exp(γ0) (11)

Parameters β0 β1 β2

Normal NL Models 80.6300 0.7137 -0.2456

s.d. (2.5390) (0.0557) (0.0459)

Beta NL models 0.8023 0.7172 -0.2521

s.d. (0.0201) (0.0579) (0.0408)

Table: Normal and Beta mean parameter estimates of nonlinear regressionEdilberto Cepeda Cuervo . ( Universidad Nacional de Colombia, )Bayesian Beta Regression 15 / 29

Beta Regression: Joint mean and variance modeling

http://www.bdigital.unal.edu.co/6207/

In this model, the mean and variance regression models are given by:

logit(µ) = xtiβ (12)

log(σ2) = ztiγ (13)

The results of fitting the mean and variance beta regression models are easily

interpretable: the mean fitted models have the usual interpretation, but the fitted

variance model is easily interpreted directly from data behavior. For example, if

the explanatory variable Z1 is associated to γ1 and γ1 > 0, increasing behavior of

Z1 is associated with increasing behavior of σ2.

In this model, the mean and variance regression models are given by:

logit(µ) = xtiβ (12)

log(σ2) = ztiγ (13)

The results of fitting the mean and variance beta regression models are easily

interpretable: the mean fitted models have the usual interpretation, but the fitted

variance model is easily interpreted directly from data behavior. For example, if

the explanatory variable Z1 is associated to γ1 and γ1 > 0, increasing behavior of

Z1 is associated with increasing behavior of σ2.

Application

Interest variable: mean performance “Performance” in Spanish of students in

second grade of secondary schools.

Explanatory variables: level of unsatisfied basic needs UBN and the percentage of

teachers with postgraduate levels of educations.

20 30 40 50 60 70 80UNB

10 20 30 40

Percentage

Figure: Plots of performance in Spanish versus explanatory variablesEdilberto Cepeda Cuervo . ( Universidad Nacional de Colombia, )Bayesian Beta Regression 17 / 29

Application

Interest variable: mean performance “Performance” in Spanish of students in

second grade of secondary schools.

Explanatory variables: level of unsatisfied basic needs UBN and the percentage of

teachers with postgraduate levels of educations.

20 30 40 50 60 70 80UNB

10 20 30 40

Percentage

Figure: Plots of performance in Spanish versus explanatory variablesEdilberto Cepeda Cuervo . ( Universidad Nacional de Colombia, )Bayesian Beta Regression 17 / 29

Application

The model

logit(µ) = β0 + β1NBI + β2PER (14)

log(σ2) = γ0 + γ2PER (15)

Mean model Variance model

DIC Parameters β0 β1 β2 γ0 γ1

-196.222 θ 0.3132 -0.0025 0.0018 -8.425 -0.0306

s.d. 0.0357 5.023E-4 7.564E-4 0.8152 0.0269

-195.769 θ 0.3026 -0.0023 0.0019 -9.287 -

s.s. 0.0316 607E-4 6.952E-4 0.2766 -

Table: Estimates of the parameters of the variance models

From Farlie-Gumbel-Morgentern copula function, a bivariate beta distribution

function is given by

FI (y1, y2) = F1(y1)F2(y2)[1 + θ[1− F1(y1)][1− F2(y2)]], (16)

and the bivariate beta density function is given by

fI (y1, y2) = f1(y1)f2(y2) + θf1(y1)f2(y2)[1− 2F1(y1)][1− 2F2(y2) (17)

The bivariate beta distribution function has five parameters: µ1 and µ2 for the

means, φ1 and φ2 for the precisions, and θ for the dependence parameter.

Bivariate beta regression model

Join mean and dispersion model

The bivariate beta regression model, as it was proposed by Cepeda et al., (2011).

logit(µki) = x′

kiβk (18)

log(φki) = z′

k = 1, 2.

A simulated study

0 2000 4000 6000 80000

0 0.5 1 1.5 20

0 2000 4000 6000 8000−1.2

−0.8

−1.2 −1.1 −1 −0.9 −0.80

0 2000 4000 6000 80000

0 0.1 0.2 0.3 0.40

0 2000 4000 6000 80001

1 1.5 2 2.5 30

0 2000 4000 6000 8000−1.5

−0.5

−1.1 −1 −0.9 −0.8 −0.70

0 2000 4000 6000 80000

0.4 0.5 0.6 0.70

Figure: Posterior chain sample for the mean regression parameters.

A simulated study

0 2000 4000 6000 80000

0 0.5 1 1.5 20

0 2000 4000 6000 8000−0.1

−0.2 0 0.2 0.4 0.60

0 2000 4000 6000 80000

0 0.5 1 1.50

0 2000 4000 6000 8000−0.2

−0.1

−0.4 −0.2 0 0.2 0.40

Figure: Posterior chain sample for the dispersion regression parameters.

Bivariate beta regression Models

0 2000 4000 6000 8000−0.2

−0.5 0 0.5 1 1.50

Figure: Posterior chain samples for the dependence parameters.

Double generalized spatial econometric models spatial beta

regression models

http://www.tandfonline.com/doi/abs/10.1080/03610918.2011.600500

1 - 39

40 - 49Edilberto Cepeda Cuervo . ( Universidad Nacional de Colombia, )Bayesian Beta Regression 24 / 29

Spatial beta regression models

Application

logit(µ) = β0 + β1RIQ + β2VIOL ++β3W1PR95,

log(φ2i ) = γ0 + γ1UBNi + γ2W1PR95i

Spatial beta regression: application

Parameter estimates

logit(µi ) = β0 + β1VIOL + β2W1PR95,

log(φi) = γ0 + γ1UBNi + γ2W1PR95i

β0 β1 β2 γ0 γ1 γ2

-1.139 -0.003 0.029 -0.043 0.028 -0.095

0.0608 0.001 0.010 2.280 0.019 0.048

Table: Models comparison.

References

Cepeda C. E. and Gamerman D.(2001). Edilberto Cepeda. Modelagem da

variabilidade em modelos lineares generalizados. 2001. Tese (Doutorado em

Matematica) - Universidade Federal do Rio de Janeiro

Cepeda C. E. and Gamerman D. (2003) Bayesian methodology for modeling

parameters in the two parameter exponential family. R.T. Na 166

(DME/IM-UFRJ).

Cepeda C. E. and Gamerman D. (2005). Bayesian methodology for modeling

parameters in the two parameter exponential family. Estadıstica, 57, 93-105.

Cepeda-Cuervo, E. and Achar, J.(2010) Heteroscedastic nonlinear regression

models: a bayesian approach. Communications In statistics - simulation and

computation.

References

(DME/IM-UFRJ).

computation.

References

(DME/IM-UFRJ).

computation.

References

(DME/IM-UFRJ).

computation.

References

Cepeda, E. C. et al., (2011). Double generalized spatial econometric models.

Communication in statistics-simulation and computation. To appear.

Cepeda, E. C. and Garrido L. (2011). Bayesian beta regression models. Submitted.

Cepeda, E. C., Achcar J. and Garrido L. (2011). Bivariate beta regression models.

Submitted.

Ferrari, S., Cribari-Neto, F. (2004). Beta regression for modeling rates and

proportions, Journal of Applied Statistics 31, 799-815.

Cribari-Neto F (2005). Improved Maximum Likelihood Estimation in a New Class

of Beta Regression Models. Brazilian Journal of Probability and Statistics, 19(1),

13-31.

Simas A. B., Barreto-Souza W., Rocha A. V. (2010). Improved Estimators for a

General Class of Beta Regression Models, Computational Statistics & Data

Analysis, 54(2), 348-366.Edilberto Cepeda Cuervo . ( Universidad Nacional de Colombia, )Bayesian Beta Regression 28 / 29

References

Submitted.

13-31.

References

Submitted.

13-31.

References

Submitted.

13-31.

References

Submitted.

13-31.

References

Submitted.

13-31.

Beta Regression Models: Joint Mean and Variance Modeling

Bivariate beta regression models: a Bayesian approach applied to educational data

Bayesian beta regression models: Joint mean and precision modeling

) Double Generalized Spatial Econometric Models

WinBugs CODE for Beta Regression Models

BAYESIAN BETA REGRESSION MODELS JOINT MEAN AND ...

Documents

Transcript of BAYESIAN BETA REGRESSION MODELS JOINT MEAN AND ...