The Bayesian Lasso for Genome-wide AssociationStudiesJiahan Li1,2, Kiranmoy Das1,2, Guifang Fu1,2, Runze Li1,2 and Rongling Wu2,1∗

1Department of Statistics, Pennsylvania State University, State College, PA 16802, USA and2Center for Statistical Genetics, Pennsylvania State University, Hershey, PA 17033, USA

ABSTRACT

Motivation: Despite their success in identifying genes that affectcomplex disease or traits, current genome-wide associationstudies (GWASs) based on a single SNP analysis are too simpleto elucidate a comprehensive picture of the genetic architectureof phenotypes. A simultaneous analysis of a large number ofSNPs, although statistically challenging, especially with a smallnumber of samples, is crucial for genetic modeling.

Method: We propose a two-stage procedure for multi-SNPmodeling and analysis in GWASs, by first producing a“preconditioned” response variable using a supervised principlecomponent analysis and then formulating Bayesian lasso toselect a subset of significant SNPs. The Bayesian lasso isimplemented with a hierarchical model, in which scale mixturesof normal are used as prior distributions for the genetic effectsand exponential priors are considered for their variances, andthen solved by using the Markov chain Monte Carlo (MCMC)algorithm. Our approach obviates the choice of the lassoparameter by imposing a diffuse hyperprior on it and estimatingit along with other parameters and is particularly powerful forselecting the most relevant SNPs for GWASs where the numberof predictors exceeds the number of observations.

Results: The new approach was examined through a simulationstudy. By using the approach to analyze a real data set fromthe Framingham Heart Study, we detected several significantgenes that are associated with body mass index (BMI). Ourfindings support the previous results about BMI-related SNPsand, meanwhile, gain new insights into the genetic control of thistrait.

Availability: The computer code for the approach developed isavailable at Penn State Center for Statistical Genetics web site,http://statgen.psu.edu.

Contact: rwu@hes.hmc.psu.edu

Supplementary information: Supplementary data are availableat Bioinformatics online.

∗To whom correspondence should be addressed

1 INTRODUCTION

Recent genotyping technologies allow the fast and accuratecollection of genotype data throughout the entire genome formany subjects. By genome-wide association studies (GWASs),the genetic variants associated with a complex disease or trait,their chromosomal distribution and individual effects, can beidentified. GWASs are based on either case-control cohorts totest the associations between SNPs and diseases or populationcohorts to estimate genetic effects of SNPs on traits. In bothcases, there are hundreds of thousands of SNPs genotyped onsamples involving thousands of subjects. This typical problem,having the number of predictors far exceeding the number ofobservations, makes it impossible to analyze the data usingtraditional multivariate regression. In current GWASs, simpleunivariate linear regression that analyzes one SNP at a time isusually used and, by adjusting for multiple comparisons, thesignificance levels of the detected genes are then calculated(McCarthy et al. 2008).

These single SNP-based GWASs have been instrumentalfor reproducibly detecting significants genes for variouscomplex diseases or traits (Donnelly 2008). However, suchstrategies have three major disadvantages, limiting the futureapplications of GWAS. First, because most complex traits arepolygenic, a single SNP analysis can only detect a very smallportion of genetic variation and, also, may not be powerful foridentifying weaker associations (Hoggart et al., 2008). Second,different genes may interact with each other to form a complexnetwork of genetic interactions, which cannot be characterizedfrom a single SNP analysis. Third, many GWASs analyzegenetic associations separately for different environments,such as males and females, and then make an across-environment comparison in genetic effects. This analysis isneither powerful nor precise for the identification of gene-environment interactions. Because of these limitations, manyauthors have developed various approaches for simultaneouslyanalyzing multiple SNPs for GWASs (Wu et al. 2009; Yang etal. 2010; Logsdon et al. 2010), although most approaches focuson case-control cohorts.

There is a daunting need on the development of avariable selection model to identify SNPs with significanteffects on quantitative traits in population cohorts and estimateall selected predictors simultaneously. Traditionally, a subset

1

Associate Editor: Dr. Jeffrey Barrett

© The Author (2010). Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Bioinformatics Advance Access published December 14, 2010 by guest on D

ecember 15, 2010

bioinformatics.oxfordjournals.org

Dow

nloaded from

of predictors in a regression model is obtained by forwardselection, backward elimination, and stepwise selection, butthese approaches are computationally expensive and unstableeven when the number of predictors is not large. Recently,alternative approaches have been developed, including ridgeregression, bridge regression (Frank and Friedman, 1993),least absolute shrinkage and selection operator (LASSO)(Tibshirani, 1996), elastic net (Zou and Hastie, 2005) andthe smoothly clipped absolute deviation (SCAD) penalty (Fanand Li, 2001). For the number of variables much larger thanthat of subjects, as commonly seen in GWASs, Fan and Lv(2008) proposed a two-stage procedure for variable selectionby first suppressing the high dimensionality of response intoits low-dimensional representation and then finding a subset ofpredictors that can predict the suppressed response. A similartwo-stage approach was also developed by Paul et al. (2008).

In this article, we for the first time integrate Paul etal.’s preconditioning procedure into LASSO to develop a two-stage strategy for identifying important SNPs in GWASs. Instep one, we find a linear combination of predictors that arestrongly correlated with the response by a supervised principlecomponent analysis and get a consistent “preconditioned”estimate of response variable. In step two, we implement theBayesian lasso (Park and Casella, 2008) for variable selectionbased on the “preconditioned” response that mitigates theobservational noise. The Markov chain Monte Carlo (MCMC)algorithm is used to estimate all the parameters. The Bayesianhierarchical model is implemented to control an issue of over-fitting that arises when too many associations are included. Ourmodel shows a great flexibility to fit many SNPs and manycovariates at the same time. The statistical properties of themodel were tested through simulation studies. We used a realGWAS data set from the Framingham Heart Study to validatethe usefulness and utilization of the new model.

2 BAYESIAN GWAS MODEL

2.1 Preconditioning

When the number of predictors far exceeds the numberof observations, preconditioning via a supervised principalcomponent analysis is recommended to reduce the effect ofobservational noise on model selection (Paul et al., 2008).In a GWAS of n subjects, we express a response variable y(assumed to be normally distributed) as a function of p SNPsgenotyped throughout the entire genome using a linear model

y = Wb + ε, (1)

where W = (w1, · · · , wn)T is a (n × p) design matrix, b =(b1, · · · , bp)T is the vector of regression coefficients, and ε ∼Nn(0, σ2In) is the residual error.

The design matrix is reduced to one that consists ofonly those predictors whose estimated regression coefficientsexceed a threshold θ in the absolute value. Thus, the reduceddesign matrix Wreduced consists of the j′-th column of W ,where j′ ∈ {j : |bj | > θ}. The principal componentsof Wreduced, called supervised principal components, are

computed. The first m supervised principal components canserve as independent variables in a linear regression model,from which a consistent predictor y of the true response isobtained. In practice, we select θ by 5-fold cross-validation.Since only the first few components are useful for prediction,in the following examples we consider the first three principalcomponents. Next, a standard variable selection procedure willbe conducted for the preconditioned response variable y.

2.2 Lasso Penalized Regression

Given phenotypical measurements and genotype information,we could obtain the preconditioned response y based on thegeneric form of linear regression (1). However in genome-wideassociation studies, a number of covariates, which are eitherdiscrete or continuous, may be measured for each subject. Inorder to estimate genetic effects precisely by adjusting for thesecovariates, a GWAS model that takes into account the effects ofimportant covariates would be more appropriate. Therefore, wedescribe the preconditioned value yi of a quantitative trait forsubject i as

yi = µ+XTi α+ZTi β+ξTi a+ζTi d+εi, i = 1, · · · , n, (2)

where µ is the overall mean, Xi is the d1-dimensional vectorof discrete covariates for subject i, α = (α1, · · · , αd1)T

is the vector of regression coefficients for discrete covariates,Zi is the d2-dimensional vector of continuous covariates forsubject i, β = (β1, · · · , βd2)T is the vector of regressioncoefficients for continuous covariates, a = (a1, · · · , ap)Tand d = (d1, · · · , dp)T are the p-dimensional vectors of theadditive and dominant effects of SNPs, respectively, ξi and ζiare the indicator vectors of the additive and dominant effectsof SNPs for subject i, and εi is the residual error assumed tofollow a N(0, σ2) distribution. The j-th elements of ξi and ζiare defined as

ξij =

1, if the genotype of SNP j is AA0, if the genotype of SNP j is Aa−1, if the genotype of SNP j is aa,

ζij =

{1, if the genotype of SNP j is Aa0, if the genotype of SNP jis AA or aa.

Despite p � n in the GWAS, most of the regressioncoefficients in (2) are expected to have no or only weak effectson the phenotype. To identify a few SNPs that may have notableeffects and enhance prediction performance, we put L1 lassopenalties on the sizes of additive effects and the dominanteffects and encourage sparse solutions using

p∑j=1

|aj | ≤ t,p∑j=1

|dj | ≤ t∗, fort ≥ 0, t∗ ≥ 0, (3)

where t and t∗ are a certain value chosen to penalize theadditive and dominant effects, respectively. Thus, parameters

2

by guest on Decem

ber 15, 2010bioinform

atics.oxfordjournals.orgD

ownloaded from

in equation (2) are estimated by the penalized least squares

1

2||y−µ−Xα−Zβ−ξa−ζd||2 +λ

p∑j=1

|aj |+λ∗p∑j=1

|dj |,

(4)where y = (y1, · · · , yn)T , µ = (µ, · · · , µ)T , X =(X1, · · · , Xn)T , Z = (Z1, · · · , Zn)T , ξ = (ξ1, · · · , ξn)T ,ζ = (ζ1, · · · , ζn)T , and λ and λ∗ are tuning parameters orlasso parameters that control the degrees of shrinkage in theestimate of the genetic effects.

2.3 Bayesian Hierarchical Model and PriorDistributions

Noting the form of the L1-penalty term in (4), Tibshirani(1996) suggested that lasso estimates can be interpreted asposterior mode estimates when the regression parameters haveindependent and identical Laplace (i.e., double-exponential)priors. Therefore, when lasso penalties are imposed on theadditive and dominant effects of SNPs, the conditional priorfor aj is a Laplace distribution with the scale parameter σ/λ:

π(a|σ2) =

p∏j=1

λ

2√σ2e−λ|aj |/

√σ2, (5)

Similarly, the conditional Laplace prior for dj is

π(d|σ2) =

p∏j=1

λ∗

2√σ2e−λ

∗|dj |/√σ2. (6)

Since the Laplace distribution can be representedas a scale mixture of a normal distribution with anexponential distribution (Andrews and Mallows, 1974), wehave the following hierarchical representation of the penalizedregression model:

y|µ,α,β,a,d, σ2 ∼ Nn(µ +Xα + Zβ + ξa + ζd, σ2In),

α ∼ Nd1(0,Σα),

β ∼ Nd2(0,Σβ),

a|σ2, τ21 , · · · , τ2p ∼ Np(0, σ2diag(τ21 , · · · , τ2p )),

τ21 , · · · , τ2p |λ ∼p∏j=1

exp(λ2

2),

d|σ2, τ∗21 , · · · , τ∗2p ∼ Np(0, σ2diag(τ∗21 , · · · , τ∗2p )),

τ∗21 , · · · , τ∗2p |λ∗ ∼p∏j=1

exp(λ∗2

2),

σ2 ∼ π(σ2),

σ2, τ21 , · · · , τ2p , τ∗21 , · · · , τ∗2p > 0.

After integrating out τ21 , · · · , τ2p and τ∗21 , · · · , τ∗2p , theconditional priors on a and d have the desired forms (5) and(6), respectively. We assign conjugate normal priors to α andβ because they are low-dimensional and not the parameters ofinterest. Finally, since the data are usually sufficient to estimate

µ and σ, we can use an independent, flat prior π(µ) = 1 for µand a noninformative scale-invariant prior π(σ2) = 1/σ2 forσ2.

The tuning parameters of the ordinary lasso can beprespecified by cross-validation, generalized cross-validation,or the idea based on Stein’s unbiased risk estimate. However, inthe Bayesian lasso, λ and λ∗ can be estimated along with otherparameters by assigning appropriate hyperpriors to them. Thisprocedure avoids the choice of lasso parameters and allowsus to determine the amount of shrinkage from the data. Inparticular, we consider the conjugate gamma priors on λ2/2and λ∗2/2,

π

(λ2

2

)∼ Gamma(a, b),

π

(λ∗2

2

)∼ Gamma(a∗, b∗).

where a, b, a∗, and b∗ are small values so that the priorsare essentially noninformative. With this specification, lassoparameters can be treated similar to the other parameters andestimated by the Gibbs sampler.

3 POSTERIOR COMPUTATION ANDINTERPRETATION

3.1 MCMC Algorithm

We estimate the parameters by sampling from their conditionalposterior distributions through the MCMC algorithm. The jointposterior distribution can be expressed as:

π(µ,α,β,a, τ21 , · · · , τ2p , λ,d, τ∗21 , · · · , τ∗2p , λ∗, σ2|y)

∝∏ni=1 π(yi|·)π(µ)π(σ2)π(α)π(β)∏p

j=1 π(aj |τ2j )π(τ2j |λ)π(λ)π(dj |τ∗2j )π(τ∗2j |λ∗)π(λ∗)

Two-level hierarchical modeling allows us to easilyderive the conditional posterior distributions of parametersand hyperparameters, from which the Gibbs samplerdraws posterior samples. Conditional on the parameters(a,d, τ21 , · · · , τ2p , τ∗21 , · · · , τ∗2p ), the model is the standardlinear regression and, thus, the conditional posteriordistributions of (α,β, σ2) are

α|· ∼ Nd1

(Σ′(∑n

i=1Xi(yi − µ− ZTi β − ξTi a− ζTi d)

σ2

),Σ′),

with Σ′ =

(∑ni=1XiX

Ti

σ2+ Σ−1

α

)−1

,

β|· ∼ Nd2

(Σ′′(∑n

i=1 Zi(yi − µ−XTi α− ξTi a− ζTi d)

σ2

),Σ′′

),

with Σ′′ =

(∑ni=1 ZiZ

Ti

σ2+ Σ−1

β

)−1

,

3

by guest on Decem

ber 15, 2010bioinform

atics.oxfordjournals.orgD

ownloaded from

σ2|· ∼ Inv−χ2

(n,

1

n

n∑i=1

(yi − µ−XTi α− ZTi β − ξTi a− ζTi d)2

).

Conditional on the parameters (τ21 , · · · , τ2p , τ∗21 , · · · , τ∗2p ,α,β),the model becomes the weighted linear regression, and thus theconditional posterior distributions of (a,d) are

a|· ∼ N(A−1a ξ(yi − µ−XT

i α− ZTi β − ζTi d), σ2A−1a

),

with A−1a =

(ξξT + diag(τ21 , · · · , τ2p )−1

)−1

,

d|· ∼ N(A−1d ζ(yi − µ−XT

i α− ZTi β − ξTi a), σ2A−1d

),

with A−1d =

(ζζT + diag(τ∗21 , · · · , τ∗2p )−1

)−1

.

Moreover, the full conditional for τ21 , · · · , τ2p , τ∗21 , · · · , τ∗2pare conditionally independent, with

1

τ2j|· ∼ Inverse-Gaussian

(√λ2σ2

β2j

, λ2

), j = 1, · · · , p,

and

1

τ∗2j|· ∼ Inverse-Gaussian

(√λ∗2σ2

β2j

, λ∗2), j = 1, · · · , p.

Finally, with the conjugate priors Gamma(a, b) andGamma(a∗, b∗), the conditional posterior distributions of thehyperparameters are gammas

λ2|· ∼ Gamma

(p+ a,

p∑j=1

τ2j2

+ b

),

and

λ∗2|· ∼ Gamma

(p+ a∗,

p∑j=1

τ∗2j2

+ b∗).

An efficient Gibbs sampler based on these full conditionalsproceeds to draw posterior samples from each full conditionalposterior distribution, given the current values of all otherparameters and the observed data. This process continues untilall chains converge. We use the potential scale reduction factorR to access the convergence (Gelman and Rubin, 1992). OnceR < 1.1 for all scalar estimands of interest, we continueto draw 15, 000 iterations to obtain samples from the jointposterior distribution.

3.2 Posterior interpretation

The proposed MCMC algorithm for our Bayesian lasso modelcan provide posterior median estimates of the additive effectsand dominant effects of individual SNPs, while adjusting forthe effects of all other SNPs and covariates. Furthermore, usingthe posterior samples of a, d, and the observed genotypes,

Fig. 1. The histograms of original and preconditioned BMI

we can calculate the proportion of the phenotypic varianceexplained by a particular SNP, i.e., heritability, by

h2j =

2p1p0(aj + (p1 − p0)dj)2 + 4p21p

20d

2j

var(y), j = 1, · · · , p,

where p1 is the estimated allele frequency for A, and p0 is theestimated allele frequency for a, aj is the median estimate ofthe additive effect for SNP j, and dj is the median estimate ofthe dominant effect for SNP j. Since heritability estimates areunitless, they could guide variable selection and identify SNPsthat have relatively large effects on the phenotype.

4 RESULTS

4.1 Worked Example

We used the newly developed model to analyze a realGWAS data set from the Framingham Heart Study (FHS),a cardiovascular study based in Framingham, Massachusetts,supported by the National Heart, Lung, and Blood Institute,in collaboration with Boston University (Dawber et al. 1951).Recently, 550,000 SNPs have been genotyped for the entireFramingham cohort (Jaquish 2007), from which 418 males and559 females were chosen for our data analysis. These subjectswere measured for body mass index (BMI) at different agesfrom 29 and 61 years. As is standard practice, SNPs with minorallele frequency < 10% were excluded from data analysis. Thenumbers and percentages of non-rare allele SNPs vary amongdifferent chromosomes and ranges from 4,417 to 28,771 andfrom 64% to 72%, respectively.

In principle, our approach can handle an extremely largenumber of SNPs at the same time. To save our computing time,however, we use those SNPs that cannot be neglected accordingto a simple single SNP analysis. We chose the phenotypic dataof BMI in a middle measure age of each subject for a singleSNP analysis, separately for males and females. SupplementaryFigure 1 gives −log10 p-values for each SNP in the two sexes

4

by guest on Decem

ber 15, 2010bioinform

atics.oxfordjournals.orgD

ownloaded from

Fig. 2. Estimated additive (A) and dominant effects (B) of 1837 SNPsfrom the Framingham heart study

from which 1837 SNPs with a −log10 p-value of > 3.5 inat least one sex were selected for Bayesian lassso analysis.Before this analysis, we imputed missing genotypes for a smallproportion of SNPs (5.16%) according to the distribution ofgenotypes in the population. A preconditional analysis withm = 3 and θ = 0.426 was used to mitigate observationalnoise, leading to the preconditioned phenotypes. Like originalmeasures, the preconditioned BMI also displays a normaldistribution (Fig. 1), which meets the normality assumptionrequired by the new approach.

By treating the sex as a discrete covariate and age asa continuous covariate, we imposed lasso penalties on theadditive effects a1, · · · , ap and dominant effects d1, · · · , dp toidentify those SNPs with notable effects on BMI. We employthe proposed MCMC algorithms to estimate all parameters andimplement variable selection, where Σα = 1, Σβ = 1, andall parameters in the conjugate gamma hyperpriors are 0.1. Inunreported tests, we find that the posteriors are not sensitive tothese prior specifications, as long as a and b are small valuesso that the priors are relatively flat (Park and Casella, 2008;Yi and Xu, 2008). Figure 2 plots the estimated additive anddominant effects of each SNP after adjusting for the effectsof other SNPs and covariates. The heritability explained by

Table 1. Information about significant SNPs

Chr Name Position Additive Heritability (%)

1 ss66185476 8445140 0.15 0.741 ss66374301 8451728 0.19 1.361 ss66295856 8578082 0.20 1.351 ss66516012 198313489 -0.13 0.891 ss66364251 198321700 0.12 0.66

10 ss66311679 32719838 0.33 4.6510 ss66293192 32903593 0.27 2.0810 ss66303064 32995111 0.36 5.9310 ss66128868 33407810 0.28 2.7520 ss66171460 22580931 -0.13 0.7822 ss66055592 23420006 0.33 5.1322 ss66164329 23420370 0.37 6.70

each SNP is shown in Fig. 3. The Bayesian hierarchical modelautomatically shrinks small coefficients to zero, and hence theposterior estimates of a, d and h2

j can guide variable selection.We claim that a genetic effect is significant if its 95% posteriorcredible interval does not contain zero. Alternatively, Hoti andSillanpaa (2006) suggested to preset a threshold value, c, suchthat one SNP is included into the final model if the heritabilityexplained by this SNP is greater than c. We usually report theSNPs with high heritabilities and, thus, this threshold can bechosen on more subjective grounds.

Table 1 tabulates the names and positions of SNPswith the heritability (h2

j ) greater than 0.5, as well as theestimated additive effects and heritabilities. We do not reportthe estimated dominant effects since they are relatively low inthis example. The Bayesian lasso tends to shrink small effectsof genes into zero. Assuming that a = d = 0.4 for a SNPwith allele frequencies of 0.5 in a population, the additive anddominant variances explained by this SNP is 1

2a2 = 0.08 and

14d2 = 0.04, respectively. Thus, there is a possibility that the

dominant effects are shrunk to a greater extent than the additiveeffects if they are of similar size. This may partly explain whythe dominant effects estimated for significant SNPs are muchsmaller than the additive effects.

The amount of shrinkage in the estimates of additive anddominant effects are quantified by two hyperparameters λ andλ∗ determined from the data. The posterior medians for λ andλ∗ are 54.474 and 54.523, respectively, with the 95% posteriorintervals being [53.325, 55.626] and [53.359, 55.678],respectively. These suggest that the tuning parameters for theadditive and dominant effects can be estimated precisely.

Since five significant SNPs are selected from chromosome1, and four from chromosome 10, we will further examinethe correlations among the significant SNPs from the samechromosome, as suggested by a referee. The correlation matrix

5

by guest on Decem

ber 15, 2010bioinform

atics.oxfordjournals.orgD

ownloaded from

Table 2. Genetic effects of 20 assumed SNPsfor data simulation.

Position Additive Position Dominant

100 1.2 50 1.2200 1.2 150 1.2300 1.2 250 1.2400 0.8 350 0.8500 0.8 450 0.8600 0.8 550 0.8700 0.4 650 0.4800 0.4 750 0.4850 1.2 850 1.2900 0.8 900 1.2950 1.2 950 0.8

1000 0.8 1000 0.8

of five significant SNPs from chromosome 1 is given by1.00 0.85∗ 0.86∗ −0.01 0.010.85∗ 1.00 0.78∗ −0.01 0.020.86∗ 0.78∗ 1.00 −0.04 0.02−0.01 −0.01 −0.04 1.00 −0.84∗

0.01 0.02 0.02 −0.84∗ 1.00

,

where star denotes significant correlations at the significancelevel 1%. Clearly, these SNPs can be classified into twogroups, and within each group, SNPs are highly correlated. Thecorrelation matrix of four significant SNPs from chromosome10

1.00 0.53∗ 0.48∗ 0.30∗

0.53∗ 1.00 0.52∗ 0.53∗

0.48∗ 0.52∗ 1.00 0.45∗

0.30∗ 0.53∗ 0.45∗ 1.00

also suggested that these SNPs are closely linked to each other.

4.2 Computer Simulation

The new approach is investigated through simulation studies.We generate data according to the model (2) with µ = 0,σ2 = 10 and n = 500. For ease of simulation, ξij is derivedfrom uij , where each uij has a standard normal distributionmarginally, and ρ =cov(uij , uik) = 0.1. Then, to mimic aSNP with equal allele frequencies, we set

ξij =

1, uij > c0, −c ≤ uij ≤ c−1, uij < −c,

where −c is the first quartile of a standard normal distribution.Finally, ζij is derived from ξij . We assume that there are 1000SNPs from which 20 are significant for a phenotypic trait.The positions and additive and dominant effects of individualsare given in Table 2. It is assumed that the trait is measuredat a subject-specific age, following the data structure of theFramingham Heart Study.

Fig. 3. Estimated additive (A) and dominant effects (B) based on 50simulations.

Figure 3 gives the estimated additive and dominant geneticeffects of different SNPs over 50 simulations, and Figure 4plots the heritability explained by each SNP. It is clear thatlasso penalties shrink small genetic effects to zeros, resultingin sparse solutions of the regression coefficients. In general, the20 assumed SNPs can be well identified and their additive anddominant effects well estimated. Also, two hyperparameters λand λ∗ whose influence the degree of shrinkage can be wellestimated. In Supplementary Fig. 2, the histograms of thesetwo hyperparameters are shown.

Then, we carry out another simulation study to comparethe performance of preconditioned Bayesian lasso, Bayesianlasso without preconditioning, and the traditional single SNPanalysis. Without loss of generality, only the additive model isconsidered. Specifically, we generate data on n = 200 and p =500 or 1000 according to the model (2), with µ = 0, σ2 = 10,ρ = 0.1, aj = 1 for 1 ≤ j ≤ 20 and aj = 0 for j > 20.

We apply three methods to the 100 simulated datasets:single SNP analysis (SSA), standard Bayesian lasso (B-lasso),and the Bayesian lasso applied to the preconditioned responsefrom supervised principal components (PB-lasso). In singleSNP analysis, we reject the null hypothesis that the genetic

6

by guest on Decem

ber 15, 2010bioinform

atics.oxfordjournals.orgD

ownloaded from

Fig. 4. Estimated heritability explained by each SNP based on 50simulations.

effect of an individual SNP equals to zero at the significancelevel of 5% with the FDR adjustment. For the Bayesianlasso and preconditioned Bayesian lasso, we reject the nullhypothesis based on 95% Bayesian credible intervals. Toameliorate the bias of the parameter estimates introduced bylasso penalties, we always refit the linear regression modelwithout the penalty term using only those SNPs selected bythe model selection procedure.

For each estimated genetic effect obtained from eachmethod, we calculate the average bias and empirical standarderror over 100 simulations. Since the first 20 genetic effectsare nonzeros with the same true value, in Table 3 we reportthe average values over the first 20 SNPs and over the restof the SNPs separately. The standard error of each averageis in parentheses. In the column labeled ”Aver. Nonzeros”,we present the average number of nonzero coefficientscorrectly identified to be nonzero, or the average number ofzero coefficients incorrectly estimated to be nonzero in 100replications. In the column ”Proportion of Correct-fit”, wepresent the proportion of replications that the exact true modelwas identified.

As can be seen from Table 3, the single SNP analysistend to overestimate the genetic effect, since when we testa SNP for the association with the phenotype, we assumethe genetic variation is solely due to this particular SNP,and ignore the effects from all other SNPs. Therefore, interms of parameter estimates, model selection methods thatsimultaneously estimate the genetic effects associated withall SNPs outperform the traditional single SNP analysis. Interms of variable selection, although preconditioned Bayesianlasso has a slightly higher false positive rate due to thepreconditioning step, it greatly improves the probabilityof correctly identifying regression coefficients with nonzeroeffects. Moreover, as the number of SNPs gets larger, singleSNP analysis detected fewer important SNPs, since this methodsubjects to severe multiple comparison adjustment. However,preconditioned Bayesian lasso is still able to identify nonzerocoefficients and zero coefficients correctly in almost every

Table 3. Simulation results for three methods based on 100simulations.

Empirical Aver. Proportion ofMethod Bias SE Nonzeros Correct-fit

n = 200, p = 500, β1 - β20SSA 4.17 1.99 16.62 0.08

(0.21) (0.19) (1.51)B-lasso 0.07 0.34 18.28 0.18

(0.04) (0.02) (1.36)PB-lasso 0.00 0.35 19.68 0.63

(0.03) (0.03) (0.65)

n = 200, p = 500, β21 - β500SSA 0.44 0.43 0.78 0.46

(0.05) (0.07) (1.09)B-lasso 0.00 0.04 0.95 0.42

(0.01) (0.01) (0.94)PB-lasso 0.00 0.03 1.25 0.30

(0.01) (0.04) (0.73)

n = 200, p = 1000, β1 - β20SSA 4.13 1.96 15.71 0.04

(0.18) (0.17) (2.73)B-lasso 0.36 0.38 17.11 0.08

(0.06) (0.07) (2.69)PB-lasso 0.00 0.36 19.24 0.51

(0.04) (0.03) (1.81)

n = 200, p = 1000, β21 - β1000SSA 0.43 0.43 0.42 0.69

(0.04) (0.06) (0.84)B-lasso 0.00 0.02 0.33 0.76

(0.01) (0.01) (0.18)PB-lasso 0.00 0.02 1.17 0.56

(0.00) (0.01) (1.38)

simulation. Supplementary Table 1 displays the simulationresults when ρ = 0.5, which are consistent with our findings.

5 DISCUSSION

When the number of predictors p is much larger than thenumber of observations n, highly regularized approaches,such as penalized regression models, are needed to identifynonzero coefficients, enhance model predictability, and avoidover-fitting (Hastie el al., 2009). The L1 penalized regressionor lasso is such one of the most popular techniques. Inthis article, we presented a Bayesian hierarchical modelwith lasso penalties to simultaneously fit and estimate allpossible genetic effects associated with all SNPs in a GWAS,adjusting for both discrete and continuous covariates. Lassopenalties are imposed on the additive and dominant effects, andimplemented by assigning double-exponential priors to theirregression coefficients. It shrinks small effects towards zeroand produces sparse solutions. In this framework, SNPs withsignificant genetic effects can be identified more accurately.

7

by guest on Decem

ber 15, 2010bioinform

atics.oxfordjournals.orgD

ownloaded from

We fit the model in a fully Bayesian approach, employingthe MCMC algorithm to generate posterior samples from thejoint posterior distribution, which can be used to make variousposterior inferences. Although computationally intensive, itis easy to implement and provides not only point estimatesbut also interval estimates of all parameters. The Bayesianlasso treats tuning parameters as unknown hyperparametersand generates their posterior samples when estimating otherparameters. This technique avoids the choice of tuningparameters, and automatically accounts for the uncertaintyin its selection that affects the estimation of the finalmodel. By contrast, standard lasso algorithms usually selecttuning parameters by K-fold cross-validation, which involvespartitioning the whole data set and refitting the model manytimes. This process may result in unstable tuning parameterestimates.

In order to improve the performance of lasso whenp is greater than n, preconditioning is considered beforevariable selection. Preconditioning encourages the principalcomponents of a reduced design matrix to be highly correlatedwith the response, and thus in most cases only the firstor first few components tend to be useful for prediction.It denonises the response variable so that variable selectionbecomes more efficient. Our simulation demonstrated thatwhen p greatly exceeds n, preconditioned Bayesian lassocould successfully identify almost all the SNPs with truegenetic effects. By analyzing real data, our approach is shownto produce biologically relevant results. For example, theapproach detected a significant SNP ss66171460 at position22580931 of chromosome 20 associated with BMI. It isinteresting to note that this SNP is within 500Kb of the FOXA2(Forkhead Box A2) gene, an important genetic variant thatregulates obesity (Wolfrum et al., 2003).

One simulation example of Paul et al. (2008) implies that,in the context of genome-wide association studies, SNPs thatare marginally independent of the phenotype could be screenedout by preconditioning, but can be identified by standardvariable selection techniques such as lasso or Bayesian lasso.In theory, if SNPs are correlated with the phenotype throughmarginal correlations, we believe the preconditioning step isworthwhile to identify more important SNPs. However, inreality, since different SNPs may display interactions, thisapproach may not work perfectly. In any case, this two-stepvariable selection procedure should always be advantageousover a single SNP analysis, because we are always testing themarginal correlation between the predictor and response whenone SNP is analyzed at a time.

Motivated by Tibshirani (1996), Park and Casella (2008)developed the Bayesian lasso and demonstrated by an thediabetes data (Efron et al., 2004) with p = 10 and n = 442. Weapplied the Bayesian lasso to the high-dimensional regressionproblem, and improved it by preconditioning. We haveconcentrated on the preconditioned Bayesian lasso methodfor continuous trait in GWASs. The proposed preconditioningprocedure and MCMC algorithm can be readily extended tosurvival data analysis and lasso penalized logistic regressionin case−control disease gene mapping. Also, we may lookfor gene-gene interaction effects after identifying main effects,

as suggested by Wu et al. (2009). The model with a capacityto identify epistatic interactions will enables geneticists todecipher a detailed picture of the genetic architecture of acomplex trait.

ACKNOWLEDGEMENT

Funding: NSF/NIH Mathematical Biology grant (No. 0540745),NIDA, NIH grants R21 DA024260 and R21 DA024266. Thecontent is solely the responsibility of the authors and does notnecessarily represent the official views of the NIDA or the NIH.

REFERENCESAndrews, D. F., and Mallows, C. L. (1974) Scale mixture of normal distributions,

J. R. Stat. Soc. Ser. B, 36, 99-102.Bair, E., Hastie, T., Paul, D. and Tibshirani, R. (2006) Prediction by supervised

principal components, J. Amer. Stat. Assoc., 101, 119-137.Chen, S. S. (1998) Atomic decomposition by basis persuit, SIAM J. Sci. Comput.,

20, 33-61.Daubechies, I., Defrise, M. and De Mol, C. (2004) An iterative thresholding

algorithm for linear inverse problems with a sparsity constraint, Comm. PureAppl. Math., 57, 1413-1457.

Donnelly, P. (2008) Progress and challenges in genome-wide association studiesin humans, Nature, 465, 728-731.

Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004) Least angleregression (with discussion), Ann. Statist., 32, 407-499.

Fan, J. and Li, R. (2001) Variable selection via nonconcave penalized likelihoodand its oracle properties, J. Am. Stat. Assoc., 96, 1348-1360.

Fan, J. and Lv, J. (2008) Sure independence screening for ultrahigh dimensionalfeature space (with discussion), J. R. Stat. Soc. Ser. B, 70, 849-911.

Frank, I. E. and Friedman, J. H. (1993) A statistical view of some chemometricsregression tools, Technometrics, 35, 109-148.

Fu, W. J. (1998) Penalized regression: the bridge versus the LASSO, J. Comput.Graph. Statist., 7, 397-416.

Gelman, A. and Rubin, D.B. (1992) Inference from iterative simulation usingmultiple sequences, Stat. Sci., 7, 457-511.

Hastie, T., Tibshirani, R. and Friedman, J. (2009) High-Dimensional Problems:p > N . The elements of statistical learning. 2nd edn. Springer, New York.

Hoggart, C., Whittaker, J., De lorio, M. and Balding, D. (2008) Simultaneousanalysis of all SNPs in genome-wide and re-sequencing association studies,PLoS Genet., 4(7), e1000130.

Hoti, F. and Sillanpaa, M. J. (2006) Bayesian mapping of genotype × expressioninteractions in quantitative and qualitative traits, Heredity, 97, 4-18.

Kim, Y., Choi, H. and Oh., H. (2008) Smoothly clipped absolute deviation onhigh dimensions, J. Am. Stat. Assoc., 103, 1665-1673.

Logsdon, B.A., Hoffman, G.E. and Mezey, J.G. (2010) A variational Bayesalgorithm for fast and accurate multiple locus genome-wide associationanalysis, BMC Bioinformatics, 27, 11-58.

McCarthy, M., Abecasis, G., Cardon, L., Goldstein, D., Little, J., Ioannidis,J. and Hirschhorn, J. (2008) Genome-wide association studies for complextraits: consensus, uncertainty and challenges, Nat. Rev. Genet., 9(5), 356-369.

Park,T., and Casella, G. (2008) The Bayesian lasso, J. Am. Stat. Assoc., 103,681-686.

Paul, D., Bair, E., Hastie, T. and Tibshirani, R. (2008) Preconditioning for featureselection and regression in high-dimensional problems, Ann. Stat., 36, 1595-1618.

Rosset, S. and Zhu, J. (2007) Piecewise Linear Regularized Solution Paths, Ann.Stat., 35, 1012-1030.

Tibshirani, R. (1996) Regression shrinkage and selction via the lasso, J. R. Stat.Soc. Ser. B, 58, 267-288.

Wolfrum, C., Shih, D. Q., Kuwajima, S., Norris, A. W., Kahn, C. R. and Stoffel,M. (2003) Role of Foxa-2 in adipocyte metabolism and differentiation, J. Clin.Invest., 112, 345-356.

Wu, T. T., Chen, Y. F., Hastie, T., Sobel, E. and Lange, K. (2009) Genome-wideassociation analysis by lasso penalized logistic regression, Bioinformatics, 25,

8

by guest on Decem

ber 15, 2010bioinform

atics.oxfordjournals.orgD

ownloaded from

714-721.Yang, J., Benyamin, B., McEvoy, B., Gordon, S., Henders, A., Nyholt,

D., Madden, P., Heath, A., Martin, N., Montgomery, G., Goddard, M.and Visscher, P. (2010) Common SNPs explain a large proportion of theheritability for human height, Nat. Rev. Genet., 42, 565-569.

Yi,N. and Xu, S. (2008) Bayesian lasso for quantitative trait loci mapping,Genetics, 179, 1045-1055.

Zhao, P. and Yu, B. (2006) On model selection consistency of Lasso, J. Mach.Learn. Res., 7, 2541-2563.

Zou, H. (2006) The adaptive lasso and its oracle properties, J. Amer. Stat. Assoc.,101, 1418-1429.

Zou, H. and Hastie, T. (2005) Regularization and variable selection via the elasticnet, J. R. Stat. Soc. Ser. B, 67, 301-320.

9

by guest on Decem

ber 15, 2010bioinform

atics.oxfordjournals.orgD

ownloaded from