On the Efficiency of Designs for Linear Models in Non-regular Regions and the Use of Standard...
Transcript of On the Efficiency of Designs for Linear Models in Non-regular Regions and the Use of Standard...
On the Efficiency of Designs for Linear Models in Non-regular Regions
and the Use of Standard Designs for Generalized Linear Models
Alyaa R. Zahran
Dissertation submitted to the Virginia Polytechnic Institute and State University in partial
fulfillment of the requirements for the degree of
Doctor of Philosophy
in
Statistics
Christine Anderson-Cook, co-chair
Raymond H. Myers, co-chair
Eric P. Smith, co-chair
John Morgan
Keying Ye
July 1, 2002
Blacksburg, Virginia
Key Words: design optimality, fraction of design space technique, non-regular design
spaces, linear models, generalized linear models
On the Efficiency of Designs for Linear Models in Non-regular Regions
and the Use of Standard Designs for Generalized Linear Models
Alyaa R. Zahran
(ABSTRACT)
The Design of an experiment involves selection of levels of one or more factor in
order to optimize one or more criteria such as prediction variance or parameter variance
criteria. Good experimental designs will have several desirable properties. Typically, one
can not achieve all the ideal properties in a single design. Therefore, there are frequently
several good designs and choosing among them involves tradeoffs.
This dissertation contains three different components centered around the area of
optimal design: developing a new graphical evaluation technique, discussing designs for
non-regular regions for first order models with interaction for the two- and three-factor
case, and using the standard designs in the case of generalized linear models (GLM).
The Fraction of Design Space (FDS) technique is proposed as a new graphical
evaluation technique that addresses good prediction. The new technique is comprised of
two tools that give the researcher more detailed information by quantifying the fraction of
design space where the scaled predicted variance is less than or equal to any pre-specified
value. The FDS technique complements Variance Dispersion Graphs (VDGs) to give the
researcher more insight about the design prediction capability. Several standard designs
are studied with both methods: VDG and FDS.
Many Standard designs are constructed for a factor space that is either a p-
dimensional hypercube or hypersphere and any point inside or on the boundary of the
1
shape is a candidate design point. However, some economic, or practical constraints may
occur that restrict factor settings and result in an irregular experimental region. For the
two- and three-factor case with one corner of the cuboidal design space excluded, three
sensible alternative designs are proposed and compared. Properties of these designs and
relative tradeoffs are discussed.
Optimum experimental designs for GLM depend on the values of the unknown
parameters. Several solutions to the dependency of the parameters of the optimality
function were suggested in the literature. However, they are often unrealistic in practice.
The behavior of the factorial designs, the well-known standard designs of the linear case,
is studied for the GLM case. Conditions under which these designs have high G-
efficiency are formulated.
v
AcknowledgementsAll praise and thanks is to Allah, the one, the only and the indivisible creator and
sustainer of the worlds. To Him, we belong and to Him, we will return. I wish to thank
Him for all that He has gifted me with, although, He can never be praised or thanked
enough.
I would like to express my deepest thanks and sincere appreciation to my
advisors: Prof. C. Anderson-Cook, Prof. R. Myers, and Prof. EP. Smith, for their strong
support throughout this study. Their high standards and goals, as well as their genuine
interest in science were both very challenging and motivating. I would like to thank Prof.
C. Anderson-Cook for her constant guidance, patience and encouragement. Grateful
acknowledgements to Prof. R. Myers for his enthusiasm, understanding, guidance and
considerable help throughout this research. I am truly thankful for Prof. E.P. Smith for
his valuable advice and assistance during this research.
I would like to express my appreciation to my committee members Prof. Morgan
and Prof. Ye for their valuable suggestions and interest during my research. Special
thanks to Prof. G. Terrell for his valuable comments in this research.
I am grateful to the Virginia Water Resources Research Center and U.S.
Environmental Protection Agency’s Science to Achieve Results (STAR) for funding this project
(Grant No. R82795301).
Finally, I would like to give my special recognition to my husband, Farouk, for
his understanding, interest and support at every stage of this research. My deepest thanks
go to my parents for their love, prayer, support, and subtle encouragement throughout
my life.
vi
Table of Contents
List of Figures ______________________________________________ ix
List of Tables _______________________________________________ xi
Chapter I Introduction and Literature Review______________________1I.1 Introduction _______________________________________________ 1
I.2 Brief Review of Some Concepts in Optimality Theory_____________ 6I.2.1 Optimality Criteria and Efficiency__________________________________ 6
I.2.1.1 D-Optimality___________________________________________________________ 6I.2.1.2 G- Optimality __________________________________________________________ 7I.2.1.3 Q-Optimality___________________________________________________________ 7
I.2.2 The General Equivalence Theorem for D- and G-optimum designs ________ 8I.2.3 Orthogonality and Rotatability_____________________________________ 8
I.2.3.1 Orthogonality __________________________________________________________ 8I.2.3.2 Rotatability ____________________________________________________________ 9
I.2.4 Graphical Methods for the Performance of the Prediction Capability in theRegion of Interest____________________________________________________ 9
I.3 Some Response Surface Designs ______________________________ 10I.3.1 Two-Level Factorial and Fractional of Resolution III, IV _______________ 10I.3.2 Second Order Model Designs ____________________________________ 11
I.3.2.1 Central Composite Designs (CCD)_________________________________________ 11I.3.2.2 Box-Behnken Designs (BBD)_____________________________________________ 12I.3.2.3 Small Central Composite Designs (SCD)____________________________________ 12I.3.2.4 Hybrid Designs ________________________________________________________ 13
I.4 Design Optimality for Generalized Linear Models_______________ 13I.4.1 Locally Optimal Designs ________________________________________ 14I.4.2 Minimax Approach ____________________________________________ 14I.4.3 Bayesian Approach ____________________________________________ 15I.4.4 Sequential Designs Approach ____________________________________ 15
I.5 Layout of Dissertation ______________________________________ 15
I.6 References ________________________________________________ 16
Chapter II Fraction of Design Space to Assess the Prediction Capability of
Response Surface Designs ___________________________21II.1 Abstract __________________________________________________ 21
II.2 Introduction ______________________________________________ 21
II.3 Review of Variance Dispersion Graphs (VDG)__________________ 25
II.4 The Fraction of Design Space Criterion (FDS) __________________ 26
II.5 Comparisons of the Standard Second-Order Designs over Spherical
Region __________________________________________________ 29
vii
II.5.1 Example: Two Factors on Spherical Region _______________________ 30II.5.2 Example: Three Factors on Spherical Region ______________________ 30II.5.3 Example: Four Factors on Spherical Region _______________________ 34II.5.4 Example: Five Factors on Spherical Region________________________ 34II.5.5 Example: Six Factors on Spherical Region ________________________ 39
II.6 Comparisons of the Standard Second-Order Models over Cuboidal
Region with Three Factors __________________________________ 41
II.7 Conclusions _______________________________________________ 43
II.8 References ________________________________________________ 43
Chapter III Modifying 22 Factorial Designs to Accommodate a Restricted
Design Space _____________________________________45III.1 Abstract __________________________________________________ 45
III.2 Introduction ______________________________________________ 46
III.3 Design Space and Possible Designs____________________________ 47III.3.1 Design I____________________________________________________ 48III.3.2 Design II ___________________________________________________ 48III.3.3 Design III __________________________________________________ 49III.3.4 Comparison of Designs________________________________________ 49
III.4 Example__________________________________________________ 55
III.5 General Design Space and Design I ___________________________ 56
III.6 Conclusions and Discussion__________________________________ 58
III.7 References ________________________________________________ 59
Supplement I: Modifying 23 Factorial to Accommodate a Restricted Design
Space ____________________________________________ 61
Supplement II: FDS Technique for the Three Designs in Restricted Design
Space ____________________________________________ 71SII.1 Two-Factor Case ______________________________________________ 71SII.2 Three-Factor Case _____________________________________________ 76
Chapter IV Use of Standard Factorial Designs with Generalized Linear
Models_________________________________________80IV.1 Abstract __________________________________________________ 80
IV.2 Introduction ______________________________________________ 80
IV.3 Implementation of Standard Designs__________________________ 82
IV.4 Characteristics of the Information Matrix in GLM ______________ 83
viii
IV.4.1 Use of the two-level Factorial___________________________________ 84IV.4.2 Examples___________________________________________________ 85
IV.5 General Results for GLM with Canonical Link and Standard 22
Factorial Design ___________________________________________ 89IV.5.1 Properties of the Scaled Prediction Variance for the 22 Factorial Design _ 89IV.5.2 Interaction Model - Equal Asymptotic Variances of the Parameters Estimates
____________________________________________________________91IV.5.3 Interaction Model -the Scaled Prediction Variance at the Design Points__ 91
IV.6 A characterization of Efficiency for the Use of 22 Factorial Designswith the Logistic and Poisson Regression Models ____________________ 91
IV.6.1 First Order Models with and without Interaction: Logistic Regression ___ 91IV.6.2 First Order Models with and without Interaction: Poisson Regression ___ 93
IV.7 Variance Stabilizing Link ___________________________________ 94
IV.8 Real Life Example _________________________________________ 95
IV.9 Conclusions _______________________________________________ 96
IV.10 References ______________________________________________ 97
Appendix A: __________________________________________________ 100
Appendix B___________________________________________________ 102
Appendix C___________________________________________________ 104
Appendix D___________________________________________________ 106
Chapter V Summary and Future Work_________________________107
ix
List of FiguresFigure II.1: Effect of Increasing Dimension on the Percentage of Volume at Radius r_ 24Figure II.2: VDG of some Two-Factor Designs _______________________________ 26Figure II.3: Volume for CCD with variance = 4 over Cuboidal Region and FDSG ____________ 28FigureII.4: Second Order Designs for Spherical Region in Two Factors ___________ 31Figure II.5: VDG for Second Order Designs for Spherical Region in Three Factors __ 32Figure II.6: Second Order Designs for Spherical Region in Three Factors _________ 33Figure II.7: VDG for Second Order Designs for Spherical Region in Four Factors___ 35Figure II.8: Second Order Designs for Spherical Region in Four Factors __________ 36Figure II.9: VDG for Second Order Designs for Spherical Region in Five Factors ___ 37Figure II.10: Second Order Designs for Spherical Region in Five Factors _________ 38FigureII.11: VDG for Second Order Designs for Spherical Region in Six Factors____ 39Figure II.12: Second Order Designs for Spherical Region in Six Factors___________ 40Figure II.13: Second Order Designs for Cuboidal Region in Three Factors_________ 42Figure III.1: Restricted Operability Region __________________________________ 47Figure III.2: Operability Region for r = 0.1 and 0.5 ___________________________ 48Figure III.3: The Alphabetical Criteria of Designs II, and III ____________________ 52Figure III.4: Contour Plots of v(x) for the three Designs at r=0.1 ________________ 53Figure III.5: Contour Plots of v(x) for the three Designs at r=0.5 ________________ 54Figure III.6: Operability Region for different d values _________________________ 57Figure SI.1: Modifying the Cuboidal Operability Region in the Three Factor Case___ 63Figure SI.2: The Definition of θi and φi _____________________________________ 63Figure SI.3: Total Space Volume of the Restricted Region and the Volume of the Design Space of Design III___________________________________________ 64Figure SI.4: Relative D-efficiency of Designs II and III to Design I _______________ 68Figure SI.5: G-efficiency of the three Designs ________________________________ 69Figure SI.6: Relative Q-efficiency of the three Designs _________________________ 69Figure SII.1: FDSG of Design I in Two Factors for Different Values of r and d=2 ___ 72Figure SII.2: Contour Plots of v(x) for Design I at r=0.1, 0.5, 1 __________________ 73Figure SII.3: FDSG of Design I, II, and III in Two Factors for r= 0.1 and d=2 _____ 74Figure SII.4: FDSG of Design I, II, and III in Two Factors for r= 0.5 and d=2 _____ 75Figure SII.5: SFDSG of Design I, II, and III in Two Factors for r= 0.5 and d=2 ____ 75Figure SII.6: FDSG of Design I for different values of r and d=2_________________ 77Figure SII.7: FDSG of Design I, II, and III in Three Factors for r= 0.1 and d=2_____ 77Figure SII.8: SFDSG of Design I, II, and III in Three Factors for r= 0.1 and d=2____ 78Figure SII.9: FDSG of Design I, II, and III in Three Factors for r= 0.5 and d=2_____________________________________________________________________ 79Figure IV.1: Contour Plot of v(x) for 6.0p4.0 ≤≤ ___________________________ 86Figure IV.2: Contour Plot of v(x) for 7.0p4.0 ≤≤ ___________________________ 87Figure IV.3: Contour Plot of v(x) for 2010 ≤≤ µ ____________________________ 88Figure IV.4: Contour Plot of v(x) for 505 ≤≤ µ _____________________________ 88Figure IV.5: Contour Plot of v(x) for 33.2=δ _______________________________ 92Figure IV.6: Contour Plot of v(x) for 3=δ __________________________________ 93
x
Figure IV.7: Contour Plot of v(x) for 10=δ _________________________________ 94Figure IV.8: Scaled Prediction Variance Contour Plot for Spermatozoa Example____ 96
xi
List of TablesTable III.1: Comparison of Designs ________________________________________ 51
Table III.2: Alphabetical Relative Efficiency of Design I to the 22 Factorial Design __ 55
Table III.3: Design matrix of Design I and II ( 8/πθ = ) for Different combinations of (d, r) _______________________________________________________ 58
Table S1.1: Comparison of Designs with Model I _____________________________ 66
Table SI.2: Quantitative Measures of Designs with Model II ____________________ 67
Table SI.3: Alphabetical Relative Efficiency of Design I to the 23 Factorial Design for Model I _____________________________________________________ 67
Table SI.4: Design matrix of Design I and III for Different values of r _____________ 70
Table IV.1: Maximum Likelihood Estimates and Wald Inference on Individual Coefficients _________________________________________________ 96
Table IV.2: G-efficiency of the 22 Factorial with Logistic Regression_____________ 104
Table IV.3: G-efficiency of the 22 Factorial with Poisson Regression_____________ 105
Table IV.4: Spermatozoa Survival Data and Design Matrix ____________________ 106
1
Chapter I Introduction and Literature Review
I.1 IntroductionExperimentation is an important part of many decision-making problems.
Usually, one performs an experiment, and hopes the outcome results in a near-optimal
decision. In order for the decision to be as accurate as possible, it is desirable that an
optimal or near-optimal experiment should be used initially. Thus, optimality theory has
found its function in experimental design. Actually, the subject has matured over the last
50 years to the point that a powerful body of theory and methodology has been produced.
The focus of optimality theory is the selection of a design, which maximizes the
information from a finite-size experiment. Implementing this selection requires a measure
of optimality. This measure is generally taken to be a real valued function of Fisher’s
information matrix. Historically, Smith (1918) appears to be the first to formally
introduce a specific optimality criterion in comparing designs in a given experimental set-
up. Types of optimality considered by Wald (1943) and Ehrenfeld (1953) together with
some new forms of optimality were explicitly named in Kiefer (1958). These criteria are
known now as the alphabetical optimality criteria. D-optimality focuses on the variances
of the estimates of the coefficients in the model, while G-optimality focuses on the
maximum variance of a predicted value over the region of interest. A major advance was
the equivalence theorem for D- and G-optimum designs (Kiefer and Wolfowitz,1960).
Since the early seventies, optimality theory was the main subject of several textbooks.
Among these are: Fedorov (1972), Silvey (1980), Pazman(1986), Shah and Sinha (1989),
Atkinson and Donev (1992), and Pukelsheim (1993).
However, a single optimality criterion is unlikely to capture all of the desirable
features of a design. A design is optimal relative to criteria, and the criteria measure the
attainment of the objectives of the underlying experiment. Hence, a variety of tools,
criteria and approaches have been developed for variety of problems as Atkinson (1996)
outlines. Accordingly, one should use the criterion that addresses a desired goal, such as
good estimation or good prediction. However, after one finds the optimal design, it is
2
important to observe how that design performs in other respects. Kiefer (1975) argues
that when selecting a design we should look at many performance criteria, since all the
optimality criteria are merely approximation to some vague notion of “Goodness”. A
slightly less efficient design in terms of some criterion, say Φ1, might be superior to the
Φ1-optimal design in terms of other criterion, say Φ2. A numerical measure of how well
one design performs relative to another is its “efficiency”.
Box and Hunter (1957) emphasized that judging a design should be on the basis
of the distribution of prediction variance. Since the experimenter does not know at the
outset where in the design space he/she might wish to predict, a reasonably stable
prediction variance over the whole region is desired. Thus, one should consider how well
the design performs over every part of the region of interest. This indicated very early
that single-number criteria might not be enough information when comparing designs. In
their paper, Box and Hunter (1957) introduced the design rotatability notion, which
requires that the variance of a predicted value remain constant at points that are
equidistant from the design center. The importance of this property evolved naturally
from the need to achieve stability in prediction variance. Rotatability was just a first step.
In the two factor case, a contour plot of the scaled prediction variance, v(x),
provides a more complete picture of the performance of the design in terms of prediction.
However, such contours are limited to three components systems. Giovannitti-Jensen and
Myers (1989) introduced the variance dispersion graphs (VDG) which allow studying the
distribution of v(x) in the region of interest for any number of design variables. Myers et
al. (1992) used such plots to compare several standard second-order designs on the basis
of their prediction capabilities over spherical and cuboidal regions. Khuri et al. (1996)
proposed the quantile plots for describing the distribution of the prediction variances.
In brief, several multifaceted aspects must be taken into account when choosing a
design for a particular situation. One should not select a design solely on the basis of a
single criterion. Good designs will have many desirable aspects. Optimal designs relative
to a single criterion are an attempt at good designing and, at the least, offer a bench-mark
3
for efficiency studies. Properties of a good response surface design are discussed in Box
and Hunter (1957), Box and Draper (1975), Atkinson and Donev (1992) and Myers and
Montgomery (2002). Typically, one can not achieve all the ideal properties in a single
design. Therefore, there are frequently several good designs and choosing among them
involves some tradeoffs. We would like to select designs that are intuitively pleasing,
relatively easy to implement, are able to estimate all effects of interest and have good
efficiencies.
The problem of optimal experimental design for linear models (i.e. normal
responses with homogenous variances) has received much attention in the literature.
Assuming that the factor space is a p-dimensional hypercube or hypersphere with any
point inside or on the boundary of the shape being a candidate design point, standard
designs for linear models were identified. However, something other than the usual
optimal design is needed in any situation where the design space is not regular! Some
economic, practical, or physical constraints may occur on the factor settings resulting in
an irregular experimental region. One often encounters situations in which it is necessary
to eliminate some portion of the design space where it is infeasible or impractical to
collect experimental data. Hence, standard designs are not always feasible and the need
arises for best possible designs under these restrictions. Kennard and Stone (1969) were
the first to discuss in the literature the problem of irregular experimental regions and
suggested computer aided searches for selecting a design. Some case-by-case examples
of non-standard design regions are discussed in Snee (1985). Johnson and Nachtsheim
(1983) discussed how single-point augmentation procedures are helpful for finding exact
D-optimal Designs on Convex Design spaces. Recognizing the importance of computer
programs to develop designs when classical designs are not appropriate, Nachtsheim
(1987) reviewed and compared the available tools for computer-aided design of
experiments. Atkinson and Donev (1992) devoted a short chapter to restricted designs.
They used some computer algorithms to find the D-optimum design for certain irregular
regions. They emphasize that whatever the shape of the experimental region the
principles of the optimality theory remain the same. Montgomery, Loredo, Jearkpaporn,
4
and Testik (2002) give a brief tutorial on computer-aided methods for constructing
designs for irregularly shaped regions.
As mentioned earlier, optimal designs for linear models have been studied
extensively in the literature. The information matrix and hence the optimality function is
independent of the unknown parameters, thus optimal designs are relatively easy to find.
However, for generalized linear models (GLMs), less work has been done, because the
optimality function is a function of the unknown parameters which complicates the
process of finding an optimal design (McCullagh and Nelder, 1989).
For the one-design-variable logistic regression models, Kalish and Rosenberger
(1978) derived a D- and G-optimal design. Abdelbasit and Plackett (1983) found a D-
optimal design using the idea of two-stage model. Myers, Myers, Carter and White
(1996) introduced a two stage D-Q design. Letsinger (1995) introduced a two-stage D-D
design using a Bayesian approach in the first stage. Chaloner and Larntz (1989) took a
Bayesian approach to find robust optimal designs. A minimax procedure for finding the
optimal designs is proposed by Sitter (1992). Sitter and Wu (1993) used a different
approach to obtain characterizations of the D-, A-, and F-optimal designs for binary
response and one single variable model. F optimality deals with optimal estimation of an
ED (Effective Dose), i.e., the level of x, the drug dose, that produce a pre-specified
probability. Jia and Myers (2001) found D-optimal designs for the two-variable logistic
model in unbounded regions. Other works that considered the logistic model are Heise
and Myers (1996) and Sitter and Wu (1999).
For the one-variable Poisson regression model, Chiacchirini (1996) developed
optimal designs using a two-stage approach. Huffman (1998) studied the robustness of
the Bayesian techniques to parameter misspecification for single- and multiple-variable
models.
For the general GLM model, Ford, Torsney and Wu (1992) developed D-optimal
designs for one design variable using the geometry of the design space. Sitter and
5
Torsney (1995) developed methods for deriving D-optimal design for GLM with multiple
design variables. Burridge and Sebastiani (1994) considered the multi-variable case of
GLM models for cases in which variances are proportional to the square of the mean
response. In addition, Burridge and Settimi (1998) and Dette and Wong (1999) deal with
a more general problem. Atkinson and Haines (1996) discussed locally D-optimal designs
for nonlinear models in general and GLM as well as the Bayesian approach for
optimality.
The above literature on optimality in the GLM case offers some valuable optimal
designs. However, designs that need good initial guesses, a complicated minimax
procedure, or estimation of the unknown parameters at each step to make the next move,
often seem unrealistic in practice. One wants experiments which are simple to implement
and also have ‘good’ efficiencies!
In this dissertation, we address three research projects related to optimal design:
1. We introduce a new graphical evaluation technique, which shows the stability of the
distribution of the prediction variance. The new technique focuses on how well the
design predicts for any fraction of the design space. Some second order response
surface designs are studied in terms of this new measure.
2. For the two-factor case with one corner of the square design space excluded, three
sensible alternatives designs are proposed. These designs involve reducing the factor
levels to make a smaller but standard factorial design fit or modifying the levels of
the variables at the excluded corner to locate it in the feasible design region.
Properties of these designs and relative tradeoffs are discussed. The work is also
extended to the three-factor case. Also, the alternative designs in both the two and the
three- factor cases are studied in terms of the new criteria introduced in the previous
point.
3. Study the performance of standard designs for generalized linear models. Some
results that are general to all GLMs are given. The logistic and Poisson regression
models are studied extensively.
6
I.2 Brief Review of Some Concepts in Optimality TheoryOriginally, optimality theory was introduced for linear models, where the model
of interest is1ppN1N
X)(E×××
= βy , with y a vector of N independent Normal responses
having constant variance σ2, X the design matrix, and β a vector of p unknown
parameters. Optimality theory centers around Fisher’s information matrix. One can show
that for the above model the information matrix is proportional to X’X. This matrix is
parameter free; thus, optimization in the linear models case depends only on the design
matrix.
I.2.1 Optimality Criteria and EfficiencyAs indicated earlier, many optimality criteria exist in the literature. In this
research we focus on three of them; namely, D-, G-, and Q-criterion. We begin our brief
review with the criterion used most by practitioners, namely the D-criterion.
I.2.1.1 D-Optimality The D-optimality criterion is the most often used criterion, because of its early
development and relative ease of calculation. For quantitative factors, the D-optimum
design remains unchanged with any change in the scale of the factors. The D-optimality
criterion focuses on the determinant of the information matrix. A D-optimal design
satisfies the following
)(MmaxNXX
maxDD
δδδ ∈∈
=′
where δ is a design in the design space D and N is the number of experimental runs.
)(M δ is called the moments matrix. Accordingly, the D-optimal design is that design
that maximizes the information per run. One should notice that, under independence and
normality, the determinant of the information matrix is inversely proportional to the
square of the coefficient confidence region volume. Hence, maximizing the determinant
is equivalent to minimizing the volume of the confidence region of the coefficients.
Therefore, the D-criterion addresses good parameter estimation (Myers and Montgomery,
2002).
7
To compare designs, the D-efficiency of a design, δ, is defined asp/1
*eff)(M
)(MD
=
δ
δ, where δ* is the D-optimal design.
I.2.1.2 G- Optimality Another important goal for any design is good prediction. Usually, the researcher wants
to predict as well as possible at any point in the region of interest. One measure of
prediction performance is the scaled prediction variance, which is defined as
0
1
02 )XX(N)ˆvar(N
)x(v xxy −′′==
σ;
where x0 is a point in the region of interest, expanded to the model space, at which we
predict. Notice that the multiplication by N gives the notion of “per observation” basis.
That is, the design with more runs is penalized in terms of larger prediction variance. The
division by σ2 makes the scaled prediction variance a scale-free quantity. A design, δ*,
that minimizes the maximum scaled prediction variance over the region of interest, i.e.
minimizes ))x(v(max Rx∈
, is the G-optimal design. In this sense, we are rewarding a design
with the best “worst case” variance. Under independence and homogeneity of the
variance, we have p))x(v(maxRx
≥∈
(Myers and Montgomery, 2002). This result leads to
the G-efficiency, defined as
Rx
eff ))x(vmax(p
G
∈
= , where R is the region of interest.
I.2.1.3 Q-Optimality A major disadvantage of the G-optimality is that one needs to calculate v(x) at each point
in the region of interest to determine the worst case variance. To eliminate this
computation, an averaging, for quantitative continues factor-levels, technique is used in
the Q-criterion. In this sense, we are examining a measure that considers all variances
throughout the design space. Q-optimality is also called V- or IV-optimality in the
literature. For more details, see Draper and St. John (1977). A design is optimal in the
sense of the Q-criterion if it minimizes the average of v(x), i.e.
8
∫∈∈==
RDD
* dx)x(vK1
min)(Qmin)(Qδδ
δδ , where ∫=R
dxK , the volume of the design region.
The Q-efficiency is defined as )(Q
)(QminQ *
Deff δ
δδ∈= .
I.2.2 The General Equivalence Theorem for D- and G-optimum designsKiefer and Wolfowitz (1960) introduced the general equivalence theorem for
linear models. It has also been generalized for the nonlinear models (White, 1973). Many
authors discussed the theorem and its generalization. Among them are Fedorov (1972),
Silvey (1980), and Atkinson and Donev (1992).
Under very mild assumptions, the theorem states the equivalence of the following
three conditions on the optimum design, *ξ :
− *ξ is D-optimum
− *ξ is G-optimum
− the maximum prediction variance is equal to p, the number of the unknown
parameters in the model, and is achieved at the design points.
I.2.3 Orthogonality and RotatabilityOrthogonality and rotatability are two desired properties of any design.
Orthogonality is a dominant property for first-order models, while rotatability is more
important for second order models. This stems from the fact that for first order models
the primary concern is typically about what variables belong in the model. In the second
order models, more emphasis placed on the quality of the prediction rather than
estimation. In what follows we shall briefly state the definition of each property.
I.2.3.1 Orthogonality The work of Fisher (1960) emphasized that orthogonality is an important design
property. Orthogonality implies that there is no linear dependency among the design
variables as far as their levels in the experiment are concerned. For first order models, if
the design contains orthogonal variables, then the variances of the coefficients are
9
minimized when the design points are set at the extremes values of their ranges (usually,
the variables are coded to be between ±1). Hence, orthogonal designs are known as
variance optimal designs.
I.2.3.2 Rotatability Box and Hunter (1957) introduced the concept of design rotatability. A rotatable design
is one for which the scaled prediction variance remains constant at points that are
equidistant from the design center. This property does not necessarily ensure stability or
even near stability in the scaled prediction variance throughout the region. Rotatability in
the case of first order models is attainable with the standard orthogonal arrays that gave
many other important properties. Designs for second order models such as the composite
designs and other designs can be made to be rotatable. The importance of the property
has historically been tied to desire to achieve stability in prediction variance.
I.2.4 Graphical Methods for the Performance of the Prediction Capability in theRegion of InterestAs mentioned earlier, when one is interested in the stability of the scaled
prediction variance, single-value criteria may not provide a true picture of the
performance of the prediction capability of the design. In the case of two factors, a
contour plot for v(x) can be used to compare designs. However, as the number of the
factors gets larger, this plot is not easy to construct. Two graphical procedures are
introduced in the literature to study design capability of prediction for any number of
factors, k: the variance dispersion graphs (VDG) of Giovannitti-Jensen and Myers (1989)
and the quantile plots of Khuri et al. (1996). We will briefly review the VDG method.
Variance Dispersion Graph (VDG) The VDG plots the maximum, minimum, and spherical average prediction variances
versus the radius r from the center of the design throughout the region of interest. The
spherical prediction variance, Vr, is the average of the variances of the estimated
responses over the surface of a sphere, i.e. dx)x(vVr
U
r ∫=ψ , where
10
}rx:x{U 2
i
2
ir== ∑ and ∫=−
rU
1 dxψ . The stability of the prediction variance at any
given radius of spheres is illustrated by comparing the maximum prediction variance to
the minimum prediction variance. The plot also displays horizontal lines at p and 2p,
which are the 100% and 50% G-efficiencies, respectively. Thus, VDG allows the user to
see the specific locations where the prediction variance is maximized and where it is
minimized. It also gives the user the G-efficiency of the design being studied. Vining
(1993) wrote a FORTRAN program to generate the VDG for any design.
I.3 Some Response Surface DesignsIn this section a brief review of some response surface designs is given. For more
details, the reader is referred to Myers and Montgomery (2002). Response surface
methodology (RSM) frequently involves fitting a first order model εββ ++= ∑=
k
1iki0
xy
or a second order model εββββ ++++= ∑∑∑∑<== ji
jiij
k
1i
2
iii
k
1iii0
xxxxy , where y is a
measured response, xi; i=1,…,k, are the design variables, and ε is a random error with
mean 0 and variance σ2.
I.3.1 Two-Level Factorial and Fractional of Resolution III, IVFactorial designs are widely used in factor screening experiments. A special class
of factorial designs is the 2k-factorial designs, where each of the k factors has just two
levels. These designs are commonly used in the response surface methodology (RSM) to
determine which variables are important and to fit a first order model. They then become
a basic building block to create the response surface designs. However, when the number
of the variables increases, the number of runs needed to perform a complete factorial
design may exceed the experimenter’s resources. If one can assume that high-order
interaction terms are negligible, one can use a fraction of the complete factorial, which
allows estimation of the effects of interest. These designs are called fractional factorial
designs and are characterized by a resolution number, say r. A design is said to be a
fractional factorial of resolution r if no p-factor effect is aliased with another effect
11
containing less than r-p factors. One should notice that the higher the resolution the less
restrictive the assumptions are in terms of the negligible interactions. Designs of
resolution III, IV, and V are frequently considered important.
For first order models without interaction, the two-level factorials and fractional
factorials of resolution ≥ III are known to be orthogonal (Myers and Montgomery, 2002).
If some (or all) interaction terms are included in the model, the two-level factorials are
still variance optimal designs. But, one must have sufficient resolution for fractional
factorials to ensure that no model terms are aliased with each other.
For cuboidal operability regions, the two-level factorials and fractional factorials
of proper resolution are D-, G-, Q-optimal designs. However, for spherical region, they
are just D- and Q-optimal (Myers and Montgomery, 2002).
I.3.2 Second Order Model DesignsA design for a second order model must have at least 2/)1k(kk21 −++ distinct
design points and at least three levels of each design variable to estimate all the
parameters in the model. Several classes of designs were introduced to fit second order
models. We will present the four most popular classes here.
I.3.2.1 Central Composite Designs (CCD) This is the most commonly used class for second order models, developed by Box and
Wilson (1951). The CCD consists of three components: a two-level factorial or resolution
V fraction with coded factor levels at ± 1, a set of axial points at distance α from the
design center along each axis, and n0 center runs. Having two parameters, α and n0, to
select gives this class great flexibility. The region of interest influences the choice of the
axial distance, while the choice of the center runs affects the distribution of the scaled
prediction variance. A rotatable design can be achieved using 4 F=α , where F is the
number of factorial points (see Myers and Montgomery, 2002). Generally, for a spherical
region of interest, we use k=α and 3-5 center runs. For Cuboidal regions, the axial
distance is one and 1-2 center runs are used. For k=2, the design matrix is
12
−
−
−−
−−
=
00α
ααα
00
001111
1111
21 xx
DCCD
I.3.2.2 Box-Behnken Designs (BBD) Box and Behnken (1960) developed this class to be a three-level alternative to the CCD.
This class of designs is very competitive to the CCD, when spherical regions are
assumed. Actually the BBD are designed for spherical regions and should not be used if
there is interest in predicting response at the extremes. The design is near rotatable if not
rotatable. A balanced incomplete block design is used to construct the BBD when k<6;
i.e. each factor does occur in a two-level factorial structure the same number of times
with every factor. For k>5 a partially balanced incomplete block designs is implemented
using different combinations of three design factors in a factorial structure (see Myers
and Montgomery, 2002). Usually, 3-5 center runs is recommended for the BBD. The
design matrix for k=3 is as follows
′
−−−−−−−−
−−−−=
111111110000111100001111000011111111
BBDD
I.3.2.3 Small Central Composite Designs (SCD) This class was developed by Hartley (1959) as a more economical copy of the CCD. A
member of this class is always saturated or near saturated. The basic construction of the
SCD is similar to the CCD, except that the factorial component is of resolution III, rather
than resolution V. As a result, it suffers in efficiency for estimating linear effects and two
factor interactions, due to some aliasing in the factorial portion. For k=3, the design
matrix is
13
′
−−−−−−
−−−=
αααα
αα
000011110000111100001111
SCDD
I.3.2.4 Hybrid Designs Roquemore (1976) developed this class of saturated or near-saturated designs for
k=3,4,6. The Hybrid designs are very efficient and, unlike the SCD, are very competitive
to the CCD. The Hybrid designs are CCD for k-1 variables and the levels of the kth
variable are supplied in such a way as to create certain symmetries in the design. We will
consider two designs from this family: 310 and 311B, which are used when k=3. The
design matrices of these two designs are as follows:
−−−−−−
−−−−
−
=
9273.0736.109273.0736.109273.00736.19273.00736.1
6386.0116386.0116386.0116386.0111360.000
2906.100x x x
D
321
310
−−−−−−−−
−−−
−−
=
00017507.01063.211063.27507.017507.01063.211063.27507.0
17507.01063.211063.27507.017507.01063.211063.27507.0
600600
x x x
D
321
B311
I.4 Design Optimality for Generalized Linear ModelsGeneralized liner models fit regression models for a univariate response that
follows any distribution of the exponential family. The model is given by
βµiii
x)]y(E[g)(g ′== , where g(.) is the link function that connects the linear
predictor βi
x′ to the natural mean of the response variable, i
µ . See McCullagh and
Nelder (1989) and Myers et al. (2002).
14
As mentioned previously, optimality is easier to assess in the linear case than the
nonlinear case. Basically, this follows because the information matrix is just a function of
the design points in the former case, while it is a function of the unknown parameters as
well as the design points in the latter case. In generalized linear models (GLM), the
information matrix is proportional to WXX);X(I ′=β , where W is the Hessian weight
matrix, which is a function of the unknown parameters. This causes increased complexity
of design optimality in GLM. However, since the nonlinear setting is needed in many
applications, such as the chemical, biological and clinical sciences, design optimality in
GLM has been investigated, but not to a great extent.
Several solutions to the dependency of the parameters of the optimality function
were suggested in the literature. We will present these in the following subsections.
I.4.1 Locally Optimal DesignsThe simplest solution proposed in the literature to solve the parameter-
dependency problem is to assume that one can find “good” initial parameter estimates.
These estimates may come from a previous experiment or subjective guesses. Designs
found using this approach are called locally optimal designs, following the terminology
of Chernoff (1953). This approach has been considered by many authors, including
Kalish and Rosenberger (1978), Abdelbasit and Plackett (1983), Minkin (1987), and
Ford, Torsney, and Wu (1992).
This approach suffers from two main disadvantages. First, good initial estimates
are seldom available. Also, a critical point is that the criteria are generally not robust to
poor initial estimates.
I.4.2 Minimax ApproachSitter (1992) proposed the minimax approach to obtain designs that are robust to
poor initial parameter guesses. To use this approach, the experimenter determines initial
guesses for the unknown parameters as well as a specific region within which he/she
wishes the design to be robust. This approach yields designs that are more robust to poor
initial estimates of the parameters. The more uncertain the experimenter is in the initial
15
points, the more spread out is the resulting design, both in terms of coverage of the design
space and number of design points. Although, the minimax approach is difficult to
implement, it is fully automatic. A computer algorithm could be easily used.
I.4.3 Bayesian Approach Another approach to the problem is to introduce a prior distribution on the
parameters and to incorporate this prior into an appropriate design criterion. Usually, the
expectation of the design criterion is maximized over the prior distribution. A weighted
sum of the criterion values evaluated at each point could be used as an alternative to the
expectation to ease the calculations. This approach is more realistic than the local
optimality approach, since it allows several parameter values to be considered, and is less
conservative than the minimax one. A good review of the Bayesian approach is found in
Chaloner and Verdinelli (1995).
I.4.4 Sequential Designs ApproachWorking in stages could protect against poor initial parameter estimates. The idea
is to use information from earlier trials to update the parameter estimates in the next trial.
Such schemes can be useful. Usually, two-stage designs are used. Chaudhuri and
Mykland (1993) discussed the inferential problems arising from sequential procedures
and provide numerous references for this approach.
I.5 Layout of DissertationChapter II introduces a new optimality criterion that addresses good prediction. In
this chapter, some first order and second order standard designs are studied to compare
this new criterion to existing methods. Chapter III deals with non-regular operability
regions in the case of linear models for the two-factor case. The chapter ends with two
supplements. Supplement I generalizes our designs from the two-factor case to the three-
factor case. Supplement II compares the designs of the restricted design space in terms of
the new criteria introduced in Chapter II. Standard designs for generalized linear models
are discussed in Chapter IV. Some general results of the use of the standard designs in
the GLM case are presented. Examples of factorials with the Logistic and Poisson
16
regression models are investigated. Applications to real life examples are presented also.
Finally, Chapter V summarizes the results and contains topics for future research.
I.6 ReferencesAbdelbasit, K. M. and Plackett, R. L. (1983), “Experimental Design for Binary Data”,
Journal of the American Statistical Association, 78, 90-98.
Atkinson, A. C. and Donev, A. N. (1992), Optimum Experimental Designs, Oxford
University Press, Oxford.
Atkinson, A. C. and Haines, L. M. (1996), “Designs for Nonlinear and Generalized
Linear Models”, In: S. Ghosh and C.R Rao, eds., Handbook of Statistics, 13,
Elsevier Science B.V, 437-475.
Box, G.E.P. and Hunter, J.S. (1957). “Multi-factor Experimental Designs for Exploring
Response Surfaces”. Annals of Mathematical Statistics, 28, 195-241.
Box, G.E.P. and Behnken, D.W. (1960). “Some New Three-Level Designs for the Study
of Quantitative Variables”, Technometrics, 2, 455-475.
Box, G.E.P. and Draper, N.R. (1975). “Robust Designs”. Biometrika, 62, 347-352.
Burridge, J. and Sebastiani, P. (1992), “D-optimal Designs for Generalised Linear
Models”, Journal Ital. Stat. Soc., 2, 182-202.
Burridge, J. and Sebastiani, P. (1994), “D-optimal Designs for Generalised Linear
Models with Variance Proportional to the Square of the Mean”, Biometrika, 81,
295-304.
Chaloner K. and Larntz, K. (1989), “Optimal Bayesian Design Applied to Logistic
Regression Experiments”, Journal of Statistical Planning and Inference, 21, 191-
208.
Chaloner, K. and Verdinelli, I. (1995), “Bayesian Experimental Design: A Review”,
University of Minnesota Technical Report.
Chaudhuri, P. and Mykland, P. A. (1993), “Nonlinear Experiments: Optimal Design and
Inference Based on Likelihood”, Journal of the American Statistical Association,
88, 583-546.
Chernoff, H. (1953), “Locally Optimal Designs for Estimating Parameters”, Annals of
Mathematical Statistics, 24, 586-602.
17
Chernoff, H. (1979), Sequential Analysis and Optimal Designs, SIAM, Philadelphia, PA.
Chiacchirini, L.M. (1996), Experimental Design Issues in Impaired Reproduction
Studies, Ph.D. Dissertation, Virginia Tech, Blacksburg,VA.
Cornell, J. (2002) Experiments with Mixtures: Designs, Models, and the Analysis of
Mixture Data, Wiley, New York.
Dette, H. and Wong, W.K. (1999), “Optimal Designs When the Variance is a Function of
the Mean”, Biometrics, 55, 925-929
Draper, N.R. and St John, R.C. (1977), “Designs in Three and Four Components for
Mixtures Models with Inverse Terms”. Technometrics, 19, 17-130.
Dykstra, O. Jr. (1971), “The Augmentation of Experimental Data to Maximize XX ′ ”,
Technometrics, 13, 682-688.
Ehrenfeld, S. (1953), “On the Efficiency of Experimental Designs”, Annals of
Mathematical Statistics, 26, 247-255.
Fedorov, V. V. (1972), Theory of Optimal Experiments, New York: Academic Press.
Fisher, R. A. (1960), The Design of Experiments, 7th ed., Edinburgh: Oliver and Boyd.
Ford, I., Torsney, B., and Wu, C.F.J (1992), “The Use of a Canonical Form in the
Construction of Locally Optimal Designs for Nonlinear Problems”, Journal of the
Royal Statistical Society, Ser B, 54, 569-583.
Giovannitti-Jensen, A. and Myers, R.H. (1989). “Graphical Assessment of the Prediction
Capability of Response Surface Designs”. Technometrics, 31, 375-384.
Hamada, M. and Nelder, J.A. (1997), ``Generalized Linear Models for Quality-
Improvement Experiments'', Journal of Quality Technology, 29, 292-304.
Hartley, H.O. (1959). “Smallest Composite Design for Quadratic Response Surfaces”.
Biometrics, 15, 159-171.
Hebble, T.L. and Mitchell, T. J. (1972), “Repairing Response Surface Designs”,
Technometrics, 14, 767-779.
Heise, M.A. and Myers, R. H. (1996), “Optimal Designs for Bivariate Logistic
Regression”, Biometrics, 52, 613-624.
Huffman, J. W. (1998), Optimal Experimental Design for Poisson Impaired
Reproduction Studies, Ph.D. Dissertation, Virginia Tech, Blacksburg, VA.
18
Jia, Y. and Myers, R. (2001), “Design Optimality for the Two Variable Logistic
Regression Case”, submitted to Journal of Statistical Planning and Inference.
Johnson, M.E. and Nachtsheim, C. J. (1983). “Some Guidelines for Constructing Exact
D-Optimal Designs on Convex Design Spaces”. Technometrics, 25, 271-277.
Kalish, L. A. and Rosenberger, J. L. (1978), “Optimal Designs for the Estimation of the
Logistic Function”, Technical Report 33, Pennsylvania State University.
Kennard, R.W. and Stone, L. (1969). “Computer Aided Design of Experiments”.
Technometrics, 11, 298-325.
Khuri, A. Kim, H.J. and Um, Y. (1996). “Quantile Plots of the Prediction Variance for
Response Surface Designs”. Computational Statistics & Data Analysis. 22, 395-
407.
Kiefer, J. (1958), “On the Nonrandomized Optimality and Randomized Nonoptimality of
Symmetrical Designs”, Annals of Mathematical Statistics, 29, 675-699.
Kiefer, J. and Wolfowitz, J. (1959), “Optimum Designs in Regression Problems”, Annals
of Mathematical Statistics, 30, 271-294.
Kiefer, J. and Wolfowitz, J. (1960), “The Equivalence of Two Extremum Problems”,
Canadian Journal of Mathemethics,12, 363-366.
Kiefer, J. (1975), “Optimal Design: Variation in Structure and Performance Under
Change of Criterion”, Biometrika, 62, 277-288.
Lesinger, W. (1995), Optimal One and Two Stage Designs for the Logistic Regression
Model, Ph.D. Dissertation, Virginia Tech, Blacksburg, VA.
Lewis, S., Montgomery, D. and Myers R. (2001), “Examples of Designed Experiments
with Nonnormal Responses”, Journal of Quality Technology, 33, 265-278.
Martin, B., Parker, D. and Zenick, L. (1987), “Minimize Slugging by Optimizing
Controllable Factors on Topaz Windshield Modeling”, Fifth Symposium on
Taguchi Methods. American Supplier Institute, Inc., Dearborn, MI, 519-526.
McCullagh, P. and Nelder, J.A. (1989), Generalized Linear Models, 2nd edition, New
York, Chapman and Hall.
Minkin, S. (1987), “Optimal Designs for Binary Data”, Journal of the American
Statistical Association, 82,1098-1103.
19
Montgomery, D. C. , Loredo, E. N., Jearkpaporn, D., Testik, M.C. (2002). “Experimental
Designs for Constrained Regions”. To appear in Quality Engineering.
Myers, R. H., Vining, G.G., Giovannitti-Jensen, A. and Myers, S.L. (1992). “Variance
Dispersion Properties of Second-Order Response Surface Designs”. Journal of
Quality Technology, 24, 1-11.
Myers, R.H. and Montgomery, D. C. (1997), “A Tutorial on Generalized Linear Models”,
Journal of Quality Technology, 29, 274-291.
Myers, R.H. and Montgomery, D. C. (2002), Response Surface Methodology: Process
and Product Optimization Using Designed Experiments, 2nd edition, Wiley.
Myers, R.H., Montgomery, D. C., and Vining, G.G. (2002), Generalized Linear Models
with Applications in Engineering and the Sciences, Wiley Series in Probability and
Statistics.
Myers, W. R., Myers, R.H., and Carter, W.H. Jr. (1994), “Some Alphabetic Optimal
Designs for the Logistic Regression Model”, Journal of Statistical Planning and
Inference, 42, 57-77.
Myers, W. R., Myers, R.H., Carter, W.H. Jr., and White, K. L. (1996), “Two Stage
Designs for the Logistic Regression Model in a Single Agent Bioassay”, Journal of
Biopharmaceutical Statistics, 6(4).
Nachtsheim, C. J. (1987). “Tools for Computer-Aided Design of Experiments”. Journal
of Quality Technology, 19, 132-160.
Pazman, A.(1986), Foundation of Optimum Experimental Design, Reidel, Dordrecht.
Pukelsheim, F. (1993), Optimal Design of Experiments, Wiley, New York.
Roquemore, K.G. (1976). “Hybrid Designs for Quadratic Response Surfaces”.
Technometrics, 18, 419-423.
Sebastiani, P. and Settimi, R. (1998), “First-order Optimal Designs for Non-linear
Models”, Journal of Statistical Planning and Inference, 74, 177-192.
Shah, K. R. and Sinha, B. K (1989), Theory of Optimum Design. Lecture Notes in
Statistics 54, Springer, Berlin.
Silvey, D. (1980), Optimal Design, Chapman and Hall, London.
Sitter R. R. (1992), “Robust Designs for Binary Data”, Biometrics, 48, 1145-1155.
20
Sitter R. R. and Torsney, B. (1995), “D-Optimal Designs for Generalized Linear
Models”, In: C.P. Kitsos and W.G. Müller, eds., MODA 4 - Advances in Modern
Data Analysis: Proceedings (the 4th international wokshop in Spetses, Greece, June
5-9, 1995). Heidelberg, Germany: Physica-Verlag, 87-102.
Sitter, R. R. and Wu, C. F. J. (1993), “On the Accuracy of Fieller Intervals for Binary
Response Data”, Journal of the American Statistical Association, 88, 1021-1025.
Sitter, R. R. and Wu, C. F. J. (1999), “Two-Stage Design of Quantal Response Studies”,
Biometrics, 55,396-402
Smith, K. (1918), “On the Standard Deviations of Adjusted and Interpolated Values of an
Observed Polynomial Function and its Constants and the Guidance they Give
Towards a Proper Choice of the Distribution of Observations”, Biometrika, 12, 1-
85.
Snee, R.D.(1985), “Computer-Aided Design of Experiments – Some Practical
Experiences”. Journal of Quality Technology, 17, 222-236.
Wald, A. (1943), “On the Efficient Design of Statistical Investigation”, Annals of
Mathematical Statistics, 14,134-140.
White, L. (1973), “An Extension of the General Equivalence Theorem to Nonlinear
Models”, Biometrika, 60, 345-348.
Zacks, S. (1971), The Theory of Statistical Inference, Wiley, NY.
21
Chapter II Fraction of Design Space to Assess the Prediction
Capability of Response Surface Designs
II.1 AbstractVariance Dispersion Graphs (VDGs) are useful summaries for comparing
competing designs on a fixed design space. However, they might not give all the
information about the prediction capability of the design. The Fraction of Design Space
(FDS) technique is proposed, which addresses some of the shortcomings of VDGs. The
new technique is comprised of two tools that give the researcher more detailed
information by quantifying the fraction of design space where the scaled predicted
variance (SPV) is less than or equal to any pre-specified value. The Fraction of Design
Space Graph (FDSG) gives the researcher information about the distribution of the SPV
in the region based on the ranges and proportions of possible SPV values. The second
tool, the Scaled FDS graph (SFDSG), is used for comparing the overall stability of the
prediction performance. The FDS technique complements the VDGs to give the
researcher more insight about the prediction capability of the design. Several standard
designs with different numbers of factors are studied with both methods: VDG and FDS.
Keywords: Alphabetical criteria, stability of scaled prediction variance, VDG, FDS
technique, FDSG, SFDSG
II.2 Introduction
One measure of prediction performance is the scaled prediction variance (SPV) or
v(x), which is defined for a particular location in the design space by
01
020 )XX(N)ˆvar(N
)x(v xxy −′′==
σ; where x0 is a point in the region of interest,
expanded to the model space, at which we predict. For example, for a design involving
two factors and a second order model the point (x10, x20) would expanded to
22
)xx,x,x,x,x,1( 2010220
21020100 =x . The use of N, the total sample size, adjusts the SPV to
be measured on a per observation basis, and allows for fair comparisons between designs
of different sizes. Two existing optimality criteria address prediction performance: G-
and Q-optimality criteria. Q-optimality is also called V- or IV-optimality in the literature
(Draper & St. John, 1977). However, these single-valued criteria do not reveal the true
complexities of design prediction capability. The approach of looking only at design
moments can be sometimes misleading, since how the moments are achieved is more
important. Box and Hunter (1957) emphasized that judging a design should be on the
basis of the distribution of SPV. Since the experimenter does not know at the outset
where in the design space he/she might wish to predict, a reasonably stable SPV over the
whole region is desired. Thus, one should consider how well the design performs over
every part of the region of interest. This highlighted very early that single-number criteria
might not be enough information when comparing designs. In their paper, Box and
Hunter (1957) introduced the notion of design rotatability, which requires that the
variance of a predicted value remain constant at points that are equidistant from the
design center. Rotatability was just a first step, as the importance of this property evolved
naturally from the need to achieve stability in SPV.
In the two factor case, a contour plot of the SPV, v(x), provides a complete picture
of the performance of the design in terms of prediction. However, the practicality of such
contours is limited to three components systems. Giovannitti-Jensen and Myers (1989)
introduced variance dispersion graphs to assess the overall prediction capability of a
response surface design inside a region of interest. These graphs consist of the maximum
and minimum SPV values and the spherical average of the SPV on spheres inside the
design region, R, against their radii. Myers, Vining, Giovannitti-Jensen and Myers (1992)
used such plots to compare several standard second–order designs on the basis of their
prediction capabilities over spherical and cuboidal regions. As with any plot that reduces
the dimensionality of the information, the VDGs can not provide complete information
concerning the distribution of the SPV on a given sphere. Thus, they may not enable the
user to discriminate between two designs that have similar VDG patterns but different
SPV distributions on the sphere.
23
Khuri et al. (1996) proposed the quantile plots for describing the distribution of
the SPV. A curve of the cumulative distribution of the SPV at each radius is created to
compare designs at each radius or to study the properties of a specific design. Although,
the quantile plots do supply more information on the distribution of the SPV at a given
radius, they do not alleviate the problems that exist with the VDGs. Also, quickly the
number of graphs becomes impractically large.
Both the VDG and the Quantile plots do not take into account the volume of the
sphere and the proportion of the design space at various distances from the center of the
design space. They deal with the SPV on a sphere of radius r but ignore the volume
associated with this information. The VDG transforms the information of the sphere to a
point at its three curves (minimum and maximum of v(x) and the spherical variance
curves), while the Quantile plot transfers this information to a single curve representing
the cumulative distribution of v(x) at each sphere. Thus, the information of each sphere is
given the same “weight” in these graphs. But the weight of each piece of information is
not equal in general, and one should weigh this information by the volume of the
corresponding sphere. Figure II.1 depicts the relative change in volume corresponding to
sphere of radius r for two-, four- and six-factor designs. The relative contribution to the
overall volume of the region is an increasing function in r. When the dimension, k, of the
design increases, the size of the relative contribution diminishes for small r and enlarges
for large r. Figure II.1 considers a spherical region and displays how a quickly increasing
fraction of the design space is associated with the outer edges of the space as k increases.
This means that fewer points at large values of radii on the VDG curves and fewer curves
corresponding to large r in the Quantile plots dominate the prediction capability; and
should be given more weight in our interpretation of these graphs.
24
Figure II.1: Effect of Increasing Dimension on the Percentage of Volume at Radius r
To gain insight about the complete picture of the prediction performance of a
design, one should look directly at the volume. The method presented here involve
calculation of the volume of the design that has v(x) less than or equal to any pre-
specified value of the SPV. The Fraction of Design Space Graph (FDSG) is a single
graph of this volume against the range of v(x) obtained for each design. To compare the
stability of different designs in terms of the overall prediction capability, the volume can
be scaled by the minimum value of SPV. The Scaled Fraction of Design Space (SFDSG)
graph reflects this information. Hence, the fraction of design space technique quantifies
the amount of the distribution of the SPV at different values. In addition, it provides the
researcher with a single plot to compare designs or study the properties of a specific
design. Accordingly, the FDS technique could be applied to regular and non-regular
design regions. Examples for non-regular design regions are found in mixture designs.
See Cornell (2002) or Montgomery, Loredo, Jearkpaporn, and Testik (2002).
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Perc
enta
ge o
f the
Incr
ease
in
the
Volu
me
Two Factors Four Factors Six Factors
radius values0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
25
The outline of the paper is as follows. Section II.3 includes a brief review of the
VDG technique. In Section II.4, the Fraction of Design Space technique with the FDSG
and SFDSG plots is described. Section II.5 evaluates some second-order designs and
compares them using FDS and VDG over spherical regions. For cuboidal regions, two
second-order designs are discussed in Section II.6.
II.3 Review of Variance Dispersion Graphs (VDG)
Giovannitti-Jensen and Myers (1989) introduced a variance-based graphical
approach to study the prediction capability of a design. The VDG plots the maximum,
and minimum over spheres of radius r from the center, as well as the spherical average of
the SPV against the radius r from the center of the design throughout the region of
interest. The spherical average variance is defined by dx)x(vVr
U
r ∫=ψ , where
}rx:x{U 2
i
2
ir== ∑ and ∫=−
rU
1 dxψ . For cuboidal regions, the above three statistics are
calculated over spheres or portions of spheres that are on or within the cube. The
rotatability of the SPV at any given radius of spheres is illustrated by comparing the
maximum to the minimum of SPV across the range of radii. The plot also displays
horizontal lines at p and 2p, which are the 100% and 50% G-efficiencies, respectively.
Vining (1993) wrote a FORTRAN program to generate the VDG for any design.
Figure II.2 shows the VDG of different second-order designs for the two-factor
case in a spherical region. The Central Composite Design (CCD), introduced by Box and
Wilson (1951), is an efficient design for estimating the unknown parameter vector in the
model. We consider two such designs with one and three center runs, respectively, both
with axial distance 2=α . The Hexagon design contains six equally spaced points on a
circle and one center run. This design is a special alternative design in the two-factor case
for the CCD for spherical region (Myers and Montgomery, 2002). In Figure II.2, the
designs have been scaled to the unit sphere. One can determine the nearness to
rotatability of a design by comparing the spread of the maximum and minimum curves.
26
As in Figure II.2, if the maximum, minimum and average curves are all identical, the
designs are rotatable. By comparing the maximum SPV value for the design to the 100%
G-efficiency line in Figure II.2, it appears that the Hexagon and the CCD with three
center runs are about 86% G-efficient, whereas the CCD with one center run is just 67%
G-efficient. The VDG also allows the user to see the specific locations where the SPV is
maximized and where it is minimized. The Hexagon performs better than the CCD with
one center run almost over the whole region. But, the CCD with three center runs
performs best when 8.0r < . If the relative change in the design volume was constant
with the change in r, one could conclude that the CCD with three center runs performs
better throughout a large portion of the design region. However, since the fraction of the
design space is changing as we change the associated radius, a precise comparison is not
easily possible.
0 0.2 0.4 0.6 0.8 1radius
5
10
15
20
CCD-1CR
HexagonCCD-3CR
50% G-eff
100% G-eff
Figure II.2: VDG of some Two-Factor Designs
II.4 The Fraction of Design Space Criterion (FDS)It is proposed that one can assess the prediction performance of a design or
compare different designs based on the fraction of the design space contained within a
27
variety of cut-off points for SPV. The larger the fraction of design space at or below a
given value, the better the design. For purposes of comparisons among the different
designs, we considered the cut-off point contour volume relative to the design region
volume. For a practitioner who does not know a priori where in the design space he/she
may wish to predict, having a large area relatively close to the minimum of the SPV is
highly desirable. Two graphs can be created to assess the prediction capability. The first
one, the Fraction of Design Space Graph (FDSG), plots the fraction of design space
values against the entire range of cut-off points ranging from the minimum to the
maximum of SPV. This allows the researcher to evaluate the performance of designs in
terms of prediction. When several designs are plotted on the same graphs, it allows the
researcher to see the global minimum and global maximum of SPV of each design. The
slope of the curve shows how quickly the design reaches the maximum value of the SPV,
with closer to vertical being preferred. The 50% and 100% G-efficiency lines are also
shown vertically on this graph, which allows the researcher to determine the approximate
G-efficiency of each design. The second graph, the Scaled Fraction of Design Space
Graph (SFDSG), compares the overall stability of different designs by first plotting the
fraction of design space against the standardized or scaled cut-off points. The scaled cut-
off point is defined as ))x(v(min
vv
0R
s = , where the minimum SPV for each individual
design is used. The steeper the slope of the curve is the more stable the SPV of the
design. This graph also allows direct access to the ratio of maximum to minimum SPV.
Consider the CCD in two variables over a cuboidal region with one center run.
We are interested in the fraction of design that has SPV no more than 4. Note that for this
design 25.7)x(v2.3 0 ≤≤ . The volume of the cut-off point (4) contour is shown in
Figure II.3a and represents 56% of the total design space. In Figure II.3b, this becomes
the point (4, 56%) on the FDSG. Similarly, for any chosen cutoff for the SPV, we can
obtain the fraction of design space at or below this value.
28
Figure II.3: Volume for CCD with variance = 4 over Cuboidal Region and FDSG
Methodology. Let v be any predetermined value of the SPV, k be the number of factors
and Ψ be the total volume of the design region. The fraction of design space criterion
(FDS) is defined as follows
∫ ∫=A
1k dx.........dx........?1
FDS (3.1)
where }v)x(v:)x,.....,x{(A k1 ≤= .
To calculate this fraction of design space volume, one needs to know the elements of the
set A to obtain the integrands of the above integrals. An approximate method uses a fine
grid and calculates how many points are satisfying the condition of set A, say mA. Then,
the above criterion could be approximated by Mm
FDS A≅ , where M is the total number
of points in the design space using the same grid. For an exact method, define
ggw −= , where v)x(vg −= ; then the fraction of design space is calculated as
1k1
1
1
1dx....dx
g w5.0
.....?1
FDS ∫ ∫− − −−
=ε
(3.2)
-1 -0.5 0 0.5 1-1
-0.5
0
0.5
1
> 4
≤ 4
(a) (b)
2 4 6 8Variance
56%
Fra
ctio
nof
Des
ign
Spa
ce
29
where ε is a small number used to ensure that the denominator is greater than zero.
Notice that the set A could be defined in terms of g as follows }0g:)x,....,x{(A k1 <= .
Hence, we are interested in negative values of g. Actually w is merely an indicator
function with two values, either -2g or 0. It filters out all the variances greater than the
cut-off points and allows integration over the entire range of the variables.
For rotatable designs in a spherical region the above criterion is simplified to
∫ −
−−
=1
0
1k drrg
w5.0kFDS
ε; where r is the radius of the region scaled to the unit sphere.
The following two sections contain some second order designs evaluated by both
the VDG and the FDS techniques over spherical and cuboidal regions. The ability of the
FDS to highlight different information of the prediction performance of the design than
the VDGs is discussed. A FORTRAN code available from the authors has been
developed for calculating the FDSG and SFDSG. It uses an IMSL multivariate numerical
integration subroutine.
II.5 Comparisons of the Standard Second-Order Designs over SphericalRegion
The CCD design, mentioned in Section II, contains three main components: a
two-level factorial, or a resolution V fraction, a set of axial points at distance α from the
center of the design along each axis and n0 center runs. Unless otherwise specified, we
will use the most commonly selected value k=α . Three other popular classes of
second-order designs are considered in this paper: the Box-Behnken (BBD), the Small
Composite (SCD) and the Hybrid designs. Box and Behnken (1960) developed the BBD
to be a three-level alternative to the CCD. These designs are competitive with the CCD
when the region of interest is spherical. Hartley (1959) introduced the Small Composite
Designs (SCD). These designs have the same construction as the CCD except they
employ a resolution III factorial design in the factorial portion. These designs are often
near-saturated and are more economical than the CCD. Another near-saturated class of
30
designs is the Hybrid class (Roquemore, 1976). These designs are available for k=3,4,
and 6. This class contains some designs that are highly efficient and near-rotatable.
II.5.1 Example: Two Factors on Spherical RegionThe FDSG for the three designs with two Factors on a spherical region discussed in
Section II and Figure II.2 is shown in Figure II.4a. Now the superiority of the CCD with
three center runs is demonstrated over the whole region since for any value of SPV, it has
the largest fraction of the design space at or below this level. Also, the fact that the
Hexagon and CCD with one center run differ consistently in overall performance is
highlighted in the FDSG. The new plot allows global comparisons more easily than the
VDGs, which encourage comparisons at fixed radii. Notice how the maximum and
minimum values of the designs occur at different radii of the VDGs and with different
associated volumes. Figure II.4b shows that the Hexagon is more stable than the CCD
with either of center runs combinations, since it has the steepest slope.
II.5.2 Example: Three Factors on Spherical RegionFigure II.5 shows the VDG for some second-order designs in three factors. The CCD
with three center runs (N=17) is near rotatable and performs consistently on a sphere for
large radii. The BBD with three center runs (N=15) is a competitor to the CCD near the
perimeter and is better at lower values of r. The SCD (N=13) is best near the center, but it
suffers badly when the radius gets bigger. For the Hybrid designs, the H311B is near
rotatable and performs better than H310 for large radii. The FDSG and SFDSG in Figure
II.6 show that the rapidly increasing maximum SPV at the edges of the design for the
SCD and H310 make the designs much less desirable. The CCD is better at the edges
relative to the BBD, which is better near the center of the design. However, there is
relatively little fraction of the design near the center. The FDSG shows that for the
majority of the SPV values the CCD design is best but the BBD, CCD, and H311B are
very close in their prediction performance. The CCD is more stable than the other designs
as shown by the SFDSG.
31
FigureII.4: Second Order Designs for Spherical Region in Two Factors
2 4 6 8 10Variance
0
0.2
0.4
0.6
0.8
1
Frac
tion
ofSp
ace
Cri
teri
on
CCD-1CR
HexagonCCD-3CR
100%
G-e
ff
a) FDSG
1 1.5 2Standardized Variance
0
0.2
0.4
0.6
0.8
1
Frac
tion
ofSp
ace
Cri
teri
on
CCD-1CR
HexagonCCD-3CR
b) SFDSG
32
Figure II.5: VDG for Second Order Designs for Spherical Region in Three Factors
0.2 0.4 0.6 0.8 1radius
0
10
20
BBD-3CR
SCD-3CRCCD-3CR
100% G-eff
50% G-eff
0.2 0.4 0.6 0.8 1radius
0
10
20
H311B-2CRH310-3CR
100% G-eff
50% G-eff
33
Figure II.6: Second Order Designs for Spherical Region in Three Factors
0 10 20 30Standardized Variance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Frac
tion
ofSp
ace
Cri
teri
on
BBD-3CR
H311B-2CRH310-3CR
SCD-3CR
CCD-3CR
50%
G-e
ff
100%
G-e
ff
a) FDSG
0 10 20 30Standardized Variance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Frac
tion
ofSp
ace
Cri
teri
on
BBD-3CR
H311B-2CRH310-3CR
SCD-3CR
CCD-3CR
b) SFDSG
34
II.5.3 Example: Four Factors on Spherical RegionFor the four factor case, the BBD is a rotation of the CCD (N=27). Therefore, their VDG
in Figure II.7 are identical, and both designs are rotatable. The designs perform
considerably better than the SCD (N=19) for large values of r. For all the above four
factor designs three center runs are used. The BBD and the CCD are highly efficient in
the G-sense (95%), while the SCD is just 25% G-efficient. The SCD suffers because it
uses a resolution III in its factor portion, which results in correlation between the linear
and the two factor interaction terms. For the Hybrids, the performance of H416A and
H416C is close to each other, with the first design nearer to rotatability. On the average
H416B performs best with lower SPV for 5.0r ≤ . In all of these Hybrids a total number
of three center runs is used (N=19). Note that in the original design matrix (N=16) of
H416C contains one center run and we added two other center runs to get a total of three.
Figure II.8 shows that the three Hybrids have smaller minimum SPV than the BBD, CCD
and SCD, but higher maximum than the BBD and the CCD. The H416C is almost
uniformly better than the other two hybrids. This information is more easily extracted
from the FDSG than from the VDGs, which are cluttered with multiple lines for each
design. From the SFDSG, one realizes that the BBD and CCD are more stable than all the
other designs. The SCD is the worst design in terms of the G-efficiency and is non-stable.
The H416C is the most stable design among the hybrid designs.
II.5.4 Example: Five Factors on Spherical RegionFigure II.9 displays the VDG of three designs for the five-variable case: BBD (N=45) and
two CCD (N=31) designs with a half fraction resolution V factorial. Different α values
are considered for the two CCD designs with 5=α and 2=α , which results in a
rotatable design. All three designs are augmented with five center runs. Both CCDs
perform better than the BBD near the center of the region, with the rotatable CCD (CCD-
R) being slightly better than the nonrotatable one (CCD-NR). The best design near the
perimeter of the region is CCD-NR. The BBD is 55% G-efficient, while CCD-R and
CCD-NR are 77% and 79% efficient, respectively.
35
Figure II.7: VDG for Second Order Designs for Spherical Region in Four Factors
0 0.2 0.4 0.6 0.8 1radius
10
20
30
40
50 BBD-3CR
SCD-3CRCCD-3CR
100% G-eff
50% G-eff
0 0.2 0.4 0.6 0.8 1radius
5
10
15
20
H416A-3CR
H416C-2CRH416B-3CR
100% G-eff
36
Figure II.8: Second Order Designs for Spherical Region in Four Factors
5 10 15 20 25 30 35 40Variance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Frac
tion
ofSp
ace
Cri
teri
on
A
A
A
A
A
A
A
B
B
B
B
B
B
B
B
C
C
C
C
C
C
C
ABC
BBD-3CR
H416B-3CRH416A-3CR
H416C-2CR
CCD-3CR
SCD-3CR10
0%G
-eff
50%
G-e
ff
a) FDSG
2 4 6 8Standardized Variance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Frac
tion
ofSp
ace
Cri
teri
on
A
A
A
A
A
A
A
B
B
B
B
B
B
B
B
C
C
C
C
C
C
C
ABC
BBD-3CR
H416B-3CRH416A-3CR
H416C-2CR
CCD-3CR
SCD-3CR
b) SFDSG
37
Although the BBD has higher minimum and maximum of v(x) than those of both CCD
designs, the FDS technique shows that the BBD is more stable than the other two designs
for all but a tiny fraction of the design space (Figure II.10a). The nonrotatable CCD
design is more stable than the rotatable CCD.
0 0.2 0.4 0.6 0.8 1radius
5
10
15
20
25
30
35BBD-5CR
SCD-R-5CRSCD-NR-5CR
100% G-eff
Figure II.9: VDG for Second Order Designs for Spherical Region in Five Factors
38
Figure II.10: Second Order Designs for Spherical Region in Five Factors
5 10 15 20 25 30 35 40Variance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Frac
tion
ofSp
ace
Cri
teri
on
BBD-5CR
SCD-R-5CRSCD-NR-5CR
100%
G-e
ff
50%
G-e
ff
a) FDSG
1 2 3 4 5Standardized Variance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Frac
tion
ofSp
ace
Cri
teri
on
BBD-5CR
SCD-R-5CRSCD-NR-5CR
b) SFDSG
39
II.5.5 Example: Six Factors on Spherical RegionTwo rotatable designs in the six-factor case are considered here: CCD with four center
runs (N=48) and the hybrid H628A with additional two center runs (N=30). Figure II.11,
shows that the hybrid is better than CCD over a considerable portion of the region
( 8.0r ≤ ), while both designs have the same prediction capability near the perimeter of
the region. This difference looks quite dramatic on the VDG but Figure II.12a, indicates
that both designs have almost the same performance over the whole region reflecting the
very small fraction of the design space right near the center of the design. Actually, the
portion of design region where the hybrid performs better contains less than 15% fraction
of design space as can be seen from Figure II.1. Figure II.12b shows that the CCD is
superior to the hybrid design in terms of the stability of the SPV distribution. This
difference is primarily attributable to the smaller minimum SPV for the Hybrid design.
0 0.2 0.4 0.6 0.8 1radius
5
10
15
20
25
30
35CCD-4CR
H628A-2CR
100% G-eff
FigureII.11: VDG for Second Order Designs for Spherical Region in Six Factors
40
Figure II.12: Second Order Designs for Spherical Region in Six Factors
5 10 15 20 25 30 35 40Variance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Frac
tion
ofSp
ace
Cri
teri
on
CCD-4CRH628-2CR
100%
G-e
ff
a) FDSG
1 2 3Standardized Variance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Frac
tion
ofSp
ace
Cri
teri
on
CCD-4CRH628-2CR
b) SFDSG
41
II.6 Comparisons of the Standard Second-Order Models over CuboidalRegion with Three FactorsAn important second-order design for cuboidal regions is the CCD with α=1 which is
known as “face center cube“. Many other highly D-efficient designs are available for
cuboidal regions but are not commonly used in practical applications. In this section we
compare two designs of equal size: the CCD with one center run and the BBD with three
center runs in three factors. Note that the BBD has no points on the corners of the cube.
All the points are on a sphere of radius 2 except the center runs, while the vertices of
the region reside on a radius of 3 . This results in large SPV at the perimeter of the
region for this design. However, the BBD is competitive with the CCD for radius values
less than 2 as the VDG indicates (see Myers et al, 1992). Figure II.13a indicates that
the CCD is better than the BBD for almost the whole cube. However, there are some SPV
values between 6 and 7 where more of the design space curve is predicted well by the
BBD. In terms of the overall stability of the prediction performance, Figure II.13b shows
that there is no clear winner.
42
Figure II.13: Second Order Designs for Cuboidal Region in Three Factors
5 10 15 20Variance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Frac
tion
ofSp
ace
Cri
teri
on
BBD-3CRCCD-1CR
100%
G-e
ff
a) FDSG
1 2 3 4 5Standardized Variance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Frac
tion
ofSp
ace
Cri
teri
on
BBD-3CRCCD-1CR
b) SFDSG
43
II.7 ConclusionsAlphabetic optimality criteria, such as G- and Q-Optimality, as well as variance
dispersion graphs are all useful measures for comparing competing designs. The Fraction
of Design Space (FDS) technique is proposed in this paper as a complement to the
existing VDG technique. The VDG is a good tool for helping visualizing the range of the
values possible for the scaled predicted variance for different designs. However, the
relative emphases that should be given to different intervals of the sphere radius can be
dramatically different depending on the dimension of the design space. The new
technique, FDS, focuses on how well the design predicts for any fraction of the design
space. It gives the fraction of the design space that is equal or less a pre-specified value of
the SPV. Two graphical summaries are then obtained. The FDS graph (FDSG) represents
the cumulative fraction of design at each value of the SPV throughout the design region.
It allows comparison of the global minimum and maximum of v(x) for different designs.
This graph produces a general summary of the design region and is not restricted to
certain radius of the design region. The second graph is the scaled FDS graph (SFDSG),
where the FDS values are plotted against the SPV values scaled by the minimum value of
v(x). This graph allows direct access to the ratio of the maximum to minimum SPV and is
useful for looking at the stability of the SPV distribution.
By considering several second order designs on different spaces, the types of
comparisons made possible by the new methods have been demonstrated. In this paper
we considered designs over spherical and cuboidal regions. However, the FDS technique
could also be applied to non-regular design regions.
II.8 ReferencesBox, G.E.P. and Behnken, D.W. (1960). “Some New Three-Level Designs for the Study
of Quantitative Variables”, Technometrics, 2, 455-475.
Box, G.E.P. and Hunter, J.S. (1957). “Multifactor experimental Designs for Exploring
Response Surfaces”. The Annals of Mathematical Statistics, 28, 195-241.
Cornell, J. (2002) Experiments with Mixtures: Designs, Models, and the Analysis of
Mixture Data, Wiley.
44
Draper, N.R. and St John, R.C. (1977), “Designs in Three and Four Components for
Mixtures Models with Inverse Terms”. Technometrics, 19, 117-130.
Giovannitti-Jensen, A. and Myers, R.H. (1989). “Graphical Assessment of the Prediction
Capability of Response Surface Designs”. Technometrics, 31, 375-384.
Hartley, H.O. (1959). “Smallest Composite Design for Quadratic Response Surfaces”.
Biometrics, 15, 159-171.
Khuri, A. Kim, H.J. and Um, Y. (1996). “Quantile Plots of the Prediction Variance for
Response Surface Designs”. Computational Statistics & Data Analysis. 22, 395-
407.
Montgomery, D. C. , Loredo, E. N., Jearkpaporn, D., Testik, M.C. (2002). “Experimental
Designs for Constrained Regions”. To appear in Quality Engineering.
Myers, R. H., Vining, G.G., Giovannitti-Jensen, A. and Myers, S.L. (1992). “Variance
Dispersion Properties of Second-Order Response Surface Designs”. Journal of
Quality Technology, 24, 1-11.
Myers, R.H. and Montgomery, D. C. (2002), Response Surface Methodology: Process
and Product Optimization Using Designed Experiments, 2nd edition, Wiley.
Roquemore, K.G. (1976). “Hybrid Designs for Quadratic Response Surfaces”.
Technometrics, 18, 419-423.
Vining, G. G. (1993). “A Computer Program for Generating Variance Dispersion
Graphs”. Journal of Quality Technology, 25, 45-48.
45
Chapter III Modifying 22 Factorial Designs toAccommodate a Restricted Design Space
III.1 AbstractStandard designs assume that the factor space is a p-dimensional hypercube or
hypersphere with any point inside or on the boundary of the shape being a candidate
design point. However, some economic, practical, or physical constraints may occur on
the factor settings resulting in an irregular experimental region. One often encounters
situations in which it is necessary to eliminate some portion of the design space where it
is infeasible or impractical to collect experimental data. Hence, standard designs are not
always feasible and the need arises for best possible designs under these restrictions.
For the two-factor case with one corner of the square design space excluded, three
sensible alternatives designs are proposed. These designs involve reducing the factor
levels to make a smaller but standard factorial design fit or modifying the levels of the
variables at the excluded corner to locate it in the feasible design region. Properties of
these designs and relative tradeoffs are discussed.
Keywords: alphabetical optimality criteria, linear models, non-regular design space.
46
III.2 IntroductionFor many designed experiments, the operability region is typically a hypercube or
a hypersphere, for which factorial designs are the standard designs in the case of first
order, or first order with interaction linear models. These designs have desirable
properties including simplicity, straightforward implementation, orthogonality and D-
and Q-optimality. Q-optimality is also called V- or IV-optimality in the literature. See
Draper and St. John (1977). For cuboidal regions, they are also G optimal designs.
Details on the definition and characteristics of these alphabet criteria are given in Myers
and Montgomery (1995). Sometimes, however, the nature of the experiment may cause
restrictions on the factor settings and hence on the design region (operability region).
Accordingly, standard designs may not be feasible. Kennard and Stone (1969) were the
first to discuss in the literature the problem of irregular experimental regions and
suggested computer aided design for selecting an experimental plan. Some case-by-case
examples of non-standard design regions are discussed in Snee (1985). Johnson and
Nachtsheim (1983) discussed how single-point augmentation procedures are helpful for
finding exact D-optimal designs on convex design spaces. Recognizing the importance of
computer programs to develop designs when classical designs are not appropriate,
Nachtsheim (1987) reviewed and compared the available tools for computer-aided design
of experiments. Atkinson and Donev (1992) devoted a short chapter to restricted designs.
They used some computer algorithms to find the D-optimum design for certain irregular
regions. They emphasize that whatever the shape of the experimental region the
principles of the optimality theory remain the same. Montgomery, Loredo, Jearkpaporn,
and Testik (2002) give a brief tutorial on computer-aided methods for constructing
designs for irregularly shaped regions.
Previous literature has focused on D-optimality and using computer searches of
candidate points in unusually shaped regions. In this paper, we consider the case of two
quantitative factors with a standard cuboidal region, where one corner of the cube is
unfeasible. For example, in a chemical application, two acids can be added to a
47
compound to improve its texture. But, if the overall pH of the compound gets too low
(acidic), the compound loses structural strength. Therefore, the high-high combination is
impractical. Another example may be in a drug interaction study, where it might be not
practical to simultaneously set the two factors at high (or low) levels, because it is known
a priori that this combination has an undesirable effect. The high-high combination might
be dangerous for the subject, while it may be unethical not to give the subject any
effective drug by giving the low-low combination. Figure III.1 shows the experimental
region for the case when the high-high corner has been removed. In this paper, we focus
on altering the high-high corner, but without loss of generality, all results obtained will
apply to altering any corner of the square design space.
Figure III.1: Restricted Operability Region
Since the feasible design space in this case consists of a square with one corner
removed, the standard factorial is not immediately feasible here, and the need for
specialized optimal designs arises. Three possible designs for this non-regular region are
proposed in Section III.3. Their properties and their efficiencies in the sense of D-, G-,
and Q-criterion are studied. Section III.4 presents an example while Section III.5 contains
a more general form of the design space boundary.
III.3 Design Space and Possible DesignsTo define the specific boundary for our design space, we chose to exclude the
high-high corner and replace it with a quarter of a circle. Depending how much we wish
to truncate the standard design space, we can adjust the size of the replacement quarter
circle. We primarily considered five radius values, namely, r= 0.1, 0.2, 0.3, 0.4, 0.5,
(1,1)(-1,1)
(1,-1)(-1,-1)X1
X2Not allowed tooperate at thiscombination
48
where r is the fraction of the range of each variable that we wish to alter. For example, if
we choose r= 0.5, then half of the range of each of x1 and x2 will have the corner square
replaced with a quarter circle. Properties and efficiency of the possible designs are
compared for the resulting design regions for different r values. Figure III.2 shows the
above design space for two values of the radius. The user would specify what value of r
is required to make the design space feasible and of practical interest. Once the region is
specified, a best design can be selected.
Figure III.2: Operability Region for r = 0.1 and 0.5
The following three designs are proposed for the above restricted region.
III.3.1 Design I We substitute the point, say (a, a), in the middle of this quarter circle (at angle π/4
radians or 45°) as a replacement point for the ineligible (1,1) point. The value of a
depends on the radius of the circle chosen. This design fills the design space and is only a
minor adjustment from the standard design.
III.3.2 Design II Another design uses two symmetric points on the quarter circle, at angles θ and (π /2-
θ). This design would appeal to practitioner who have already done one-at-a-time studies
on each factor and now want to study the interaction of the two more thoroughly by
looking at two new combinations of the drugs. We considered five sets of points ranging
from the edges of the quarter circle with angle 0=θ to two points on top of each other at
(1,1)(0,1)
(1,0)(0,0)X1
X2
r =0.1
r =0.5
49
the point (a, a) with 4/πθ = . This latter matches the design point of Design I, which
has just four design points whereas Design II has five points.
III.3.3 Design III If we wish to preserve an orthogonal design, we can use a standard factorial design in
the reduced space with the excluded high-high corner. This design might seem intuitively
pleasing to a practitioner who wishes to use a standard design and just changes the scale
of the coded variables to make it fit in the admissible region. The new cube region has the
points: (-1, -1), (-1, a), (a, -1), and (a, a), where a is the same point defined in Design I.
Although, the resulting design is orthogonal, it requires extrapolation to provide estimates
for the entire region. As the radius used for altering the design space increases, the
amount of extrapolation required will increase.
III.3.4 Comparison of DesignsTo select a single “best” design for this restricted region, there are several aspects
to consider. Properties of a good response surface design are discussed in Box and Hunter
(1957), Box and Draper (1975), Atkinson and Donev (1992) and Myers and Montgomery
(1995). Typically, one can not achieve all the ideal properties in a single design. Hence,
there are frequently several good designs and choosing among them involves some
tradeoffs. All of the designs above are able to estimate all of effects in the usual first
order model with interaction
εββββ ++++=211222110
xxxxy with ),0(N~ 2iid
σε .
To compare the three designs, we considered two sets of measures:
- Descriptive measures, which include number of design points, orthogonality,
how many levels per factor, ability to measure pure error.
- Quantitative measures including D-, G-, and Q-efficiency, maximum correlation
between any two parameter estimates, and the variance of the interaction term, which
may be of particular interest if the main effects are well understood.
50
Orthogonality allows independent estimates of effects. A high D-efficiency results
in good joint estimation of parameters in terms of generalized variance. High G-
efficiency indicates good prediction capability in terms of minimizing the maximum
prediction variance in the region of interest. A high Q-efficiency results in minimum
average of scaled prediction variance in the region of interest. All the above three
alphabetical criteria do account for the number of design runs in the calculations by
looking at results on a per observation basis. A design that minimizes the maximum
correlation between any two parameter estimates means the design is nearly orthogonal.
Finally, a design that estimates the interaction term well is desirable, when the interaction
term is of primary interest. Hence, the D-s optimality criterion, which optimizes the D-
efficiency for a subset of the parameters, for the interaction term may be considered. See
Atkinson and Donev (1992).
Table III.1 shows the comparison of all three designs. All of our three designs
have different desirable features and there is a need to consider tradeoffs between
designs. Design III, the reduced area full factorial, is orthogonal and has the fewest
numbers of levels per factor. Using the Equivalence Theorem (Kiefer and Wolfowitz,
1960), Design I is found to be optimal in the D- and G- sense. The Equivalence Theorem
states that a design with the maximum prediction variance equal to the number of the
unknown parameters in the model is D- and G-optimum design, if the maximum is
achieved at the design points. Design I is also the Q- optimum design since given a
certain value of r, the point on the quarter of the circle that minimizes the integration of
the scaled prediction variance over the region of interest is (a, a), the one used in Design
I.
The alphabetical efficiency of the other two designs decreases as the radius
increases and within a particular radius for Design II the efficiency increases as the angle
selected gets closer to the point (a, a) at 4/πθ = . The maximum correlation between
any two terms in the model remains relatively small for all designs. For Designs I and II,
the correlation increases as we alter more of the design space with a large r value. For
Design II the largest correlation is minimized at 4/πθ = . As well, the variance of the
51
interaction term is comparable for all designs with Design II with two points at θ = π/4
being best. However, design II uses more design points than the other two designs, which
makes it necessary to scale the variance of the interaction term by the number of runs to
make a fair comparison. When the variance is scaled, Design I turns to be the best in
terms of estimating the interaction term. Note that the scaled variance of the interaction
term is related to the Ds-criterion. Ds-optimum designs are appropriate when interest is in
estimating a subset of s of the parameters of the model as precisely as possible. This
might be the case if the researcher has already done one-at-a-time studies on each factor
and now wants to study the interaction of the factors more thoroughly, in which case a
Ds-efficient design is preferable, where s is the number of the interaction terms in the
model.
Table III.1: Comparison of Designs
* denotes best design for each criteria at a given radius valueDesign I Design II Design III
Description of Design (1,1) → (a,a) 2 pts on the quarter circle OrthogonalDescriptive Measures
# design points 4* 5 4*Orthogonality close Orthogonal*
Levels per factor 3 4 2*
Quantitative MeasuresThe first entry gives the value for r= 0.1; Second entry gives the value for r= 0.5
θ
D-efficiency 1.00* – 1.00* 0π/4
0.93 – 0.830.95 – 0.95
0.97 – 0.86
G-efficiency 1.00* – 1.00* 0π/4
0.80 – 0.680.80 – 0.80
0.94– 0.71
Q-efficiency 1.00* – 1.00* 0π/4
0.92 – 0.870.93 – 0.95
0.98 – 0.91
MaximumCorrelation
0.04 – 0.24 0π/4
0.14 – 0.400.14 – 0.09
Orthogonal*
Variance ofInteraction term
0.27 – 0.35 0π/4
0.24 – 0.470.23* – 0.27*
0.28 – 0.47
Scaled Variance ofInteraction term
1.08*-1.4* 0π/4
1.2 – 2.351.15 – 1.35
1.12-1.88
Figure III.3 shows the efficiency of Designs II (at 0 and π/4 radians) and III at all
the radius values. These are also the relative efficiencies of these designs to Design I,
since it is optimal for all D-, G- and Q-criteria. Design III is better than Design II in the
alphabetical sense at low radius values. Design II with angle π/4 radians is most efficient
52
at the high radius values. From Figure III.3a, Design III is very close to optimality for
small adjustments with r small, since the value a is very close to +1. Figure III.3b shows
that Design III has high efficiency in the G-sense for small r as well. However, as r
increases Design II emerges as better than Design III with the double point at (a, a) being
best. The Q-efficiency is depicted in Figure III.3c. Design III is still better than Design II
in the Q-sense as long as r is not too large, however, for large values of the radius,
Design II with two points at angle π/4 radians is better than Design III.
Figure III.3: The Alphabetical Criteria of Designs II, and III
If the stability of the prediction variance over the design region is of interest, a
single number measure like G- or Q-efficiency may not capture the true prediction
capability of the design. A contour plot of the prediction variance, v(x), can give a more
complete picture of the performance of the design in terms of the prediction sense. Figure
III.4 and III.5 show the contour plots of v(x) for all the three designs at r= 0.1 and 0.5,
0.1 0.2 0.3 0.4 0.5
0.85
0.90
0.95
r
D-e
ff
II, 0
II,π/4
III
0.1 0.2 0.3 0.4 0.5
0.90
0.95
1.00
r
Q-e
ff
0.50.40.30.20.1
0.9
0.8
0.7
r
G-e
ff
(a) D-efficiency (b) G-efficiency
(c) Q-efficiency
53
respectively. For small adjustments to the region ( 1.0r = ) in Figure III.4, Design I is
nearly rotatable. The effect of excluding the high-high corner results in having a slightly
higher prediction variance than any other corner. Design III is rotatable, while Design II
is the least rotatable. Since Design I is optimal in both G- and Q-sense, it has the smallest
levels of prediction variance contours in the region of interest. However, if the researcher
is primarily interested in the adjusted corner, Design II performs best. Figure III.5 shows
that Design II with angle π/4 radians has the lowest values of the prediction variance at
the adjusted corner, which helps it in the Q-sense. For large radii, Design III loses its
good Q-efficiency because of the large region with extrapolation and high prediction
variances.
Figure III.4: Contour Plots of v(x) for the three Designs at r=0.1
All of the above designs, of course, can accommodate center runs. As with the
standard designs, the effect of center runs result in the alphabet optimality efficiencies
10-1
1
0
-1
x1
x2
Design I
-1 0 1
-1
0
1
x1
Design II ,
x2
-1 0 1
-1
0
1
x1
x2
Design III
1.5
2.0
3.0
4.0
5.0
6.0
7.0
θ=π/4
54
being reduced, but an extra degree of freedom is available for lack of fit. The ability to
measure lack of fit gives information about possible curvature for either of the effects. If
multiple center runs are used then an estimate of pure error can be obtained. Note that for
Design III, the center run would not be at (0,0), however, the effect of using the point
(0,0) instead of the true center run is very small.
Figure III.5: Contour Plots of v(x) for the three Designs at r=0.5
For any experiment there are two regions: operability region and the region of
interest. The operability (experimental) region is defined on the basis of the capability of
the process to operate at certain settings of the independent variables. However, the
researcher may have primary interest in a sub-region of the operability region, which is
-1 0 1
-1
0
1
x1
x2
Design III
1.5
2.0
3.0
4.0
5.0
6.0
7.0
-1 0 1
-1
0
1
x1
x2
Design II ,
10-1
1
0
-1
x1
x2
Design I θ=π/4
55
called the region of interest. Typically, the two regions are the same. Consider now that
the region of interest is the whole cube, for which the 22 Factorial design is optimal.
Table III.2 shows the relative efficiency of Design I, the optimal design for the restricted
region, to the 22 Factorial for three values of r. This efficiency gives some sense of what
we are losing by having the restriction on the design space.
Table III.2: Alphabetical Relative Efficiency of Design I to the 22 Factorial Design
r =.1 r =.5 r =1
D-efficiency 0.97 0.85 0.71
G-efficiency 0.89 0.51 0.23
Q-efficiency 0.98 0.83 0.49
III.4 ExampleSuppose an experimenter has a design region with two explanatory variables,
]300 ,200[X 1 ∈ and ]57 ,50[X 2 ∈ , but has reason to believe that combinations of the
variables that lie outside of the location (270,70) will cause problems. Then, we can
convert to coded variables *1X and *
2X in the [0,1] range using the following
relationships, 100
2001*1
−=
XX and
25502*
2
−=
XX . This gives a point on the boundary of
the quarter circle as (x1, x2)=(270,70) in coded variables as ( *1X , *
2X )=(0.7,0.8). We can
then solve for the appropriate radius, r, with the equation 2222
211 )()( rxxxx cc =−+−
where the center of the circle defining the boundary curve is )1,1(),( 21 rrxx cc −−= .
There are two solutions to the equation 222 ))1(8(.))1(7(. rrr =−−+−− , namely, r
=0.846 and r =0.154. However, the second solution is not sensible, since it gives the
center of the circle outside of the point on our boundary curve. In general, we would
choose the solution with (1-r) value less than both of the coded values on the boundary
point. Hence, in this case we obtain a solution of r = 0.846, which suggests an optimal
56
design point using Design I in the restricted region of )752,.752(.)1,1(22
=+−+− rr rr
in coded variables or (275.2, 68.8) in the original variables.
If it were not possible to run the experiment with these precise values at these
experimental conditions, but instead needed to round to the nearest unit, then it would be
necessary to round down for both variables to ensure that we remain in the feasible
region. This would yield the location of (275, 68) in the restricted corner, as well as the
standard design points of (200,50), (200, 75), (300, 50) in the other corners of the design
space. However, this rounding results in a different design than Design I. If one would
like to use the optimal design (Design I), then it is better to round down the radius value
or to change the coded values a little bit. In our case, one could use (0.76,0.76) which is
(276,69) in the original scale and has a radius value of 0.819.
III.5 General Design Space and Design IThe optimality of Design I in the sense of all three alphabetical criteria
encourages us to check its optimality for other related design spaces. So far, we
considered replacing the high-high corner of the cube with a quarter of a circle of radius
r, i.e., we implemented the equation 2222
211 )()( rxxxx cc =−+− , where (x1c, x2c) = (1-
r,1-r) the center of our circle. A more general equation that gives flexibility of the design
space is ddc
dc rxxxx =−+− |||| 2211 , for d > 0. Figure III.6 shows the effect of
changing the value of d on the design space. For d=1 we have a straight line truncating
the corner. To make the design space as close as possible to the cuboidal standard design
space, we could use increasing powers of d. This is beneficial only if the interaction
between the two factors is known a priori to be positive, or synergistic. However, if the
interaction between the two factors is known to be negative it may be more appropriate to
use a d value less than 1.
57
Figure III.6: Operability Region for different d values
Design I replaces the high-high point with a point at angle π/4 radians on the
boundary of the restricted design space for the chosen value of d. Using the equivalence
theorem, this design is D- and G- optimal for 99.0d ≥ . The G-efficiency decreases as d
gets less than 0.99. Design I remains optimum in the Q-sense as long as 75.0d ≥ . That is
for a certain value of r, the only point on the boundary of the design space at the altered
corner that minimizes the integration of the scaled prediction variance is the point (a, a).
Note that one could implement the general formula for the boundary in any of our
three designs. Table III.3 represents some possible design matrices for Designs I and II
( 8/πθ = ) for different combinations of d and r.
(0,1)
(1,0)(0,0)
d=3
d=2
d=1
d=0.5
58
Table III.3: Design matrix of Design I and II ( 8/πθ = ) for Different combinations of (d, r)
(d, r) Design I Design II (θ = π /8)
(0.5, 0.1)
−−
−−=′
85.011185.0111
D
−−
−−=′
8740.08307.01118307.08740.0111
D
(0.5, 0.5)
−−
−−=′
25.011125.0111
D
−−
−−=′
3702.01533.01111533.03702.0111
D
(0.5, 1.0)
−−−−−−
=′5.01115.0111
D
−−−−−−−−
=′2596.06933.01116933.02596.0111
D
(1.0, 0.1)
−−
−−=′
9.01119.0111
D
−−
−−=′
9414.08586.01118586.09414.0111
D
(1.0, 0.5)
−−
−−=′
5.01115.0111
D
−−
−−=′
7071.02929.01112929.07071.0111
D
(1.0,1.0)
−−
−−=′
01110111
D
−−−
−−−=′
4142.04142.01114142.04142.0111
D
(2.0, 0.1)
−−
−−=′
9414.01119414.0111
D
−−
−−=′
9847.08765.01118765.09847.0111
D
(2.0, 0.5)
−−
−−=′
7071.01117071.0111
D
−−
−−=′
9239.03829.01113827.09239.0111
D
(2.0, 1.0)
−−
−−=′
4142.01114142.0111
D
−−−
−−−=′
8478.02346.01112346.08478.0111
D
III.6 Conclusions and DiscussionFor the first order with interaction model in the two-factor case, three sensible
designs are proposed to suit the restricted design space that excludes the high-high
combination of the factors. To define the boundary of the restricted region, the high-high
corner was replaced with a quarter of a circle. The radius of the circle specifies what
59
fraction of design space needs to be altered. Although, we have considered the problem
of excluding the high-high combination, all our results hold for the problem of excluding
any combination of the two factors.
Using formal optimality criteria, Design I with its minimal number of points,
maximal space filling, and near orthogonality performed best. But each of the other
designs has some other desirable aspects of estimation or implementation.
Design I was considered in a more general form of defining the design space
boundary. This general form is intuitive since it gives the practitioner more flexibility to
define the design space. Depending on prior information about restrictions of feasible
points or the nature of the interaction term, one can choose the power in the general form
equation. Design I remains optimal in the D- and G-sense for power values greater than
or equal to 0.99. Design I is also Q-optimum for power values greater than or equal to
0.75.
Overall, Design I, which shifts the (1,1) point to get it into the feasibility region,
is the preferred design using the various optimality criteria. Design II is to be preferred if
we are primarily interested in estimating the interaction term precisely, if cost was not of
an issue (i.e. there is no need to scale by number of runs). Note also that Design II would
appeal to practitioner who have already done one-at-a-time studies on each factor and
now want to study the interaction of the two more thoroughly by looking at two new
combinations of the drugs. Maintaining the orthogonal design in the reduced region is an
appropriate strategy for regions with moderate or small truncations of a corner. Any of
the suggested designs can be supplemented with center runs to estimate lack of fit and
pure error.
III.7 ReferencesAtkinson, A.C. and Donev, A.N. (1992). Optimum Experimental Designs. Oxford
University Press, Oxford.
Box, G.E.P. and Draper, N.R. (1975). “Robust Designs”. Biometrika, 62, 347-352.
60
Box, G.E.P. and Hunter, J.S. (1957). “Multi-factor Experimental designs for Exploring
Response Surfaces”. Annals of Mathematical Statistics, 28, 195-241.
Draper, N.R. and St John, R.C. (1977), “Designs in Three and Four Components for
Mixtures Models with Inverse Terms”. Technometrics, 19, 117-130.
Johnson, M.E. and Nachtsheim, C. J. (1983). “Some guidelines for Constructing Exact D-
Optimal Designs on Convex Design Spaces”. Technometrics, 25, 271-277.
Kennard, R.W. and Stone, L. (1969). “Computer Aided Design of Experiments”.
Technometrics, 11, 298-325.
Kiefer, J. and Wolfowitz, J. (1960). “The Equivalence of Two Extremum Problems”.
Canadian Journal of Mathematics,12, 363-366.
Montgomery, D. C. , Loredo, E. N., Jearkpaporn, D., Testik, M.C. (2002). “Experimental
Designs for Constrained Regions”. To appear in Quality Engineering.
Myers, R.H. and Montgomery, D.C. (1995). Response Surface Methodology: Process
and Product Optimization Using Designed Experiments. Wiley.
Nachtsheim, C. J. (1987). “Tools for Computer-Aided Design of Experiments”. Journal
of Quality Technology, 19, 132-160.
Snee, R.D. (1985). “Computer-Aided Design of Experiments – Some Practical
Experiences”. Journal of Quality Technology, 17, 222-236.
61
Supplement I: Modifying 23 Factorial to Accommodate a Restricted DesignSpace
In this section, we generalize the three designs (I, II, III) presented in Chapter III
to the three-factor setup. For the quantitative three-factor case the standard cuboidal
experimental region is the cube. Consider for some reason the high-high-high (H-H-H)
combination of the factors is prohibited. Analogous to the previous chapter, we chose to
exclude a cube of side length r from the restricted H-H-H corner and replace it with a
sector of a sphere of radius 2r=ρ centered at )r1 ,r1 ,r1( −−− . Figure SI.1
represents the operability region in this case. The total volume of this restricted design
space in the [0,1]2 scale is calculated as follows
)1( 31
1
1
1
1
1123 rdxdxdxwvolume
r r r
−+= ∫ ∫ ∫− − −
where ε+
+=
g
ggw
21
, ε is a small number used to ensure that the denominator is greater
than zero and ∑=
−−−=r
ii rxg
1
22 ))1((ρ . The function w is an indicator function to filter
out the points outside the sphere in the restricted corner.
Design I replaces the H-H-H point in [0,1]2 scale with the point
)a,a,a(a* = where 3/r1a ρ+−= . Design II uses three points to replace the H-H-H
point: )x,x,x(a i3i2i1*i = , where )r1(sinsinx iii1 −+= θφρ ,
)r1(cossinx iii2 −+= θφρ and )r1(cos x ii3 −+= φρ , i=1,2,3. The angles (θi , φi) are
defined in Figure SI.2. We considered two sets of the angles [(θ1, φ1), (θ2, φ2), (θ3, φ3)].
Set 1 is )]90,45(),45,0(),45,90[( °°°°°° , this set gives us three edge-points: (1-r, 1, 1),
(1,1-r,1), (1, 1,1-r), respectively, corresponding to the angle value o0 in the two-factor
case. Set 2 is °=°= 7356.54,45 ii φθ , i=1,2,3, which results in three points on top of
each other corresponding to the angle value o45 in the two-factor case and the point
)a,a,a(a* = of Design I. Design III uses the same point a* of Design I, but it also
62
modifies all the other corners that include the high level. The design matrix in [-1,1]2
scale is as following
−−
−−−
−−−−
−−−
=
bbbbb
bbb
bbb
b
xxx
DIII
11
111
1111
111321
where b is the corresponding value of a in the [-1,1] scale. Note that Design III is just a
smaller regular factorial, which does not fill the whole experimental space shown in
Figure SI.1. An extrapolation space of [volume-a3] is present in the [0,1]2 scale. Figure
SI.3 shows the total volume of the experimental region and the volume of the cube
implemented in Design III (a3). The figure shows how the extrapolation area increases
rapidly as r increases while the volume of the experimental decreases slowly.
63
Figure SI.1: Modifying the Cuboidal Operability Region in the Three Factor Case
Figure SI.2: The Definition of θi and φi
(1-r , 1-r , 1-r )
X2
rX1
( xi , yi ,zi )
φi
θi
ρ
X3
(0,1,0)
(0,1,0)
r
(1,1,0)
(0,0,1)(0,0,0)
(0,1,1)1-r
(0,0,1)(1,0,1)
(1,1,0)
(0,0,1)(0,0,0)
(0,1,1)1-r
(0,0,1)(1,0,1)
64
0 0.2 0.4 0.6 0.8 1r
0.4
0.6
0.8
1
Volu
me
Experimental Region
Design III Region
Figure SI.3: Total Space Volume of the Restricted Region and the Volume of the Design Space ofDesign III
We will discuss two first order models with interaction in the three-factor case:
Model I considers only the main effects with two-way interactions (without the three-way
interaction term) while Model II contains main effects and all possible interactions (two-
and three-way interactions). Equations SI.1 and SI.2 show the functional forms of Model
I and Model II, respectively.
∑∑∑<=
++=33
10
jijiij
iii xxxy βββ (SI.1)
321123
33
10 xxxxxxy
jijiij
iii ββββ +++= ∑∑∑
<=
(SI.2)
Model I would be the more common choice in response surface modeling, but there may
be situations when we wish to consider a three-way interaction term in the model. Design
I with Model II is saturated and is the D- and G-optimum design for our restricted region.
65
For Model I, Table SI.1 compares the three designs in terms of the descriptive
and quantitative measures, discussed in the two-factor case. Design III, the reduced area
full factorial, is orthogonal and has the fewest numbers of levels per factor. Design I has
the highest determinant of the information matrix and the lowest SPV among the three
designs. Therefore, its D- and Q- efficiency in Table SI.1 is set to one and the relative D-
and Q-efficiency of the other two designs relative to Design I is calculated. Table SI.1
shows the G-efficiency of all the designs relative to the G-optimum design, which by
theorem has maximum scaled prediction variance equal to P. Design I is the most G-
efficient design. Note that the same behavior of the Q-efficiency of Design II here is as it
was in the two factor case, namely an increasing function in the side length of the
excluded cube. In general, the alphabetical efficiency decreases as the side length of the
excluded cube, r, increases and within a particular value of r for Design II the efficiency
increases as the three points gets closer to the point a*. The maximum correlation
between any two terms in the model remains relatively small for all designs. For Designs
I and II, the correlation increases as we alter more of the design space with a large r
value. For Design II the largest correlation between any two terms in the model is
minimized with Set 2, the three points at a*. Design I also is the most Ds-efficient design.
Ds-optimum designs are appropriate when interest is in estimating a subset of s of the
parameters of the model as precisely as possible. This might be the case if the researcher
has already done one-at-a-time studies on each factor and now wants to study the
interaction of the factors more thoroughly, in which case a Ds-efficient design is
preferable, where s is the number of the interaction terms in the model. The Ds-optimum
design is that design that maximizes 22
111 );(
M
XMM
β=
−, where the moment matrix is
partitioned as
′
==2212
1211);(1
);(MMMM
XIN
XM ββ and M11, the s×s left upper sub-
matrix of ),(1 XM β− , is the covariance matrix for the least squares estimates of the s
parameters we are interested in (Atkinson and Donev, 1992). Table SI.1 shows the Ds-
efficiency of Design II and III relative to Design I . Design III is the second best design in
terms of the Ds-sense for small values of r, but it is the worst design for large values of r.
66
Table SI.1: Comparison of Designs with Model I
* denotes best design for each criteria at a given side length of the excluded cube value
Design I Design II Design IIIDescription of Design (1,1,1) →
(a*,a*,a*)3 pts on the portion of a
sphereorthogonal
Descriptive Measures# design points 8* 10 8*Orthogonality close Orthogonal*Levels per factor 3 5 2*
Quantitative Measuresthe first entry gives the value for r= 0.1; Second entry gives the value for r= 0.5
Set
Relative D-efficiencyto Design I
1.00* – 1.00* 12
0.90 – 0.890.91 – 0.90
0.97 – 0.85
G-efficiency 0.995* – 0.98* 12
0.81 – 0.780.82 – 0.80
0.94 – 0.71
Relative Q-efficiencyto Design I
1.00*-1.00* 12
0.89 – 0.930.89 – 0.91
0.98 – 0.89
MaximumCorrelation
0.018 – 0.099 12
0.107 – 0.1450.103 – 0.112
Orthogonal*
Relative Ds-efficiencyto Design I
1.00* – 1.00* 12
0.68 – 0.610.71 – 0.72
0.84 – 0.42
If the three-way interaction term is included in the model, the D- and G-
efficiencies of Design I improve. For Model II, Design I is optimum in the D- and G-
sense by the Equivalence Theorem. Table SI.2 shows the quantitative measures of the
three designs for Model II. The D- and G-efficiencies of Designs II and III are again
decreasing functions in r. For Design II at a certain value of r, both efficiencies increase
as the three points get closer to the point a*. For large values of r, Design II with Set 1
using Model II is worse in the G-sense than it was when using Model I. Design I is the
most Ds-efficient design. Table SI.2 shows the Ds-efficiency of all the other designs
relative to Design I. Design III is the second best design in terms of the Ds-sense for
small values of r, but the worst one for large values of r.
67
Table SI.2: Quantitative Measures of Designs with Model II
1) the first entry gives the value for r= 0.1; Second entry gives the value for r= 0.5;
2) * denotes best design for each criteria at a given side length of the excluded cube
Design I Design II Design IIISet
Description of Design (1,1,1) →(a*,a*,a*)
3 pts on the portion of asphere
orthogonal
D-efficiency 1.00* – 1.00* 12
0.91 – 0.890.92 – 0.92
0.96 – 0.81
G-efficiency 1.00* – 1.00* 12
0.80 – 0.480.80 – 0.80
0.96 – 0.81
MaximumCorrelation
0.024 – 0.130 12
0.101 – 0.2060.096 – 0.111
Orthogonal*
Relative Ds-efficiency 1.00* – 1.00* 12
0.58 – 0.430.62 – 0.66
0.77 – 0.27
For any experiment there are two regions: operability region and the region of
interest. The operability (experimental) region is defined on the basis of the capability of
the process to operate at certain settings of the independent variables. However, the
researcher may have primary interest in a sub-region of the operability region, which is
called the region of interest. Typically, the two regions are the same. Consider now that
the region of interest is the whole cube, for which the 23 Factorial design is optimal.
Table SI.3 shows the relative efficiency of Design I using Model I to the 23 Factorial for
three values of r. This kind of efficiency gives some sense of what we are losing by
having the restriction on the design space. It seems that we do not lose much by
implementing the restriction on the region in terms of the alphabetic criteria.
Table SI.3: Alphabetical Relative Efficiency of Design I to the 23 Factorial Design for Model I
r =.1 r =.5 r =1D-efficiency 0.99 0.94 0.89G-efficiency 0.995 0.98 0.95Q-efficiency 0.99 0.96 0.93
For Model I, Figure SI.4 shows the D-efficiency of Designs II (Set 1 and 2) and III
relative to Design I at different length values of the excluded cube. Design III is better
than Design II in the D-sense at low values of r. Design II with Set 2 is most efficient at
the high values of r. Design II performs best when we use Set 2. From Figure SI.5,
Design I is very close to G-optimality at all the side length values. Design III has high
68
efficiency in the G-sense for small r as well. However, as r increases the efficiency of
Design II with both angle sets begins to dominate the efficiency of Design III. Again,
Design II performs best with Set 2. Figure SI.6 presents the relative Q-efficiency of
Design II and III relative to Design I. Design III is better in the Q-sense than Design II at
both of the angle sets for small and moderate truncation of the region. As the side length
of the excluded cube value gets larger Design II at both angle sets begins to dominate
Design III.
Table SI.4 represents some possible design matrices for Designs I and III for
different values of r.
Figure SI.4: Relative D-efficiency of Designs II and III to Design I
0.1 0.2 0.3 0.4 0.5Radius
0.8
0.85
0.9
0.95
1
Rela
tive
D-e
ffici
ency
Design I
Design II (Set 1)
Design II (Set 2)
Design III
69
Figure SI.5: G-efficiency of the three Designs
Figure SI.6: Relative Q-efficiency of the three Designs
0.1 0.2 0.3 0.4 0.5Radius
0.8
0.85
0.9
0.95
1
Rela
tive
Q-e
ffici
ency
Design I
Design II (Set 1)
Design II (Set 2)
Design III
0.1 0.2 0.3 0.4 0.5Radius
0.8
0.9
1
G-e
ffici
ency
Design I
Design II (Set 1)
Design II (Set 2)
Design III
70
Table SI.4: Design matrix of Design I and III for Different values of r
r Design I Design III
0.1
−−−−−
−−−
−−−
−
=
111111111
111111
111111
9633.09633.09633.0
D
−−−−−
−−−
−−−
−
=
1119633.011
19633.019633.09633.01
119633.09633.019633.0
19633.09633.09633.09633.09633.0
D
0.3
−−−−−
−−−
−−−
−
=
111111111
111111
111111
0.88990.88990.8899
D
−−−−−
−−−
−−−
−
=
1110.889911
10.889910.88990.88991
110.88990.889910.8899
10.88990.88990.88990.88990.8899
D
0.5
−−−−−
−−−
−−−
−
=
111111111
111111
111111
0.81650.81650.8165
D
−−−−−
−−−
−−−
−
=
1110.816511
10.816510.81650.81651
110.81650.816510.8165
10.81650.81650.81650.81650.8165
D
1
−−−−−
−−−
−−−
−
=
111111111
111111
111111
0.6330.6330.633
D
−−−−−
−−−
−−−
−
=
1110.63311
10.63310.6330.6331
110.6330.63310.633
10.6330.6330.6330.6330.633
D
71
Supplement II: FDS Technique for the Three Designs in Restricted DesignSpace
In this section, we compare our three designs using the FDS technique for the
two- and three-factor case. Subsection SII.1 presents the FDSG and SFDSG for the two-
factor case, while Subsection SII.2 contains the same graphs for the three-factor case. We
should note that we could not use the VDG to compare these designs since the VDG is
not easily applicable to irregular (restricted) regions.
SII.1 Two-Factor CaseIn this section we use the FDS technique to compare our three designs: Design I,
which shifts the (1,1) point to get it into the feasibility region, Design II that uses two
symmetric points on the quarter circle, and Design III, the reduced area full factorial.
Figure SII.1 shows the FDSG of Design I in two factors for d=2 and different values of r.
The graph shows that the design is 100% G-optimum for all values of r; and we know
from the Equivalence Theorem that it is D- and G-optimum. The minimum SPV value of
Design I does not change with r. Recall that changing r changes the region of operability,
so for different r we have different regions of interest as well as different designs.
Truncating more from the region results in having a less stable distribution of scaled
prediction variance (SPV). This is obvious from the SPV contour plots in Figure SII.2.
The contours are pushed toward the (-1,-1) corner when we increase the value of r, which
is a natural effect of pushing the border of our design space operability towards the low-
low corner.
72
0 1 2 3 4Variance
0
0.2
0.4
0.6
0.8
1
Frac
tion
ofSp
ace
Cri
teri
on
r=.1
r=.5
r=1
100%
G-e
ff
Figure SII.1: FDSG of Design I in Two Factors for Different Values of r and d=2
73
Figure SII.2: Contour Plots of v(x) for Design I at r=0.1, 0.5, 1
Figure SII.3 presents the FDSG of Design I, II ( 4/ ,0 πθ = ) and III for small
changes to the region, namely for radius value equal to 0.1 and d=2. All the designs have
almost the same minimum SPV. Design III is more G-efficient than Design II at its both
angles. All the designs perform virtually equally in terms of prediction within the first
20% of the design space. Thereafter, the FDS curves begin to separate indicating
different prediction capability of each design. Design III is a real competitor to Design I,
the best design in terms of prediction. Design II at both angles has the worst overall
prediction. The angle selected for the location of the points has very small effect on the
prediction of Design II at r=1 and d=2. Since the minimum SPV of all the designs is the
same, the appearance of the SFDSG (not shown here) is exactly the same as the FDSG.
10-1
1
0
-1
x1
x2
r=1
1.5
2.0
7.0
6.0
5.0
4.0
3.0
-1 0 1
-1
0
1
x1
x2
r=0.5
10-1
1
0
-1
x1
x2r=0.1
74
This means that, from Figure SII.2, Design I and III provide us with a more stable SPV
distribution than either choice of Design II.
1 2 3 4 5Variance
0
0.2
0.4
0.6
0.8
1
Frac
tion
ofSp
ace
Cri
teri
on
D I
D II - 45
D II - 0
D III
100%
G-e
ff
Figure SII.3: FDSG of Design I, II, and III in Two Factors for r= 0.1 and d=2
Figure SII.4 shows the FDSG for our different designs when the space is more
substantially altered, namely for r=0.5 and d=2. Design II at 0=θ has the largest
minimum SPV. The other designs have almost the same minimum value of SPV. The
Figure shows that Design I is 100% G-efficient, while Design II at 0=θ is the worst G-
efficient design (approximately 68%) among all the other designs. Design II at π/4 is
better than Design III in the G-sense. Beyond the first 30% of design space, Design I is
superior in prediction capability. Design II with the two points on top of each other is
better than Design II at the zero angle and Design III in terms of prediction. The SFDSG
is shown in Figure SII.5. Since the minimum variances for the designs differ, the SFDSG
changes slightly from the FDSG. Design I again is the most stable design, since its curve
has the steepest slope and is closest to vertical. Design II with either of its two angles is
more stable than Design III over almost 90% of the design space.
75
1 2 3 4 5 6Variance
0
0.2
0.4
0.6
0.8
1
Frac
tion
ofSp
ace
Cri
teri
on
D I
D II - 45
D II - 0
D III
Figure SII.4: FDSG of Design I, II, and III in Two Factors for r= 0.5 and d=2
1 2 3 4 5 6Variance
0
0.2
0.4
0.6
0.8
1
Frac
tion
ofSp
ace
Cri
teri
on
D I
D II - 45
D II - 0
D III
Figure SII.5: SFDSG of Design I, II, and III in Two Factors for r= 0.5 and d=2
76
SII.2 Three-Factor CaseFor the three-factor case, we compare our three designs in terms of the FDS
technique when fitting Model I. Recall that Design I shifts the (1,1,1) point to (a, a, a)
where 1<a , Design II uses three points on the sphere portion and Design III is a reduced
area full factorial. Figure SII.6 shows the FDSG of Design I in three factors for d=2 and
different values of r. The graph shows that the design is no longer G-optimal. However,
the design is highly G-efficient as the figure shows. The minimum SPV value of Design I
does not change with r. The stability of the SPV is a decreasing function of r.
Figure SII.7 depicts the FDSG of Design I, II (for angle sets 1 and 2) and III for
r=0.1 and d=2. All the designs have almost the same minimum SPV. Design III is more
G-efficient than Design II at both angles. All the designs perform almost equally in terms
of prediction within the first 25% of the design space. Design III is a real competitor to
Design I, the best design in terms of prediction. Design II at both angle sets has the least
prediction capability. Changing the angle sets does not affect the prediction capability of
Design II at r=1 and d=2 greatly. The SFDSG, in Figure SII.8 shows the superiority of
Design I in terms of the overall prediction capability. Design III is more stable than
Design II with its two angle sets.
77
0 2 4 6 8Variance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Frac
tion
ofSp
ace
Cri
teri
on
Design I
Design II (Set 2)
Design II (Set 1)
Design III
100%
G-e
ffFigure SII.6: FDSG of Design I for different values of r and d=2
0 1 2 3 4 5 6 7 8 9Variance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Frac
tion
ofSp
ace
Cri
teri
on
Design I
Design II (Set 2)
Design II (Set 1)
Design III
100%
G-e
ff
Figure SII.7: FDSG of Design I, II, and III in Three Factors for r= 0.1 and d=2
78
0 2 4 6 8Variance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Frac
tion
ofSp
ace
Cri
teri
on
Design I
Design II (Set 2)
Design II (Set 1)
Design III
Figure SII.8: SFDSG of Design I, II, and III in Three Factors for r= 0.1 and d=2
Figure SII.9 shows the FDSG for our different designs when we change the
design space more, namely for r=0.5 and d=2. Again Design I is the most G-efficient
design. Design II at both angle sets is more G-efficient than Design III. Truncating more
from the design space hurts Design III in terms of the G-sense. Within Design II, Set 2 is
better than Set 1, i.e. having all three points in the adjusted corner on top of each other is
better than putting them far apart from each other. Since all the designs have the same
minimum SPV, the SFDSG (not shown here) has the same appearance as the FDSG.
Design I is the most stable design among all of our designs. The least stable design is
Design III.
79
0 1 2 3 4 5 6 7 8 9Variance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Frac
tion
ofSp
ace
Cri
teri
on
Design I
Design II (Set 2)
Design II (Set 1)
Design III
100%
G-e
ff
Figure SII.9: FDSG of Design I, II, and III in Three Factors for r= 0.5 and d=2
80
Chapter IV Use of Standard Factorial Designs withGeneralized Linear Models
IV.1 AbstractOptimum experimental designs for generalized linear models (GLM) depend on the
values of the unknown parameters. Several solutions to the dependency of the parameters
of the optimality function were suggested in the literature. However, designs that need
good initial guesses, a complicated minimax procedure, or estimation of the unknown
parameters at each step to make the next move, are often unrealistic in practice. Rather,
one seeks designs which are simple to implement and have ‘good’ efficiencies! In this
paper, the behavior of the factorial designs, the well-known standard designs of the linear
case, is studied in the GLM case. Conditions under which these standard designs have
high G-efficiency are formulated.
IV.2 IntroductionExperimental design techniques in nonlinear model settings are needed for a
broad range of applications in the chemical, biological and clinical sciences. Often, one
faces responses such as the number of defectives out of N sampled items, the number of
defects that occur in a piece of yarn in the textile industry, or the number of organisms
that survive a pesticide or chemical treatment. For more examples, see Myers and
Montgomery (2002), Hamada and Nelder (1997), Lewis et al. (2001), or Myers,
Montgomery and Vining (2002).
Tools for design choice or design optimality in the nonlinear setting arise in
applications with non-normal responses and generalized linear models are commonly
used. But it is well known that design optimality in the nonlinear case is very difficult to
implement. The information matrix is a function of the design levels in the linear models
case, while it is also a function of the unknown parameters in the nonlinear case. In
generalized linear models (GLM), the information matrix, apart from a constant, is
81
WXX);X(I ′=β , where W is the Hessian weight matrix, which is a function of
unknown parameters (McCullagh and Nelder, 1989). In spite of this, design optimality in
GLM has been investigated, but not to a great extent. Two GLM models have received
considerable attention in the literature: the logistic and the Poisson regression models.
The logistic regression model is defined as ßx′−+
=e1
1p , where ßx′ is the so-called
“linear predictor” and p is the binomial probability that is modeled against the regresssors
in x. The Poisson model with the log or canonical link is defined as ßx′= eµ , where µ is
the Poisson mean. We will consider these two models in the current paper.
For the one-variable logistic regression model, Kalish and Rosenberger (1978)
derived locally D- and G-optimal designs. Abdelbasit and Plackett (1983) found a D-
optimal design using the idea of a two-stage model. Myers et al. (1996) introduced a two
stage D-Q design. Chaloner and Larntz (1989) took a Bayesian approach to find robust
optimal designs. A minimax procedure for finding the optimal designs is proposed by
Sitter (1992). Ford et al. (1992) developed D-optimal designs for GLM with one design
variable using the geometry of the design space. Sitter and Wu (1993) used a different
approach to obtain characterizations of the D-, A-, and F-optimal designs for binary
response and one single variable model. F optimality deals with optimal estimation of an
ED, i.e., the level of x that produce a pre-specified probability. Sitter and Torsney (1995)
developed methods for deriving D-optimal design for GLM with multiple design
variables. Other works that considered the logistic model are Heise and Myers (1996) and
Sitter and Wu (1999). Burridge and Sebastiani (1994) considered the use of standard
designs for GLM models when variances are proportional to the square of the mean
response. In addition, Burridge and Settimi (1998) and Dette and Wong (1999) deal with
a more general problem. Atkinson and Haines (1996) discussed locally D-optimal designs
for nonlinear models in general and GLM as well as the Bayesian approach for
optimality. Atkinson and Donev (1992) discussed in a very brief section the D-optimality
for the one variable logistic regression case.
The above literature offers valuable optimal designs in the GLM setup. However,
designs that need good initial guesses, a complicated minimax procedure, or estimation of
82
the unknown parameters at each step to make the next move, are often unrealistic in
practice. Rather, one may seek designs which are simple to implement and have ‘good’
efficiencies! In this paper, the behavior of factorial designs, the well-known standard
designs of the linear case, is studied in the case of logistic and Poisson regression models.
The motivation for the use of these designs is clear. They require neither prior knowledge
nor sequential estimation of the parameters. Conditions under which standard designs
enjoy high efficiency are discussed.
The outline of the paper is as follows. Section IV.3 includes a brief discussion of the
motivation for use of standard designs with GLM. Section IV.4 discusses the structure of
the information matrix in the GLM case. It finishes with some examples of the efficiency
of the factorial designs with the logistic and Poisson models. Section IV.5 presents
general results for GLM with canonical link and the 22 factorial design. Section IV.6
discusses the use of the 22 factorial designs with the logistic and the Poisson regression
models. Section IV.7 discusses the optimality of the factorials when variance-stabilizing
link is used. An illustration with a real life example is presented in Section IV.8. Finally,
Section IV.9 concludes the paper.
IV.3 Implementation of Standard DesignsQuite often in engineering and the physical sciences, the engineer or the scientist
has an a priori notion of a region of operability. This is true even if the experimental plan
includes a sequential approach. This choice is clearly made for scientific reasons based
on knowledge of the system under study. As a result, the efficiency of interest for a
design should be that which is relevant to the restricted region. The idea here is to let the
researcher determine his/her experimental region, with any restrictions on the region that
might exist. Frequently, this region is often characterized by a cube and thus it is
convenient to use the standard factorial (or fraction) as the design. Thereby, the choice of
design will not depend on parameters. It is known that the factorial design, which was
developed for linear models with homogenous variances, has desirable properties. The
design is often orthogonal and is D- and G- optimal for linear models with restriction to
points inside or on the surface of a cube. See St. John and Draper (1975) and Myers and
83
Montgomery (2002). However, the question is how efficient is the two level Factorial
design in the GLM case? The design should not be compared to any type of globally
optimal design, which will not be known and quite likely can not be implemented in the
relevant region of interest anyway.
It should be noted that despite the clear appropriateness of this class of designs for
the case of linear models, it is not clear for the GLM case. The use of generalized linear
models often involves a nonlinear model, which involves a situation where the residual
variance is a function of the mean, and thus is non-constant. While the nature of these
relationships depends on the distribution involved and the link function used, these
nonideal conditions are not the ones under which the factorial or fractional factorials
were intended. Nevertheless, we shall demonstrate that often these standard designs are
not only efficient in the GLM case but for certain situations they are D and G optimal
under the operability region restrictions assumed. We shall highlight and review some
results that are general to all GLMs and then use logistic regression and Poisson
regression as illustrations.
IV.4 Characteristics of the Information Matrix in GLMThe nature of the Fisher information matrix is an important factor in determining
the efficiency of a design for the GLM case as it is in the linear model case. The
information matrix is a quantitative measure of the quality of the information on the
parameters available in the data. It is known that the asymptotic variance covariance
matrix of the maximum likelihood estimator (MLE) is the inverse of Fisher information
matrix. The MLE of the parameter is the solution of the likelihood equations (score
function)
−=
=
∂∂
−=∂∂
∂∂
=∂
∂ ∑ 0)()(a
1)(llog)(llog in
1iii ß
yßß
θµ
φθ
θθθ
. (3.1)
where θ is the natural location parameter and φ is the dispersion parameter of the
distribution of yi, β is the vector of the unknown parameters in the model and µ is the
mean response function. For more details see McCullagh and Nelder (1989) or Myers
Montgomery and Vining (2002).
84
Using the chain rule to write ii
i
xß
ßxßxß i
)()(
?=∂
′∂′∂
∂=
∂∂ θθ
, one can write the score
function, apart from a(φ), in matrix form as 0? =−′ )(X µY , where )(diag inn ?? =× .
Apart from a(φ), the Fisher information matrix, I(X;β), is the variance of the score
function
WXXXVX))(Xvar())(llog
var();X(I ′=′=−′=∂
∂= ??? µ
θY
ßß (3.2)
where W is a diagonal matrix that contains the Hessian weights, wi, i=1,…,n. These
weights are functions of the unknown parameters. For the canonical link, the Hessian
weight matrix, W, reduces to V, a diagonal matrix that contains the variances of the
response distribution.
IV.4.1 Use of the two-level FactorialDefining XWZ 2/1= , one can write the Fisher information matrix of the GLMs
as ZZ);X(I ′=β . This has the same appearance as the information matrix for linear
models except that the Z matrix is a function of the unknown parameters. For two-level
factorials and fractional factorials with levels at 1± , the diagonal elements of I(X;β) are
sums of the Hessian weights, ∑=
n
1ii
w , while the off-diagonals are natural contrasts in the
Hessian weights, the same contrasts that produce “effects” in the linear model analysis.
For example, in the case of logistic regression these off diagonal elements are contrasts in
the binomial variances. For an illustration, consider the two-level factorial in two factors,
A and B, with only main effects
−−
−−
=
111111111111
X
=
4
3
2
1
w0000w0000w0000w
W
The information matrix is then
=
∑∑
∑
i
i
i
wContrABContrBContrABwContrAContrBContrAw
)(I β
85
where ContrA is a contrast of A on the Hessian weights, ContrAB is a contrast of the
interaction effect of the two factors A and B on the Hessian weights, which appears in the
information matrix despite the fact that there is no interaction in the model. For more
details, see Myers et al. (2002). The Fisher information matrix will be well conditioned if
these off-diagonals are near zero. Usually, the sum of the Hessian weights will dominate
the off diagonal elements. The Hessian weights for the case of canonical link are merely
the variances of the distribution; and the contrasts in the distribution variance are
certainly dominated by the sum, resulting in a well-conditioned information matrix.
Hence, in this case, good efficiency of the two-level factorial design is expected.
IV.4.2 ExamplesIn this section some hypothetical examples are used to show the G-efficiency of
the 22 factorial design for the case of the logistic and Poisson regression models. The
measure of design efficiency proposed by Myers et al. (2002), which compares the
asymptotic variances of individual coefficients with that obtained if the same Hessian
weights were observed with an orthogonal design, is calculated. This kind of efficiency
gives a sense of the ill conditioning level of the information matrix. In the present article,
we will refer to this kind of efficiency as efficiency of parameter estimates. These
values are very much like variance inflation factors in regression analysis. The
asymptotic variances of the coefficients are also computed and the contour plots of the
scaled prediction variance are shown.
Example 1: Consider the 22 factorial design with a first order model in the linear
predictor in the case of logistic regression. Parameter values are chosen so that the
binomial parameter in the design region is 6.0p4.0 ≤≤ . The G-efficiency of the
factorial design is 99.4%. This reflects the high level of conditioning of the information
matrix in this case. The asymptotic variance covariance matrix is
−−=−
05115.000092.0000092.005115.000005113.0
);X(I 1 β
86
Figure IV.1: Contour Plot of v(x) for 6.0p4.0 ≤≤
Note that asymptotic variances of coefficients are very close. In terms of the efficiency
of the parameters estimates, the efficiencies are 100%, 99.9%, and 99.9% respectively
for iβ̂ , i=0,1,2. In addition, it is of interest to compute the D-efficiency relative to an
orthogonal design given by 999.0)/();( =∑ piwXI β . So, the factorial design behaves
very much like that of an orthogonal design in the case of a linear model. This is also
reflected in the plot of scaled prediction variance, v(x), even though the efficiency is not
100%. See Figure IV.1. The reader should recall that for a linear model the plot of scaled
prediction variance for an orthogonal design contains concentric spheres.
Example 2: Consider the 22 factorial with a “first order plus interaction” model for the
linear predictor in the case of logistic regression, where 7.0p4.0 ≤≤ . With four runs,
the design is saturated. The maximum prediction variance in this case is equal to the
number of parameters (=4) at all the design points. Using the General Equivalence
theorem, the design is both G- and D-optimum. See White (1973) and Silvey (1980).
Another interesting feature of the design is that all the asymptotic variances of the
coefficients are exactly the same (0.05378). We will show in Section (4.4) that for
saturated designs this holds for any GLM model. The contour plot of v(x) is depicted in
-1 -0.5 0 0.5 1-1
-0.5
0
0.5
1
1.45
2.95
87
Figure IV.2. Again this appearance is the same as that for a factorial design used in
conjunction with a linear model involving first order terms and an interaction term.
Example 3: Consider the 22 factorial design used with a first order model for the linear
predictor in the case of Poisson regression using the log link. The parameter values were
chosen so that the Poisson mean satisfies 2010 ≤≤ µ within the design region. The G-
efficiency of the factorial design is 90.5%. The asymptotic variance covariance matrix is
−
−=−
01733.0000199.0001803.00040409.000199.00040409.001826.0
);X(I 1 β
Notice again how the asymptotic variances of the coefficients are close to each other. In
terms of the efficiency of the parameters estimates, the efficiencies are 93.6%, 94.8%,
and 98.7%, respectively for iβ̂ , i=0,1,2. Again, the contour plot of v(x) is nearly the
same as it is in the case of linear models. See Figure IV.3.
Figure IV.2: Contour Plot of v(x) for 7.0p4.0 ≤≤
-1 -0.5 0 0.5 1-1
-0.5
0
0.5
1
1.6
4
88
Figure IV.3: Contour Plot of v(x) for 2010 ≤≤ µ
Example 4: Consider now the 22 factorial design with a “first order plus interaction”
model for the Poisson regression case, where 505 ≤≤ µ and again the log link is
applied. The design again is saturated and if we apply the general equivalence theorem,
the design is both G- and D-optimum, even though the range on the Poisson variance (µ)
is large. All the asymptotic variances of the coefficients are exactly the same, namely
0.0216. The contour plot of v(x) for this region is given in Figure IV.4. Note that the
maximum v(x) is equal to four which is the number of the parameters in the model; and
occurs at the design points.
Figure IV.4: Contour Plot of v(x) for 505 ≤≤ µ
-1 -0.5 0 0.5 1-1
-0.5
0
0.5
1
1.6
4
-1 -0.5 0 0.5 1-1
-0.5
0
0.5
1
1.5
3.3
89
The foregoing examples illustrate that the 22 factorial can be very efficient and even
optimal in some cases for Poisson and logistic regression. Obviously, the efficiency
depends on the values of the mean (and thus the variance) of the distribution. In what
follows we shall provide some more formal details.
IV.5 General Results for GLM with Canonical Link and Standard 22
Factorial DesignOne can show that in the GLM case, the factorial design is quite efficient both in
the D and G sense if the model contains first order and interaction terms except in certain
rather unusual cases. Actually, we considered only the G-efficiency since the D
efficiency is difficult to calculate. It is well known that the G-efficiency is a lower bound
for the D-efficiency, see Atwood (1969). One can show also that under some conditions
the factorial design achieves optimality.
Let us define v4 as the maximum variance of the distribution among all the design
locations. Similarly, let v1 be the minimum. We will assume without loss of generality
that these variances are achieved at (1,1) and (-1,-1). Assume further that v2 and v3 are the
variances at the other two design points. In what follows we characterize how the
distribution variance at the design points influences the scaled prediction variance.
Clearly, if the distribution variance throughout the design region has a very small
dispersion, one would expect the factorial design to have high efficiency. The scaled
prediction variance, employing the canonical link, can be written as1
00 )VXX(v)n4()(v −′=x , where v0 is the distribution variance at the point 0x in question.
IV.5.1 Properties of the Scaled Prediction Variance for the 22 Factorial DesignLet us first assume that the linear predictor contains a first order model. In Appendix
A, we present an equivalent expression for )(v 0x which is written in terms of ratios of
the distribution variances, v1, v2, v3, v4 and v0. The result is found in Appendix A equation
(A.1). Assume further that 40,321
vvv,vv ≤≤ , then the following important results
become clear.
90
1) the scaled prediction variance is a function of the ratio 1
4
v
v=δ
2) the G-efficiency of the design is a decreasing function of δ .
Now, let us suppose that v0 is larger than any one of the variances at the design
points. For example in the case of logistic regression, the ED50 may be inside the design
region. Then we have max04321
vvvvvv ≤≤≤≤≤ , where vmax is the largest variance in
the design region. It turns out that )(v 0x is a function of 4max4 v/v=δ and δ. Hence, the
G-efficiency is also a function of both of these ratios. In fact it is a decreasing function of
both ratios. Our empirical studies show that the impact of δ is much greater than δ4.
Assume now that the linear predictor contains first order plus interaction terms. As in
the case of the no interaction model we develop the expression for the scaled prediction
variance in terms of ratios of the distribution variances, v1, v2, v3, v4 and v0 in Appendix
A. In this case, the expression is simpler and is given by
]}ffvv
fvv
f[v1
{4
nv )x(v 43
3
42
2
41
4
00 +++= δ
where 22
211 )1x()1x(f −−= , 2
12
22 )1x()1x(f −+= , 22
213 )1x()1x(f −+= and
21
224 )1x()1x(f ++= .
Using the same argument as in the no interaction model, the G-efficiency of the 22
design is a decreasing function of the ratio δ. In this case, the value of v4 and its
relationship to vmax does not play a role (see Appendix A).
In the Section V, we will use this important result for the logistic and Poisson case to
ascertain conditions in which standard factorials are highly efficient. However, we first
reveal some highly desirable properties of the 22 factorial design in the case of the
interaction model for GLMs in which the canonical link is used. We noticed these
conditions in examples given in Section 3.2. The properties relate to the coefficient
variances and the nature of the maximum prediction variance.
91
IV.5.2 Interaction Model - Equal Asymptotic Variances of the ParametersEstimates
One can show that all the asymptotic variances of coefficients are equal for the
first order plus interaction GLM model when the 2k Factorial design is used. In the case
of the 22 Factorial design, this variance is given by
4321
314324421321i vvvv16
vvvvvvvvvvvv)ˆvar(
+++=β ,
where vi, i=1,2,3,4, is the variance at the four design points. For non-saturated factorial
designs these asymptotic variances are generally close to each other, as it appeared in the
case of the empirical studies illustrated earlier.
IV.5.3 Interaction Model -the Scaled Prediction Variance at the Design Points
One can prove that the scaled prediction variances at the design points are exactly
P, the number of the parameters in the model, see Appendix B. This is important since
this is one of the necessary conditions for both D- and G-optimality, as given by the
General Equivalence Theorem. When this value of the prediction variance is also the
maximum one in the region of interest, then the above optimality holds. This
phenomenon holds under most practical conditions, but not in general. For large values of
the ratio, δ, the factorial design is not fully G-efficient. We further clarify this in the
following section.
IV.6 A characterization of Efficiency for the Use of 22 Factorial Designswith the Logistic and Poisson Regression Models
In this section we illustrate more carefully the efficiency of the 22 factorial in both
logistic and Poisson regression models with and without interaction by using the results
in Section 4.1.
IV.6.1 First Order Models with and without Interaction: Logistic RegressionConsider first the without interaction model in which the range in p is from 0.7 to
0.9, i.e., a ratio of 2.33, the G-efficiency is 89.6%. The plot of the scaled prediction
92
variance, v(x), takes on nearly the same appearance as it does in the case of linear
models, see Figure IV.5 for 33.2=δ .
If we choose a region with the range in p from 0.2 to 0.7 (clearly a very progratic
situation), then ED50% is clearly inside the region of interest. In this case two ratios affect
the G-efficiency: 56.1=δ , and 19.14
=δ . The G-efficiency here is 94.5%.
Figure IV.5: Contour Plot of v(x) for 33.2=δ
As the ratio, δ, increases, the G-efficiency decreases as we indicated earlier.
Consider now the region 5.0p015.0 ≤≤ , i.e.,δ=17. This scenario does illustrate what
causes low efficiency for the factorial design. The G-efficiency is 77.2%. On the other
hand, for the situation in which 2.0p05.0 ≤≤ , with a ratio of 3.37. The efficiency is
85.8%.
Now consider the interaction model. With four runs, the design is saturated. It turns
out that the factorial design in this case is D- and G-optimum for nearly all the regions
considered. After considerable empirical study, it becomes apparent that the factorial is
optimal both in the D and G sense if the ratio δ is approximately ≤6. In fact, the ratio can
become slightly less than 7 before the efficiency is reduced below 90%.
-1 -0.5 0 0.5 1-1
-0.5
0
0.5
1
1.5
3.3
93
Table IV.2 in Appendix C shows some of the empirical examples we investigated
for the logistic regression model. Note that even when δ is as large as 10, the G-
efficiency for the no interaction model is quite good (approximately 80%).
IV.6.2 First Order Models with and without Interaction: Poisson RegressionIn the case of the no interaction model, if the range in the Poisson parameter µi
results in a ratio, δ=3, the G-efficiency is 86.5%. But if the ratio is as large as ten, the
efficiency is 79.5%. The appearance of the contour plot of scaled prediction variance is
nearly like the standard appearance in the case of linear models for small δ, but it loses
this feature when the ratio gets large. Figure IV.6 and Figure IV.7 show the contour plots
for the case of δ=3 and δ=10, respectively.
Figure IV.6: Contour Plot of v(x) for 3=δ
In the case of the interaction model the saturated 22 factorial design is D- and G-
optimum for nearly all the regions considered for ratios less than or equal to 10. Table
IV.3 in Appendix C shows some of the empirical examples we investigated for the
Poisson regression model.
-1 -0.5 0 0.5 1-1
-0.5
0
0.5
1
1.5
3.3
94
Figure IV.7: Contour Plot of v(x) for 10=δ
IV.7 Variance Stabilizing LinkThe variance stabilizing link function is a function that when applied to the raw
response stabilizes the variance, when the transformed response is fit with a linear model.
However, in the case of the GLM models it is just a transformation on the population
mean. When this link is used, it turns out that all the Hessian weights reduce to a
constant, say k, and 111 )XX(k1
)WXX()ZZ( −−− ′=′=′ . For the entire proof, see Myers
et. al (2002). Thus, for the GLM model with variance stabilizing link, the two-level
factorial design for any number of factors is orthogonal.
Consider now the scaled prediction variance for a GLM model with variance
stabilizing link
xx
xx
x
1
1
1
)(
)]()(1
[
)];();([ )(
−
−
−×
′′=
′′=
=
XXN
kXXk
trN
xJXItrNv PP ββ
-1 -0.5 0 0.5 1-1
-0.5
0
0.5
1
1.7
3.5
95
where J(x;β) is the information matrix due to a single observation, xp×1. For a factorial
design we have P
1 IN1
)XX( =′ − . Thereby the scaled prediction variance of this design in
this case is xx′=)x(v . This is maximized at the extreme values of the variables, i.e., for
a factorial design it is maximized at the design points. For coded variable-levels at ±1,
P=)x(v , the number of the parameters in the model, i.e., the number of columns in x.
Thus, by the Equivalence Theorem, the 2k factorial design in the case of any GLM model
with variance stabilizing link is a D- and G-optimal design. This becomes particularly
important in the case of Poisson, exponential or gamma responses. For the Poisson, the
variance stabilizing link is the square root link and for the later two distributions the
variance stabilizing link is the log link.
IV.8 Real Life ExampleIn this section, we consider a binary real life example. The interest is in the
number of spermatozoa that survived out of 50 samples in a sperm bank. Three factors
are considered here: sodium citrate (x1), glycerol (x2), and equilibrium time (x3). This data
set is published by Myers et al (2002). Table IV.4 Appendix D contains this data set.
Myers et al (2002) initially fit the logistic regression model that includes the linear main
effects and all the two factor interactions. The only significant effects are x2, x1x2. But to
maintain hierarchy we include x1 in our analysis as well. Table IV.1 shows the analysis of
this model. The estimated variance ratio in this case is 68.1ˆ =δ . The sum of the Hessian
weights is 86.641 and hence the estimated efficiencies of the parameters estimates are
approximately 96% for all parameters and the relative D-efficiency compared to an
orthogonal design is 92%. The maximum scaled prediction variance is equal to 4, the
number of the parameters, and is achieved at all design points. Thus, the design is both
D- and G- optimum. Figure IV.8 depicts the contour plots for the scaled prediction
variance. One should note that since parameters are not known but estimated, all
efficiencies are estimates. This example illustrates one of many situations in which a
standard factorial is either optimal or highly efficient in the case of generalized linear
models.
96
Table IV.1: Maximum Likelihood Estimates and Wald Inference on Individual Coefficients
Figure IV.8: Scaled Prediction Variance Contour Plot for Spermatozoa Example
IV.9 ConclusionsOptimality is easier to assess in the linear case than the nonlinear case. Basically,
this follows because the information matrix is just a function of the design points in the
former case, while it is a function of the unknown parameters as well as the design points
in the latter case. In generalized linear models, the information matrix is proportional to
WXX ′ , where W is the Hessian weight matrix, which is a function of the unknown
parameters. Several solutions to the dependency of the parameters of the optimality
function were suggested in the literature. However, these solutions are either complicated
or unrealistic in practice.
Standard Wald 95% Chi-Parameter DF Estimate Error Confidence Limits Square Pr> ChiSq
Intercept 1 -0.3767 0.1099 -0.5920 -0.1614 11.76 0.0006x1 1 0.0938 0.1099 -0.1215 0.3091 0.73 0.3932x2 1 -0.4616 0.1099 -0.6770 -0.2463 17.66 <.0001x1*x2 1 0.5842 0.1099 0.3689 0.7995 28.28 <.0001Scale 0 1.0000 0.0000 1.0000 1.0000
-1 -0.5 0 0.5 1-1
-0.5
0
0.5
1
1
4
97
In this paper, the behavior of factorial designs, the well-known standard designs
of the linear case, has been studied. Although the generalized linear models often results
in non-ideal conditions under which the standard designs were never intended to be used,
these designs are not only efficient but for certain situations they are D and G optimal.
We formulated conditions under which these standard designs have high G-efficiency.
The use of these designs in the GLM case is illustrated with examples from the logistic
and Poisson regression models.
Other desirable properties of the factorial designs are discussed in this paper. In the
case of the interaction model for GLMs in which the canonical link is used, the 2k
factorial design ensures equal asymptotic variances of the estimated coefficients and that
the maximum prediction variance at the design point is equal to the number of the
parameters. Also, by the Equivalence Theorem, the 2k factorial design in the case of any
GLM model with variance stabilizing link is a D- and G-optimal design. This becomes
particularly important in the case of Poisson, exponential or gamma responses.
IV.10 ReferencesAbdelbasit, K. M. and Plackett, R. L. (1983), “Experimental Design for Binary Data”,
Journal of the American Statistical Association, 78, 90-98.
Atkinson, A. C. and Donev, A. N. (1992), Optimum Experimental Designs, Oxford
University Press, Oxford.
Atkinson, A. C. and Haines, L. M. (1996), “Designs for Nonlinear and Generalized
Linear Models”, In: S. Ghosh and C.R Rao, eds., Handbook of Statistics, 13,
Elsevier Science B.V, 437-475.
Burridge, J. and Sebastiani, P. (1992), “D-optimal Designs for Generalised Linear
Models”, Journal Ital. Stat. Soc., 2, 182-202.
Burridge, J. and Sebastiani, P. (1994), “D-optimal Designs for Generalised Linear
Models with Variance Proportional to the Square of the Mean”, Biometrika, 81,
295-304.
98
Chaloner K. and Larntz, K. (1989), “Optimal Bayesian Design Applied to Logistic
Regression Experiments”, Journal of Statistical Planning and Inference, 21, 191-
208.
Chaloner, K. and Verdinelli, I. (1995), “Bayesian Experimental Design: A Review”,
University of Minnesota Technical Report.
Chaudhuri, P. and Mykland, P. A. (1993), “Nonlinear Experiments: Optimal Design and
Inference Based on Likelihood”, Journal of the American Statistical Association,
88, 583-546.
Chernoff, H. (1953), “Locally Optimal Designs for Estimating Parameters”, Annals of
Mathematical Statistics, 24, 586-602.
Chernoff, H. (1979), Sequential Analysis and Optimal Designs, SIAM, Philadelphia, PA.
Dette, H. and Wong, W.K. (1999), “Optimal Designs When the Variance is a Function of
the Mean”, Biometrics, 55, 925-929
Ford, I., Torsney, B., and Wu, C.F.J (1992), “The Use of a Canonical Form in the
Construction of Locally Optimal Designs for Nonlinear Problems”, Journal of the
Royal Statistical Society, Ser B, 54, 569-583.
Hamada, M. and Nelder, J.A. (1997), ``Generalized Linear Models for Quality-
Improvement Experiments'', Journal of Quality Technology, 29, 292-304.
Heise, M.A. and Myers, R. H. (1996), “Optimal Designs for Bivariate Logistic
Regression”, Biometrics, 52, 613-624.
Kalish, L. A. and Rosenberger, J. L. (1978), “Optimal Designs for the Estimation of the
Logistic Function”, Technical Report 33, Pennsylvania State University.
Kiefer, J. and Wolfowitz, J. (1960), “The Equivalence of Two Extremum Problems”,
Cand. J. Math,12, 363-366.
Lewis, S., Montgomery, D. and Myers R. (2001), “Examples of Designed Experiments
with Nonnormal Responses”, Journal of Quality Technology, 33, 265-278.
Martin, B., Parker, D. and Zenick, L. (1987), “Minimize Slugging by Optimizing
Controllable Factors on Topaz Windshield Modeling”, Fifth Symposium on
Taguchi Methods. American Supplier Institute, Inc., Dearborn, MI, 519-526.
McCullagh, P. and Nelder, J.A. (1989), Generalized Linear Models, 2nd edition, New
York, Chapman and Hall.
99
Minkin, S. (1987), “Optimal Designs for Binary Data”, Journal of the American
Statistical Association, 82,1098-1103.
Myers, R.H. and Montgomery, D. C. (1997), “A Tutorial on Generalized Linear Models”,
Journal of Quality Technology, 29, 274-291.
Myers, R.H. and Montgomery, D. C. (2002), Response Surface Methodology: Process
and Product Optimization Using Designed Experiments, 2nd edition, Wiley.
Myers, R.H., Montgomery, D. C., and Vining, G.G. (2002), Generalized Linear Models
with Applications in Engineering and the Sciences, Wiley Series in Probability and
Statistics.
Myers, W. R., Myers, R.H., and Carter, W.H. Jr. (1994), “Some Alphabetic Optimal
Designs for the Logistic Regression Model”, Journal of Statistical Planning and
Inference, 42, 57-77.
Myers, W. R., Myers, R.H., Carter, W.H. Jr., and White, K. L. (1996), “Two Stage
Designs for the Logistic Regression Model in a Single Agent Bioassay”, Journal of
Biopharmaceutical Statistics, 6(4).
Sebastiani, P. and Settimi, R. (1998), “First-order Optimal Designs for Non-linear
Models”, Journal of Statistical Planning and Inference, 74, 177-192.
Sitter R. R. (1992), “Robust Designs for Binary Data”, Biometrics, 48, 1145-1155.
Sitter R. R. and Torsney, B. (1995), “D-Optimal Designs for Generalized Linear
Models”, In: C.P. Kitsos and W.G. Müller, eds., MODA 4 - Advances in Modern
Data Analysis: Proceedings (the 4th international wokshop in Spetses, Greece, June
5-9, 1995). Heidelberg, Germany: Physica-Verlag, 87-102.
Sitter, R. R. and Wu, C. F. J. (1993), “On the Accuracy of Fieller Intervals for Binary
Response Data”, Journal of the American Statistical Association, 88, 1021-1025.
Sitter, R. R. and Wu, C. F. J. (1999), “Two-Stage Design of Quantal Response Studies”,
Biometrics, 55,396-402
100
Appendix A:A.1 First Order Model: The Scaled Prediction Variance of a 22 Factorial and a
Canonical Link in the GLM Case
16
);X(I
)xx(vv)1x(vv)1x(vv)xx(vv)1x(vv)1x(vvnv
)VXX(v)n4()x(v2
2132
2
242
2
213
2
2141
2
121
2
1430
1
00
β
++−+++−+++−=
′= −
where )vvvvvvvvvvvv(16);X(I421324341321
+++=β , v0 is the distribution variance at
the point x0 and n is the number of runs at each design point.
3223432
2
211
322
214
2
23
2
12
2
22
2
13
0
3242341321
2
2132
2
2141
2
23
2
121
2
22
2
134
00
vv)vv(vvv
)xx(v
vv)xx(v})1x(v)1x(v{})1x(v)1x(v{
nv
vvv)vv(vvvvv
)xx(vv)xx(vv})1x(v)1x(v{v})1x(v)1x(v{vnv)x(v
δ
δ
+++
++−+++++−+−
=
+++
++−+++++−+−=
)v
v
v
v(
v
v
v
v)1(
)xx(v
v
v
v)1x(
v
v)1x(
v
v})xx(
v1
)1x(v
v)1x(
v
v{
v
vn)x(v
4
3
4
2
4
3
4
2
2
214
3
4
22
24
32
14
22
214
2
24
22
14
3
4
0
0
+++
++++++−+−+−
=
δ
δ
(A.1)
A.2 First Order plus Interaction Model: The Scaled Prediction Variance of a 22
Factorial and a Canonical Link in the GLM Case
16
);X(I
])1x(v)1x(v[)1x(vv])1x(v)1x(v[)1x(vvnv4
)VXX(v)n4()x(v2
13
2
24
2
121
2
21
2
22
2
1430
1
00
β
++−++++−−=
′= −
where )vvvv(256);X(I4321
=β .
101
]}ffv
vf
v
vf[
v1
{4
nv
}v
)1x()1x(
v
)1x()1x(
v
)1x()1x(
v
)1x()1x({
4
nv)x(v
433
42
2
41
4
0
4
2
2
2
1
3
2
2
2
1
2
2
1
2
2
1
2
2
2
100
+++=
+++
−++
−++
−−=
δ
(A.2)
where 22
211 )1x()1x(f −−= , 2
12
22 )1x()1x(f −+= , 22
213 )1x()1x(f −+= and
21
224 )1x()1x(f ++=
1) Assume 40,321
vvv,vv ≤≤
The prediction variance is an increasing function in the ratio δ. Therefore, the G-
efficiency is a decreasing function in this ratio. Note that
1v
v1
4
0 ≤≤δ
, δ≤≤2
4
v
v1 and δ≤≤
3
4
v
v1 .
2) Assume max04321
vvvvvv ≤≤≤≤≤
The prediction variance is an increasing function of the ratio δ; hence as before the G-
efficiency is a decreasing function in this ratio. Note that
4
max4
4
0
v
v
v
v1 =≤≤ δ , δ≤≤
2
4
v
v1 and δ≤≤
3
4
v
v1 .
102
Appendix BTheorem: For the interaction model, the scaled prediction variance at the design points
of the 2k factorial design when used with any GLM model equals P, the number of the
parameters in the model if the design is saturated.
Proof:
The GLM model is given by ßxy ′== )](E[g)(g µ , where g(.) is the link function that
connects the linear predictor, β′x , to the natural mean of the response variable, µ. This
GLM model can be linearized using Taylor series as follows:
)()(
)()(
)()(
µµµ
β
µµµ
µ
−∂
∂+′=
−∂
∂+≅
yg
x
yg
gyg
Let )()(
)(0
* µµµ
−∂
∂−= y
gygy , and regress y* against x. Note that the variance of y* is
not homogenous, i.e., )var(])(
[)var( 2* yg
yµµ
∂∂
≅ . Hence consider weighted least
squares with weight 12 )}var(])(
{[w −
∂∂
= yg
µµ
. That is, at each iteration, implement the
model *21
21
*21
** www εβεβ +=+== Zxyy . Thus,
iiiiiixWXXxzZZzy 11** )(w)()ˆvar( −− ′′=′′= . However, this is the i-th diagonal element
of the hat matrix, hii, where P==′′== −
=∑ )I(tr)Z)ZZ(Z(rank)H(rankh P
1N
1iii .
Hence, the sum of the scaled predicted variances apart from N is equal to the number of
the parameters. It is also well known that 1h0ii
≤≤ . Therefore, N
hiiP
≤ . Consider now
the saturated case for the factorial design, where the number of design points, d, equals P,
r dN = and r is the number of replicates at each design point, then from the above the
103
ith diagonal element r1
hii = , thus the scaled prediction variance at the ith design point is
P==rN
)x(v .
104
Appendix C
Table IV.2: G-efficiency of the 22 Factorial with Logistic Regression
intervals onp
ED50%
Inside? n
v1
n
v4
1
4
v
v=δ δ4
Range ofvariance
in theregion
Withoutinteraction
Withinteraction
.015-.01553 .0147 .015288 1.04 .000588n 99.4 1000.4-0.6 yes .24 .24 1.04 1.0416 .01n 99.4 1000.3-0.5 .21 .25 1.19 .04n 97.8 1000.5-0.7 .21 .25 1.19 .04n 97.8 1000.4-0.7 yes .21 .24 1.19 1.0416 .04n 97.9 1000.3-0.7 yes .21 .21 1.19 1.1904 .04n 97.5 100
0.2-0.45 .16 .2475 1.55 .0875n 94.3 1000.2-0.7 yes .16 .21 1.56 1.1904 .09n 94.5 1000.3-0.8 yes .16 .21 1.56 1.1904 .09n 94.5 100
0.21-0.8 yes .16 .1659 1.56 1.5069 .09n 93.9 1000.9-0.95 .0475 .09 1.89 .0425n 91.1 1000.05-0.1 .0475 .09 1.89 .0425n 91.1 1000.1-0.3 .09 .21 2.33 .12n 89.6 1000.7-0.9 .09 .21 2.33 .12n 89.6 100
.1-.5 .09 .25 2.78 .16n 88.5 1000.095-.5 .0859 .25 2.91 .1641n 87.6 1000.085-0.5 .0777 .25 3.22 .1723n 86.7 1000.05-0.2 .0475 .16 3.37 .1125 85.8 1000.8-0.95 .0475 .16 3.37 .1125 85.8 1000.075-0.5 .0693 .25 3.61 .1807n 85.7 1000.065-0.5 .0607 .25 4.11 .1893n 84.7 1000.055-0.5 .0525 .25 4.76 .1975n 83.5 1000.045-0.5 .0429 .25 5.83 .2071n 82.2 1000.044-0.5 .0420 .25 5.94 .208n 99.40.043-0.5 .0412 .25 6.075 .2088n 99.10.04-0.5 .0384 .25 6.51 .216n 98.10.035-0.5 .0337 .25 7.42 .2163n 80.7 93.70.025-0.5 .0243 .25 10.28 .2257n 79.2 84.70.015-0.5 .0147 .25 17.01 .2353n 77.2 66.6
)nqp,nqpmax(v22114
= ;
)nqp,nqpmin(v22111
= ;
21ppp ≤≤
105
Table IV.3: G-efficiency of the 22 Factorial with Poisson Regression
1
4
v
v=δ Without interaction With interaction
80 75.3 58.650 76.1 70.320 77.5 92.112 99.410 79.5 1005 82.8 1004 84.3 1003 86.5 1002 90.5 100
),max(v214
µµ= ;
),min(v211
µµ= ;
21µµµ ≤≤
106
Appendix D
Table IV.4: Spermatozoa Survival Data and Design Matrix
Run x1 x2 x3 y1 -1 -1 -1 342 1 -1 -1 203 -1 1 -1 84 1 1 -1 215 -1 -1 1 306 1 -1 1 207 -1 1 1 108 1 1 1 25
107
Chapter V Summary and Future Work
The research contains three main contributions. First, we introduce a new
graphical technique, the Fraction of Design Space (FDS) technique, which shows how the
performance of the prediction variance changes from the center of the design out to the
perimeter. Some second order response surface designs are studied in terms of this new
measure. Secondly, for the two- and three-factor case with one corner of the cuboidal
design space excluded, three sensible alternative designs are studied and compared.
These designs involve reducing the factor levels to make a smaller but standard factorial
design fit or modifying the levels of the variables at the excluded corner to locate it in the
feasible design region. Properties of these designs and relative tradeoffs are discussed.
The alternative designs are studied in terms of the new criteria FDS. Thirdly, we study
the performance of standard designs for generalized linear models. Some results that are
general to all GLMs are given. The logistic and Poisson regression models are studied
extensively.
The Fraction of Design Space (FDS) technique is proposed in this research as a
complement to the existing Variance Dispersion Graph (VDG) technique. Although the
VDG is a good tool for visualizing the range of the scaled predicted variance values
(SPV) for different designs, the relative emphases that should be given to different
intervals of the sphere radius can be dramatically different depending on the dimension of
the design space. The new technique, FDS, focuses on how well the design predicts for
any fraction of the design space. It gives the fraction of the design space that is equal or
less than a pre-specified value of the SPV. Two graphical summaries are then obtained.
The FDS graph (FDSG) gives the cumulative fraction of design at each value of the SPV
throughout the design region. It allows comparison of the global minimum and maximum
of SPV for different designs. This graph produces a general summary of the design
region and is not restricted to certain radii of the design region. The second graph is the
scaled FDS graph (SFDSG), where the FDS values are plotted against the SPV values
scaled by the minimum value of SPV. This graph allows direct access to the ratio of the
108
maximum to minimum SPV and is useful for looking at the stability of the SPV
distribution. In this research we compared second order designs over spherical and
cuboidal regions using the FDS technique. However, the FDS technique could also be
applied to non-regular design regions, which are found when there are restrictions on the
region of operability and also extensively in mixture designs.
For the first order with interaction model in the two-factor case, three sensible
designs are discussed, which suit the restricted design space that excludes the (1,1)
combination of the factors. To define the boundary of the restricted region, the (1,1)
corner was replaced with a quarter of a circle of which its radius specifies what fraction
of design space needs to be altered. Although, we have considered the problem of
excluding the high-high combination, all our results hold for the problem of excluding
any combination of the two factors. Overall, Design I, which shifts the (1,1) point to get it
into the feasibility region, is the preferred design using the various optimality criteria.
Design I is to be preferred if we are primarily interested in estimating the interaction term
precisely. Maintaining the orthogonal design in the reduced region is an appropriate
strategy for regions with moderate or small truncations of a corner. Any of the suggested
designs can be supplemented with center runs to estimate lack of fit and pure error.
Design I was considered in a more general form of defining the design space boundary,
which gives the practitioner more flexibility to define the design space. Depending on
prior information about restrictions of feasible points or the nature of the interaction term,
one can choose the power in the general form equation. Design I remains optimal in the
D- and G-sense for certain power values. Design I is also Q-optimum for some power
values of the general form equation.
The above work is generalized to the three-factor case. To define the boundary of
the restricted region in this case, a cube of radius r from the (1,1) corner was replaced
with a portion of sphere of radius 2r . Design I remains the best design in terms of the
alphabetic criteria. Design I is best in terms of the Ds criteria to estimate the interaction
terms.
109
For both the two- and the three-factor cases, Design I performs best in terms of
the FDS technique.
In the case of the three-factor design, we considered excluding the (1,1,1) corner.
However, there are other possibilities of restricting the region like excluding more than
just one corner. For example, one could face a situation in which the (1,1,1) and the (1,
-1,1) are excluded. Generalizing the three designs discussed in this research to the case of
restricting more than one corner is of interest.
The last part of the dissertation discusses the behavior of factorial designs, the
well-known standard designs of the linear case, when used with the generalized linear
models. These designs are not only efficient in the GLM case but for certain situations
they are D and G optimal. We formulated conditions under which these standard designs
have high G-efficiency. The use of these designs in the GLM case is illustrated with
examples from the logistic and Poisson regression models. The Factorial designs have
other desirable properties in the GLM case. For the interaction model and GLMs in
which the canonical link is used, the 2k factorial design ensures equal asymptotic
variances of the estimated coefficients and that the maximum prediction variance at the
design point is equal to the number of the parameters. Also, by the Equivalence Theorem,
the 2k factorial design in the case of any GLM model with variance stabilizing link is a
D- and G-optimal design. This becomes particularly important in the case of Poisson,
exponential or gamma responses.
There exists considerable empirical information that design augmentation along
the lines of a DETMAX like algorithm has good potential for augmentation of factorial
and fractional factorial experiments with logistic regression. This may be particularly
helpful if the design region is such that unusually small binomial variances are present at
one or more design points. This algorithm of course involves repeatedly adding points
(from vertices of the cube or within the cube) in which the scaled prediction variance is
largest. It is conjectured that this produces the maximum increase in the determinant of
the information matrix. Questions arise regarding the utility of doing this, particularly
110
since the scaled prediction variances and binomial variances are only estimates. The
desire is to determine conditions under which augmentation is desirable. In addition, we
need to be reassured that the method “works” as it does in the case of the linear model.
It is interesting that while augmentation appears to increase D-efficiency
considerably, it does not greatly impact G-efficiency. There are preliminary conjectures
as to why this is true but we need more investigation.
It has been noticed that even when it is clear that fractional factorials have high
D-efficiency, the G-efficiency is low. We need to determine the importance of G-
efficiency for fractional factorials. Our conjecture is that G-efficiency is not important
here since prediction is of less importance in the case of screening designs. Precision of
coefficients becomes more important. We feel as if the fractional factorials are efficient
in a G-sense if the region of the design is confined to the region covered by the design
points, rather than the whole experimental region.
111
VITA
Alyaa R. Zahran
The author was born on September 18, 1970 in Cairo, Egypt. She earned the
German Language Diploma (Deutsche Sprachdiplom der Kultursminister Konfernz,
Zweite Stufe) in April 1988 from the Christian German High School (DEO), Cairo,
Egypt. In 1992, she graduated from Cairo University, Faculty of Economics and Political
Sciences, Department of Statistics with a B.Sc. with distinction in Statistics. After
graduation, she worked as a Teaching Assistant at the Department of Statistics, Cairo
University. She received a M.Sc. in Statistics in 1995. She worked as a Teaching
Assistant at Old Dominion University, Applied Mathematics and Statistics Department,
Norfolk, Virginia, in December, 1997. In 1999, she received her second M.Sc. in
Statistics at Old Dominion University. Her graduate studies at Virginia Polytechnic
Institute and State University started in August, 1999. She worked as a Teaching
Assistant and a Consultant at Statistics department consulting center. Upon graduation,
she expected to join the Department of Statistics, Cairo University, as a Professor.