Chessboard Distributions and Random Vectors with Speci�ed Marginalsand Covariance MatrixSoumyadip Ghosh and Shane G. HendersonDepartment of Industrial and Operations EngineeringUniversity of MichiganAnn Arbor, MI 48109-2117, U.S.A.August 29, 2000AbstractThere is a growing need for the ability to specify and generate correlated random variables as primitiveinputs to stochastic models. Motivated by this need, several authors have explored the generation ofrandom vectors with speci�ed marginals, together with a speci�ed covariance matrix, through the useof a transformation of a multivariate normal random vector.A covariance matrix is said to be feasible for a given set of marginal distributions if a random vectorexists with these characteristics. We develop a computational approach for establishing whether a givencovariance matrix is feasible for a given set of marginals. The approach is used to rigorously establishthat there are sets of marginals with feasible covariance matrix that the normal transformation techniquereferred to above cannot match.An important feature of our analysis is that we show that for almost any covariance matrix (in acertain precise sense), our computational procedure either explicitly provides a construction of a randomvector with the required properties, or establishes that no such random vector exists.We also provide two new methodologies that may be used to deal with the situation where one cannotexactly match the desired marginals and covariance matrix using the normal transformation technique.The new methodologies possess certain advantages over other approaches that have been suggested inthe past. 1

IntroductionThere is a growing need for the ability to specify and generate random vectors consisting of correlatedobservations as primitive inputs to stochastic models. For example, in a manufacturing setting, theprocessing times of a single job at di�erent stations may be correlated due to characteristics of thejob such as size. In determining reservoir release rules, the in ows of water to di�erent reservoirs areinvariably correlated. In generating random test problems for a given algorithm, it is advantageousto ensure that some elements of the problem are correlated (Hill and Reilly 1994, 2000). Furtherapplications have recently been reported in cost analysis (Lurie and Goldberg 1998), and in decision andrisk analysis (Clemen and Reilly 1999).Perhaps the \ideal" approach is to specify the full joint distribution of the random vector. Thisapproach is typically limited to situations where the marginal distributions are all from the same para-metric family. For methods of this type see, for example, Devroye (1986) and Johnson (1987). Butthe case where the marginals are not all from the same parametric family a�ords far greater modelinggenerality, and is perhaps the case of more interest from a practical standpoint.The primary di�culty in this case is that a tremendous amount of information is typically requiredto specify (and �t) such a joint distribution. Furthermore, special methods must be devised to generaterandom vectors with the given joint distribution, and this can be a practically insurmountable problemfor a model of even moderate complexity (Law and Kelton 2000, p. 479).A practical alternative is to only specify the marginal distributions of the random variables, togetherwith the correlation matrix or covariance matrix. (Note that this information does not necessarilyuniquely specify the distribution.) The covariance measure could be Spearman's rank covariance, Pear-son's product-moment covariance, Kendall's � , or any other convenient covariance measure. In thispaper, we will focus on Pearson's product-moment covariance, and Spearman's rank covariance, becauseof their wide use and acceptance in application settings.It is important to note that we are restricting attention to the generation of �nite-dimensional randomvectors. As such, we are not attempting to generate a time series with given correlation properties. Forsuch studies, see for example Cario and Nelson (1996), Melamed et al. (1992), and Lewis et al. (1989).Hill and Reilly (1994) describe a method for generating random vectors with speci�ed marginalsand covariances through mixtures of extreme correlation distributions. The approach is very e�ectivefor random vectors of low dimension (d � 3 say), but the computational requirements quickly becomeexcessive for higher dimensional random vectors. There is another di�culty with this approach. We2

say that a covariance matrix is feasible for a given set of marginal distributions if a random vectorexists with the prescribed marginals and covariance matrix. We show (Section 1) that there are sets ofmarginals with feasible covariance matrix that cannot be matched using the technique of Hill and Reilly(see Section 1 below).Cario and Nelson (1997) described the \NORmal To Anything" (NORTA) method for generatingrandom vectors with prescribed covariance matrix. The NORTA method basically involves a component-wise transformation of a multivariate normal random vector, and capitalizes on the fact that multivariatenormal random vectors are easily generated; see Law and Kelton 2000, p. 480. Cario and Nelson tracedthe roots of the method back to Mardia (1970) who looked at bivariate distributions, and to Li andHammond (1975) who concentrated on the case where all of the marginals have densities (with respectto Lebesgue measure). Iman and Conover (1982) implemented the same transformation procedure toinduce a given rank correlation in the output. Their method is only approximate, in that the outputwill have only very approximately the desired rank correlation. Clemen and Reilly (1999) describedhow to use the NORTA procedure to induce a desired rank correlation in the context of decision andrisk analysis. Lurie and Goldberg (1998) implemented a variant of the NORTA method for generatingsamples of a predetermined size.A natural question to ask is whether the NORTA procedure can match any feasible covariance matrixfor a given set of marginals. Both Li and Hammond (1975) and Lurie and Goldberg (1998) give exampleswhere this does not appear to be the case. However, the random vectors that are proposed in thesepapers as counterexamples are not proved to exist, and so the question has not yet been completelysettled.In this paper, we prove that there are feasible covariance matrices for a given set of marginalsthat the NORTA method cannot match. To establish this result we derive a computational procedurefor establishing whether or not a given covariance matrix is feasible for a given set of marginals, andif so, explicitly providing a joint distribution with the required properties. We call the constructeddistributions \chessboard" distributions because of their structure; see Section 2.It should be noted though, that in the case of 2-dimensional random vectors, the class of feasiblecovariance matrices has been completely characterized; see Whitt (1976). However, for dimensions 3and greater, little is known.Remark 1 In the case where all of the marginal distributions have densities with respect to Lebesguemeasure, the chessboard distribution we construct has a joint density with respect to d-dimensionalLebesgue measure. In this case, we can, and do, refer to a chessboard density.3

If NORTA cannot precisely match a feasible covariance matrix, it is still possible to use NORTAto obtain the desired marginals exactly, and the desired covariance matrix approximately. Lurie andGoldberg (1998) gave an alternative approach to this problem, but we believe that our solution hasproperties that make it more desirable; see Section 5.We also provide a new method for generating random vectors with speci�ed marginals and covariancematrix based on chessboard distributions. The new method requires a nontrivial amount of computa-tional e�ort during initialization, but will typically be quite competitive with the NORTA method whileactually generating the required random vectors. It has the potential to generate virtually any (in acertain precise sense; see Section 6) feasible rank covariance for a given set of marginals. The abilityto more closely match certain covariance matrices than the NORTA method should be seen as a clearadvantage of this approach. However, it should be noted that there are covariance matrices that canbe matched by NORTA, but cannot be matched exactly by a chessboard distribution, even thoughchessboard distributions can come arbitrarily close to the desired covariance matrix.The philosophy of specifying marginals and correlations to model dependent random variates isclearly an approximate one, since the joint distribution is not completely speci�ed. Therefore, oneshould be willing to live with reasonable discrepancies in the covariance matrix from that desired. Whatis \reasonable" depends on the application at hand, and this is the reason that we describe the newmethod for random vector generation alluded to above.If one is not willing to live with reasonable (again, this is a relative term) discrepancies from thedesired covariance matrix, then perhaps a more careful approach to specifying the dependence structureis warranted. So while we present an alternative method that can be used in cases where NORTA doesnot exactly match the required covariances, we still believe that NORTA should typically remain themethod of choice for this kind of variate generation.We view the primary contributions of this paper as follows.1. We provide a computational procedure for determining whether a given covariance matrix is feasibleor not for a copula, i.e., for a random vector with uniform(0; 1] marginals. (In this case the rankcovariance and product-moment covariance are identical.) If the covariance matrix is feasible, thenan explicit construction of a joint density with these properties, that we call a chessboard density,is provided. The method works for almost all covariance matrices; see Section 2. To the best of ourknowledge, this is the �rst example of such a characterization of feasible covariance matrices. Thiscase is important because it is central to the analysis and use of rank covariance for continuousmarginals; see Section 6 for example. 4

2. We provide a computational procedure for determining whether a given Pearson product-momentcovariance matrix is feasible or not for a given set of more general marginal distributions. If thecovariance matrix is feasible for the given set of marginals, we provide an explicit construction of achessboard distribution with the desired properties. Again, we believe that this is the �rst exampleof such a characterization.3. We rigorously establish that there are feasible covariance matrices that cannot be matched usingthe NORTA method.4. We provide a simple modi�cation to the initialization phase of the NORTA method that enables oneto use the NORTA method to closely approximate the desired covariance matrix. The modi�cationinvolves the solution of a semide�nite program, and works in both the rank correlation and Pearsonproduct-moment correlation cases without any specialization. Based on a small computationalstudy, it appears that when one cannot exactly match a desired covariance, the discrepancy betweenthe desired and realized covariance matrices is quite small.5. We develop a new method for generating random vectors with speci�ed covariance structure usingchessboard distributions that can exactly match covariances that the NORTA method cannot. Thenew method could involve a nontrivial amount of initialization e�ort, but certainly be competitive(computationally speaking) with the NORTA method while actually generating random vectors.The remainder of this paper is organized as follows. In Section 1 we review the NORTA method,and describe how it may be used to match a given Pearson product-moment covariance matrix, or agiven rank covariance matrix. In Section 2 we develop the theory of chessboard distributions in the casewhere all of the marginal distributions are uniform(0; 1]. The chessboard distribution concept is thenextended to more general marginals in Section 3. Next, in Section 4 we present a small computationalstudy that sheds light on when we might expect the NORTA method to be unable to match a feasiblecovariance matrix. We also provide several examples where this occurs. Then, in Section 5, we presentour modi�cation of the NORTA method that involves semide�nite programming. Finally, in Section 6 wepresent an alternative method for generating a random vector with speci�ed marginals and covariancematrix, based on chessboard distributions.We are developing a software package that implements the overall NORTA process, incorporating thesemide�nite programming modi�cation suggested in Section 5. The software is similar in spirit to theARTAFACTS/ARTAGEN package developed by Cario and Nelson (1997b). Indeed, we are adaptingtheir software for the task. When the software is ready, it will be freely available from the second5

author's website Henderson (2000).1 The NORTA methodCario and Nelson (1997) described the \NORmal To Anything" (NORTA) method for generating i.i.d.replicates of random vectors with speci�ed marginals and covariance structure. In this method, onestarts by generating a random vector Z with a multivariate normal distribution and transforms Z toobtain a random vector X = (X1; : : : ; Xd) with the desired marginals and covariance structure. Let Fibe the distribution function of Xi, for i = 1; : : : ; d.The NORTA method generates i.i.d. replicates of X by the following procedure.1. Generate an IRd valued standard normal random vector Z = (Z1; : : : ; Zd) with mean vector 0 andcovariance matrix �Z = (�Z(i; j) : 1 � i; j � d), where �Z(i; i) = 1 for i = 1; : : : ; d.2. Compute the vector X = (X1; : : : ; Xd) viaXi = F�1i (�(Zi)); (1)for i = 1; : : : ; d, where � is the distribution function of a standard normal random variable, andF�1i (u) = inffx : Fi(x) � ug: (2)A vector X generated by this procedure will have the prescribed marginal distributions. To see this,note that each Zi has a standard normal distribution, so that �(Zi) is uniformly distributed on (0; 1),and so F�1i (�(Zi)) will have the required marginal distribution.The covariance matrix �Z should be chosen so that it induces the required correlation structure onX . There are many measures of correlation between two random variables, but perhaps the two mostpopular are Pearson's product-moment correlation, and Spearman's rank correlation.1.1 Pearson's Product-Moment CorrelationIn this section we specialize to the case where one wishes to generate X with prespeci�ed Pearsonproduct-moment covariance matrix �X , where�X(i; j) = EXiXj �EXiEXjfor 1 � i; j � d. This is the case that Cario and Nelson (1997) examined. Note that this is equivalentto prespecifying the correlation matrix, since the marginal distributions are also prespeci�ed. To ensure6

that the required correlations are de�ned, we make the assumption that EX2i < 1 for i = 1; : : : ; d. Itturns out that choosing �Z to ensure the correct covariance matrix �X is a nontrivial problem.Each term cov(Xi; Xj) is a function of only cov(Zi; Zj) (Cario and Nelson 1997), and so the problemreduces to d(d � 1)=2 separate problems of selecting cov(Zi; Zj) to match cov(Xi; Xj) = �X(i; j).Unfortunately, there is no general analytical expression relating these two quantities, and hence wecannot determine the exact �Z that is to be used analytically.De�ne the function cij(cov(Zi; Zj)) = cov(Xi; Xj), where Xi and Xj are de�ned via (1). Carioand Nelson (1997) established that under very mild conditions, the function cij is a continuous non-decreasing function of �Z(i; j). This result allows us to perform an e�cient numerical search for values�Z(i; j) that yield cij(�Z(i; j)) = �X(i; j) for i < j: (3)We take �Z(i; i) = 1 for i = 1; : : : ; d, and for i > j, set �Z(i; j) = �Z(j; i) to ensure that �Z issymmetric. Alternatives to the numerical search suggested by Cario and Nelson (1997) include the usea stochastic root-�nding algorithm (Chen 2000), or polynomial expansions (van der Geest 1998). Unlessotherwise stated, we henceforth assume that a solution to (3) exists.One might hope that if the matrix �Z satis�es (3), then �Z could be used in the NORTA method togenerate i.i.d. replicates of X . Unfortunately, the results of this paper prove that this is not always thecase. In fact, there exists a feasible covariance matrix for a 3-dimensional random vector with uniformmarginals on (0; 1] that cannot be generated with the NORTA procedure. The problem arises when thematrix �Z as determined from (3) is not positive semide�nite, in which case it is not a valid covariancematrix.Li and Hammond (1975) suggested the following example to illustrate this important fact. Let X1; X2and X3 be 3 uniformly distributed random variables on (0; 1] with correlation matrixRX = 0BBB@ 1 �0:4 0:2�0:4 1 0:80:2 0:8 1 1CCCA :In the special case when X has uniform marginals, the equations (3) can be solved analytically. Inparticular, Kruskal (1958) showed that the (unique) solution to (3) is given by�Z(i; j) = 2 sin[2��X(i; j)]: (4)For the Li and Hammond example, the (unique) matrix �Z that can be determined using (4) (with7

�X = RX=12) is not positive semide�nite. It is important to observe though, that this is a counterex-ample only if the postulated random vector exists, and Li and Hammond did not show this.Remark 2 It is straightforward to show that the Li and Hammond example cannot be generated usingthe extremal distributions method of Hill and Reilly (1994). One simply attempts to solve the linearprogram suggested by Hill and Reilly (1994), which turns out to be infeasible. Therefore, if the Li andHammond example exists, it shows that there are feasible covariance matrices that cannot be matchedusing the extremal distributions technique.Lurie and Goldberg (1998) gave an example with nonuniform marginals and positive de�nite covari-ance matrix for which the solution to (3) is also not positive semide�nite. They did not establish thatthe postulated random vector exists.When all of the marginals have continuous distribution functions, a natural alternative to the nu-merical search procedure mentioned earlier is to \work in the Gaussian space". In other words given aset of data with known (or �tted) marginals with continuous distribution functions, we �rst transformthe data set into normal random variates using the inverse of the transformation (1). We can thencompute an empirical covariance matrix �Z and use this covariance matrix in the NORTA procedure.(If the distribution function F of a random variable X is not continuous, then F (X) does not have auniform distribution on (0; 1), and so one will not obtain a normally distributed random variable using��1(F (X)). Therefore, the continuity of the marginal distribution functions is needed.)This approach is certainly simpler than a numerical search procedure, but it has two importantdrawbacks. First, it requires a set of input data, which may not be available in general. But second,and perhaps more importantly, this procedure does not necessarily ensure that the resulting X variateswill have the required covariance structure. To see why, observe that the transformed normal randomvariables mentioned above are unlikely to have a joint normal distribution. Therefore, the correlations ofthe jointly normal random variables used in the NORTA method using �Z will be unlikely to transformthrough the NORTA procedure to yield the desired covariance matrix for X , as one might otherwiseexpect. This is a subtle point, but one that is worth bearing in mind.1.2 Spearman's Rank CorrelationIn this section we specialize to the case where one wishes to generate X with prespeci�ed Spearman'srank covariance matrix �X , where�X(i; j) = rcov(Xi; Xj) = EFi(Xi)Fj(Xj)�EFi(Xi)EFj(Xj)8

for 1 � i; j � d. This is the case treated by Clemen and Reilly (1999). In contrast to product-momentcovariance, the rank covariance is always de�ned. Note that when the Xi's all have nondegeneratedistributions, specifying the rank covariance matrix is equivalent to prespecifying the rank correlationmatrix with (i; j)th elementrcor(Xi; Xj) = cor(Fi(Xi); Fj(Xj)) = rcov(Xi; Xj)[varFi(Xi)varFj(Xj)]1=2 :It turns out that choosing �Z to ensure the correct rank covariance matrix �X is easy in the specialcase that all of the distribution functions Fi are continuous (i = 1; : : : ; d).To see why this is the case, observe that if Xi and Xj have continuous distribution functions Fiand Fj , then Fi(F�1i (u)) = u for all u 2 (0; 1), and similarly for Fj . The rank covariance rcov(Xi; Xj)between Xi and Xj as generated by a NORTA procedure is therefore given byrcov(Xi; Xj) = cov(Fi(Xi); Fj(Xj))= cov(Fi(F�1i (�(Zi))); Fj(F�1j (�(Zj)))) (5)= cov(�(Zi);�(Zj)): (6)But (6) is precisely the quantity �X(i; j) in (4). Therefore, given a desired rank covariance matrix �X ,we simply compute �Z = �Z via (4) and use this within the NORTA procedure.Observe that if the random vector in the Li and Hammond example (given above) exists, then it isagain an example showing that there are feasible rank covariance matrices for a given set of marginalsthat cannot be matched using a NORTA procedure.In the case where Fi (say) is not continuous, equation (6) no longer holds, because Fi(F�1i (x)) 6= xfor some x 2 (0; 1). Therefore, the analytical expression (4) cannot be used. However, one could use anumerical search procedure as in Cario and Nelson (1997) to identify the covariance �Z(i; j) that yieldsthe required rank covariance rcov(Xi; Xj). This follows since the rank covariance (5) between Xi andXj is a nondecreasing continuous function of the covariance between Zi and Zj . The nondecreasingproperty follows immediately from the proof of Theorem 1 in Cario and Nelson (1997), and the fact thatthe function Fi(F�1i (�(�))) is nondecreasing. Continuity follows from Theorem 2 of Cario and Nelson.1.3 To summarizeWe have just seen that the NORTA procedure can be used to generate a random vector with the desiredproduct-moment, or rank, correlation. The same is true when correlation is measured by Kendall's � ,9

as Clemen and Reilly (1999) note. The di�erence lies in how the covariance matrix �Z of the initialnormal random vector is determined.We have already noted that if the Li and Hammond example exists, then we may conclude thatthere are sets of marginals with feasible (product-moment or rank) correlation matrices that cannotbe matched using the NORTA procedure. In the following section, we introduce a linear programmingbased method that may be used to investigate the existence or not of random vectors with uniformmarginals and speci�ed covariance matrix.2 CopulasA copula can be characterized as the joint distribution function of a random vector with uniformmarginals on (0; 1] (Nelsen 1999). In this section, we develop a method for constructing the jointdensity of a 3-dimensional copula with prescribed covariance matrix. (Note that Pearson's product-moment covariance and Spearman's rank covariance coincide in the copula case.) The approach is easilycarried over to the general case of a d-dimensional copula with prescribed covariance matrix.We will let X = (X1; X2; X3) denote a random variable with such a density and let � = (�ij : 1 �i; j � 3) be the desired covariance matrix. We �rst construct the probability mass function (pmf) of arandom vector Y = (Y1; Y2; Y3) whose marginals are discretized versions of the marginals of X . The pmfwill be constructed to try to ensure that Y has covariance matrix � (except for the diagonal entries).From this pmf, we then construct the density of X in such a way that the o�-diagonal entries in thecovariance matrix are maintained. (The diagonal elements are determined by the marginals of X .) Thisthen yields the required construction.Our notation will appear partly redundant at times, but this is done to ensure consistency withSection 3 where we will extend these ideas to more general marginal distributions.Let n � 1 be an integral parameter that determines the level of discretization that will be performed.Let yi;k = kn ; k = 0; : : : ; n be the set of points that divide the range (0,1] of the ith variable into n equallength sub-intervals. For k = 1; : : : ; n and i = 1; 2 and 3, let�Yi;k = E[Xi jXi 2 (yi;k�1; yi;k] ] = 2k � 12n ; (7)be the conditional mean of Xi given that it lies in the kth sub-interval.The support of the random vector Y is the mesh of pointsf( �Y1;i; �Y2;j ; �Y3;k) : 1 � i; j; k � ng:10

Let q(i; j; k) = P (Y1 = �Y1;i; Y2 = �Y2;j ; Y3 = �Y3;k)be the probability that Y equals the (i; j; k)th point in the support of Y , so that q represents the pmfof the random vector Y . (Note that it is not the pmf itself, since the function q is de�ned on integers,while the domain of the pmf is contained in the unit cube.)Consistent with the notion that Y is a discretized version of X , we also have thatq(i; j; k) = P (X 2 C(i; j; k));where the cell C(i; j; k) represents the cube of points surrounding the (i; j; k)th point in the support ofY . More precisely,C(i; j; k) = f(x1; x2; x3) : y1;i�1 < x1 � y1;i; y2;j�1 < x2 � y2;j ; y3;k�1 < x3 � y3;kg:We then see thatnXj;k=1 q(i; j; k) = P (Y1 = �Y1;i) = P (X1 2 (y1;i�1; y1;i]) = 1n; 8i = 1; : : : ; n; (8)nXi;k=1 q(i; j; k) = P (Y2 = �Y2;j) = P (X2 2 (y2;j�1; y2;j ]) = 1n; 8j = 1; : : : ; n; (9)nXi;j=1 q(i; j; k) = P (Y3 = �Y3;k) = P (X3 2 (y3;k�1; y3;k]) = 1n; 8k = 1; : : : ; n; (10)q(i; j; k) � 0 8i; j; k = 1; : : : ; n: (11)With these constraints satis�ed, we then have that EYi = 1=2 = EXi for i = 1; : : : ; 3. To see this,note that for Y1, we have thatEY1 = nXi;j;k=1 �Y1;iq(i; j; k)= nXi;j;k=1E[X1 jX1 2 (y1;i�1; y1;i] ]P (X 2 C(i; j; k))= nXi=1 E[X1 jX1 2 (y1;i�1; y1;i] ]P (X1 2 (y1;i�1; y1;i])= EX1:Recall that our intermediate goal is to match the covariance matrix of Y to that of X (with theexception of the diagonal elements), and we propose to do this using linear programming methods. IfCij = cov(Yi; Yj), then we want to minimizejC12 ��12j+ jC13 ��13j+ jC23 ��23j: (12)11

Now C12 = nXi;j;k=1 �Y1;i �Y2;jq(i; j; k)�EY1EY2;which is a linear function of the q(i; j; k)'s, with similar linear expressions for C13 and C23. Furthermore,the matrix � is simply a parameter, and so, for example, C12��12 is a linear function of the q(i; j; k)'s.Using a standard trick in linear programming, we can also represent jC12��12j in a linear fashion, andsimilarly for the other terms in (12) as follows.De�ne Z+ij to be the positive part of the di�erence Cij ��ij , i.e.,Z+ij = (Cij ��ij)+ = maxfCij ��ij ; 0g;and Z�ij to be (Cij ��ij)� = �minfCij ��ij ; 0g:We can now attempt to match the covariances of Y to those of X using the LPmin P2i=1P3j=i+1(Z+ij + Z�ij )subject to Cij ��ij = Z+ij � Z�ij ; i = 1 to 2 and j = i+ 1 to 3Z+ij � 0; Z�ij � 0; together with constraints (8), (9), (10) and (11).This LP is always feasible since a product copula where the Yi's are independent can be easilyconstructed by setting all q(i; j; k) = n�3. Also, the objective function of the LP is bounded below by0. Hence an optimal solution exists. We also know from standard results in linear programming that inany optimal solution to the LP, Z+ij � Z�ij = 0, i.e., at most one of Z+ij and Z�ij is > 0.If the optimal objective value for the above LP turns out to be 0, then we have constructed adiscretized joint probability mass function for Y that has the desired covariance structure. So, for i 6= j,we have cov(Yi; Yj) = �ij : (13)Now, the discretized random vector Y does not possess continuous uniform (0,1] marginals. However,we can construct a random vector X with continuous uniform marginals from Y in such a way thatcov(Yi; Yj) = cov(Xi; Xj) for i 6= j, i.e, the covariances are preserved. Assuming the optimal objectivevalue of the LP is 0, this then yields an explicit construction of a random vector with the desiredmarginals and convariance matrix. 12

By conditioning on the cell containingX , we see that the requirement that cov(Y1; Y2) = cov(X1; X2)is equivalent tonXi;j;k=1 q(i; j; k) � �Y1;i �Y2;j �EY1EY2 = nXi;j;k=1E[X1X2jX 2 C(i; j; k)] � P (X 2 C(i; j; k))�EX1EX2 (14)But, EY1 = EX1 and EY2 = EX2, and so (14) can be re-expressed asnXi;j;k=1 q(i; j; k) � (E[X1jX 2 C(i; j; k)] � E[X2jX 2 C(i; j; k)]�E[X1X2jX 2 C(i; j; k)]) = 0: (15)Equation (15) could be satis�ed in many ways, but perhaps the simplest is to note that (15) will holdif, conditional on X lying in C(i; j; k), X1; X2 and X3 are independent. In that case, each term in thesum (15) is 0. One can ensure that this conditional independence holds, while simultaneously ensuringthat X has the correct marginal distributions, by setting the density of X within the cell C(i; j; k) tothat of independent, uniformly distributed random variables, scaled so that the total mass in the cell isq(i; j; k). To be precise, if f is the density of X , then for any x 2 C(i; j; k), we setf(x) = n3q(i; j; k): (16)In a sense, we are \smearing" the mass q(i; j; k) uniformly over the cell C(i; j; k).Theorem 1 below proves that if the optimal objective value of the LP is 0, then the density f soconstructed has the desired marginals and covariance matrix.Theorem 1 If the optimal objective value of the LP is 0, then the density f de�ned via (16) has uniform(0,1] marginals and covariance matrix �.Proof: Clearly, f is nonnegative and integrates to 1. Next, we need to show that the marginals of fare uniform. Let the marginal density function of X1 be denoted by g1(�), which we have not yet provedis equal to f1 as desired. For any x 2 (y1;i�1; y1;i), we have thatg1(x)dx = nXj;k=1P (X1 2 [x; x+ dx)jX 2 C(i; j; k))P (X 2 C(i; j; k))= nXj;k=1P (X1 2 [x; x+ dx)jX1 2 (y1;i�1; y1;i])q(i; j; k)= nXj;k=1 dxR y1;iy1;i�1 1 dy q(i; j; k)= n nXj;k=1 q(i; j; k)dx = 1dx13

The �rst equation follows by conditioning on the cell in which the random vector lies, and the secondby the conditional independence of X1; X2 and X3 given that X lies in C(i; j; k). The third follows fromthe assumption of uniform \smearing" of q(i; j; k) on the cell C(i; j; k). A similar result holds for themarginals of X2 and X3, and so the joint density f has the right marginals.Next we need to show that the obtained covariances are indeed the desired ones. Take the case ofcov(X1; X2). Starting with its de�niton, we havecov(X1; X2) = EX1X2 �EY1EY2= EY1Y2 �EY1EY2= �12:The �rst equality follows from the fact that EY1 = EX1 and EY2 = EX2, and the second is just arestatement of (15). The �nal equation follows from the fact that the optimal objective value is 0, asnoted in (13). The same follows for cov(X2; X3) and cov(X1; X3). Hence, f has the covariances asdesired and this completes the proof. 2Remark 3 The name \chessboard" distribution is motivated by the form of (16) in a 2 dimensionalproblem. In this case, the unit square is broken down in n2 squares, and the density f is constant oneach square, with value n2q(i; j).Remark 4 There is no need for the cells used in the above construction to be of equal size. Indeed,Theorem 1 remains true for more general discretizations; see Theorem 10 in Section 3.In practice, this LP can become quite expensive to solve even for moderate values of n like 20, sincewe will then have 8000 q(i; j; k)s to contend with. However, the feasible region of the LP can be reducedby the inclusion of some more constraints on the Z+ijs and Z�ij s. This not only improves the performanceof the LP, but as we shall see, also provides us with a new feasibility criterion to test for the existenceof a random vector with the given covariance matrix.The constraints are developed by assuming that a random vector X with uniform marginals andcovariance matrix � exists, discretizing X to obtain a new random vector ~X say, and then bounding thechange in the covariances resulting from the discretization.So suppose that we discretize X to obtain ~X . Let~q(i; j; k) = P ( ~X = (�Y1;i; �Y2;j ; �Y3;k));14

and observe that ~q provides a feasible solution to the above LP. We now wish to bound the change inthe covariance resulting from this discretization. Observe thatcov( ~X1; ~X2)��12 = E ~X1 ~X2 �EX1X2= nXi;j;k=1( �Y1;i �Y2;j �E[X1X2jX 2 C(i; j; k)])~q(i; j; k): (17)But y1;i�1y2;j�1 � E[X1X2jX 2 C(i; j; k)] � y1;iy2;j : (18)Combining (17) with (18) we see thatcov( ~X1; ~X2)��12 � nXi;j;k=1 ~q(i; j; k)( �Y1;i �Y2;j � y1;i�1y2;j�1) and (19)cov( ~X1; ~X2)��12 � nXi;j;k=1 ~q(i; j; k)( �Y1;i �Y2;j � y1;iy2;j): (20)To summarize then, we assumed that X exists, and under this assumption have shown that there isa feasible solution ~q to the LP satisfying the bounds (19) and (20). Therefore, we can add these boundsto our LP. In particular, (19) gives an upper bound on Z+12, and (20) gives an upper bound on Z�12.Similar bounds may be obtained for the other covariances. After substituting in the explicit expressionsfor yi;k and �Yi;k, we see that these bounds simplify toZ+ij � 12n � 14n2 and Z�ij � 12n + 14n2 1 � i < j � 3: (21)Once the above LP is augmented with the bounds (21), it is no longer guaranteed to be feasible. Infact, Theorem 2 below establishes that if the augmented LP is infeasible for any value of n � 1, thenthe covariance matrix � is not feasible for uniform marginals. The proof is basically a summary of theabove discussion, and is given to help clarify these ideas.Theorem 2 If the augmented LP is infeasible for some n � 1, then there cannot exist a random vectorX with uniform marginals and the desired covariance matrix �.Proof: Suppose there exists a random vectorX with uniform marginals and covariance matrix �. Then,as above, we can construct a solution ~q by discretizing X that satis�es all of the constraints, includingthe bounds (21). Thus the augmented LP is feasible, which is a contradiction. 2In fact, one can prove a converse to Theorem 2.15

Theorem 3 If the covariance matrix � is not feasible for uniform (0,1] marginals, then there exists ann � 1 such that the augmented LP is infeasible.Proof: On the contrary, suppose that the augmented LP is feasible for all n � 1. Let qn denote anoptimal solution to the nth augmented LP, and let �n denote the probability measure corresponding tothe density resulting from the smearing operation (16) applied to qn. Then each �n is the distributionof a random vector with support contained in (0; 1]3 with uniform(0; 1] marginals. Hence, the sequence(�n : n � 1) is tight, and by Theorem 29.3 on p. 392 of Billingsley (1986), it possesses a weaklyconvergent subsequence (�n(k) : k � 1), converging to � say.Now, � has uniform (0; 1] marginals. This follows from Theorem 29.2, p. 391 of Billingsley (1986)since each �n(k) has uniform(0; 1] marginals, �n(k) ) � as k !1, and the projection map �j : IR3 ! IRthat returns the jth coordinate of a vector in IR3 is continuous.Now, if Cn is the covariance matrix of the distribution qn, then2Xi=1 nXj=i+1 jCnij ��ij j � 32n + 34n2 ! 0as n ! 1. This follows from the bounds (21), and the fact that in any optimal solution, it is not thecase that both Z+ij and Z�ij are strictly positive.Finally, if Xn(k) has distribution �n(k), then (Xn(k)i Xn(k)j : k � 1) is a bounded sequence of randomvariables, and therefore uniformly integrable. It immediately follows that the covariance matrix � of �is given by � = limk!1Cn(k) = �ij :Thus, � has the required marginals and covariance matrix, which is a contradiction, and the result isproved. 2Combining Theorems 2 and Theorem 3, we see that a covariance matrix is infeasible for uniformmarginals if, and only if, the augmented LP is infeasible for some n � 1.Given this very sharp characterization of infeasible covariance matrices, it is natural to ask whethera similar result holds for feasible covariance matrices. We would then have the result that a covariancematrix is feasible for a given set of marginals if and only if there is some �nite n such that the optimalobjective value of the augmented LP is zero. Unfortunately, this conjecture is false, as shown by thefollowing counterexample in 2 dimensions.Suppose that X1 = X2 and hence cov(X1; X2) = var(X1) = 1=12. For given n, the covariancebetween Y1 and Y2 is maximized by concentrating all mass on the cells (i; i), and so q(i; i) = n�1 for16

1 � i � n. In that case, we have thatcov(Y1; Y2) = nXi=1 �2i� 12n �2 1n � 12 12 = 112 � 112n2Therefore, cov(Y1; Y2) < 1=12 for all �nite n, and so the conjecture is false.Notice that the covariance matrix in this example is singular. This example is a special case of thefollowing result.Theorem 4 All chessboard densities have nonsingular covariance matrices.Proof: On the contrary, suppose that f is a chessboard density with singular covariance matrix �, andlet X have density f . Since � is singular, there exists a nonzero vector � such that �� = 0. Hence,var(�0X) = �0�� = 0, and so �0X = �0EX a.s. Since � is nonzero, we may, by relabelling variables ifnecessary, write X1 as a linear function of X2; X3, say X1 = �0+�2X2+�3X3. This equality must alsohold conditional on X 2 C(i; j; k). But the components of X are conditionally independent given thatX 2 C(i; j; k) because f is a chessboard density, which is the required contradiction. 2The importance of Theorem 4 is that if � is feasible for the given marginals and singular, then nomatter how large n may be, the optimal objective value of the LP will always be > 0, i.e., we cannotexactly match the covariance matrix �. However, we can come arbitrarily close, as the following resultshows.Theorem 5 Suppose that the covariance matrix � is feasible for uniform (0; 1] marginals. Then forall n � 1, the augmented LP is feasible, and if z(n) is the optimal objective value of the nth LP, thenz(n)! 0 as n!1.Proof: Since � is feasible for uniform marginals, the augmented LP is feasible for all n � 1. (This isjust the contrapositive of Theorem 2.) Let qn denote an optimal solution to the nth LP, and let fn bethe corresponding smeared density. If Cn is the covariance matrix corresponding to fn, then the bounds(21) imply that z(n) = 2Xi=1 nXj=i+1 jCnij ��ij j � 32n + 34n2 ! 0as n!1. 2Therefore, chessboard densities can come arbitrarily close to any required � that is feasible foruniform marginals. In fact, one can prove that chessboard densities can exactly match a (very) slightlyrestricted class of feasible covariance matrices. To state this result we need some notation.17

We can and do easily state and prove Proposition 6 for a general dimension d (i.e., not just d = 3)without any notational di�culty. Any covariance matrix � of a d dimensional random vector withuniform[0; 1) marginals can be characterized by d(d � 1)=2 covariances, since the diagonal entries aredetermined by the marginals, and the matrix is symmetric. Hence we can, with an abuse of notation,think of � as a d(d� 1)=2 dimensional vector in some contexts, and as a d� d matrix in others.Let � [�1=12; 1=12]d(d�1)=2 denote the space of feasible covariance matrices, so that � 2 impliesthat there exists a random vector with uniform(0; 1] marginals, and covariance matrix �. We will showbelow that is nonempty and convex (this is well-known), but also closed and full-dimensional (thisappears to be new). In particular then, any covariance matrix on the boundary of is feasible. We willalso show that � is contained in the interior of if, and only if, there is some �nite n for which theaugmented LP has objective value 0. The collective implications of this and our previous results will bediscussed after the statement and proof of these results.Proposition 6 The set is nonempty, convex, closed and full-dimensional.Proof: If the components of X are independent, then the covariance matrix � is diagonal, and so contains the zero vector, and is therefore nonempty.It is well-known that is convex. For if �1;�2 2 , then there exist random vectors X;Y withuniform(0; 1] marginals, and covariance matrices �1 and �2 respectively. For � 2 (0; 1), let Z be givenby X with probability �, and Y with probability 1��. Then Z has covariance matrix ��1+(1��)�2.The proof that is closed is virtually identical to that of Theorem 3 and is omitted.We use the NORTA method to prove that is full-dimensional. We will show that each of thevectors �ek=12 are contained in , where ek is the vector whose components are all 0 except for a 1 inthe kth position, for k = 1; : : : ; d(d� 1)=2. The convexity of then ensures that is full-dimensional.Let Z be a multivariate normal random vector with mean 0 and covariance matrix consisting of 1'son the diagonal, and also in the (i; j)th and (j; i)th position (i 6= j), with the remaining componentsbeing 0. That is, Z consists of 2 perfectly correlated standard normal random variables Zi and Zj , andd � 2 independent standard normal random variables. Now let U be the random vector with uniform(0; 1) marginals obtained by setting Um = �(Zm) for m = 1; : : : ; d. Then Ui and Uj are perfectlycorrelated, and independent of all of the remaining components of U . Thus, U has covariance matrixwhose components are all 0 except for the diagonal elements, and the (i; j), and (j; i)th elements, whichare equal to 1=12. Thus, ek=12 lies in , where k corresponds to the position (i; j). A similar argumentwith perfectly negatively correlated Zi and Zj shows that �ek=12 2 . Since i 6= j were arbitrary, the18

proof is complete. 2In Theorem 4 we showed that all chessboard densities have nonsingular covariance matrices. This isalmost su�cient to establish that all boundary points of do not have chessboard densities. However,it is certainly conceivable that the boundary of contains nonsingular, as well as singular, covariancematrices. So we strengthen Theorem 4 with the following result.Theorem 7 If fn is a chessboard density with covariance matrix �, then fn is contained in the interiorof .Proof: Let X have density fn. We will show that we can both increase, and decrease, the covariancebetween X1 and X2. Symmetry then allows us to conclude that the same result holds for Xi and Xjwith i 6= j. The convexity of then completes the proof.Let qn be the discretization of fn into its n3 cells, and let C(i; j; k) be a cell with qn(i; j; k) > 0.Now, divide the cell C(i; j; k) into 4 (equal size) subcells,Cab(i; j; k) = f(x; y; z) 2 C(i; j; k) : 2i� (3� a)2n < x � 2i� (2� a)2n ;2j � (3� b)2n < y � 2j � (2� b)2n g;for 1 � a; b � 2.Now, generate a new density gn by the usual smearing (16) in all cells except C(i; j; k). Withinthe cell C(i; j; k), assign a mass of qn(i; j; k)=2 to each of the cells C11(i; j; k), and C22(i; j; k), andthen uniformly smear within these cells. In other words, for (x; y; z) contained in these two cells, setgn(x; y; z) = 2n3qn(i; j; k) and set gn to be 0 in the cells Cab(i; j; k) for a 6= b. Then it is straightforwardto show that gn has uniform marginals, that the (1; 2)th covariance is strictly increased, and that theother covariances remain unchanged.A similar argument placing the mass in the cells Cab(i; j; k) with a 6= b shows that the covariancecan be strictly decreased, and so the proof is complete. 2We have thus far shown that if a covariance matrix � is not in , then the augmented LP will beinfeasible for some n � 1, and if � is on the boundary of , then the LP approach will yield distributionswith covariance matrices that arbitrarily closely approximate �, but never actually achieve it. Our �nalresult shows that if � is contained in the interior of , then there is some n � 1 for which the optimalobjective value of the augmented LP is 0, and so one can exactly match � using a chessboard density.Before proving this result, we need the following lemma. This lemma basically states that given a �xed19

vector x, we can choose certain other vectors arbitrarily close to x, so that x is a convex combination ofthese \close" vectors, and if we perturb the close vectors slightly, then x is still a convex combination ofthe perturbed vectors.For x 2 IRm and � > 0, let B(x; �) denote the (open) set of vectorsfy 2 IRm : �(x; y) < �g;where � is the L1 distance �(x; y) = mXi=1 jxi � yij:Lemma 8 Let x 2 IRm, and let � > 0 be arbitrary. There exist m + 1 points x1; : : : ; xm+1 2 B(x; �),and a � > 0 such that if �(xi; x0i) < � 8i = 1; : : : ;m+ 1;then x may be written as a convex combination of x01; : : : ; x0m+1.Proof: Basically, one chooses the xi's to be the vertices of a simplex centered at x. To be precise, letr > 0 be a parameter, and setx1 = ( �a1 �a2 � � � �am�1 �am )0 + xx2 = ( a1 �a2 � � � �am�1 �am )0 + xx3 = ( 0 2a2 � � � �am�1 �am )0 + x... ... ... ... ... ... ... ... ...xm = ( 0 0 � � � (m� 1)am�1 �am )0 + xxm+1 = ( 0 0 � � � 0 mam )0 + x;where ai = rr mm+ 1s 1i(i+ 1) :Then, (Dantzig 1991), the xi's de�ne the vertices of an equilateral simplex whose center is x, and whosevertices are a (Euclidean) distance rm=(m+ 1) from x. Choose r so that xi 2 B(x; �) for all i.Observe that the average of the xi's is x. In fact, it is easy to show that the (m+1)� (m+1) matrixB consisting of the xi's in columns, supplemented with a row of 1's is nonsingular, and soy = B�1x = (m+ 1)�1(1; 1; : : : ; 1)0:Now, observe that B�1 is a continuous function of B, at least in a neighbourhood of B, and so y = B�1xis locally a continuous function of x1; : : : ; xm+1. Hence, there is a � > 0 such that if �(xi; x0i) < � for20

all i = 1; : : : ;m+ 1, and D consists of the vectors x0i in columns supplemented with a row of 1's, theny = D�1x consists of all positive components, and the elements of y sum to 1. 2We are now ready to state the �nal result of this section. As in Proposition 6, there is no loss ofclarity if we state this result for a general dimension d rather than just d = 3.Theorem 9 If � is contained in the interior of , then there exists an n � 1 such that the optimalobjective value of the augmented LP is 0.Proof: Let m = d(d�1)=2, and for now, consider � as an m-vector. Let � > 0 be such that B(�; �) � ,and choose �1;�2; : : : ;�m+1 2 B(�; �) and � as in Lemma 8.Since �i 2 , from Theorem 5 there exists an n(i) such that the augmented LP with targetcovariance matrix �i has optimal objective value smaller than �, for each i = 1; : : : ;m + 1. Letn = n(1)n(2) � � �n(m + 1), and let qi denote a solution to the augmented LP with target matrix �iand discretization level n for i = 1; : : : ;m+ 1. Then the optimal objective value corresponding to qi isalso less than �. (Note that if k; n � 1 are integers, then the optimal objective values z(n) and z(kn)satisfy the relationship z(kn) � z(n), since the chessboard density obtained from the solution to the nthLP can also be obtained from the (kn)th LP.)Let �0i denote the covariance matrix corresponding to the chessboard density f i for the solution qi,for i = 1; : : : ;m+ 1. Then, by Lemma 8, there exist nonnegative multipliers �1; �2; : : : ; �m+1 summingto 1 such that � = m+1Xi=1 �i�0i: (22)If we set f = m+1Xi=1 �if i;then f is also a chessboard density with discretization level n, and from (22), its covariance matrix isexactly �. 2In summary then, we have shown that if � is infeasible for uniform marginals, then the augmentedLP will be infeasible for some n � 1. This includes the case where � is singular and infeasible for uniformmarginals. Furthermore, we have shown that if � is contained in the interior of , then the augmentedLP will have optimal objective value 0 for some n � 1, and so one can construct a chessboard densityfrom the solution to the augmented LP with the required marginals and covariance matrix. So if � isnot contained in the boundary of , then we have an algorithm for determining, in �nite time, whether21

� is feasible for the given marginals or not. One simply solves the augmented LP for n = 1; 2; 3; : : : untilthe augmented LP is either infeasible, or has an optimal objective value of 0. In the latter case, we candeliver an explicit construction of the desired distribution.The case where � lies on the boundary of is more problematical. We have shown that in this case,� is feasible for uniform marginals, but that a chessboard density cannot be constructed with uniformmarginals and covariance matrix �. Therefore, for such matrices, the algorithm outlined above will notterminate in �nite time. However, a chessboard distribution can come arbitrarily close to the requiredcovariance matrix.These results shed a great deal of light on the class of feasible covariance matrices for random vectorswith uniform marginals. It is natural to ask whether a similar approach may be applied to a moregeneral class of marginal distributions, and this is the subject of the following section.3 Joint Distributions with more general marginalsThe LP method used in Section 2 to evaluate the existence of a random vector with uniform marginalsand given covariance matrix can be adapted to investigate the existence of random vectors havingarbitrary marginal distributions and given Pearson product-moment covariance matrix. We will stickto the case of a 3-dimensional random vector for notational simplicity but it should be noted that theapproach is easily extended to the general d-dimensional case.Let X = (X1; X2; X3) represent the random vector that is to be constructed, and let � be thedesired covariance matrix. Let Fi(�) denote the distribution function of Xi, for i = 1; 2; 3. For ease ofexposition we assume that each of the Fi's has a density fi with respect to Lebesgue measure, althoughthe approach applies more generally, and in particular, can be applied when some or all of the Xi's havediscrete distributions.In the spirit of the method developed in Section 2, we will �rst construct the probability mass functionof a discretized random vector Y = (Y1; Y2; Y3) with a covariance structure as close to the desired oneas possible, and then derive a joint distribution function for X .Let n1; n2 and n3 represent the levels of discretization of the random variables X1; X2 and X3respectively, and hence the number of points that form the support of Y1; Y2 and Y3. Let the range ofthe variable Xi be divided into ni subintervals (which may, or may not, be equal in length) by the setof points fyi;0; yi;1; : : : ; yi;nig, with�1 � yi;0 < yi;1 < � � � < yi;ni � 1:22

Note that we explicitly allow yi;0 and yi;ni to be in�nite and the spacing between the yi;ks to be arbitrary.Let �Yi;k denote the conditional mean of Xi, given that it lies in the subinterval (yi;k�1; yi;k]. In otherwords, we set �Yi;k = E[XijXi 2 (yi;k�1; yi;k]] = Z yi;kyi;k�1 x fi(x)Pi(k) dx;where Pi(k) = Fi(yik) � Fi(yi;k�1) represents the probability that Xi lies in the kth subinterval. Thesupport for the discretized random variable Y is then f �Y1;i; �Y2;j ; �Y3;k : 1 � i � n1; 1 � j � n2; 1 � k �n3g.Let q(i; j; k) represent P (Y = (�Y1;i; �Y2;j ; �Y3;k)), i.e., the probability that X 2 C(i; j; k), whereC(i; j; k) is de�ned as in Section 2 to be the cell corresponding to q(i; j; k). We now give constraints onthe q(i; j; k)s analogous to (8) through (11). Speci�cally, we have thatnXj;k=1 q(i; j; k) = P (Y1 = �Y1;i) = P (X1 2 (y1;i�1; y1;i]) = P1(i); 8i = 1; : : : ; n1; (23)nXi;k=1 q(i; j; k) = P (Y2 = �Y2;j) = P (X2 2 (y2;j�1; y2;j ]) = P2(j); 8j = 1; : : : ; n2; (24)nXi;j=1 q(i; j; k) = P (Y3 = �Y3;k) = P (X3 2 (y3;k�1; y3;k]) = P3(k); 8k = 1; : : : ; n3; (25)q(i; j; k) � 0 8i = 1; : : : ; n1; j = 1; : : : ; n2; k = 1; : : : ; n3: (26)When these constraints are satis�ed, we have that EYi = EXi for each i, just as in the case ofuniform marginals. We can now formulate an LP along the lines of that given in Section 2 to attempt tomatch the covariances of Y to those required of X . If Cij is de�ned as in Section 2, then the formulationof the LP ismin P2i=1P3j=i+1(Z+ij + Z�ij )subject to Cij ��ij = Z+ij � Z�ij ; i = 1 to 2 and j = i+ 1 to 3Z+ij � 0; Z�ij � 0; together with constraints (23), (24), (25) and (26).This LP is always feasible, since we can take the Yi's to be independent, and the objective value isbounded below by 0. Hence, an optimal solution exists for every discretization.If the optimal objective value is 0, then we have been able to construct a probability mass functionfor Y with the required covariance structure. Constructing a joint distribution function for X from thispmf for Y is similar in avor to the method of uniform \smearing" used in Section 2.23

Speci�cally, the \smearing" process should be able to satisfy (15). Again, the easiest method fordoing so is perhaps to ensure that the variables are conditionally independent, given that X lies withinthe cell. To ensure that this conditional independence holds while simultaneously ensuring that X hasthe right marginals, we set the density of X within the cell C(i; j; k) to be that of independent randomvariables with the right marginals, scaled so that the total mass in the cell is q(i; j; k). To be precise, iff is the density of X , then for any x = (x1; x2; x3) 2 C(i; j; k), we setf(x) = f1(x1)P1(i) f2(x2)P2(j) f3(x3)P3(k) q(i; j; k): (27)We can now prove an analogous result to Theorem 1, which states that if the optimal objectivevalue of the LP is 0, then the density f constructed using (27) has the desired marginals and covariancematrix. The proof is virtually identical to that of Theorem 1 and is omitted.Theorem 10 If the optimal objective value of the LP is 0, then the density f de�ned via (27) has therequired marginals and covariance matrix �.We can also obtain analogues to the other results of Section 2. As in Section 2, let � IRd(d�1)=2denote the set of feasible covariance matrices for the given marginals. Note that, as before, we think ofa given covariance matrix as a vector in d(d � 1)=2 dimensional space in some contexts, and as a d� dmatrix in others. We have the following result on the structure of .Proposition 11 The set is nonempty, convex and full-dimensional.The proof of this result is virtually identical to that of Proposition 6 and is omitted.We also have the following analogue of Theorems 4 and 7.Theorem 12 Any chessboard density has a nonsingular covariance matrix. Furthermore, if fn is achessboard density with covariance matrix �, then � is contained in the interior of .Again, the proofs are very similar to those in the previous section and omitted.To extend the other results of the previous section, we assume that yi;0 and yi;ni are �nite for all i,i.e., that all of the distribution functions Fi have bounded support. We further assume that all of theni's are equal to n say, and that all subintervals are of equal length. Thus, we will discretize on a regulargrid containing n3 cells.In particular, suppose that a random vector X with the desired marginals and covariance matrixexists, and let ~X denote its discretization. Let ~q(i; j; k) be the probability that X lies in the cell C(i; j; k).We can now bound the di�erence in the covariance between X1 and X2. But �rst it is convenient to let24

ai = yi;0 and �i = yi;1 � yi;0, so that �i is the width of the cells in the ith coordinate direction, fori = 1; 2; 3. With this notation, we have thatcov( ~X1; ~X2) � �12= nXi;j;k=1 ~q(i; j; k)[ �Y1;i �Y2;j �E[X1X2jX 2 C(i; j; k)]]� nXi;j;k=1 ~q(i; j; k)[y1;iy2;j � y1;i�1y2;j�1]= nXi;j;k=1 ~q(i; j; k)[(a1 + i�1)(a2 + j�2)� (a1 + (i� 1)�1)(a2 + (j � 1)�2)]= nXi;j;k=1 ~q(i; j; k)[�1(a2 + (j � 1)�2) + �2(a1 + (i� 1)�1) + �1�2]� �1EX2 +�2EX1 +�1�2:A similar lower bound can be derived, and so the LP can be augmented by the boundsZ+ij ; Z�ij � �iEXj +�jEXi +�i�j ;for 1 � i < j � 3.Observe that as n!1, these bounds converge to 0. This is the �nal ingredient required to strengthenthe other results of the previous section to the more general case of distributions with bounded supportand densities. In particular, we now have the following results, which we state without proof becausethe proofs are similar to the case of uniform marginals. The implications of these results are given afterthe statement of the results.Remark 5 The above bounds were derived assuming a regularly spaced grid of cells, so that the cellswere all of identical size. However, similar bounds can be expected to hold when the cells are not of equalsize. Indeed, one should be able to obtain bounds on Z+ij and Z�ij which converge to 0 as long as themaximum sidelength of the cells converges to 0.Proposition 13 Suppose that � is feasible for the given marginals. If all of the densities fi havebounded support, then as n ! 1, the optimal objective value of the LP converges to 0. Furthermore,the set is closed.Theorem 14 Suppose that all of the densities fi have bounded support. Then 9n � 1 such that the nthLP is infeasible if and only if the matrix � is infeasible for the given marginals.25

Theorem 15 Suppose that all of the densities fi have bounded support. Then 9n � 1 such that that theoptimal objective value of the nth LP is 0 if and only if the matrix � is contained in the interior of .So assuming that all of the densities fi have bounded support, and � does not lie on the boundary of, then we have a �nite algorithm for determining whether � is feasible or not, and if feasible, supplyingan explicit joint density with the required properties. The algorithm is simply to solve a sequence ofLPs for n = 1; 2; : : : until either the LP is infeasible, or has an optimal objective value of 0. If � lieson the boundary of , then we know that it is feasible, and we can approach it arbitrarily closely withchessboard distributions, but never exactly reach it.It is natural to ask how these results might be generalized to densities with in�nite support. Theprimary di�culty seems to lie in coming up with a satisfactory bound on the objective value of theaugmented LP that converges to 0 in some sense, and we leave this as an open problem. In the case ofdistributions with in�nite support then, the utility of this approach is limited to generating chessboarddistributions for covariance matrices that lie in the interior of .4 Unit Cube ExplorationThe characterizations of Section 3 are certainly potentially useful, especially given the wide use ofproduct-moment correlation in practice. But it is a widely held contention (and one that we share) thatrank covariance is a more meaningful measure of dependence than product-moment covariance whendealing with nonnormal distributions. Therefore, while we view the results of the previous section asuseful and interesting, we prefer to emphasize the case where one is attempting to induce a given rankcorrelation matrix � over a set of marginal distributions.In the case where all of the marginal distribution functions are continuous, this problem reduces tothat of inducing the required product-moment covariance matrix over uniform marginals; see (6), notingthat �(Zi) has a uniform distribution on (0; 1). Therefore, it is important to gain some understandingof the covariance structure of uniform random variables, and in particular, to gain some understandingof the class of covariance matrices that can be achieved using the NORTA method.We present the results we have obtained by applying the LP method introduced in Section 2 to con-struct joint distributions for a random vector X = (X1; X2; X3) with uniform marginals and covariance26

matrix �. For notational convenience, we will use the correlation matrixR = 0BBBB@ 1 �12 �13�12 1 �23�13 �23 1 1CCCCAinstead of the covariance matrix � in the rest of this section.The matrix R is characterized by �12; �13 and �23. The potential set � of all possible values of� = (�12; �13; �23) that can constitute correlation matrices is just a rescaling of the set in Section 2,and is a proper subset of the cube [�1; 1]3 because a correlation matrix R is constrained to be positivesemide�nite.We examined all correlation matrices whose components �12; �13 and �23 lay on the grid f-1.0, -0.9, : : :, -0.1, 0, 0.1, : : :, 0.9, 1.0g3. Each matrix R was �rst checked for positive semide�niteness anddiscarded if not, thus leaving us 4897 matrices from the 9261 matrices that were tested.The remaining 4897 positive semide�nite matrices were further tested to see whether they wereNORTA feasible. We de�ne the matrix R to be NORTA feasible if the covariance matrix � found via(4) is positive semide�nite. In this case, a multivariate normal random vector with covariance matrix �will be transformed via the NORTA method to a multivariate uniform random vector with the requiredcorrelations �.In this manner, a total of 160 sample matrices were identi�ed to be NORTA defective. Note thatsince X1; X2 and X3 are identically distributed, many di�erent �'s form the same e�ective correlationmatrix for X . For example, � =(0.5,-0.5,0.5), (-0.5,0.5,0.5) and (0.5,0.5,-0.5) constitute the same jointdistribution for X up to a symmetry. If we eliminate such multiple occurences, the number of NORTAdefective matrices reduces to 31 cases.The question that remains to be answered is whether these NORTA defective matrices are feasiblefor uniform marginals. This is where the LP method of Section 2 comes in handy. We apply the LPmethod to each NORTA defective case iteratively for increasing values of n, the level of discretization, todetermine whether a chessboard density can be constructed in each case. For the sake of computationaltractability, we limited the value of n to a maximum of 80. The results obtained have been tabulatedin Table 1.Table 1 indicates that the LP method was able to construct a chess-board distribution for almost allof the cases for low to moderate values of n. In fact, cases 26 to 31 are the only cases where the methodfailed to tie down a chessboard density for X with the given correlation matrix R. This is expected fromTheorem 4 since each of these R matrices are singular.27

Correlations Final Optimal Level of Discretization Determinant Smallest EigenvalueNo. �12 �13 �23 Objective Value n jRj of R1 -0.9 -0.6 0.2 0 18 0.0060 0.00342 -0.9 -0.5 0.1 0 11 0.0200 0.01053 -0.9 -0.2 0.6 0 18 0.0060 0.00344 -0.9 -0.1 0.5 0 11 0.0200 0.01055 -0.8 -0.8 0.3 0 12 0.0140 0.00876 -0.8 -0.5 -0.1 0 11 0.0200 0.00977 -0.8 -0.4 -0.2 0 10 0.0320 0.01518 -0.8 -0.3 -0.3 0 10 0.0360 0.01699 -0.8 -0.3 0.8 0 12 0.0140 0.008710 -0.8 0.1 0.5 0 11 0.0200 0.009711 -0.8 0.2 0.4 0 10 0.0320 0.015112 -0.8 0.3 0.3 0 10 0.0360 0.016913 -0.7 -0.7 0.0 0 12 0.0200 0.010114 -0.7 0.0 0.7 0 12 0.0200 0.010115 -0.6 -0.2 0.9 0 18 0.0060 0.003416 -0.5 -0.1 0.9 0 11 0.0200 0.010517 -0.5 0.1 0.8 0 11 0.0200 0.009718 -0.4 0.2 0.8 0 10 0.0320 0.015119 -0.3 0.3 0.8 0 10 0.0360 0.016920 -0.2 0.4 0.8 0 10 0.0320 0.015121 -0.1 0.5 0.8 0 11 0.0200 0.009722 0.0 0.7 0.7 0 12 0.0200 0.010123 0.1 0.5 0.9 0 11 0.0200 0.010524 0.2 0.6 0.9 0 18 0.0060 0.003425 0.3 0.8 0.8 0 12 0.0140 0.008726 -0.8 -0.6 0.0 0.0000159 80 0 027 -0.8 0.0 0.6 0.0000159 80 0 028 -0.6 0.0 0.8 0.0000159 80 0 029 -0.5 -0.5 -0.5 0.0000159 80 0 030 -0.5 0.5 0.5 0.0000159 80 0 031 0.0 0.6 0.8 0.0000159 80 0 0Table 1: Results of the LP method when run for the NORTA defective R matrices28

Hence, almost all of these NORTA defective matrices are feasible for uniform marginals. In particular,case 18 is the example given by Li and Hammond (1975). So we have rigorously established that thereare feasible covariance matrices for a given set of marginals which cannot be generated via the NORTAmethod.It is interesting to question why the NORTA defective matrices are, in fact, NORTA defective.An inspection of the NORTA defective matrices in Table 1 shows that the determinants and thesmallest eigenvalues of all these matrices are quite close to zero in magnitude. This means that thecorresponding points � lie either on the boundary of � or in its close proximity, i.e., the NORTAdefective matrices lie close to, or on, the boundary of the set of achievable correlations.Indeed, we can expect that this will be the case for more general distributions, and higher dimen-sional problems. We reason heuristically as follows. The set �Z of feasible correlation matrices for amultivariate standard normal random vector is the set of all positive semide�nite matrices with unitdiagonal entries. The NORTA method transforms each element of this set into an element of �. There-fore, the set �Z is mapped into a subset of �. Assuming the transformation is continuous, which indeedit is in great generality (see Cario and Nelson 1997), and not \too nonlinear", it is reasonable to expectthat any elements of � that are not covered by the transformation of �Z will be those that are close tothe boundary of �. Indeed, if one plots the set of NORTA defective vectors using a three dimensionalplotting package, this is exactly what we see.The numbers quoted above seem to suggest that the occurence of NORTA defective R matrices isrelatively rare. However, Lurie and Goldberg (1998) believe that singular and near singular correlationmatrices actually represent a common situation in cost analysis for example. This is because correlationsbetween cost elements are typically estimated from unbalanced data sets. This is likely to lead toinde�nite target correlation matrices, so that any least adjustment to them is almost certainly going toresult in an adjusted target matrix that is singular, or very nearly so.So then, we can either modify the NORTA method to generate random vectors with the requiredmarginals and approximately (in NORTA defective cases) the right covariance matrix, or turn to someother method. In the following sections we describe such a modi�cation to the initialization phase ofthe NORTA method, and outline an alternative approach to the NORTA method that relies on thegeneration of iterates with a chessboard distribution.29

5 Semide�nite Programming and NORTAAs seen in the previous section, the Li and Hammond counterexample does exist, as do others, andso there are sets of marginals with feasible (product-moment or rank) correlation matrices that cannotbe exactly matched using the NORTA method. Nevertheless, the NORTA method is very general, andquite easily implemented, and so we may wish to employ the method to generate random vectors withthe required marginals, and at least approximately, the right covariance matrix.Lurie and Goldberg (1998) described a method for identifying a positive semide�nite covariancematrix �Z for use within the NORTA method, that yields approximately the desired product-momentcovariance matrix �X . Their approach involves a complicated nonlinear optimization, and must bespecialized for approximating the rank correlation or product-moment correlation, depending on the casedesired. In contrast, the method we present below does not require such specialization. Furthermore,although they report that their optimization procedure always converges in practice, they do not have aproof of this result. The technique below does not share such a problem because we are using standardsemide�nite programming techniques. Finally, their suggested technique appears to be limited to �xedsample sizes, whereas our adjustment to the initialization phase of NORTA does not share this limitation.Let �Z be the covariance matrix that we wish to use in the NORTA procedure. In what follows, wedo not distinguish between the cases where �Z is chosen to induce a given rank, product-moment, orother correlation in the output random vector X .Our procedure uses semide�nite programming (SDP) to search for a positive semide�nite matrix�Z that is \close" (in a certain sense) to the matrix �Z . We will then employ �Z within the NORTAmethod to generate the required random vectors. Our formulation then falls under the broad class ofmatrix completion problems; see Alfakih and Wolkowicz (2000), or Johnson (1990).Why is this approach reasonable? In Theorem 2 of Cario and Nelson (1997), it is shown that undera certain moment condition, the target covariance matrix �X is a continuous function of the inputcovariance matrix �Z used in the NORTA procedure. This moment condition always holds when we areattempting to match rank covariances, and we can expect it to hold almost invariably when matchingproduct-moment correlations. Therefore, it is eminently reasonable to try and minimize some measureof distance d(�Z ;�Z) say, between �Z and �Z .Given �Z as data, and assuming that we are operating in dimension d = 3, we wish to choose a30

symmetric matrix �Z tominimize j�Z(1; 2)� �Z(1; 2)j+ j�Z(1; 3)� �Z(1; 3)j+ j�Z(2; 3)� �Z(2; 3)jsubject to �Z � 0;�Z(i; i) = 1;where the matrix inequality A � 0 signi�es a constraint that the matrix A be positive semide�nite.We can use the same trick used in formulating the LPs in Section 2 and write each component of theobjective function above as j�Z(i; j)� �Z(i; j)j = w+ij + w�ij ;where w+ij and w�ij are the positive and negative parts respectively of the di�erence �Z(i; j)� �Z(i; j),with w+ij ; w�ij � 0. In particular, we have that for 1 � i < j � 3, �Z(i; j) = �Z(i; j) + w+ij � w�ij .For a vector y, let diag(y) denote the diagonal matrix with diagonal entries equal to the elementsof y. It is well known in SDP formulations that any set of linear inequalities of the form Ax+ b � 0 canbe transformed to a matrix inequality of the form diag(Ax + b) � 0. The nonnegativity constraints onthe variables w+ij and w�ij can be easily handled in this manner.The complete SDP formulation of the problem in the case of a 3 dimensional problem is then tomin w+12 + w�12 + w+13 + w�13 + w+23 + w�23s/t 24 �Z 00 D 35 = 26666664 1 �Z(1; 2) + w+12 � w�12 �Z(1; 3) + w+13 � w�13�Z(1; 2) + w+12 � w�12 1 �Z(2; 3) + w+23 � w�23�Z(1; 3) + w+13 � w�13 �Z(2; 3) + w+23 � w�23 1 D37777775 � 0;where D is the diagonal matrix with diagonal elements fw+12; w�12; w+13; w�13; w+23; w�23g. The matrix wedesire is an optimal solution �Z to this SDP.There is considerable exibility in formulating the problem using this SDP framework that allows usto include preferences on how the search for �Z is performed. For example, if we believe that �Z(i; j)should not be smaller than �Z(i; j), we simply omit the variable w�ij from the formulation. Alternatively,if we believe that the value �Z(i; j) should not be changed, we simply omit the variables w+ij and w�ij .Alternatively, if we believe that �Z(i; j) should not be changed by more than � > 0 say, then we addthe bounds w+ij � � and w�ij � � in the usual (diagonal) fashion.E�cient algorithms are available for solving semide�nite problems; see Wolkowicz, Saigal and Van-denberghe (2000). All of the semide�nite programs in this paper were solved using the CSDP solver,available at the NEOS Optimization Server (2000) website.31

We formulated SDPs as shown above for all of the 31 cases of NORTA defective correlation matricesthat we identi�ed in Section 4 and solved them using the CSDP solver. The optimal objective valuesd(�Z ;�Z) obtained for each of these cases is shown in Table 2. The covariance matrix �0X for the vectorX that can be achieved using �Z can be calculated from (4). The values calculated for d(�0X ;�X) arealso shown in Table 2.Now, since the SDP essentially solves a convex optimization problem, the optimal �Z obtained isguaranteed to be the nearest we can get to the desired matrix �Z . In fact, �Z is a projection of �Z ontothe set of positive semide�nite matrices Z with diagonal elements equal to 1. It follows that �Z lieson the boundary of Z and hence is singular, which indeed is true of all the 31 cases we solved. It doesnot immediately follow though, that the induced covariance matrix �0X is singular, since the NORTAtransformation alters covariances in a nonlinear fashion.It should be pointed out, in fact, that �0X may not be the closest NORTA feasible covariance matrixto �X , because the optimization was performed \in Gaussian space". This is in contrast to the Lurieand Goldberg (1998) procedure. But the values shown in Table 2 seem to suggest that the di�erenced(�0X ;�X) is usually very small.In summary then, we are simply proposing an additional step in the initialization phase of theNORTA method. The revised NORTA procedure is as follows.1. Identify �Z to match some aspect of the desired covariance structure of X in any fashion.2. If �Z is positive semide�nite, then one can proceed directly with the NORTA procedure.3. If not, solve an SDP of the form outlined above to identify a matrix �Z that is \close" to �Z .4. Use the matrix �Z in the NORTA procedure.The additional work involved in this modi�cation only shows up in the initialization phase of theNORTA method, and so there is no additional computational overhead while the method is being usedto generate replicates of X . Furthermore, public-domain algorithms are available for solving SDPs, andthese algorithms can handle very large dimensional problems with relative ease. Finally, the method doesnot require tailoring to di�erent correlation measures. The only step that depends on the correlationmeasure being used is the �rst step when �Z is identi�ed.32

Case Optimal di�erences Case Optimal di�erencesNo. d(�Z ;�Z) d(�0X ;�X) No. d(�Z ;�Z) d(�0X ;�X)1 0.009997 0.010699 16 0.002899 0.0031062 0.002898 0.003106 17 0.016755 0.0174793 0.009997 0.010699 18 0.011054 0.0115394 0.002899 0.003106 19 0.009247 0.0096555 0.006534 0.006827 20 0.011054 0.0115396 0.016755 0.017479 21 0.016755 0.0174797 0.011054 0.011539 22 0.019258 0.0196798 0.009247 0.009655 23 0.002899 0.0031069 0.006534 0.006827 24 0.009997 0.01069910 0.016755 0.017479 25 0.006534 0.00682711 0.011054 0.011539 26 0.027322 0.02846612 0.009247 0.009655 27 0.027322 0.02846613 0.019258 0.019679 28 0.027322 0.02846614 0.019258 0.019679 29 0.052914 0.05224915 0.009997 0.010699 30 0.052914 0.052249- - - 31 0.027322 0.028466Table 2: Results of the SDP when run for the NORTA defective R matrices33

6 Generating From Chessboard DistributionsSuppose, for convenience, that all of the marginals of X have continuous distribution functions, and thatwe wish to match a given (feasible) rank correlation matrix �X . If the corresponding �Z given by (4)is not positive semide�nite, then the approximate method given in the previous section may be used toproduce random vectors with approximately the desired rank covariance matrix.It may be the case, however, that one is not willing to live with such an approximation. An alternativemethod based on chessboard distributions described below could be used in such cases. Furthermore,the chessboard method could be competitive (from a numerical e�ciency standpoint) with the NORTAmethod in many cases, and so it is certainly worth exploring such an approach in this paper.Our assumption of continuous marginal distribution functions ensures that the given rank covariancematrix �X can be matched by �rst generating a random vector U of uniform(0; 1] random variables withPearson product-moment covariance �X , and then setting Xi = F�1i (Ui) for i = 1; : : : ; d.Therefore, the main di�culty lies in generating the random vector U . Since �X is feasible for thegiven marginals, it is also feasible for U . Therefore, �X lies in the set of feasible covariance matricesfor uniform marginals. If �X does not lie on the boundary of , then as in Section 2, we can identifya chessboard density that exactly matches �X . If �X lies on the boundary of , then we can getarbitrarily close to �X , but never exactly match it.In any case, suppose that we have determined an acceptable chessboard density fn for U with uniformmarginals, and covariance matrix �X . How can we generate from this density?One approach is to �rst generate the cell in which U will reside. Then, conditional on U lying in theselected cell, we generate the components of U independently. These components are simply uniformrandom variables on a range that depends on the cell. So the approach is as follows.1. Formulate the problem of constructing a chessboard distribution as an augmented LP as indicatedin Section 2. Solve this LP iteratively for increasing values of the level of discretization n until theoptimal objective value is su�ciently close to 0. Record the �nal optimal solution as the optimalchessboard distribution qn.2. Generate i.i.d. replicates of X by �rst generating the cell containing U , then generating the individ-ual components of U within the cell independently, and �nally transforming U into X as outlinedabove.The computational demands of this method of random vector generation may be competitive withthat of the NORTA procedure, depending on the discretization level n involved. To see why, �rst34

note that perhaps the most taxing part of this method (apart from the initialization - see below) isto determine the cell containing U . This involves generating the cell from the discrete probabilitydistribution speci�ed by qn.An optimal extreme point solution to the augmented LP will yield a solution qn with at mostnd + d(d � 1)=2 strictly positive qn(i1; : : : ; id) terms, because this is the number of constraints in theLP. Notice that this bound will typically be far smaller than nd, the number of cells in the nth LP.Therefore, if n is not too large, we can use the Alias method (see Law and Kelton 2000, p. 474) forexample, to generate the cell containing U in constant time. The storage requirements for this methodare directly related to the number of \bins" needed, which is at most nd+ d(d � 1)=2.It is the initial setup for �nding the optimal distribution qn that could potentially be computationallyexpensive, because this involves solving a succession of LPs in an attempt to match the covariance matrix�X . Nevertheless, the approach is certainly feasible.It is worth noting that a similar method to that described above could also be used in conjunctionwith the LPs of Section 3 to generate random vectors with nonuniform marginals and speci�ed Pearsonproduct-moment correlation matrix. However, the same cautionary remarks apply to the computationalrequirements of the initialization phase.AcknowledgmentsWe would like to thank Marina Epelman for a discussion that was helpful in proving Lemma 8. Thiswork was partially supported by NSF grant DMI-9984717.ReferencesAlfakih, A. and H. Wolkowicz. 2000. Matrix completion problems. In Handbook of Semide�nite Pro-gramming: Theory, Algorithms and Applications. H. Wolkowicz, R. Saigal, L. Vandenberghe, eds,Kluwer, Boston, 533{545.Billingsley, P. 1986. Probability and Measure. Wiley, New York.Cario, M. C. and B. L. Nelson. 1996. Autoregressive to anything: time-series input processes forsimulation. Operations Research Letters 19:51{58.Cario, M. C. and B. L. Nelson. 1997. Modeling and generating random vectors with arbitrary marginaldistributions and correlation matrix. Technical Report, Department of Industrial Engineering and35

Management Sciences, Northwestern University, Evanston, Illinois.Cario, M. C. and B. L. Nelson. 1997b. Numerical methods for �tting and simulating autoregressive-to-anything processes. INFORMS Journal on Computing 10:72{81.Chen, H. 2000. Initialization for NORTA: generation of random vectors with speci�ed marginals and cor-relations. Working paper, Department of Industrial Engineering, Chung Yuan Christian University,Chung Li, Taiwan.Clemen, R. T., and T. Reilly. 1999. Correlations and copulas for decision and risk analysis. ManagementScience. 45:208{224.NEOS Optimization Server. 2000. Available at http://www-neos.mcs.anl.gov/neos/ [accessed Au-gust 2, 2000].Dantzig, G. B. 1991. Converting a converging algorithm into a polynomially bounded algorithm. Techni-cal report 91-5, Systems Optimization Laboratory, Dept of Operations Research, Stanford University,Stanford, California.Devroye, L. 1986. Non-Uniform Random Variate Generation. Springer-Verlag, New York.Henderson, S. G. 2000. Personal Website. Department of Industrial and Operations Engineering,University of Michigan, Michigan. http://www-personal.umich.edu/ shaneioe [accessed August2, 2000].Hill, R. R., and C. H. Reilly. 1994. Composition for multivariate random vectors. In Proceedings of the1994 Winter Simulation Conference, J. D. Tew, S. Manivannan, D. A. Sadowsky, A. F. Seila, eds.IEEE, Piscataway New Jersey, 332 { 339.Hill, R. R., and C. H. Reilly. 2000. The e�ects of coe�cient correlation structure in two-dimensionalknapsack problems on solution procedure performance. Management Science, 46: 302{317.Iman, R. and W. Conover. 1982. A distribution-free approach to inducing rank correlation among inputvariables, Communications in Statistics: Simulation and Computation, 11: 311-334.Johnson, C. R. 1990. Matrix completion problems: a survey. Proceedings of Symposia in AppliedMathematics 40:171{198.Johnson, M. E. 1987. Multivariate Statistical Simulation. Wiley, New York.Kruskal, W. 1958. Ordinal measures of associaton. J. Amer. Statist. Assoc. 53:814{861.Law, A. M. and W. D. Kelton. 2000. Simulation Modeling and Analysis. McGraw Hill, Boston.Lewis, P. A. W., E. McKenzie, and D. K. Hugus. 1989. Gamma processes. Communications in Statistics:36

Stochastic Models 5:1{30.Li, S. T., and J. L. Hammond. 1975. Generation of pseudorandom numbers with speci�ed univariatedistributions and correlation coe�cients. IEEE Transactions on Systems, Man, and Cybernetics.5:557{561.Lurie, P. M., and M. S. Goldberg. 1998. An approximate method for sampling correlated randomvariables from partially-speci�ed distributions. Management Science. 44:203{218.Mardia, K. V. 1970. A translation family of bivariate distributions and Fr�echet's bounds. Sankhya.A32:119{122.Melamed, B., J. R. Hill, and D. Goldsman. 1992. The TES methodology: modeling empirical stationarytime series. In Proceedings of the 1992 Winter Simulation Conference IEEE, Piscataway, New Jersey,135{144.Nelsen, R. B. 1999. An Introduction to Copulas. Lecture Notes in Statistics, 139. Springer-Verlag, NewYork.van der Geest, P. A. G. 1998. An algorithm to generate samples of multi-variate distributions withcorrelated marginals Computational Statistics and Data Analysis 27: 271-289.Whitt,W. 1976. Bivariate distributions with given marginals. The Annals of Statistics. 4:1280{1289.Wolkowicz, H., R. Saigal, L. Vandenberghe, eds. 2000. Handbook of Semide�nite Programming: Theory,Algorithms and Applications. Kluwer, Boston.

37