Physica D 161 (2002) 163–186

Exploring archetypal dynamics of patternformation in cellular flames

Emily StoneDepartment of Mathematics and Statistics, Utah State University, Logan, UT 84322-3900, USA

Received 29 September 2000; received in revised form 16 October 2001; accepted 18 October 2001Communicated by C.K.R.T. Jones

Abstract

The application of archetypal analysis to high-dimensional data arising from video-taped images is presented. Included inthe analysis are intermittent regimes which have not been analyzed previously by other statistical methods such as principalcomponent analysis (PCA). A hybrid PCA/archetypes technique has been developed to overcome the difficulties of applyingarchetypes to data sets with points living in a space of dimension higher than about 500. The advantages of the method lie inthe creation of patterns typical of the set as a whole, and an expression of the dynamics in terms of these patterns. Archetypesare particularly useful in identifying intermittent regimes, where low energy events that might be missed by a severe principalcomponent truncation are none-the-less crucial to understanding the dynamics. They are part of a suite of data analysistechniques that can be used on dynamic data sets (such as FFT, PCA and other spectral decompositions). This hybrid methodextends the application of archetypes to spatio-temporal dynamics in two-dimensional patterns. © 2002 Elsevier Science B.V.All rights reserved.

Keywords: Archetypal analysis; Cellular flames; Principal component analysis

1. Introduction

The study of spatially extended dynamical systems,through the analysis of partial differential equations,experiments in fluid dynamics and granular flow, andintermediate models such as coupled map lattices, iscurrently an active field of physical and mathematicalresearch. Advances in computing technology havemade it possible to efficiently process and store largequantities of data from spatially extended systems.Ways of reducing this vast amount of data to somesmaller descriptive subset have been pursued by re-searchers in statistics and dynamical systems alike.One such statistical method is principal component

E-mail address: stone@sunfs.math.usu.edu (E. Stone).

analysis (PCA) and its application to data sets fromdynamical systems has received much attention inthe past 20 years [1–3,12,16]. It is possible withPCA to compress the data, in a certain sense, to itsmost efficient form. If the governing equations ofmotion are known, the eigenvectors that result fromPCA can be used as a basis for the reduction of aninfinite-dimensional system to a finite-dimensionalsystem [1,10]. Other examples of applications arepattern classification and normal form elucidation[8,9,13].

Recently, PCA has been applied to a physical exper-iment in combustion carried out in Michael Gorman’slab at the University of Houston [7,13–15]. In this ex-periment, combustion takes place on the surface ofa circular porous plug burner, where the fuel and air

0167-2789/02/$ – see front matter © 2002 Elsevier Science B.V. All rights reserved.PII: S0167-2789(01)00361-X

164 E. Stone / Physica D 161 (2002) 163–186

are mixed prior to their entrance to the burner. Byvarying pressure, flow rate and fuel/oxidizer ratio thephysicists are able to generate a variety of stationaryand moving patterns in the flame front. The circularsymmetry of the domain plays an important role inthe selection of these patterns, and Palacios et al. [13]discuss characteristic aspects of cellular patterns on acircular domain. They speculate that certain featuresarise primarily from spontaneous symmetry breakingand the domain geometry, rather than involving thenecessarily complicated physical mechanisms of thecombustion process.

We present here an application of a statisticalmethod known asarchetypes to the cellular flamedata. This paper is part of a sequence of papers uti-lizing archetypes to analyze data from dynamicalsystems [6,17,18]. The archetype algorithm searchesfor points in the data set that are representative ofthe entire set, in a way that we describe below. Forthe flame experiment, archetypes are similar to actualpatterns observed in the sequence of images analyzedfrom a video recording. Archetypal analysis allowsthe description of such a data set in terms of theseempirically determined patterns, rather than a morequalitative description. In some cases, archetypes canisolate patterns and dynamics that are not evidentfrom a visual investigation.

Archetypes were formulated as a variation of PCAby Cutler and Breiman [5]. By confining the linearcombinations allowed in the principal component con-struction to non-negativemixtures of the data points,the archetype algorithm searches for points on the con-vex hull of the data set that minimize the residual sumof squares (RSS) of an archetypal reconstruction ofthe data set. In adding this constraint, much of the con-venient formalism of PCA is lost and the archetypesmust be determined by a time consuming alternatingoptimization algorithm. However, the result may beworth the wait; archetypes themselves resemble datapoints in the set (and in some cases may actually be thedata points), and a reconstructed data point can neverfall outside the convex hull of the original data set.

The time intensive aspect of the archetype algo-rithm has made it difficult, if not impossible, to ap-ply it to data sets of very high-dimensional vectors

such as video images. Principal components have beenused successfully in the analysis of images of faces[8], lip reading [9] and other video data, including theflame data set to be examined in this paper [14,13].Archetypes cannot be applied directly to this data,whose vectors can lie inR4096. We outline here a tech-nique to arrive at an archetypal decomposition of datasets that lie in such high-dimensional spaces, and il-lustrate its use with three examples from the flamedata. The first is a very regular rotation of a singlecell flame front, but the other two are from regimeswhere the patterns break-up and reform in an inter-mittent fashion. These intermittent regimes have beenstudied qualitatively [19] in the past, but this is the firstquantitative analysis of the data sets using principalcomponents and archetypes. Archetypes have provenuseful in analyzing other data sets from intermittentregimes [17], and we show here how they can be usedto separate the flame dynamics into patterns and ex-cursions between patterns.

We are motivated in this analysis by the study ofPalacios et al. [15], in which they analyzed the hoppingstates of the cellular flame experiment. By perform-ing PCA on the data sets and further decomposingeach eigenmode in terms of Fourier–Bessel (FB) func-tions, they could identify the active FB modes in eachregime, and deduce the symmetry in each pair ofeigenmodes. They were able to make a connectionto a simplified dynamic model through an interplayof symmetry considerations and experimental obser-vation of the dynamics of the hopping regime, creat-ing a normal form equation for the active FB modes.These ODEs explain and predict the onset of hop-ping through the interaction of two traveling wave so-lutions with axisymmetric cells moving at differentspeeds. We would like to extend this sort of analysisto more complex, intermittent regimes. In these cases,a severe truncation of the PC eigenmodes to a set offour or six (two or three pairs) could miss states vis-ited briefly, and the transition to these states, sincethe energy contained in these states is low. Here iswhere an archetypal decomposition might really beuseful, since archetypes, being sensitive to outlyersin a data set, can pick up the intermittent events inreduced number sets. We suggest that an FB decom-

E. Stone / Physica D 161 (2002) 163–186 165

position of the archetypes could then be performedto aid creating normal form equations for the bifur-cations to intermittency seen in the experiments. Tothat end we have developed the hybrid technique offirst reducing the data with PCA, but retaining a largeenough number of modes to capture the intermittentdynamics (much more than would be helpful in a di-rect modal analysis, generally on order 50–100), andthen doing an archetypal decomposition of the datarepresented in terms of the PCA modal coefficients.This is the case of using PCA in a more standard man-ner, to reduce the dimension of the data set, ratherthan to uncover coherent dynamics. And to reiterate,this hybrid technique will be useful in cases where thePCA spectrum is broad, but visually it is clear thatthe interesting dynamics can be captured by a fewrepresentative patterns and the transitions betweenthem.

2. The mathematical formulation

In order to use the archetypal algorithm to arriveat archetypal images for the flame data, we must firstreduce the dimension of the data set to a feasible range.To this end we employ PCA. A brief description ofPCA follows. For more details see, e.g. [3,16].

Consider a data set{xi , i = 1, . . . , n}, where eachxi is anm-vectorxi = (x1i , . . . , xmi)

T. In the case ofthe flame data, each vector would be a video image,andm would be the total number of pixels sampled, forinstance a 64× 64 image would havem = 4096. Theordering of the vectors is important in this application,since it holds the time information of the data set,i.e. the dynamics. To clarify, the spatial information iscontained in the elements of the vector (the value ateach pixel of the image), whilei = 1, . . . , n indicatesthe frame number—each vector is a image sampled inthe flame experiment at 30 frames/s.

The problem is to find a set of vectors that formsan “optimal” basis, by which we mean that, given abasis set of�’s for the data vectors:

xi =m∑

j=1

aij�j ,

the approximation (with a truncated set of vectors)

x̂i =p∑

j=1

aij�j , p < m,

has minimum error, defined:

εp = 〈‖xi − x̂i‖2〉,where〈 〉 denotes an ensemble average. Minimizingεp

subject to the constraint(�i , �j ) = δij, via Lagrangemultipliers results in an eigenvalue problem for the�’s:

C�j = λj �j ,

where

C = 〈xixTi 〉

is the ensemble averaged covariance matrix.C isan m × m symmetric non-negative matrix and con-sequently determines a complete set of orthogonaleigenvectors, and real, non-negative, eigenvalues.These eigenvalues can be ordered:λ1 ≥ λ2 ≥ λ3 ≥· · · ≥ λm and the statistical variance of the data set inthe direction of thejth eigenvector is proportional tothe jth eigenvalue ofC. In fact, it is easily shown that

εp =m∑

j=p+1

λj .

The �’s, which appear in many contexts, are calledthe empirical eigenfunctions [12], coherent structures[16], etc., by the dynamical systems community. Thedata can be reconstructed in terms of these eigenfunc-tions

x̂i =p∑

j=1

aij�j , p < m,

and theaij are the reconstruction coefficients, one foreach vector(j = 1, . . . , p), for every data point(i =1, . . . , n).

This is a description of what is known as the “directmethod” of computing the eigenfunctions, and sinceC is m × m its eigenvectors arem-dimensional andcan be viewed as video images themselves. A moreefficient algorithm for computing the eigen-images is

166 E. Stone / Physica D 161 (2002) 163–186

the “snap-shot method” [8], which capitalizes on thefact that the eigenvectors ofC can be more efficientlycomputed from a matrix formed from a space aver-age over temporal cross-correlations, i.e. an averageover 4096n × n matrices. In cases where the num-ber of images is much smaller than the dimensionof the data set, this is obviously a much faster com-putation, and sinceC is then degenerate the methodprovides then eigenvectors ofC that have non-zeroeigenvalues.

2.1. Archetypal analysis

Archetypal analysis is a statistical technique, whosecreation was motivated by some of the shortcomingsof principal components. For instance, if the dataset in question is not elliptically distributed, the re-construction of a data point with a truncated set ofeigenvectors could produce a point outside of thebounds of the original data set. If the data are per-haps restricted by some physical property to lie in asingle quadrant of the data space, this means that areconstructed point could lie outside that quadrant.The creators of archetypes, Cutler and Breiman [5],discuss the algorithm, the motivation behind its cre-ation, and some of the advantages of the methodin more detail. There are also several papers onthe application of archetypal analysis to dynamicalsystems. For a more complete description of themethod, and for an in depth comparison to PCA, werefer readers to [5], and to the papers that followed[6,17,18].

Archetypal analysis represents each point in adata set as a mixture of “individuals of pure type”or archetypes. The archetypes themselves are re-stricted to be mixtures of the individual points in thedata set. Archetypes are selected by minimizing thesquared error in representing each point as a mixtureof archetypes. As mentioned in the introduction, thisminimization must be carried out via a numericaloptimization scheme, due to the added constraint ofpositive coefficients in the mixtures.

The problem can be stated as follows: given multi-variate data as in the previous section

{xi , i = 1, . . . , n},

where eachxi is anm-vector

xi = (x1i , . . . , xmi)T.

Findp m-vectorsz1, . . . , zp that characterize “archety-pal patterns” in the data such that data can be rep-resented as mixtures of the archetypal patterns. Thearchetypal patterns are then specified: findz1, . . . , zp

to minimize

εp = 〈‖xi − x̂i‖2〉,where

x̂i =p∑

k=1

αikzk, αik ≥ 0,∑

k

αik = 1

and

zk =n∑

j=1

βkjxj , βkj ≥ 0,∑

i

βki = 1.

The z1, . . . , zp solving this optimization problem arethen archetypes. Unlike PCA, there is no intrinsic or-der in the archetypes, success levels of truncation donot “nest”, the set generated at each level of truncationp can contain different archetypes, though an approx-imate form of nesting does occur in the cases beingstudied here (and, in fact, in all the cases the authoris familiar with). The data can be approximated by aconvex combination of the archetypes

x̂i =p∑

k=1

αikzk, αik ≥ 0,∑

k

αik = 1

and theαik are denoted “mixture coefficients”.It can be shown that forp > 1 the archetypes lie

on the convex hull of the data set, i.e. they representextreme data-values (or mixtures thereof) such that allof the data can be well-represented as convex mixturesof the archetypes [5]. The archetypes themselves areoften more readily interpreted in terms of the data thanPCA eigenfunctions, since each is constrained to be aconvex combination of the data points. Computing thearchetypes is a nonlinear least squares problem whichis solved using an alternating minimization algorithm.For details on the algorithm, see [5].

To quantitatively compute the effectiveness ofeach truncation level, we plot the relative error,Λp,

E. Stone / Physica D 161 (2002) 163–186 167

incurred in approximating the data. The relative er-ror is the mean square error,εp in truncating to 2parchetypes, divided by the total variance of the data set

Λp = εp

ε1= 〈‖xi − x̂i‖2〉

〈‖xi − 〈xi〉‖2〉 .

Typically the value ofp is chosen to be the point wherethe relative error levels off; adding more archetypes tothe set does not substantially reduce the error incurredin reconstructing the data. This is the criteria used inthe examples presented below.

For data sets of the size that we will be considering,the archetype algorithm is ungainly. The CPU time ofcomputation increases roughly linearly with the sizeof the data space [4], so that even an image with di-mensions 64×64 pixels lives in the unreasonably highdimensionedR4096. Archetypes could be computed fora snap-shot formulation of the data, i.e. with vectors ofthe time evolution of each pixel, but the interpretationof the results would be much less straightforward, sosome of the advantages of using archetypes would belost. Instead we adopt a hybrid technique, where thedata are first compressed using PCA, and the resultis analyzed via archetypes. If the original data set is{xi , i = 1, . . . , n}, where eachxi is anm-vector (inour examplesm is 4096), principal components gener-ates a set ofn eigenvectors (those with non-zero eigen-values of〈xxT〉). A truncation is performed to the setof eigenvectors that contain some large fraction of thevariance (typically 85–90%), say a numberq, whereq � n. The reconstruction coefficients for the origi-nal data set live then inRq , so aq-dimensional set ofvectors (as many as frames in the data set) is passedto the archetype algorithm. Archetypal sets are com-puted, and each archetype is a vector in the truncatedreconstruction coefficient space, and so can be visu-alized as a flame cell by compiling the reconstructioncoefficient with the PCA eigenvectors.

Mixture coefficients for the archetypes are best un-derstood by considering the archetypes shapes and thedata together, where it is evident how different pointsin the data are made up of mixtures of archetypalflame patterns. The time series created from the mix-ture coefficients as the frame number is advanced car-ries important information about time evolution of the

pattern. In one instance, we projected a part of a dataset onto archetypes computed from another subset ofthe data; in this way, new data can be described byalready formulated archetypes which provides a sig-nificant savings in time.

The error in the computation is the combined er-ror created by the archetypal reconstruction and thetruncation in the principal component step. Archetypalanalysis will find points on the convex hull of the dataset “smoothed” by PCA, and ideally the informationlost is “in the noise”, and the essential dynamics areunaltered. If that is not the case more PCA modes mustbe considered, and for very high-dimensional dynam-ics the archetype representation would not be appro-priate. The utility of the method lies less in actuallyreconstructing compressed data points (PCA does thisbest), but in extracting flame cell shapes that can bedesignated as archetypes and, therefore, descriptive ofthe entire data set.

3. Analyzing flame cell dynamics

The flame cell experiment dynamics has beencaptured by infrared video recording, and here westudy three distinct regimes. For movies of these se-quences see the webpage maintained by Kay Robbinsat http://vip.cs.utsa.edu/flames/klvisual/. The data arein the form of sequential video images, each imagebeing 64× 64 pixels, sampled at 30 frames/s. Theoriginal data sets are thus clouds of points inR4096.We first tested the method on a simple regime inwhich a single flame cell rotates clockwise aroundthe center of the burner in a smooth and predictablemanner at approximately 3.3 rev/s. An in-depth cov-erage of the PCA for this set is presented in [14]. Thedecomposition of this spatially homogeneous dataset produces, not surprisingly, eigenfunctions thatresemble vibrating drum-head modes: the best basisfor a system with circular symmetry. As documentedin [14], the first pair has approximateD1 symmetry,and the second, approximateD2 symmetry, and thereconstruction coefficient time series for the first twomodes oscillate at once per revolution of the pattern,with a phase shift of half-a-period, and the second

168 E. Stone / Physica D 161 (2002) 163–186

pair of modes oscillate at twice the frequency of theoriginal pair. All this is a consequence of the uniformrotation of the cell, and while a reconstruction ofthe data from the modes is very close to the original(see [14]), it would be a mistake to identify any ofthe modes with a particular image in the sequence (amistake sometimes made by identifying the modeswith “coherent structures” in the data). And while thecell itself is rotating, it is also deforming as it does

Fig. 1. Five archetype set from single rotating cell data.

so, it is not simply a rigid body rotation of a staticshape.

We performed an archetype analysis on a com-pressed data set formed from a PCA analysis of thevideo image sequence. From the evidence of theeigenvalue spectrum, we chose to truncate at eighteigenvectors, capturing about 89% of the variance.The reconstruction coefficients for each data point(frame) were then passed to the archetype routine,

E. Stone / Physica D 161 (2002) 163–186 169

which was therefore presented with a data set in eightdimensions. Archetypes themselves are points in theeight-dimensional reconstruction coefficient space,but they can be visualized by projecting each ontothe principal component eigenvectors. A representa-tive set of five archetypes is shown in Fig. 1. At five

Fig. 2. Sample frames from intermittent regime HC-C (frames 120–139).

archetypes the relative error begins to level off, indi-cating that much of the variance that can be capturedhas been captured. Each archetype is the flame cellshape, captured at various points in its rotation aboutthe center. Sets with more archetypes contain a finerbreakdown of the rotation of the cell. The archetypes

170 E. Stone / Physica D 161 (2002) 163–186

are representative points in the data set, i.e. frames inthe video tape. The effect of the rotational symmetrydoes not smear the result into highly symmetrical pat-terns on a disk, which is what happens in the creationof the eigenmodes. If an identification of significantpatterns is being sought, an archetypal analysis shouldbe used, the PCA reduction alone will not give thisinformation.

The rotation of the patterns suggests a possible ap-plication of moving archetypes [6], where the rotationwould be subtracted out by the algorithm and the fi-nal result would be a single archetype representingthe pattern, along with the variation of the rotation intime. We are considering ways of extending movingarchetypes to this case. Moving archetypes would beeven more useful for data sets where patterns rotateand vary in shape, a situation in which a straight for-ward application of some image preprocessing tech-

Fig. 3. HC-C: PCA spectrum and archetype RSS for first 150 frames.

nique would not work as well (for instance, doing atwo-dimensional FFT and locating the rotation in theangular variable of the decomposition).

This flame data set is very regular and dynamicallyrather uninteresting, but provides a good test for themethod. The results of the PCA and the archetypeanalysis are quite different, and provide complimen-tary information about the dynamics of the patterns.PCA gives a great deal of information about the sym-metries involved in the uniformly rotating cell, whilethe archetypes retain the integrity of the patterns them-selves. We next consider data sets where PCA cannotreproduce the dynamics in such a small number ofmodes, and the utility of archetypes in these cases isexplored. In these regimes, cells break-up and reform;the number of cells is not constant in time. In the firstexample (HC-C), the number of outer cells is alwaysnine, but the number of inner cells varies from one,

E. Stone / Physica D 161 (2002) 163–186 171

two or three. The regime is typified by an intermit-tent aspect: the pattern can remain quasi-stationary formany seconds, after which the inner pattern of cellsloses stability. Then, after an interval of disorder ofthe inner ring, a stationary pattern again emerges. Thetime during which a pattern is quasi-stationary varies,as does the interval of disorder. An example is shownin Fig. 2, where an one-inner-cell pattern breaks intotwo-inner-cells, the two cells then rotate by mutatingthrough an one-inner-cell pattern, and finally one ofthe two cells splits generating the three-inner-cell pat-

Fig. 4. HC-C: first six PCA eigenmodes.

terns which dominates the dynamics. This is a partic-ularly active sequence in the data set; over a sequenceof 600 frames, the pattern is in the three-inner-cellstate for over 50% of the time.

This sort of dynamics is reminiscent of heteroclinicexcursions between saddle fixed points, a conjecturethat was explored in a different way in an earlier paper[19]. One hope is to be able to reduce the completehigh-dimensional system to a low-dimensional sub-space, one possibly provided by PCA. However, sincethe underlying equations of motion for the flames are

172 E. Stone / Physica D 161 (2002) 163–186

not known, or at least not agreed upon, analysis musttake another turn for the moment. One very nice ap-proach introduced by Palacios et al. [15], is to studythe normal form equations generated through an in-terplay of symmetry considerations and experimentalobservations. The relevant FB modes involved in theinteraction are deduced through an FB decompositionof the highest energy eigenmodes. The problem withusing PCA alone for these intermittent cases is that thespectrum is broad, and many eigenmodes would beneeded to capture the relevant dynamics. This is wherearchetypes come in handy, since archetypes have beenfound to be useful in dealing with intermittent data,where the intermittent excursion away from the fixed

Fig. 5. HC-C: six archetype set from 15 PCA modes.

point extends to the convex hull of the data set (whichit often does, since it usually is an outlyer of the set).Here we test that utility for two-dimensional imagedata. Later in the section, we will present a new way ofvisualizing the dynamics by projecting onto the spaceof archetype mixture coefficients. In Section 4, we ex-pand on the idea of extending the analysis performedby Palacios et al. [15], to these intermittent regimesby using the hybrid PCA/archetype technique.

3.1. HC-C data

A principal component decomposition on thefirst 150 frames of the HC-C data set yields the

E. Stone / Physica D 161 (2002) 163–186 173

eigenvalue spectrum shown in Fig. 3. The firstsix most energetic eigenmodes (capturing 73% ofthe variance) are presented in Fig. 4. The primarynine-outer-cells/three-inner-cell patterns is evident inthe first two eigenmodes, but the other quasi-stationarypatterns observed in the sequence are not as eas-ily identified. They would be constructed as linearcombinations of all the eigenmodes. The advantageof archetypes is clear—the archetypes look like thequasi-stationary patterns. A representation of thedata set with 15 eigenvectors (90% of the variance)was then analyzed with the archetype routine. TheRSS for this is also presented in Fig. 3. The levelingoff of the RSS happens around six archetypes, anda set of six archetypes does capture the predomi-nant patterns seen in the data set (see Fig. 5). Theone-inner-cell pattern appears, and two orientationsof the two-inner-cell patterns (archetype 3 is obvi-ously more of a transition pattern than archetype 4),and three copies of the three-inner-cell patterns. Twoorientations of that pattern are captured, and one du-

Fig. 6. HC-C: mixture coefficients vs. frame number for six archetype set.

plicate. The duplicate indicates we have taken morearchetypes than is really needed, but this was the bestset we derived none-the-less, and duplicate patternscan be dealt with easily enough by identifying thetwo, and adding together their mixture coefficients.

The entire 600 frame data set was projected ontothese six archetypes to generate the time series shownin Fig. 6. As mentioned above, we grouped the twoidentical three-inner-cell patterns together by addingtheir mixture coefficients, hence there are only fivetraces. For comparison, we also show the principalcomponent reconstruction coefficients for the same600 frame data set (Fig. 7). Here is one place wherethe representation in terms of archetype mixture co-efficients could be useful: measuring the time intervalspent near quasi-stationary states (e.g. whereα6 = 1,from about frames 135–230). These intervals are notat all obvious from the principal component recon-struction coefficient time series.

Describing the dynamics in terms of convex com-binations of patterns is what archetypes can do, and

174 E. Stone / Physica D 161 (2002) 163–186

Fig. 7. HC-C: PCA reconstruction coefficients vs. frame number.

we are working on ways of organizing that informa-tion to aid building dynamics models. One way topresent the data that highlights connections betweenquasi-stationary patterns (possible heteroclinic con-nections in a model, loosely inspired by heteroclinicnetworks [11]) is with mixture coefficient triangles.For a set of three archetypes, an equilateral triangleis constructed where the vertices are identified witheach archetypes. Data points represented by all onearchetypes fall on the vertices. Data points mappedto the edges of the triangle are mixtures of the twoarchetypes defining the edge. A data point that is anequal mixture of all three is mapped to the centerof the triangle. For a large part of the data set, onlythree archetypes are “active” for significant intervalsof time. During such intervals representing the data onsuch a triangle gives a reasonable idea of the dynamicsconstrained to that particular simplex. A square could

be used to present the data decomposed with fourarchetypes, a pentagon for five, etc., but we found thatthe triangle (or rather two triangles) gave the clearestpicture of the dynamics.

We constructed two mixture coefficient trianglesfor the data decomposed with six archetypes, butthey share a common edge, one connecting the cleartwo-inner-cell patterns (archetype 4) and the transi-tional two-inner-cell patterns (archetype 3). The threearchetypes representing the three-inner-cell patternsare combined and put at the top of one of the trian-gles, and the one-inner-cell pattern (archetype 2) atthe top of the other. These choices were made aftersome preliminary consideration of the data. The twothree-inner-cell patterns that are rotations of one an-other are seen to not be active (meaning their mixturecoefficient is non-zero) at the same time, they can becombined without violating the sum-to-one rule on the

E. Stone / Physica D 161 (2002) 163–186 175

alphas. And direct transition between one-inner-celland three-inner-cell were not seen, hence not havingan edge between them is reasonable.

The two triangles for four different “epochs” in thedata set are graphed in Fig. 8. The first runs from thefirst frame until the end of the first long residence nearthe three-inner-cell patterns, at about the 117th frame.The one-cell triangle only has points until the 45th

Fig. 8. HC-C: mixture coefficient triangles; each row corresponds to an epoch.

frame, since the one-inner-cell pattern is inactive afterthat. The next epoch includes the transition from thatthree-inner-cell patterns to the rotated one, until theend of the second long residence spent near it, aboutframe 232. Again the one-cell triangle only includespoints where the one-inner-cell mixture coefficient isnon-zero, until the 145th frame. The third epoch con-tains the interval of disorder from the time when the

176 E. Stone / Physica D 161 (2002) 163–186

three-inner-cell quasi-stationary patterns breaks up,until it reforms again at about frame 350. In the fourthepoch, the three-inner-cell patterns predominates untilabout frame 500, after which there is a direct transitionto the two-inner-cell patterns, followed by a transitionto one-inner-cell through the transitional direction.The remaining frames are a complicated excursionending at the rotated three-inner-cell patterns.

Naturally, this is an imperfect method of projectinga hypersurface described byα1 +α2 +α3 +α4 +α5 +α6 = 1, and what is seen to be on the plane in threedimensions that each triangle describes, is actually oc-curring in the region under the plane, and the trajec-tory on the triangle is a projection up onto that plane.To further aid understanding, we give some approxi-

Fig. 9. HC-C: trajectory cartoons of mixture coefficient triangles for the first two epochs: (a) and (b) are the first epoch; (c) and (d) arethe second epoch.

mate line drawings of the trajectories in Fig. 9. De-spite its obvious limitations, we are able to form someconclusions about the dynamics from these diagrams.

In the first epoch, we see the first transition tothe three-inner-cell stationary patterns. The trajectoryloops near the line connecting the two-inner-cell pat-terns and the transitional state several times in thethree-inner-cell triangle, before heading up to the topvertex and the quasi-stationary three-inner-cell pat-terns. On the one-inner-cell triangle the trajectory firstgoes to the top vertex (as seen on theα2 trace inFig. 9), after which it descends to the transition direc-tion and executes a loop to the two-inner-cell state andback. One distinction to make is whether a loop staysnear the base of a triangle; meaning that the pattern is

E. Stone / Physica D 161 (2002) 163–186 177

mostly described by the two archetypes on the associ-ated vertices, or if it moves into the center of the trian-gle. During the first epoch, transitions along the edgefrom the three-inner-cell to the two-inner-cell state areseen, but not the reverse. Also, direct transitions (not

Fig. 10. Sample frames from intermittent regime HC-A (frames 610–644). Increasing time reads to the right across the page and down.

through another state) between the one-inner-cell andthe two-inner-cell state are not seen.

During the second epoch, the transition to the ro-tated three-inner-cell stationary state occurs. On thethree-inner-cell triangle this involves a direct transition

178 E. Stone / Physica D 161 (2002) 163–186

first to the two-inner-cell patterns (presented in thefirst epoch for clarity), and several loops again fromthe two-inner-cell patterns out in the transition direc-tion and back (we call it a transition direction becausethe archetype associated with it is not a clear patternseen in the quasi-stationary states).

During the third epoch, all the patterns are involvedand it is clear how this method of presenting the datacan fail. The dynamics is very complicated, but stillsome information can be gleaned from these diagrams.Transitions directly from the two-inner-cell patternsand the one-inner-cell pattern are not seen even duringthis complex regime, and the top half of the three-celltriangle is hardly occupied, indicating an absenceof patterns made up of mixtures of three-inner-cellswith lesser parts two-inner-cell and transitionalpatterns.

Fig. 11. HC-A: PCA spectrum and archetype fractional RSS.

The final epoch runs from frame 350 to 600. Duringthis time the three-inner-cell patterns persists for ap-proximately 120 frames, after which a switch is madeto the rotated three-inner-cell patterns. The switch isaccomplished over more than 50 frames and involvesall the other patterns. In the mixture triangles, we seethat much of the time the dynamics is restricted to theedge of the triangles, and in particular to the connec-tions between the three/one-inner-cell pattern and thetransition state.

The triangles are an alternative to examining thetime series directly, and hopefully organize the in-formation in a way that clarifies the dynamics. Thecreation of the two triangles was dictated by the dataitself, and the best representation to use will dependon the dynamics in any particular case. The two tri-angles, however, are also reminiscent of heteroclinic

E. Stone / Physica D 161 (2002) 163–186 179

networks [11], and we were aiming to uncover poten-tial heteroclinic connections between states and theinvariant subspaces they might live in. For instance,from the data there is no direct connection from anone-inner-cell state to a two-inner-cell state, but thereis between three-inner-cell and the other two. Over a

Fig. 12. HC-A: eight archetype set from 90 PCA modes.

longer data set this sort of information could be builtup to help fine-tune a model built through symmetryconsiderations of the quasi-stationary states (throughan FB representation of the archetypes involved, andan associated set of normal form equations). We dis-cuss other model building ideas in the final section.

180 E. Stone / Physica D 161 (2002) 163–186

A final comment: the validity of this representationwill depend on the completeness of the archetype set.If a quasi-stationary pattern observed in the data is notpicked up by the archetype algorithm, any conclusionsabout direct transitions and transitions along edgeswill be incorrect. We have identified the two observedrotations of three-inner-cell state, and archetypes ontheir own do not recognize patterns that are identicalexcept for a rigid body rotation. If a whole group orbitof patterns was observed in an experiment, variation ofmoving archetypes for two-dimensional images wouldbe a more appropriate choice (as in the rotating cell inthe first example). This is not the case with this dataset, so we are content to leave a more thorough studyof archetypes and data sets with symmetry to a laterproject.

Fig. 13. HC-A: mixture coefficients vs. frame number for the eight archetype set.

3.2. HC-A data

The second set we analyzed is considerablymore complex than the first. The residence times inquasi-stationary states are much shorter on average,and the degree of disorder in general much greater(using the broadness of the KL spectrum as a roughindicator of entropy or disorder). A representative se-quence is shown is Fig. 10. Using the nomenclatureof Gorman et al. [7], we see first in the sequence a7/1 pattern that undergoes a metamorphosis to an 8/2pattern over the first five frames. The two cells in thecenter then join to form an 8/1 pattern in about frame621. Two of the outer cells join to form 7/1 at frame627, and this goes through a complicated transition to8/3 through frames 632–639.

E. Stone / Physica D 161 (2002) 163–186 181

A similar description can be constructed for the restof the 900 frame data set, but is omitted for obvi-ous reasons. Suffice to say the dynamics is compli-cated, but a qualitative understanding of it is possi-ble through the archetypal analysis, without resortingto looking at the data frame-by-frame. To this end,we performed a PCA analysis of the 900 frames, andthe eigenvalue spectrum is presented in Fig. 11. Be-cause the first few modes appear to be much largerthan the rest, one might be tempted to truncate af-ter only four or five modes. This, however, wouldcapture only a small amount of the variance, sincethe spectrum is quite broad. For the archetypal de-composition, we truncated to 90 PC modes, whichcaptures 94% of the variance. The RSS (Σ) plot is

Fig. 14. HC-A: PCA reconstruction coefficients vs. frame number.

also shown in Fig. 11. The ratio levels off aroundsix or eight archetypes, so we consider here an eightarchetype set (Fig. 12). It was also found that theseeight archetypes captured the essential behavior ina reconstruction of the data, a conclusion that wasreached by comparing reconstructions at various levelsto confirm the RSS cut-off. The one-center-cell pat-tern predominates, though with a little study the dif-ferences between the archetypes are clear. Archetype1 is pattern 8/1, archetype 4 is also 8/1, with blurredouter cells on the right hand side. Archetype 8 is again8/1, but the position of the outer cells is rotated by ap-proximatelyπ /8. Archetypes 6 and 7 are 7/1 patterns,and archetype 7 has an outer ring cell at 9 o’clock thatis starting to split, as would be needed in a transition

182 E. Stone / Physica D 161 (2002) 163–186

between seven- and eight-outer-cells. Archetype 2 isa 9/2 pattern, archetype 3 is 8/3 and archetype 5 is atransition between 9/3 and 9/1.

Combining mixture coefficients for similararchetypes greatly simplifies the resulting time series.For instance,α1, α4 andα8 all represent 8/1 patternsand are combined in the first time series in Fig. 13.Similarly, we combine the 7/1 patterns in the secondtrace of Fig. 13. The third trace is the mixture coeffi-cient for the two-inner-cell patterns, archetype 2, thefourth is for pattern 8/3, and the fifth for pattern 9/3.From the time series, it is clear that the one-inner-cellpattern dominates, with intervals in which the other

Fig. 15. HC-A clip: PCA spectrum and archetype fractional RSS.

patterns stabilize. In fact, what is decomposed asthe one-inner-cell pattern is actually a complicated“boiling” of cells that returns intermittently to anone-inner-cell pattern (see the first frames of Fig. 10and compare with time series). For comparison, weplot the PCA reconstruction coefficients (Fig. 14).Plotting 90 traces is not feasible, so we consider onlythe first six. It is much less obvious what the dynamicsof the patterns are from this representation.

We found it useful to analyze a clip taken fromthe above data, using 100 frames between frames 600and 700. The PCA spectrum is understandably lessbroad than before (Fig. 15), and we only used 15

E. Stone / Physica D 161 (2002) 163–186 183

Fig. 16. HC-A clip: six archetype set from 15 PCA modes.

eigenmodes (for 87% of the variance) in an archetypedecomposition. Fewer patterns are visited and thenumber of archetypes needed to capture the dynamicsdecreases. For this set (see Fig. 10 for a 35 framesample), we found that six archetypes worked well toseparate the clip into patterns and the time sequenceof their appearance, and the RSS measurement,Σ ,levels off at six as well. The archetype set is shownFig. 16. The first archetype is the 8/3 pattern, sec-ond is 8/2, third captures 9/3 and fourth is transi-

tion between seven- and eight-outer-cell and one- totwo-inner-cell. The fifth is a transition between seven-and eight-outer-cell with one-inner-cell, and the sixthis the three-inner-cell patterns with eight-, possiblynine-outer-cells. The progression through the patternsin time is clear from the mixture coefficients overthis short sequence (Fig. 17), and is easily confirmedby comparison with the frames themselves (Fig. 10).The sequence begins with 10 frames of a transitionfrom 7/1 to 7/2, and then to 8/2. A direct transition to

184 E. Stone / Physica D 161 (2002) 163–186

Fig. 17. HC-A clip: mixture coefficients vs. time for the six archetype set.

Fig. 18. HC-A clip: a comparison of the PCA reconstruction, archetype reconstruction and the archetype itself for a sample data frame.

E. Stone / Physica D 161 (2002) 163–186 185

8/1 then occurs, followed by a complicated mixtureof all the patterns between frames 630 and 640. Aftera brief appearance of 9/3, the pattern stabilizes in 8/3(frames 645–660). A direct transition is then made to9/3, when one of the outer cells splits to add anothercell to the outer ring. In Fig. 18, we compare the PCAreconstruction with 15 eigenmodes of frame 619 tothe frame itself, and the archetype reconstruction. Themixture is almost all in archetype 2 for this frame,so that is shown too, for comparison. The archetypalmethod’s strength lies in capturing the quasi-stationarypatterns in the data set, as demonstrated in thisfigure.

4. Discussion and conclusion

The application of archetypal analysis to high-dimensional data sets requires some modification tomake it time effective (and even possible). We havedescribed here a hybrid method utilizing principalcomponents to first reduce the size of the data set,which is then analyzed with the archetype algorithm.There are situations in which archetypes have decidedadvantages over PCA, such as the representation ofintermittent regimes, and this technique allows usto extend archetypes to higher-dimensional data setsthan previously thought possible. We analyzed threedata sets to test this utility, and two of the three datasets are intermittent regimes which have not previ-ously had a quantitative analysis presented in theliterature.

For the uniformly rotating cell regime, the resultswere obvious. The archetypes were simply rotationsof the single cell pattern, and increasing the numberof archetypes in the set added more angles of rota-tion and subsequently decreased the error in a recon-struction. This is a case where a variation of mov-ing archetypes, which for one-dimensional data sub-tract out translations of structures, might be very suc-cessful. The data could be separated into the singlecell pattern and a time series that described its rota-tion. Moving archetypes are not limited to situationsin which the rotation is uniform, or the structure isstatic. For instance, a modulated wave is decomposed

by moving archetypes into several archetypes that de-scribe the modulation of the wave, mixture coefficientsthat record the time variation, and a shift vector thathandles the translation. Such a thing could be imag-ined for two-dimensional data as well, and will be thesubject of future work.

The two other data sets were dynamically moreinteresting. Cells break-up and reform, with some pat-terns lingering over much longer time intervals thanothers. Archetypal analysis finds these patterns objec-tively and then gives a description of the dynamics interms of the patterns. This can be viewed as time seriesof mixture coefficients, or on triangles defined by therelationship that three of the mixture coefficients sumto one. In cases where the dynamics is well describedby three patterns, this gives a way to analyze transi-tions between patterns: Is the transition completelydescribed by the two patterns involved, or is anotherdirection needed to capture it? This information wouldbe very hard to obtain by visually examining the videotype. Another use for the decomposition is in the com-putation of “residence times” near quasi-stationarypatterns. From the mixture coefficient time series, itis clear when the dynamics is near a fixed pattern fora significant interval—its mixture coefficient is closeto 1, and the rest are close to zero. The time can beread off the diagram directly. A PCA reconstructioncoefficient time series does not give such a clear in-dication of transitions between patterns, since at anyone time a video frame is made up of many PCA vec-tors. In [19], the residence times were determined byexamining the video tape frame-by-frame and count-ing the number of frames where a certain pattern isquasi-stationary. Naturally, this is labor intensive andprone to error, and an archetypal time series wouldhave been very helpful. And, since the projection ofdata onto already-determined archetypes is relativelyfast, it may be possible to use the method on-the-fly inexperiments to detect transitions between archetypes,and the pathways between patterns involved in thetransitions.

Archetypes should be seen as another tool in theset of techniques used to analyze spatio-temporal datasets. If it makes sense to use archetypes will depend onthe data set being reduced and the kind of information

186 E. Stone / Physica D 161 (2002) 163–186

required. For more on this issue see [17]. In the datasets presented here, it was useful to do an archetypaldecomposition because of its ability to capture low en-ergy/variance features of intermittent dynamics. Thearchetypal patterns uncovered in this analysis couldbe used to create normal form equations in a mannersimilar to that presented in [15], allowing for the ex-tension of the method to situations in which the PCAspectrum is too broad for a low-dimensional PCA rep-resentation. In addition, the dynamics described bythese patterns in the data set could be used to ex-tract more information about the underlying model,such as the possible existence of invariant subspacesspanned by FB modes that can be combined to cre-ate the patterns. The archetypes would provide a wayto extract this information quantitatively, rather thanqualitatively through a visual examination of the videotape.

To summarize, PCA analysis has advantages overFourier or FB decomposition in that it generates anoptimal reduced set of modes, optimal in that theenergy in each mode is maximized. Archetypes, be-yond the qualitative advantages of producing a setof typical patterns and transitions between them, hasquantitative advantages over PCA for representingintermittent dynamics. In order to extend archetypesto two-dimensional intermittent patterns such as theones seen in the flame experiment, it was neces-sary to create the hybrid technique described here.Our results are promising, and open up many newavenues of investigation for both classifying and pre-dicting the complex dynamics of the cellular flameexperiment.

Acknowledgements

We gratefully acknowledge the support of NSFgrant #9622642 and REU supplement for BrandonOlson during this project. We also want to thankAdele Cutler for archetype expertise, and AntonioPalacios and Kay Robbins for useful conservationsand for the raw data.

References

[1] N. Aubry, P. Holmes, J.L. Lumley, E. Stone, The dynamics ofcoherent structures in the wall region of a turbulent boundarylayer, J. Fluid Mech. 192 (1988) 115.

[2] N. Aubry, R. Guyonnet, R. Lima, Spatio-temporal analysisof complex signals: theory and applications, J. Stat. Phys. 63(1991) 683.

[3] G. Berkooz, P. Holmes, J.L. Lumley, The proper orthogonaldecomposition in the analysis of turbulent flow, Ann. Rev.Fluid Mech. 25 (75) (1993) 115.

[4] A. Cutler, Optimization methods in statistics, Ph.D. Thesis,University of California, Berkeley, CA, 1988.

[5] A. Cutler, L. Breiman, Archetypal analysis, Technometrics36 (1994) 338.

[6] A. Cutler, E. Stone, Moving archetypes, Physica D 107 (1)(1996) 1.

[7] M. Gorman, M. El-Hamdi, K.A. Robbins, Spatio-temporalchaotic dynamics of premixed flames, in: S. Vohra, M. Spano,M. Schlesinger, L. Pecora, W. Ditto (Eds.), Proceedings ofthe First Experimental Chaos Conference, World Scientific,Singapore, 1992, p. 403.

[8] M. Kirby, L. Sirovich, Application of the Karhunen–Loèveprocedure for the characterization of human faces, IEEETrans. PAMI 12 (1) (1990) 103.

[9] M. Kirby, F. Weisser, G. Dangelmayr, A model problem inthe representation of digital image sequences, Pattern Recogn.26 (1) (1993) 63.

[10] M. Kirby, D. Armbruster, Reconstructing phase-space fromPDE simulations, ZAMP 43 (1992) 999.

[11] V. Kirk, M. Silber, A competition between heteroclinic cycles,Nonlinearity 7 (1994) 1605.

[12] J.L. Lumley, The structure of inhomogeneous turbulentflows, in: A.M. Yaglom, V.I. Tatarski (Eds.), AtmosphericTurbulence and Radio Wave Propagation, Nauka, Moscow,1967, p. 166.

[13] A. Palacios, G. Gunaratne, M. Gorman, K.A. Robbins,Cellular pattern formation in circular domains, Chaos 7 (3)(1997) 463.

[14] A. Palacios, G. Gunaratne, M. Gorman, K.A. Robbins,Karhunen–Loève analysis of spatiotemporal flame patterns,Phys. Rev. E 57 (4) (1998) 1.

[15] A. Palacios, G. Gunaratne, M. Gorman, Modal decompositionof hopping states in cellular flames, Chaos 9 (3) (1999) 755.

[16] L. Sirovich, Turbulence and the dynamics of coherentstructures. Parts I–III, Quart. Appl. Math. XLV (3) (1987)561.

[17] E. Stone, A. Cutler, Archetypal analysis of spatio-temporaldynamics, Physica D 90 (1996) 209.

[18] E. Stone, A. Cutler, Introduction to archetypal analysis ofspatio-temporal dynamics, Physica D 96 (1996) 110.

[19] E. Stone, M. Gorman, M. El-Hamdi, K. Robbins,Identification of intermittent ordered patterns as heteroclinicconnections, Phys. Rev. Lett. 76 (12) (1996) 2061.