Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Hierarchical Clustering on Special Manifolds

Angelos Markos1 George Menexes2

1Democritus University of Thrace, Greece2Aristotle University of Thessaloniki, Greece

February 11th - CARME 2011

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Motivation

Let Xi, i = 1, . . . , n denote n matrices of size (m× k)such that X ′X = Ik. The rows of each Xi correspond toa low-dimensional representation of different “views” ofthe same object and the columns refer to the samevariables. The whole dataset can be seen as a collection oforthonormal bases or subspaces.

For example, suppose we have multiple images of thesame person where each image is represented by a vector.A low-dimensional representation of this image set spans asubspace in the so-called image space.

We address the problem of clustering sets of objects,where object-specific subspaces instead of vectors arecompared.We should take into account the specificgeometry of the space of orthonormal matrices.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Motivation

Let Xi, i = 1, . . . , n denote n matrices of size (m× k)such that X ′X = Ik. The rows of each Xi correspond toa low-dimensional representation of different “views” ofthe same object and the columns refer to the samevariables. The whole dataset can be seen as a collection oforthonormal bases or subspaces.

For example, suppose we have multiple images of thesame person where each image is represented by a vector.A low-dimensional representation of this image set spans asubspace in the so-called image space.

We address the problem of clustering sets of objects,where object-specific subspaces instead of vectors arecompared.We should take into account the specificgeometry of the space of orthonormal matrices.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Motivation

Let Xi, i = 1, . . . , n denote n matrices of size (m× k)such that X ′X = Ik. The rows of each Xi correspond toa low-dimensional representation of different “views” ofthe same object and the columns refer to the samevariables. The whole dataset can be seen as a collection oforthonormal bases or subspaces.

For example, suppose we have multiple images of thesame person where each image is represented by a vector.A low-dimensional representation of this image set spans asubspace in the so-called image space.

We address the problem of clustering sets of objects,where object-specific subspaces instead of vectors arecompared.We should take into account the specificgeometry of the space of orthonormal matrices.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Motivation

Let Xi, i = 1, . . . , n denote n matrices of size (m× k)such that X ′X = Ik. The rows of each Xi correspond toa low-dimensional representation of different “views” ofthe same object and the columns refer to the samevariables. The whole dataset can be seen as a collection oforthonormal bases or subspaces.

For example, suppose we have multiple images of thesame person where each image is represented by a vector.A low-dimensional representation of this image set spans asubspace in the so-called image space.

We address the problem of clustering sets of objects,where object-specific subspaces instead of vectors arecompared.We should take into account the specificgeometry of the space of orthonormal matrices.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Motivation

Let Xi, i = 1, . . . , n denote n matrices of size (m× k)such that X ′X = Ik. The rows of each Xi correspond toa low-dimensional representation of different “views” ofthe same object and the columns refer to the samevariables. The whole dataset can be seen as a collection oforthonormal bases or subspaces.

For example, suppose we have multiple images of thesame person where each image is represented by a vector.A low-dimensional representation of this image set spans asubspace in the so-called image space.

We address the problem of clustering sets of objects,where object-specific subspaces instead of vectors arecompared.We should take into account the specificgeometry of the space of orthonormal matrices.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Outline

1 Motivation

2 BackgroundManifoldsSpecial Manifolds

3 HCA on Special ManifoldsDistance & MeanIntrinsic CaseExtrinsic CaseHCA Algorithms

4 Experiments

5 Summary & Future Work

6 References

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Manifolds (Boothby, 2002)

A manifold is a topological space that is locally similar(homeomorphic) to an open set in a Euclidean space. Theshortest distance between two points is a geodesic distance.

Figure: A two dimensional manifold embedded in R3

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Differentiable Manifolds (Boothby, 2002)

For differentiable manifolds, it is possible to define thederivatives of the curves on the manifold. The derivatives at apoint X on the manifold M lie in a vector space TX , which isthe tangent space at that point.

Figure: Basic geometry of a manifold and its tangent space at a point

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Riemannian Manifolds 1/4

A Riemmanian manifold can be defined as a manifold with aninner product structure (Riemmanian metric) at each point(Chikuse, 2003). The inner product induces a norm for thetangent vectors in the tangent space.

Figure: Basic geometry of a manifold and its tangent space at a point

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Riemannian Manifolds 2/4

A geodesic is a smooth curve that locally joins their pointsalong the shortest path. The length of the geodesic is definedto be the Riemannian distance between the two points.

Figure: Basic geometry of a manifold and its tangent space at a point

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Riemannian Manifolds 3/4

The exponential map, expX : TX 7→ M , maps the vector y inthe tangent space to the point on the manifold reached by thegeodesic after unit time expX(y) = 1.

Figure: Basic geometry of a manifold and its tangent space at a point

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Riemannian Manifolds 4/4

The inverse exponential mapping logX : M 7→ TX takes thepoint Y on the manifold and returns it on the tangent space. Itis uniquely defined only around the neighborhood of the pointX.

Figure: Basic geometry of a manifold and its tangent space at a point

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Outline

1 Motivation

2 BackgroundManifoldsSpecial Manifolds

3 HCA on Special ManifoldsDistance & MeanIntrinsic CaseExtrinsic CaseHCA Algorithms

4 Experiments

5 Summary & Future Work

6 References

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Special Manifolds

We consider two differentiable manifolds with well-definedmathematical properties, the Stiefel and Grassmannmanifolds. A good introduction to their geometry can befound in Edelman et al. (1999).

In terms of their (differential) topology, the specialmanifolds can be describeda. as embedded submanifolds of the real Euclidean spaceb. as quotient spaces of the orthogonal group underdifferent equivalence relations.

The equivalence classes on special manifolds induce somenice mathematical properties and make geodesic distancecomputable.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Special Manifolds

We consider two differentiable manifolds with well-definedmathematical properties, the Stiefel and Grassmannmanifolds. A good introduction to their geometry can befound in Edelman et al. (1999).

In terms of their (differential) topology, the specialmanifolds can be describeda. as embedded submanifolds of the real Euclidean spaceb. as quotient spaces of the orthogonal group underdifferent equivalence relations.

The equivalence classes on special manifolds induce somenice mathematical properties and make geodesic distancecomputable.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Special Manifolds

We consider two differentiable manifolds with well-definedmathematical properties, the Stiefel and Grassmannmanifolds. A good introduction to their geometry can befound in Edelman et al. (1999).

In terms of their (differential) topology, the specialmanifolds can be describeda. as embedded submanifolds of the real Euclidean spaceb. as quotient spaces of the orthogonal group underdifferent equivalence relations.

The equivalence classes on special manifolds induce somenice mathematical properties and make geodesic distancecomputable.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Special Manifolds

We consider two differentiable manifolds with well-definedmathematical properties, the Stiefel and Grassmannmanifolds. A good introduction to their geometry can befound in Edelman et al. (1999).

In terms of their (differential) topology, the specialmanifolds can be describeda. as embedded submanifolds of the real Euclidean spaceb. as quotient spaces of the orthogonal group underdifferent equivalence relations.

The equivalence classes on special manifolds induce somenice mathematical properties and make geodesic distancecomputable.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Special Manifolds

We consider two differentiable manifolds with well-definedmathematical properties, the Stiefel and Grassmannmanifolds. A good introduction to their geometry can befound in Edelman et al. (1999).

In terms of their (differential) topology, the specialmanifolds can be describeda. as embedded submanifolds of the real Euclidean spaceb. as quotient spaces of the orthogonal group underdifferent equivalence relations.

The equivalence classes on special manifolds induce somenice mathematical properties and make geodesic distancecomputable.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Stiefel Manifold

Let Vk,m denote the Stiefel manifold which is the space ofk orthonormal vectors in R

m represented by the set ofm× k (k ≤ m) matrices X such that X ′X = Ik, where Ikis the k × k identity matrix (Chikuse, 2003).

For m = k, Vk,m is the orthogonal group O(m) of m×morthonormal matrices.

The Stiefel manifold may be thought of as the quotientspace O(m)/O(m− k) with respect to the group ofleft-orthogonal transformations X → HX for H ∈ O(m).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Stiefel Manifold

Let Vk,m denote the Stiefel manifold which is the space ofk orthonormal vectors in R

m represented by the set ofm× k (k ≤ m) matrices X such that X ′X = Ik, where Ikis the k × k identity matrix (Chikuse, 2003).

For m = k, Vk,m is the orthogonal group O(m) of m×morthonormal matrices.

The Stiefel manifold may be thought of as the quotientspace O(m)/O(m− k) with respect to the group ofleft-orthogonal transformations X → HX for H ∈ O(m).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Stiefel Manifold

Let Vk,m denote the Stiefel manifold which is the space ofk orthonormal vectors in R

m represented by the set ofm× k (k ≤ m) matrices X such that X ′X = Ik, where Ikis the k × k identity matrix (Chikuse, 2003).

For m = k, Vk,m is the orthogonal group O(m) of m×morthonormal matrices.

The Stiefel manifold may be thought of as the quotientspace O(m)/O(m− k) with respect to the group ofleft-orthogonal transformations X → HX for H ∈ O(m).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Stiefel Manifold

Let Vk,m denote the Stiefel manifold which is the space ofk orthonormal vectors in R

m represented by the set ofm× k (k ≤ m) matrices X such that X ′X = Ik, where Ikis the k × k identity matrix (Chikuse, 2003).

For m = k, Vk,m is the orthogonal group O(m) of m×morthonormal matrices.

The Stiefel manifold may be thought of as the quotientspace O(m)/O(m− k) with respect to the group ofleft-orthogonal transformations X → HX for H ∈ O(m).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Stiefel Manifold

Let Vk,m denote the Stiefel manifold which is the space ofk orthonormal vectors in R

m represented by the set ofm× k (k ≤ m) matrices X such that X ′X = Ik, where Ikis the k × k identity matrix (Chikuse, 2003).

For m = k, Vk,m is the orthogonal group O(m) of m×morthonormal matrices.

The Stiefel manifold may be thought of as the quotientspace O(m)/O(m− k) with respect to the group ofleft-orthogonal transformations X → HX for H ∈ O(m).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Grassmann Manifold

The Grassmann manifold Gk,m is the space whose pointsare k-dimensional linear subspaces of Rm (k-planes in R

m,containing the origin). To each k-plane ν in Gk,m

corresponds a unique m×m orthogonal projection matrixP idempotent of rank k onto ν. If the columns of anm× k matrix Y span ν, then Y Y ′ = P . (Mardia & Jupp,2009)

The Grassmann manifold can be identified by a quotientrepresentation O(m)/O(k) ×O(m− k).

Using the quotient representation of Stiefel manifolds,Gk,m = Vk,m/O(k) with respect to the group ofright-orthogonal transformations X → XH for H ∈ O(k).

We can represent subspaces using their unique orthogonalprojections

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Grassmann Manifold

The Grassmann manifold Gk,m is the space whose pointsare k-dimensional linear subspaces of Rm (k-planes in R

m,containing the origin). To each k-plane ν in Gk,m

corresponds a unique m×m orthogonal projection matrixP idempotent of rank k onto ν. If the columns of anm× k matrix Y span ν, then Y Y ′ = P . (Mardia & Jupp,2009)

The Grassmann manifold can be identified by a quotientrepresentation O(m)/O(k) ×O(m− k).

Using the quotient representation of Stiefel manifolds,Gk,m = Vk,m/O(k) with respect to the group ofright-orthogonal transformations X → XH for H ∈ O(k).

We can represent subspaces using their unique orthogonalprojections

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Grassmann Manifold

The Grassmann manifold Gk,m is the space whose pointsare k-dimensional linear subspaces of Rm (k-planes in R

m,containing the origin). To each k-plane ν in Gk,m

corresponds a unique m×m orthogonal projection matrixP idempotent of rank k onto ν. If the columns of anm× k matrix Y span ν, then Y Y ′ = P . (Mardia & Jupp,2009)

The Grassmann manifold can be identified by a quotientrepresentation O(m)/O(k) ×O(m− k).

Using the quotient representation of Stiefel manifolds,Gk,m = Vk,m/O(k) with respect to the group ofright-orthogonal transformations X → XH for H ∈ O(k).

We can represent subspaces using their unique orthogonalprojections

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Grassmann Manifold

The Grassmann manifold Gk,m is the space whose pointsare k-dimensional linear subspaces of Rm (k-planes in R

m,containing the origin). To each k-plane ν in Gk,m

corresponds a unique m×m orthogonal projection matrixP idempotent of rank k onto ν. If the columns of anm× k matrix Y span ν, then Y Y ′ = P . (Mardia & Jupp,2009)

The Grassmann manifold can be identified by a quotientrepresentation O(m)/O(k) ×O(m− k).

Using the quotient representation of Stiefel manifolds,Gk,m = Vk,m/O(k) with respect to the group ofright-orthogonal transformations X → XH for H ∈ O(k).

We can represent subspaces using their unique orthogonalprojections

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Grassmann Manifold

The Grassmann manifold Gk,m is the space whose pointsare k-dimensional linear subspaces of Rm (k-planes in R

m,containing the origin). To each k-plane ν in Gk,m

corresponds a unique m×m orthogonal projection matrixP idempotent of rank k onto ν. If the columns of anm× k matrix Y span ν, then Y Y ′ = P . (Mardia & Jupp,2009)

The Grassmann manifold can be identified by a quotientrepresentation O(m)/O(k) ×O(m− k).

Using the quotient representation of Stiefel manifolds,Gk,m = Vk,m/O(k) with respect to the group ofright-orthogonal transformations X → XH for H ∈ O(k).

We can represent subspaces using their unique orthogonalprojections

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Grassmann Manifold

The Grassmann manifold Gk,m is the space whose pointsare k-dimensional linear subspaces of Rm (k-planes in R

m,containing the origin). To each k-plane ν in Gk,m

corresponds a unique m×m orthogonal projection matrixP idempotent of rank k onto ν. If the columns of anm× k matrix Y span ν, then Y Y ′ = P . (Mardia & Jupp,2009)

The Grassmann manifold can be identified by a quotientrepresentation O(m)/O(k) ×O(m− k).

Using the quotient representation of Stiefel manifolds,Gk,m = Vk,m/O(k) with respect to the group ofright-orthogonal transformations X → XH for H ∈ O(k).

We can represent subspaces using their unique orthogonalprojections

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Grassmann Manifold

The Grassmann manifold Gk,m is the space whose pointsare k-dimensional linear subspaces of Rm (k-planes in R

m,containing the origin). To each k-plane ν in Gk,m

corresponds a unique m×m orthogonal projection matrixP idempotent of rank k onto ν. If the columns of anm× k matrix Y span ν, then Y Y ′ = P . (Mardia & Jupp,2009)

The Grassmann manifold can be identified by a quotientrepresentation O(m)/O(k) ×O(m− k).

Using the quotient representation of Stiefel manifolds,Gk,m = Vk,m/O(k) with respect to the group ofright-orthogonal transformations X → XH for H ∈ O(k).

We can represent subspaces using their unique orthogonalprojections

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Statistics on Special Manifolds - Applications

Statistics on Riemannian manifolds have found wideapplicability in Shape Analysis (Goodall & Mardia 1999,Patrangenaru & Mardia 2003).

The Grassmann manifold structure of the affine shapespace was exploited in Begelfor & Werman (2006) toperform affine invariant clustering of shapes.

Srivasatava & Klassen (2004) exploited the geometry ofthe Grassmann manifold for subspace tracking in arraysignal processing applications.

Turaga & Srivastava (2010) showed how a large class ofproblems drawn from face, activity, and object recognitioncan be recast as statistical inference problems on theStiefel and/or Grassmann manifolds.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Statistics on Special Manifolds - Applications

Statistics on Riemannian manifolds have found wideapplicability in Shape Analysis (Goodall & Mardia 1999,Patrangenaru & Mardia 2003).

The Grassmann manifold structure of the affine shapespace was exploited in Begelfor & Werman (2006) toperform affine invariant clustering of shapes.

Srivasatava & Klassen (2004) exploited the geometry ofthe Grassmann manifold for subspace tracking in arraysignal processing applications.

Turaga & Srivastava (2010) showed how a large class ofproblems drawn from face, activity, and object recognitioncan be recast as statistical inference problems on theStiefel and/or Grassmann manifolds.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Statistics on Special Manifolds - Applications

Statistics on Riemannian manifolds have found wideapplicability in Shape Analysis (Goodall & Mardia 1999,Patrangenaru & Mardia 2003).

The Grassmann manifold structure of the affine shapespace was exploited in Begelfor & Werman (2006) toperform affine invariant clustering of shapes.

Srivasatava & Klassen (2004) exploited the geometry ofthe Grassmann manifold for subspace tracking in arraysignal processing applications.

Turaga & Srivastava (2010) showed how a large class ofproblems drawn from face, activity, and object recognitioncan be recast as statistical inference problems on theStiefel and/or Grassmann manifolds.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Statistics on Special Manifolds - Applications

Statistics on Riemannian manifolds have found wideapplicability in Shape Analysis (Goodall & Mardia 1999,Patrangenaru & Mardia 2003).

The Grassmann manifold structure of the affine shapespace was exploited in Begelfor & Werman (2006) toperform affine invariant clustering of shapes.

Srivasatava & Klassen (2004) exploited the geometry ofthe Grassmann manifold for subspace tracking in arraysignal processing applications.

Turaga & Srivastava (2010) showed how a large class ofproblems drawn from face, activity, and object recognitioncan be recast as statistical inference problems on theStiefel and/or Grassmann manifolds.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Statistics on Special Manifolds - Applications

Statistics on Riemannian manifolds have found wideapplicability in Shape Analysis (Goodall & Mardia 1999,Patrangenaru & Mardia 2003).

The Grassmann manifold structure of the affine shapespace was exploited in Begelfor & Werman (2006) toperform affine invariant clustering of shapes.

Srivasatava & Klassen (2004) exploited the geometry ofthe Grassmann manifold for subspace tracking in arraysignal processing applications.

Turaga & Srivastava (2010) showed how a large class ofproblems drawn from face, activity, and object recognitioncan be recast as statistical inference problems on theStiefel and/or Grassmann manifolds.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Statistics on Special Manifolds - Applications

Statistics on Riemannian manifolds have found wideapplicability in Shape Analysis (Goodall & Mardia 1999,Patrangenaru & Mardia 2003).

The Grassmann manifold structure of the affine shapespace was exploited in Begelfor & Werman (2006) toperform affine invariant clustering of shapes.

Srivasatava & Klassen (2004) exploited the geometry ofthe Grassmann manifold for subspace tracking in arraysignal processing applications.

Turaga & Srivastava (2010) showed how a large class ofproblems drawn from face, activity, and object recognitioncan be recast as statistical inference problems on theStiefel and/or Grassmann manifolds.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Statistics on Special Manifolds - Applications

Statistics on Riemannian manifolds have found wideapplicability in Shape Analysis (Goodall & Mardia 1999,Patrangenaru & Mardia 2003).

The Grassmann manifold structure of the affine shapespace was exploited in Begelfor & Werman (2006) toperform affine invariant clustering of shapes.

Srivasatava & Klassen (2004) exploited the geometry ofthe Grassmann manifold for subspace tracking in arraysignal processing applications.

Turaga & Srivastava (2010) showed how a large class ofproblems drawn from face, activity, and object recognitioncan be recast as statistical inference problems on theStiefel and/or Grassmann manifolds.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Hierarchical Clustering on Special Manifolds

Need to define:

a distance metric applicable to subspaces (points onGk,m). Single, complete and average linkage hierarchicalclustering.

a suitable notion of the mean on Riemannian manifolds(intrinsic or extrinsic). Centroid-linkage, Ward-likeClustering (?).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Hierarchical Clustering on Special Manifolds

Need to define:

a distance metric applicable to subspaces (points onGk,m). Single, complete and average linkage hierarchicalclustering.

a suitable notion of the mean on Riemannian manifolds(intrinsic or extrinsic). Centroid-linkage, Ward-likeClustering (?).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Outline

1 Motivation

2 BackgroundManifoldsSpecial Manifolds

3 HCA on Special ManifoldsDistance & MeanIntrinsic CaseExtrinsic CaseHCA Algorithms

4 Experiments

5 Summary & Future Work

6 References

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Measuring the Distance on Manifolds

Traditional distance measures, such as the Euclideanmeasure, are not reasonable to use when measuringdistances between spaces. This point has been eithermissed or ignored in many simulation studies whereinappropriate distance measures have been used (Larsson& Villani, 2001).

The concept of principal angles is fundamental tounderstand the measure of closeness or similarity betweentwo subspaces.

Principal angles reflect the closeness of two subspaces ineach individual dimension, while subspace distances reflectthe distance of two subspaces along the Grassmannmanifold or in embedding space. Distances on Gk,m haveclear geometrical interpretation.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Measuring the Distance on Manifolds

Traditional distance measures, such as the Euclideanmeasure, are not reasonable to use when measuringdistances between spaces. This point has been eithermissed or ignored in many simulation studies whereinappropriate distance measures have been used (Larsson& Villani, 2001).

The concept of principal angles is fundamental tounderstand the measure of closeness or similarity betweentwo subspaces.

Principal angles reflect the closeness of two subspaces ineach individual dimension, while subspace distances reflectthe distance of two subspaces along the Grassmannmanifold or in embedding space. Distances on Gk,m haveclear geometrical interpretation.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Measuring the Distance on Manifolds

Traditional distance measures, such as the Euclideanmeasure, are not reasonable to use when measuringdistances between spaces. This point has been eithermissed or ignored in many simulation studies whereinappropriate distance measures have been used (Larsson& Villani, 2001).

The concept of principal angles is fundamental tounderstand the measure of closeness or similarity betweentwo subspaces.

Principal angles reflect the closeness of two subspaces ineach individual dimension, while subspace distances reflectthe distance of two subspaces along the Grassmannmanifold or in embedding space. Distances on Gk,m haveclear geometrical interpretation.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Measuring the Distance on Manifolds

Traditional distance measures, such as the Euclideanmeasure, are not reasonable to use when measuringdistances between spaces. This point has been eithermissed or ignored in many simulation studies whereinappropriate distance measures have been used (Larsson& Villani, 2001).

The concept of principal angles is fundamental tounderstand the measure of closeness or similarity betweentwo subspaces.

Principal angles reflect the closeness of two subspaces ineach individual dimension, while subspace distances reflectthe distance of two subspaces along the Grassmannmanifold or in embedding space. Distances on Gk,m haveclear geometrical interpretation.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Measuring the Distance on Manifolds

Traditional distance measures, such as the Euclideanmeasure, are not reasonable to use when measuringdistances between spaces. This point has been eithermissed or ignored in many simulation studies whereinappropriate distance measures have been used (Larsson& Villani, 2001).

The concept of principal angles is fundamental tounderstand the measure of closeness or similarity betweentwo subspaces.

Principal angles reflect the closeness of two subspaces ineach individual dimension, while subspace distances reflectthe distance of two subspaces along the Grassmannmanifold or in embedding space. Distances on Gk,m haveclear geometrical interpretation.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Principal angles

Let X and Y be two orthonormal matrices of size m by k. Theprincipal angles or canonical angles 0 ≤ θ1 ≤ . . . θm ≤ π/2between span(X) and span(Y ) are defined recursively by

cos(θk) = maxuk∈span(X)

maxvk∈span(Y )

u′kvk

subject to u′kuk = 1, v′kvk = 1,u′kui = 0, v′kvi = 0, (i = 1, . . . , k − 1).The principal angles can be computed from the SVD of X ′Y(Bjorck & Golub, 1973),

X ′Y = U(cosΘ)V ′

where U = [u1 . . . um], V = [v1 . . . vm], and cosΘ is thediagonal matrix cosΘ = diag(cos θ1 . . . cos θm).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Principal angles

Let X and Y be two orthonormal matrices of size m by k. Theprincipal angles or canonical angles 0 ≤ θ1 ≤ . . . θm ≤ π/2between span(X) and span(Y ) are defined recursively by

cos(θk) = maxuk∈span(X)

maxvk∈span(Y )

u′kvk

subject to u′kuk = 1, v′kvk = 1,u′kui = 0, v′kvi = 0, (i = 1, . . . , k − 1).The principal angles can be computed from the SVD of X ′Y(Bjorck & Golub, 1973),

X ′Y = U(cosΘ)V ′

where U = [u1 . . . um], V = [v1 . . . vm], and cosΘ is thediagonal matrix cosΘ = diag(cos θ1 . . . cos θm).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Outline

1 Motivation

2 BackgroundManifoldsSpecial Manifolds

3 HCA on Special ManifoldsDistance & MeanIntrinsic CaseExtrinsic CaseHCA Algorithms

4 Experiments

5 Summary & Future Work

6 References

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The geodesic distance

The geodesic distance or arc length is derived from the intrinsicgeometry of Grassmann manifold. It is the length of thegeodesic curve connecting two subspaces along theGrassmannian surface.

dg(X,Y ) =

(

q∑

i=1

θ2i

)1/2

= ‖θ‖2

Instead of using only the first principal angle, the full geometryof manifolds is taken into account. However, this distance isnot differentiable everywhere.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Karcher Mean

The Karcher Mean (Karcher, 1977) is an intrinsic mean onmanifolds which minimizes the sum of squared geodesic

distances.

Algorithm (Begelfor & Werman 2006)Input: Points p1, . . . , pn ∈ G(k,m), ǫ (machine zero)Output: Karcher mean q

1 Set q = p12 Find A = 1

n

∑ni=1 Logq(pi)

3 ‖A‖ < ǫ return q else, go to Step 4.

4 Find the SVD UΣV T = A and updateq → qV cos(Σ) + U sin(Σ)Go to Step 2.

The Karcher Mean is unique if the points are clustered closetogether on the manifold (Berger, 2003).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Karcher Mean

The Karcher Mean (Karcher, 1977) is an intrinsic mean onmanifolds which minimizes the sum of squared geodesic

distances.

Algorithm (Begelfor & Werman 2006)Input: Points p1, . . . , pn ∈ G(k,m), ǫ (machine zero)Output: Karcher mean q

1 Set q = p12 Find A = 1

n

∑ni=1 Logq(pi)

3 ‖A‖ < ǫ return q else, go to Step 4.

4 Find the SVD UΣV T = A and updateq → qV cos(Σ) + U sin(Σ)Go to Step 2.

The Karcher Mean is unique if the points are clustered closetogether on the manifold (Berger, 2003).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Karcher Mean

The Karcher Mean (Karcher, 1977) is an intrinsic mean onmanifolds which minimizes the sum of squared geodesic

distances.

Algorithm (Begelfor & Werman 2006)Input: Points p1, . . . , pn ∈ G(k,m), ǫ (machine zero)Output: Karcher mean q

1 Set q = p12 Find A = 1

n

∑ni=1 Logq(pi)

3 ‖A‖ < ǫ return q else, go to Step 4.

4 Find the SVD UΣV T = A and updateq → qV cos(Σ) + U sin(Σ)Go to Step 2.

The Karcher Mean is unique if the points are clustered closetogether on the manifold (Berger, 2003).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Karcher Mean

The Karcher Mean (Karcher, 1977) is an intrinsic mean onmanifolds which minimizes the sum of squared geodesic

distances.

Algorithm (Begelfor & Werman 2006)Input: Points p1, . . . , pn ∈ G(k,m), ǫ (machine zero)Output: Karcher mean q

1 Set q = p12 Find A = 1

n

∑ni=1 Logq(pi)

3 ‖A‖ < ǫ return q else, go to Step 4.

4 Find the SVD UΣV T = A and updateq → qV cos(Σ) + U sin(Σ)Go to Step 2.

The Karcher Mean is unique if the points are clustered closetogether on the manifold (Berger, 2003).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Karcher Mean

The Karcher Mean (Karcher, 1977) is an intrinsic mean onmanifolds which minimizes the sum of squared geodesic

distances.

Algorithm (Begelfor & Werman 2006)Input: Points p1, . . . , pn ∈ G(k,m), ǫ (machine zero)Output: Karcher mean q

1 Set q = p12 Find A = 1

n

∑ni=1 Logq(pi)

3 ‖A‖ < ǫ return q else, go to Step 4.

4 Find the SVD UΣV T = A and updateq → qV cos(Σ) + U sin(Σ)Go to Step 2.

The Karcher Mean is unique if the points are clustered closetogether on the manifold (Berger, 2003).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Karcher Mean

The Karcher Mean (Karcher, 1977) is an intrinsic mean onmanifolds which minimizes the sum of squared geodesic

distances.

Algorithm (Begelfor & Werman 2006)Input: Points p1, . . . , pn ∈ G(k,m), ǫ (machine zero)Output: Karcher mean q

1 Set q = p12 Find A = 1

n

∑ni=1 Logq(pi)

3 ‖A‖ < ǫ return q else, go to Step 4.

4 Find the SVD UΣV T = A and updateq → qV cos(Σ) + U sin(Σ)Go to Step 2.

The Karcher Mean is unique if the points are clustered closetogether on the manifold (Berger, 2003).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The Karcher Mean

The Karcher Mean (Karcher, 1977) is an intrinsic mean onmanifolds which minimizes the sum of squared geodesic

distances.

Algorithm (Begelfor & Werman 2006)Input: Points p1, . . . , pn ∈ G(k,m), ǫ (machine zero)Output: Karcher mean q

1 Set q = p12 Find A = 1

n

∑ni=1 Logq(pi)

3 ‖A‖ < ǫ return q else, go to Step 4.

4 Find the SVD UΣV T = A and updateq → qV cos(Σ) + U sin(Σ)Go to Step 2.

The Karcher Mean is unique if the points are clustered closetogether on the manifold (Berger, 2003).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Outline

1 Motivation

2 BackgroundManifoldsSpecial Manifolds

3 HCA on Special ManifoldsDistance & MeanIntrinsic CaseExtrinsic CaseHCA Algorithms

4 Experiments

5 Summary & Future Work

6 References

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The projection metric 1/2

When Gk,m is defined as a submanifold of Euclidean space viaa projection embedding, the projection metric on Gk,m is givenin terms of the principal angles by (Edelman et al., 1999):

dP (X,Y ) = ‖ sin θ‖2

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The projection metric 2/2

dP (X,Y ) is the distance proposed by Larsson & Villani(2001) as a measure of the distance between twocointegration spaces.

dP (X,Y ) =√

m− tr(XX ′Y Y ′) =√

m− ‖X ′Y ‖2F

Note that tr(XX ′Y Y ′) corresponds to a scalar productbetween two positive semidefinite matrices.

d2P (X,Y ) = tr(X⊥X⊥′

Y Y ′) = COI(X⊥, Y ), where COIis the co-inertia criterion, the numerator of both the RVand Tucker’s congruence coefficient for positivesemidefinite matrices (Dray, 2008).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The projection metric 2/2

dP (X,Y ) is the distance proposed by Larsson & Villani(2001) as a measure of the distance between twocointegration spaces.

dP (X,Y ) =√

m− tr(XX ′Y Y ′) =√

m− ‖X ′Y ‖2F

Note that tr(XX ′Y Y ′) corresponds to a scalar productbetween two positive semidefinite matrices.

d2P (X,Y ) = tr(X⊥X⊥′

Y Y ′) = COI(X⊥, Y ), where COIis the co-inertia criterion, the numerator of both the RVand Tucker’s congruence coefficient for positivesemidefinite matrices (Dray, 2008).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The projection metric 2/2

dP (X,Y ) is the distance proposed by Larsson & Villani(2001) as a measure of the distance between twocointegration spaces.

dP (X,Y ) =√

m− tr(XX ′Y Y ′) =√

m− ‖X ′Y ‖2F

Note that tr(XX ′Y Y ′) corresponds to a scalar productbetween two positive semidefinite matrices.

d2P (X,Y ) = tr(X⊥X⊥′

Y Y ′) = COI(X⊥, Y ), where COIis the co-inertia criterion, the numerator of both the RVand Tucker’s congruence coefficient for positivesemidefinite matrices (Dray, 2008).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The projection metric 2/2

dP (X,Y ) is the distance proposed by Larsson & Villani(2001) as a measure of the distance between twocointegration spaces.

dP (X,Y ) =√

m− tr(XX ′Y Y ′) =√

m− ‖X ′Y ‖2F

Note that tr(XX ′Y Y ′) corresponds to a scalar productbetween two positive semidefinite matrices.

d2P (X,Y ) = tr(X⊥X⊥′

Y Y ′) = COI(X⊥, Y ), where COIis the co-inertia criterion, the numerator of both the RVand Tucker’s congruence coefficient for positivesemidefinite matrices (Dray, 2008).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The projection metric 2/2

dP (X,Y ) is the distance proposed by Larsson & Villani(2001) as a measure of the distance between twocointegration spaces.

dP (X,Y ) =√

m− tr(XX ′Y Y ′) =√

m− ‖X ′Y ‖2F

Note that tr(XX ′Y Y ′) corresponds to a scalar productbetween two positive semidefinite matrices.

d2P (X,Y ) = tr(X⊥X⊥′

Y Y ′) = COI(X⊥, Y ), where COIis the co-inertia criterion, the numerator of both the RVand Tucker’s congruence coefficient for positivesemidefinite matrices (Dray, 2008).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Other Subspace distances 1/2

As a slight variation of the projection metric, we may alsoconsider the so called chordal Frobenius distance (Edelman et.al, 1999) or Procrustes metric (Chikuse, 2003), given by:

dF (X,Y ) = ‖2 sin1

2θ‖2

Note that the geodesic, projection and Procrustes metrics areasymptotically equivalent for small principal angles i.e. theseembeddings are isometries (Edelman et al., 1999).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Other Subspace distances 2/2

Max Correlation (Golub & Van Loan, 1996)

dmax(X,Y ) = ‖XX ′ − Y Y ′‖2 = sin(θ1)

dmax(X,Y ) is a distance based on only the largestcanonical correlation cos θ1 (or the smallest principal angleθ1)

Fubiny-Study metric (Edelman et al., 1999)

dFS(X,Y ) = arccos|detX ′Y | = arccos(∏

i

cos θi)

Binet-Cauchy metric (Wolf & Shashua, 2003)

dBC(X,Y ) =

(

1−∏

i

cos2 θi

)1/2

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Other Subspace distances 2/2

Max Correlation (Golub & Van Loan, 1996)

dmax(X,Y ) = ‖XX ′ − Y Y ′‖2 = sin(θ1)

dmax(X,Y ) is a distance based on only the largestcanonical correlation cos θ1 (or the smallest principal angleθ1)

Fubiny-Study metric (Edelman et al., 1999)

dFS(X,Y ) = arccos|detX ′Y | = arccos(∏

i

cos θi)

Binet-Cauchy metric (Wolf & Shashua, 2003)

dBC(X,Y ) =

(

1−∏

i

cos2 θi

)1/2

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Other Subspace distances 2/2

Max Correlation (Golub & Van Loan, 1996)

dmax(X,Y ) = ‖XX ′ − Y Y ′‖2 = sin(θ1)

dmax(X,Y ) is a distance based on only the largestcanonical correlation cos θ1 (or the smallest principal angleθ1)

Fubiny-Study metric (Edelman et al., 1999)

dFS(X,Y ) = arccos|detX ′Y | = arccos(∏

i

cos θi)

Binet-Cauchy metric (Wolf & Shashua, 2003)

dBC(X,Y ) =

(

1−∏

i

cos2 θi

)1/2

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

An Extrinsic Mean

Chikuse (2003) proposed an extrinsic mean which minimizes

the sum of squared projection distances on Gk,m:Given a set of matrices Pi = XiX

′i on Gk,m, for

Xi ∈ Vk,m, i = 1, . . . , n, a natural mean P ∈ Gk,m is definedby minimizing:

n∑

i=1

(trIk − trPiP )

Letting the spectral decomposition of S =∑n

i=1 Pi beS = HDsH

′, where H ∈ O(m),Ds = diag(s1, . . . , sm), s1 > . . . sm > 0, and puttingH = (H1H2), with H1 being m× k,we obtain P = H1H

′1 and min = kn−

∑ki=1 si.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

An Extrinsic Mean

Chikuse (2003) proposed an extrinsic mean which minimizes

the sum of squared projection distances on Gk,m:Given a set of matrices Pi = XiX

′i on Gk,m, for

Xi ∈ Vk,m, i = 1, . . . , n, a natural mean P ∈ Gk,m is definedby minimizing:

n∑

i=1

(trIk − trPiP )

Letting the spectral decomposition of S =∑n

i=1 Pi beS = HDsH

′, where H ∈ O(m),Ds = diag(s1, . . . , sm), s1 > . . . sm > 0, and puttingH = (H1H2), with H1 being m× k,we obtain P = H1H

′1 and min = kn−

∑ki=1 si.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Outline

1 Motivation

2 BackgroundManifoldsSpecial Manifolds

3 HCA on Special ManifoldsDistance & MeanIntrinsic CaseExtrinsic CaseHCA Algorithms

4 Experiments

5 Summary & Future Work

6 References

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

HCA Algorithms 1/2 (Rencher, 2002)

Single-Link. Defines the distance between any two clustersr and s as the minimum distance between them:

d(r, s) = min(d(xri, ysj)), i ∈ (1, . . . nr), i ∈ (1, . . . ns)

Complete-Link. Defines the distance as the maximum

distance between them:

d(r, s) = max(d(xri, ysj)), i ∈ (1, . . . nr), i ∈ (1, . . . ns)

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

HCA Algorithms 1/2 (Rencher, 2002)

Single-Link. Defines the distance between any two clustersr and s as the minimum distance between them:

d(r, s) = min(d(xri, ysj)), i ∈ (1, . . . nr), i ∈ (1, . . . ns)

Complete-Link. Defines the distance as the maximum

distance between them:

d(r, s) = max(d(xri, ysj)), i ∈ (1, . . . nr), i ∈ (1, . . . ns)

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

HCA Algorithms 2/2

Average. Uses the average distance between all pairs ofobjects in any two clusters:

d(r, s) =1

nrns

nr∑

i=1

ns∑

i=1

d(xri, xsj)

Centroid. Uses a Grassmannian distance between thecentroids of the two clusters, e.g.:d(r, s) = d2g(xr, xs) (squared geodesic)d(r, s) = d2P (xr, xs) (squared projection metric)where x is either the Karcher or the Extrinsic mean.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

HCA Algorithms 2/2

Average. Uses the average distance between all pairs ofobjects in any two clusters:

d(r, s) =1

nrns

nr∑

i=1

ns∑

i=1

d(xri, xsj)

Centroid. Uses a Grassmannian distance between thecentroids of the two clusters, e.g.:d(r, s) = d2g(xr, xs) (squared geodesic)d(r, s) = d2P (xr, xs) (squared projection metric)where x is either the Karcher or the Extrinsic mean.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

A Toy Example

- 15 Original matrices of size 14× 6, orthonormalized viaP = X(X ′X)1/2

- 15 Geographic Areas- 14 Crop Farming Systems- 6 Outputs and Inputs [Height, Fertilizer (MJ/ha), Labor(MJ/ha), Machinery (MJ/ha), Fuel (MJ/ha), Transportation(MJ/ha)]

- Three groups with high, medium, and low within-group, lowbetween-group correlations- Distance and Mean: a.Karcher mean with squared geodesicdistance b.squared projection metric with the Extrinsic meanExperiments were performed in Matlab,http://utopia.duth.gr/~amarkos/subspacehca

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

A Toy Example

- 15 Original matrices of size 14× 6, orthonormalized viaP = X(X ′X)1/2

- 15 Geographic Areas- 14 Crop Farming Systems- 6 Outputs and Inputs [Height, Fertilizer (MJ/ha), Labor(MJ/ha), Machinery (MJ/ha), Fuel (MJ/ha), Transportation(MJ/ha)]

- Three groups with high, medium, and low within-group, lowbetween-group correlations- Distance and Mean: a.Karcher mean with squared geodesicdistance b.squared projection metric with the Extrinsic meanExperiments were performed in Matlab,http://utopia.duth.gr/~amarkos/subspacehca

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

A Toy Example

- 15 Original matrices of size 14× 6, orthonormalized viaP = X(X ′X)1/2

- 15 Geographic Areas- 14 Crop Farming Systems- 6 Outputs and Inputs [Height, Fertilizer (MJ/ha), Labor(MJ/ha), Machinery (MJ/ha), Fuel (MJ/ha), Transportation(MJ/ha)]

- Three groups with high, medium, and low within-group, lowbetween-group correlations- Distance and Mean: a.Karcher mean with squared geodesicdistance b.squared projection metric with the Extrinsic meanExperiments were performed in Matlab,http://utopia.duth.gr/~amarkos/subspacehca

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The 20NG Dataset (1/4)

Settings of the experiment:

The 20News-Group collection consists of newsgroupdocuments that were manually classified into 20 differentcategories, related (e.g. comp.sys.ibm.pc.hardware /comp.sys.mac.hardware) or not (e.g misc.forsale /soc.religion.christian).http://people.csail.mit.edu/jrennie/20Newsgroups/.Each category includes 1, 000 documents, for a totalcollection size of about 20, 000 documents.

We consider a particular instance of a Semantic Space, theHyperspace Analogue to Language (HAL). The HAL spaceis created through the co-occurrence statistics within acorpus of documents (see Lund & Burgess, 1996).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The 20NG Dataset (1/4)

Settings of the experiment:

The 20News-Group collection consists of newsgroupdocuments that were manually classified into 20 differentcategories, related (e.g. comp.sys.ibm.pc.hardware /comp.sys.mac.hardware) or not (e.g misc.forsale /soc.religion.christian).http://people.csail.mit.edu/jrennie/20Newsgroups/.Each category includes 1, 000 documents, for a totalcollection size of about 20, 000 documents.

We consider a particular instance of a Semantic Space, theHyperspace Analogue to Language (HAL). The HAL spaceis created through the co-occurrence statistics within acorpus of documents (see Lund & Burgess, 1996).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The 20NG Dataset (2/4)

Procedure:- combine the documents of each category (20 sets)- compute the SS representation of each document set(co-occurence matrix based on the HAL model)- Agglomerative HCA (square projection metric, extrinsic mean,centroid linkage)

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The 20NG Dataset (3/4)

Results: 6 - cluster solution

C1: comp.graphics, comp.os.ms-windows.misc,comp.sys.ibm.pc.hardware, comp.sys.mac.hardware,comp.windows.xC2: rec.autos, rec.motorcycles, rec.sport.baseball,rec.sport.hockeyC3: sci.crypt, sci.electronics, sci.med, sci.spaceC4: misc.forsaleC5: talk.politics.misc, talk.politics.guns, talk.politics.mideastC6: talk.religion.misc, alt.atheism, soc.religion.christian

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

The 20NG Dataset (4/4)

Results: 4 - cluster solution

C1: comp. & sci. (computers & science)C2: talk. (politics & religion)C3: rec. (sports)C4: misc.forsale

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Summary

Affine invariance can be treated robustly and effectivelyusing a Riemmanian framework, by viewing subspaces aspoints on special manifolds.

New geometric insights in designing data analysisalgorithms that incorporate the geometry manifolds.

We reviewed actual distance measures and notions of themean naturally available on the Grassmann manifold anddefined algorithms for hierarchical clustering on theGrassmann manifold, providing empirical results.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Summary

Affine invariance can be treated robustly and effectivelyusing a Riemmanian framework, by viewing subspaces aspoints on special manifolds.

New geometric insights in designing data analysisalgorithms that incorporate the geometry manifolds.

We reviewed actual distance measures and notions of themean naturally available on the Grassmann manifold anddefined algorithms for hierarchical clustering on theGrassmann manifold, providing empirical results.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Summary

Affine invariance can be treated robustly and effectivelyusing a Riemmanian framework, by viewing subspaces aspoints on special manifolds.

New geometric insights in designing data analysisalgorithms that incorporate the geometry manifolds.

We reviewed actual distance measures and notions of themean naturally available on the Grassmann manifold anddefined algorithms for hierarchical clustering on theGrassmann manifold, providing empirical results.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Future Directions

Methods which rely on distance matrices or on centroids(e.g. MDS, k-means)

Interesting applications (further experiments)

Define a non-linear extension using the kernel trick

Relaxing the orthonormality condition (the invariantproperty is lost, a uniform way of choosing the basis isthen needed)

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Future Directions

Methods which rely on distance matrices or on centroids(e.g. MDS, k-means)

Interesting applications (further experiments)

Define a non-linear extension using the kernel trick

Relaxing the orthonormality condition (the invariantproperty is lost, a uniform way of choosing the basis isthen needed)

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Future Directions

Methods which rely on distance matrices or on centroids(e.g. MDS, k-means)

Interesting applications (further experiments)

Define a non-linear extension using the kernel trick

Relaxing the orthonormality condition (the invariantproperty is lost, a uniform way of choosing the basis isthen needed)

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Future Directions

Methods which rely on distance matrices or on centroids(e.g. MDS, k-means)

Interesting applications (further experiments)

Define a non-linear extension using the kernel trick

Relaxing the orthonormality condition (the invariantproperty is lost, a uniform way of choosing the basis isthen needed)

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 1/4

Barg, A. and Nogin, D. (2002). Bounds on packings ofspheres in the Grassmann manifold. IEEE Trans.

Information Theory, 48(9), 2450-2454.

Begelfor E. & Werman M. (2006). Affine invariancerevisited. In Proc. IEEE Conf. on Computer Vision andPattern Recognition, New York, NY, 2, 2087-2094.

Berger, M. (2003). A Panoramic View of Riemannian

Geometry. Springer, Berlin.

Bjorck, A. & Golub, G.H. (1973) Numerical methods forcomputing the angles between linear subspaces, Math.

Comp. 27, 579-594

Boothby, W.M. (2002). An Introduction to Differentiable

Manifolds and Riemannian Geometry. Academic Press,2002.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 1/4

Barg, A. and Nogin, D. (2002). Bounds on packings ofspheres in the Grassmann manifold. IEEE Trans.

Information Theory, 48(9), 2450-2454.

Begelfor E. & Werman M. (2006). Affine invariancerevisited. In Proc. IEEE Conf. on Computer Vision andPattern Recognition, New York, NY, 2, 2087-2094.

Berger, M. (2003). A Panoramic View of Riemannian

Geometry. Springer, Berlin.

Bjorck, A. & Golub, G.H. (1973) Numerical methods forcomputing the angles between linear subspaces, Math.

Comp. 27, 579-594

Boothby, W.M. (2002). An Introduction to Differentiable

Manifolds and Riemannian Geometry. Academic Press,2002.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 1/4

Barg, A. and Nogin, D. (2002). Bounds on packings ofspheres in the Grassmann manifold. IEEE Trans.

Information Theory, 48(9), 2450-2454.

Begelfor E. & Werman M. (2006). Affine invariancerevisited. In Proc. IEEE Conf. on Computer Vision andPattern Recognition, New York, NY, 2, 2087-2094.

Berger, M. (2003). A Panoramic View of Riemannian

Geometry. Springer, Berlin.

Bjorck, A. & Golub, G.H. (1973) Numerical methods forcomputing the angles between linear subspaces, Math.

Comp. 27, 579-594

Boothby, W.M. (2002). An Introduction to Differentiable

Manifolds and Riemannian Geometry. Academic Press,2002.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 1/4

Barg, A. and Nogin, D. (2002). Bounds on packings ofspheres in the Grassmann manifold. IEEE Trans.

Information Theory, 48(9), 2450-2454.

Begelfor E. & Werman M. (2006). Affine invariancerevisited. In Proc. IEEE Conf. on Computer Vision andPattern Recognition, New York, NY, 2, 2087-2094.

Berger, M. (2003). A Panoramic View of Riemannian

Geometry. Springer, Berlin.

Bjorck, A. & Golub, G.H. (1973) Numerical methods forcomputing the angles between linear subspaces, Math.

Comp. 27, 579-594

Boothby, W.M. (2002). An Introduction to Differentiable

Manifolds and Riemannian Geometry. Academic Press,2002.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 1/4

Barg, A. and Nogin, D. (2002). Bounds on packings ofspheres in the Grassmann manifold. IEEE Trans.

Information Theory, 48(9), 2450-2454.

Begelfor E. & Werman M. (2006). Affine invariancerevisited. In Proc. IEEE Conf. on Computer Vision andPattern Recognition, New York, NY, 2, 2087-2094.

Berger, M. (2003). A Panoramic View of Riemannian

Geometry. Springer, Berlin.

Bjorck, A. & Golub, G.H. (1973) Numerical methods forcomputing the angles between linear subspaces, Math.

Comp. 27, 579-594

Boothby, W.M. (2002). An Introduction to Differentiable

Manifolds and Riemannian Geometry. Academic Press,2002.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 2/4

Conway, J., Hardin, R. & Sloane, N. (1996). Packinglines, planes, etc.: Packings in Grassmannian spaces.Experimental Mathematics, 5, 139-159.

Edelman, A., Arias, T. & Smith, S. (1998). The geometryof algorithms with orthogonality constrains. SIAM J.

Matrix Anal. Appl, 20(2), 303-353.

Goodall, C. R. & Mardia, K.V. (1999). Projective shapeanalysis, Journal of Computational and Graphical

Statistics, 8(2), 143-168.

Karcher, H. (1977). Riemannian center of mass andmollifier smoothing, Communications on Pure and Applied

Mathematics, 30, 509-541.

Larsson, R. & Villani, M. (2001). A distance measurebetween cointegration spaces, Economics Letters, 70(1),21–27.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 2/4

Conway, J., Hardin, R. & Sloane, N. (1996). Packinglines, planes, etc.: Packings in Grassmannian spaces.Experimental Mathematics, 5, 139-159.

Edelman, A., Arias, T. & Smith, S. (1998). The geometryof algorithms with orthogonality constrains. SIAM J.

Matrix Anal. Appl, 20(2), 303-353.

Goodall, C. R. & Mardia, K.V. (1999). Projective shapeanalysis, Journal of Computational and Graphical

Statistics, 8(2), 143-168.

Karcher, H. (1977). Riemannian center of mass andmollifier smoothing, Communications on Pure and Applied

Mathematics, 30, 509-541.

Larsson, R. & Villani, M. (2001). A distance measurebetween cointegration spaces, Economics Letters, 70(1),21–27.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 2/4

Conway, J., Hardin, R. & Sloane, N. (1996). Packinglines, planes, etc.: Packings in Grassmannian spaces.Experimental Mathematics, 5, 139-159.

Edelman, A., Arias, T. & Smith, S. (1998). The geometryof algorithms with orthogonality constrains. SIAM J.

Matrix Anal. Appl, 20(2), 303-353.

Goodall, C. R. & Mardia, K.V. (1999). Projective shapeanalysis, Journal of Computational and Graphical

Statistics, 8(2), 143-168.

Karcher, H. (1977). Riemannian center of mass andmollifier smoothing, Communications on Pure and Applied

Mathematics, 30, 509-541.

Larsson, R. & Villani, M. (2001). A distance measurebetween cointegration spaces, Economics Letters, 70(1),21–27.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 2/4

Conway, J., Hardin, R. & Sloane, N. (1996). Packinglines, planes, etc.: Packings in Grassmannian spaces.Experimental Mathematics, 5, 139-159.

Edelman, A., Arias, T. & Smith, S. (1998). The geometryof algorithms with orthogonality constrains. SIAM J.

Matrix Anal. Appl, 20(2), 303-353.

Goodall, C. R. & Mardia, K.V. (1999). Projective shapeanalysis, Journal of Computational and Graphical

Statistics, 8(2), 143-168.

Karcher, H. (1977). Riemannian center of mass andmollifier smoothing, Communications on Pure and Applied

Mathematics, 30, 509-541.

Larsson, R. & Villani, M. (2001). A distance measurebetween cointegration spaces, Economics Letters, 70(1),21–27.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 2/4

Conway, J., Hardin, R. & Sloane, N. (1996). Packinglines, planes, etc.: Packings in Grassmannian spaces.Experimental Mathematics, 5, 139-159.

Edelman, A., Arias, T. & Smith, S. (1998). The geometryof algorithms with orthogonality constrains. SIAM J.

Matrix Anal. Appl, 20(2), 303-353.

Goodall, C. R. & Mardia, K.V. (1999). Projective shapeanalysis, Journal of Computational and Graphical

Statistics, 8(2), 143-168.

Karcher, H. (1977). Riemannian center of mass andmollifier smoothing, Communications on Pure and Applied

Mathematics, 30, 509-541.

Larsson, R. & Villani, M. (2001). A distance measurebetween cointegration spaces, Economics Letters, 70(1),21–27.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 3/4

Lund, K., Burgess, C. (1996). Producing High-dimensionalSemantic Spaces from Lexical Co-occurrence. BehaviorResearch Methods 28(2), 203-208.

Rencher, A.C. (2002). Methods of Multivariate Analysis,2nd ed. New York: John Wiley & Sons.

Srivasatava, A. & Klassen, E. (2004). Bayesian geometricsubspace tracking, Advances in Applied Probability, 36,43–56.

Wang, L., Wang, X., & Feng, J. (2006). SubspaceDistance Analysis with Application to Adaptive BayesianAlgorithm for Face Recognition. Pattern Recognition,39(3).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 3/4

Lund, K., Burgess, C. (1996). Producing High-dimensionalSemantic Spaces from Lexical Co-occurrence. BehaviorResearch Methods 28(2), 203-208.

Rencher, A.C. (2002). Methods of Multivariate Analysis,2nd ed. New York: John Wiley & Sons.

Srivasatava, A. & Klassen, E. (2004). Bayesian geometricsubspace tracking, Advances in Applied Probability, 36,43–56.

Wang, L., Wang, X., & Feng, J. (2006). SubspaceDistance Analysis with Application to Adaptive BayesianAlgorithm for Face Recognition. Pattern Recognition,39(3).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 3/4

Lund, K., Burgess, C. (1996). Producing High-dimensionalSemantic Spaces from Lexical Co-occurrence. BehaviorResearch Methods 28(2), 203-208.

Rencher, A.C. (2002). Methods of Multivariate Analysis,2nd ed. New York: John Wiley & Sons.

Srivasatava, A. & Klassen, E. (2004). Bayesian geometricsubspace tracking, Advances in Applied Probability, 36,43–56.

Wang, L., Wang, X., & Feng, J. (2006). SubspaceDistance Analysis with Application to Adaptive BayesianAlgorithm for Face Recognition. Pattern Recognition,39(3).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 3/4

Lund, K., Burgess, C. (1996). Producing High-dimensionalSemantic Spaces from Lexical Co-occurrence. BehaviorResearch Methods 28(2), 203-208.

Rencher, A.C. (2002). Methods of Multivariate Analysis,2nd ed. New York: John Wiley & Sons.

Srivasatava, A. & Klassen, E. (2004). Bayesian geometricsubspace tracking, Advances in Applied Probability, 36,43–56.

Wang, L., Wang, X., & Feng, J. (2006). SubspaceDistance Analysis with Application to Adaptive BayesianAlgorithm for Face Recognition. Pattern Recognition,39(3).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 4/4

Wolf, L., & Shashua, A. (2003). Learning over sets usingkernel principal angles. J. Mach. Learn. Res., 4, 913-931.

Wong, Y.-C. (1967). Differential geometry of Grassmannmanifolds. Proc. of the Nat. Acad. of Sci., 57, 589-594.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

References 4/4

Wolf, L., & Shashua, A. (2003). Learning over sets usingkernel principal angles. J. Mach. Learn. Res., 4, 913-931.

Wong, Y.-C. (1967). Differential geometry of Grassmannmanifolds. Proc. of the Nat. Acad. of Sci., 57, 589-594.

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Thank you for your attention!

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Bonus Slide - The HAL Model

A window of text is passed over each document in thecollection in order to capture co-occurrences of words.The length of the window is set to l: a typical value of l is10; different values capture different levels of relationshipbetween words.Words that co-occur into a window do so with a strengthinversely proportional to the distance between the twoco-occurring words. By sliding the window over the wholecollection and recording the co-occurrence values, aco-occurrence matrix A can be created.Since in our approach, we are not interested in the orderof the co-occurrences therefore we can compute asymmetric matrix by means of S = AA′ and thenorthonormalize the columns.For a similar experiment see Zuccon (2009).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Bonus Slide - The HAL Model

A window of text is passed over each document in thecollection in order to capture co-occurrences of words.The length of the window is set to l: a typical value of l is10; different values capture different levels of relationshipbetween words.Words that co-occur into a window do so with a strengthinversely proportional to the distance between the twoco-occurring words. By sliding the window over the wholecollection and recording the co-occurrence values, aco-occurrence matrix A can be created.Since in our approach, we are not interested in the orderof the co-occurrences therefore we can compute asymmetric matrix by means of S = AA′ and thenorthonormalize the columns.For a similar experiment see Zuccon (2009).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Bonus Slide - The HAL Model

A window of text is passed over each document in thecollection in order to capture co-occurrences of words.The length of the window is set to l: a typical value of l is10; different values capture different levels of relationshipbetween words.Words that co-occur into a window do so with a strengthinversely proportional to the distance between the twoco-occurring words. By sliding the window over the wholecollection and recording the co-occurrence values, aco-occurrence matrix A can be created.Since in our approach, we are not interested in the orderof the co-occurrences therefore we can compute asymmetric matrix by means of S = AA′ and thenorthonormalize the columns.For a similar experiment see Zuccon (2009).

Markos & Menexes Hierarchical Clustering on Special Manifolds

Hierarchical

Clustering on

Special

Manifolds

Markos &

Menexes

Motivation

Background

Manifolds

SpecialManifolds

HCA on

Special

Manifolds

Distance &Mean

Intrinsic Case

Extrinsic Case

HCA Algorithms

Experiments

Summary &

Future Work

References

Bonus Slide - The HAL Model

A window of text is passed over each document in thecollection in order to capture co-occurrences of words.The length of the window is set to l: a typical value of l is10; different values capture different levels of relationshipbetween words.Words that co-occur into a window do so with a strengthinversely proportional to the distance between the twoco-occurring words. By sliding the window over the wholecollection and recording the co-occurrence values, aco-occurrence matrix A can be created.Since in our approach, we are not interested in the orderof the co-occurrences therefore we can compute asymmetric matrix by means of S = AA′ and thenorthonormalize the columns.For a similar experiment see Zuccon (2009).

Markos & Menexes Hierarchical Clustering on Special Manifolds