Implicit Surface Visualization of

Reconstructed Biological Molecules

Edgar Garduno and Gabor T. Herman

presented by Adriana Wise

Thursday, March 18th, 2004

Contents

*List of Slides

1 Contents

2 Introduction

6 Blob Representation/Implicit Surfaces

7 Reconstruction

9 Blobs and Grids

11 Basic Raycasting-Blobs Technique for Visualization

13 Main Blob Raycasting Algorithm

14 First Experiment Set. Visualization of Reconstructed

Molecules through Plain Blob Raycasting Algorithm

15 Improved Method to Speed Up the Blob Raycasting

Technique

16 Second Experiment Set. Visualization of Reconstructed

Molecules through Improved Blob Raycasting Algorithm

17 Conclusion

Introduction

The transmission electron microscope (TEM) captures projec-

tions from different directions of a molecule, as two-dimensional

density functions. These are processed via a reconstruction al-

gorithm, which outputs a three-dimensional electron density dis-

tribution.

(TEMs work the same way as slide projectors, except that they

shine (transmit) a beam of electrons (instead of light) through

the specimen (instead of the slide). Whatever part is transmitted

is projected onto a phosphor screen for the user to see.)

The three-dimensional density distribution is represented as a lin-

ear combination of basis functions, or blobs, chosen a priori to

have certain desired particularities, such as smoothness (differ-

entiable in all points) and zero values outside a bounded support

(their Fourier Transform are close-to-zero sufficiently far away

from the origin).

The reconstruction algorithm departs from a set of chosen such

functions and calculates the coefficients in the linear combina-

tion:

ν(x) =J∑

j=1

(cj · bj(x)) (1)

where:

x – a point in the three-dimensional space, x ∈ R3;

ν(x) – three-dimensional density function, output by ART;

J – the number of basis functions;

bj(x) – the jth basis function (blob).

The scope of this paper is to visualize the three-dimensional sur-

face describing the density function resulted from reconstruction,

through a procedure that departs directly from over a million ba-

sis functions (blobs) and produces two-dimensional projections

on the visual screen of implicit 3-D surfaces of equal density.

In contrast, the existing methods produced projections on the

visual screen based on an intermediate stage of polygonalization,

which can introduce substantial inaccuracies, both in the position

of the 3-D surface being visualized, and in the angle of normals

to that surface at various points.

The main idea behind using directly the blob-based representa-

tion of the density function for visualization as a 2-D projection is

taking “depth cues to deliver the three-dimensional information

contained in the density functions” (p.3). The method used,

namely raycasting, calculates the angles between a multitude of

rays being cast from the visual screen and the normals to the

isosurface at the points of intersection. Isosurfaces are sets of

points where the density function has a particular value. It also

uses, to a lesser extent, the lengths of these rays (i.e. distance

between screen and intersection point). This information is as-

sociated with corresponding intensities of color on the display

screen, that create the image of the 3-D object being investi-

gated.

Blob Representation/Implicit Surfaces

An implicit surface S of a density function ν is defined as

S ={x | ν(x) = t

}(2)

where:

x – a point in space, x ∈ R3

t – threshold of density, such that all points in space of densitygreater than or equal to t define the object of interest;

ν(x) – the density function.

If, in addition, the total volume V of the object is known throughother methods (as is the case with electron microscopy), then thethreshold t is uniquely determined by the additional constraintthat the surface S should enclose precisely the volume V .

Reconstruction

We reminded before that the density function can be expressed

as

ν(x) =J∑

j=1

(cj · bj(x)) (3)

A series of papers indicate that a good choice for the basis

functions bj are the generalized Kaiser-Bessel window functions,

or blobs, of the form:

b(m, α, a, r) =

Im

(α

√1−

(ra

)2)

Im(α)

(√1−

(ra

)2)m

if 0 ≤ r ≤ a,

0 otherwise.

(4)

where:

r – distance from blob center of a point x ∈ R3

whose local density is returned by corresponding bj(m, α, a, r)

(all points of equal r in blob bj have equal density);

a – the radius of the sphere that is the support of the blob;

α – blob shape parameter;

Im – modified Bessel function of order m:

Im(x) = (−i)m · Jm(ix)

Jm(x) =∞∑

n=0

(−1)n · x2n+m

n!(n + m)!22n+m

The ART phase can be summarized as follows:

1. Measurements in a number of points x ∈ R3. This is the dataacquisition stage as electron density mapping;

The following steps aim to produce a set of coefficientsc1, . . . , cJ which, when combined with the pre-selected setof basis functions b1, . . . , bJ, placed at successive grid pointsin the grid pattern selected, will compose into a function ν

resembling the originally acquired data.

2. Solve for cj the system of linear equations

yi ≈J∑

j=1

(li,j · cj) (5)

(or y = L · c), where

li,j = [Pν](−→o , x) (6)

and

[Pν](−→o , x) =∫R

ν(x + τ−→o )dτ (7)

is the ray transform of ν along vector −→o ;

3. Calculate ν(x) in xj ∈ X ⊂ R3 from grid using ν(x) =∑J

j=1 cj ·bj(x);

4. Determine surface St ={x | ν(x) = t

}such that the volume

enclosed by St is the measured volume V .

Blobs and Grids

Considering a body-centered cubic grid (bcc), the choices of the

bj parameters will relate to the bcc ∆ distance and to other

desirable basis functions properties as follows:

a – it has been empirically shown that small a values are OK.

Big blobs will not yield results that preserve concavities

in a known original 3-D object. Blobs with large support

can still be ”skinny” by choosing α appropriately.

The main problem with large support is the computational cost.

m – order of modified Bessel function m = 2;

α – argument of I2. For considerations related to

the convolution of the blob bj with a truncated version of

the train of pulses XB∆, where F{b} should be

zero-valued at the locations of the pulses in F{XB∆}

closest to origin, α should be selected as below:

α =

√2π2

( a

∆

)2− 6.9879322

Basic Raycasting-Blobs Technique for

Visualization

As briefly mentioned in the introduction, the raycasting tech-

nique consists of intersecting rays cast perpendicularly from the

viewing plane onto S, the implicit surface defined by ν(x). The

lengths of such rays hitting a portion of the implicit surface

“closest” to the viewing plane are mapped onto different color

intensities to create the illusion of a 3D object in a 2D environ-

ment.

The innovative contribution of this paper is twofold: it applies

raycasting directly to blobs, and not to polygons which appear

as an intermediate step in existing raycasting procedures. This

will significantly reduce the level of inaccuracy, already present at

the stages of data capture and reconstruction. Secondly, it pro-

poses an optimization algorithm for the rendering stage,

which the new blob raycasting technique lends itself to, and

which spectacularly reduces the display time from hours to sec-

onds.

The challenge is, then, finding points q ∈ R3 of intersection of

rays R with the implicit surface S, such that the distance between

q and the viewing screen is minimal:

minq=R

⋂S

{dist(q, screen)

}(8)

There are procedures published in the literature that guarantee

finding such intersections, where they exist. These procedures

are dependent on ν(x)’s derivatives, which are easy to compute

analytically from the formula of ν(x) =∑J

j=0 cj · bj(x), and, in

turn, bj(x) can be also analytically differentiated from the formula

of bj(m, α, a, r). However, in these procedures, the step size for

the search is inversely proportional to the large number of basis

functions contributing to the value of ν in a given point, which

makes them computationally unacceptable.

The alternative, presented in this paper, consists of preprocessing

the set of grid centers{pj

}such that, for every distinct R, all

possible intersections q = R⋂

S of R with the isosurface S are

ordered by the distance dist(q, screen). Then all but the smallest

such distances are eliminated (intuitively, thus ensuring that only

the surface that should contribute to the projection on the visual

screen is retained).

This minimal distance minR⋂

S

{dist(qt, screen)

}is determined

as the intersection point qt, nearest to the screen, encountered

along the projection ray R, that satisfies ν(q) = t (i.e. that is on

the isosurface determined by the density threshold t):

dR = minq∈R3

{dq | ν(q) = t

}(9)

In other words, instead of a set of intersections of purely geo-

metric entities (as is the case with raytracing techniques applied

to surfaces approximated through polygonalization), this paper

proposes a set of points at the intersection of rays R with im-

plicit surfaces. Further optimizations can be made to reduce

the computational complexity, by reducing the number of grid

centers whose blobs might influence the density function at the

intersection with the projection ray.

We will try to present in what follows a sketch of the algorithmic

steps considered in the new blob raytracing method.

Main Blob Raycasting Algorithm

Prior to applying the algorithm, which iterates after the number

of rays cast from the visual screen: create 1-dimensional tables,

one table per blob, mapping radial distance (of various points

inside the blob from the grid center) onto pre-calculated pairs

(bj, cj) of basis functions and appropriate coefficients for each

blob bj. Due to radial symmetry, different points x equidistant

to the grid center of bj will have the same pair (bj, cj) (the value

varying with r being bj).

The composite value of the density function for any point x ∈ R3

can be then simply calculated by the sum of the products bj(x)·cj

that are found in the look-up tables by the key x.

1. For each R, eliminate all blobs of grid centers farther than a

from R, which cannot possibly influence ν(q) along R (elim-

inate blobs with null intersection bj⋂

R = ∅);

2. Find projections of all blobs (still remaining after (1)), cut

by R, in order of closeness to the visual screen;

3. For all remaining grid points whose blobs cut R, now also

conveniently ordered, as per step (2), in decreasing distance

from the screen, evaluate ν at the intersection between R

and the normal of the grid point on R.

ray R

blob b j

grid center p j

blob b j+1

grid center p j+1

projection q j

projection q j+1

loci of points of equal density in b j (spheres)

ray R

blob b j

p j blob b j+1

p j+1

projection q j

projection q j+1

section of simple isosurface from

intersection of b j and b j+1 ; t=6, and is obtained from

local contributions of points of densities (3,3) and (1,5)

1 3 5 9

1 3 5 9

6 6

6

4. When two adjacent projections are found, let those qa and

qb, which satisfy

ν(qa) < t but ν(qb) > t, (10)

then go to next step of finding q ∈[qa; qb

]of exact t;

Note: A point x for a grid with bcc distance ∆ and blobs

of radii a is influenced by at most 51 blobs, if a and ∆ are

chosen of particular values.

5. Assuming that points of projection of adjacent grid centers

on R, qa and qb, have been found such that

ν(qa) < t but ν(qb) > t,

use the Newton-Raphson or/and bisection methods on the

interval[qa; qb

]to find points of density exactly t on R be-

tween qa and qb.

Newton-Raphson Root Finding Method — Quick Re-

minder

The N-R method finds roots of polynomials of degree ≥ 5 or

of transcendentals (e.g. cosx = x), r such that f(r) = 0.

f(x)

Y-A

xis

X-Axis

f(x 1 )

x 1

r

L

x 2

f(x 2 )

x 3

L 1

The method chooses an initial approximation of r, as x1,

randomly (or optimized) along f(x). It builds a tangent L1

to f(x) in x1 and finds x2 as L1’s intersection with the x axis.

Now x2 is closer to r than x1, and will be chosen as the next

approximation for r and so on:

f ′(x1) =f(x)− f(x1)

x− x1equation of L1

To find the x2 as the x-intercept of L1, L1(x2) = 0:

x2 = x1 −f(x1)

f ′(x1). . .

In general,

xn+1 = xn −f(xn)

f ′(xn),

with limn→∞ xn = r, if convergence exists.

In our case, finding qt ∈[qa; qb

]is possible if the fact that

ν(qa) < t ∧ ν(qb) < t indicates a local monotony of ν(q) on[qa; qb

]. This is intuitively the case if qa and qb are sufficiently

close, which is (more or less) guaranteed by the grid structureand by the small value(s) chosen for a.

In particular, if convergence exists, x− ν(x)ν′(x) converges quadrat-

ically.

If N-R fails due to non-convergence, then the bisection method(which, albeit slower, is guaranteed to converge) will be ap-plied.

The Bisection Root Finding Method — Quick Reminder

The bisection method chooses two random points x1 andx2 within f ’s domain, such that f(x1) and f(x2) are on ei-ther side of 0. Then it calculates x3 as the middle of the

interval [x1;x2], and the corresponding f(x3). Then a new

interval is chosen, such that both values in the new pair (ei-

ther(f(x1), f(x3)

), or

(f(x3), f(x2)

)) are on different sides

of 0. The iteration goes on until the interval(f(xm), f(xn)

)becomes very small.

In our case, this method presents the potential for an ad-

ditional optimization: the initial interval does not need to

be chosen as big as the difference along R between the pro-

jections[qa, qb

]. Instead, only that interval where the N-R

method was abandoned before it deconverged can be chosen

as a first approximation.

The guarantee presented by the particular properties of the

basis functions bj that at least one of the two methods will

succeed, lies in the fact that, by choice, ν(x) is continuouslydifferentiable, with gradient:

∇ν(x) =J∑

j=1

cj · ∇bj(x) (11)

(Quick reminder: If f is a function of two variables x and y,then the gradient of f is the vector function ∇f :

Duf = ∇f(x, y) =∂f

∂xi +

∂f

∂yj, (12)

where u = 〈i, j〉 is a vector. So ∇f(x, y) is the rate of changeof f in the direction of vector u = 〈i, j〉).

6. Collect all points q as found at (6) (i.e. that are projectionson R of grid centers and are of equal density t), for all projec-

tion rays R cast from the display screen, and color the projec-

tions on the screen according to the distances dist(q, screen).

The result will be a 2-D projection of all points from the 3-D

original object, of chosen equal density t, colored according

to the distance from the display screen.

First Experiment Set. Visualization of

Reconstructed Molecules through Plain Blob

Raycasting Algorithm

Experiment 1:

IN: – molecule: Escherichia Coli, DnaB and DnaC (which “dock”

during E. Coli DNA replication, and thus are of interest);

– ∆ = 12;

– a = 2.40;

– α = 13.26

OUT: – time reconstruction: 65h 22′ 48′′;

– set of coefficients cj: 1,600,065 distinct values

– time visualization: 1h 28′ 12′′

HW: – processor: Pentium IV, 2GHz

– OS: Linux

– RAM: 1GB

– resolution: 512x512

Note: Although the resolution was chosen significantly lowerthan the performance of current display monitors, it would not

have made sense to choose a higher value due to the quality

of the data from which the reconstructions were obtained (you

acquire a density map, which you then try to approximate with a

function; both the acquisition and the approximation/ART are

prone to inaccuracies). A comparison with a known shape was

not available.

Experiment 2:

IN: – molecule: Bacteria Rhodopsin, for which a protein database

(PDB) description is also available (i.e. the density data

is obtained by knowing a priori the shape of the function);

– intermediate step: the so-called “conical tilt scheme” is

used to convert the PDB data into usable input for ART.

The conical tilt scheme yields data resembling/simulatingdata obtained from the e microscope. It introduces someinaccuracy (some dispersion) of the sharp values from thePDB;

– ∆ = 12 (same as for Experiment 1);

– a = 2.40 (same);

– α = 13.26 (same)

OUT: – time reconstruction: 36h 13′ 36′′;

– set of coefficients cj: 1,482,624 distinct values

– time visualization: 1h 37′ 55′′

HW: – processor: Pentium IV, 2GHz

– OS: Linux

– RAM: 1GB

– resolution: 512x512

Preliminary conclusion: Visualization through blob raycasting,

in the absence of further optimizations, is computationally ex-

pensive due to the ν(q)− t = 0 searches for all rays R. And this

— only for one projection!!!

Improved Method to Speed Up the Blob

Raycasting Technique

Within the algorithm presented, the longest computing time is

spent by the search aimed at finding the pairs of consecutive

(adjacent) projections qa, qb for which the images through ν are

on either side of t (i.e. finding even rough initial approximations

for the N-R/bisection methods).

On the other hand, finding q ∈[qa; qb

]such that ν(q) = t is

standard, and cannot be further optimized.

The proposed optimization method is presented below:

1. Calculates for every grid center pj the local value of ν at

points pj:

vj = ν(pj)

2. Uses the set vj and the subset of grid centers for which vj ≥ t

in a “Z-buffer algorithm” that operates as follows:

for all R

length(R) ←∞for all j (i.e. for all blobs) s.t. vj ≥ t

calculate dj = dist(pj, screen)

for all R s.t. (R⋂

bj 6= ∅)R ← min

{R, dj

}

The Z-buffer algorithm approximates, for each blob j whose

bj(pj) ≥ t, the lengths of all rays that intersect it with the

distance of the grid center pj to the screen. By doing so,

it generates a good initial approximation for one of the ex-

tremities of the interval[qa; qb

](more precisely the positive

one, i.e. the one yielding ν(qa)− t ≥ 0).

The change of ν(x) to a value ≤ t will occur between adjacent

grid points, so the other extremity is likely to be found by

taking the value bj(pj+1) of one of the adjacent blobs.

3. Given the set of grid centers with vj ≥ t and their projec-

tions qc on rays intersecting bj as first approximations for

the N-R/bisection methods, apply the modified raycasting

algorithm, described as follows (the projections on R of pj

grid centers are hereafter denoted qc):

for all R s.t. ∃qc = projection of pj on R

check that ν(qc) ≥ t

if (ν(qc) � t)

do the search for[qa; qb

]around qc, by checking

values for ν of all projections

of grid centers pj on R,

(i.o.w. apply plain blob raycasting() )

else

limit search for[qa; qb

]only to points such that{

q ∈ R3 | dist(q, screen) ≤ dist(qc; screen)}

apply blob raycasting()

This is saying that once initial approximations qc have been found

through the Z-buffer algorithm, the N-R/bisection algorithm(s)

will be applied as per the following cases:around qc if ν(qc) � t,

between qc and the plane otherwise.(13)

Remember that, in the absence of the optimization, all adjacent

projections of grid centers were searched, until a pair was found

such that its density values were on different sides of t.

The savings in computation time occur once we start off with

only those projections whose density values are above t, and then

either apply N-R (which only requires one initial approximation),

or find the other extremity of the initial approximation interval

to apply bisection in the vicinity of the current qc, towards the

plane (where the function ν(qc)− t changes sign). Intuitively, the

density function is guaranteed to decrease to a value below t as

we get farther from qc and towards the plane.

Second Experiment Set. Visualization of

Reconstructed Molecules through Improved

Blob Raycasting Algorithm

Experiment 1:

IN: – molecule: Escherichia Coli;

– ∆ = 12;

– a = 2.40;

– α = 13.26

OUT: – time reconstruction: 65h 22′ 48′′ (same as Set 1);

– set of coefficients cj: 1,600,065 distinct values (same);

– Z-buffer pre-processing time: 5′′;

– time visualization: 94′′

Experiment 2:

IN: – molecule: Bacteria Rhodopsin with PDB available;

– intermediate step: “conical tilt scheme” to convert the

PDB data into input for ART including simulated ran-

domness;

– ∆ = 12;

– a = 2.40;

– α = 13.26

OUT: – time reconstruction: 36h 13′ 36′′ (same as Set 1);

– set of coefficients cj: 1,482,624 distinct values (same);

– Z-buffer pre-processing time: 6′′

– time visualization: 86′′

Conclusion

Through careful use of various improvements to the implicit sur-

face intersection search algorithm, a substantial speed-up in vi-

sualization times for data which has already been reconstructed

can be achieved (about 60 times, from hours to only seconds),

bringing such visualizations into the speed range where multiple

views and even interactive exploration become possible.