J Math Imaging Vis (2010) 38: 83–94DOI 10.1007/s10851-010-0214-6

Estimating the Duration of Overlapping Events from ImageSequences Using Cylindrical Temporal Boolean Models

María Elena Díaz · Guillermo Ayala · Ester Díaz

Published online: 20 July 2010© Springer Science+Business Media, LLC 2010

Abstract Recent advances in microscopy jointly to the de-velopment of fluorescent probes have enabled to image dy-namic processes with very high spatial-temporal resolution,for instance in Cell Biology. In some applications, the seg-mented areas associated with different events overlap spa-tially and temporally forming random clumps. In order tostudy the shape-size features and durations of the events, itis a usual practice to analyze only isolated episodes. How-ever, this sample is biased, because faster and smaller eventstend to be isolated. We model the images as a realizationof a cylindrical temporal Boolean model. We evaluate thebias introduced when ruling out non-isolated episodes. Wepropose an estimator of the duration distribution and per-form a simulation study to assess its accuracy. The methodis applied to fluorescent-tagged proteins image sequences.Results show that this procedure is effective for analyzing

This paper has been supported by Human Frontier ScienceOrganization Program (RGY40/2003-first author) and SpanishMinistry of Science and Education (TIN2007-67587-first author andTIN2006-10143-second author). The authors thank one of thereferees, who also reviewed our paper [1], for his/her suggestionswhich contributed to the genesis of this paper. We would like to thankthe anonymous reviewer who made a substantial contribution to therevision of our paper.

M.E. Díaz (�) · E. DíazComputer Science Department, University of Valencia,Avda. Vicent Andrés Estellés, 1, 46100 Burjasot, Spaine-mail: elena.diaz@uv.es

E. Díaze-mail: ester.diaz@uv.es

G. AyalaDepartamento de Estadística e Investigación Operativa,University of Valencia, Avda. Vicent Andrés Estellés, 1,46100 Burjasot, Spaine-mail: guillermo.ayala@uv.es

dynamic processes where spatial and temporal overlappingoccurs.

Keywords Temporal Boolean model · Temporal randomsets · Covariance · Image analysis

1 Introduction

The problem of estimating shape-size characteristics ofoverlapping objects from images is common in many realapplications. Boolean models explicitly consider and as-sume this overlap, providing a good description for very ir-regular patterns observed in Microscopy, Material Sciences,Biology, Medicine, Chemistry, Geostatistics, Cellular Com-munications Networks or Image Processing, since they for-malize the configuration of independent and randomly lo-cated particles. What is observed is a pattern of overlap-ping random shapes, as shown in Fig. 1. More precisely,a Boolean model is a random closed set consisting of aPoisson point process producing the germs coupled with anindependent random shape process, i.e. a sequence of in-dependent and identically distributed random closed sets,the grains. The connected components made of overlappingshapes are called clumps. Definitions and statistical analy-sis of these models can be found in [6–8, 13, 14]. Previousapplications of Boolean models within Image Processing, inparticular for texture analysis and classification, are [2, 4, 5].

In spite of the considerable number and variety of appli-cations of Boolean models, the problem of analyzing over-lapping events in space and time from image sequences re-mains elusive. More sophisticated mathematical models andstatistical methods are needed, i.e. hybrid models that cap-ture both time and geometric properties while formalizinga configuration of independent randomly placed particles

84 J Math Imaging Vis (2010) 38: 83–94

with random durations. Figure 1 displays several consecu-tive frames of a simulated model where discs are randomlylocated in space and time with random radii and random du-rations (see Video1 in supplementary material). Figure 2 de-picts a spatial-temporal reconstruction of the model. Thesesophisticated data require specialized methods for effectiveanalysis, in particular for extracting useful information andfor the interpretation of results.

In applied fields, such as Cell Biology, the estimation ofshape-size features and durations of overlapping events fromimage sequences is often confined to visual inspection orlimited statistical analysis typically done manually on a one-

Fig. 1 Several consecutive frames of a simulated temporal Booleanmodel with cylindrical grains. Arrows point to an isolated episode

Fig. 2 A spatial-temporal reconstruction of a cylindrical temporalBoolean model

by-one basis of isolated events, those that can be completelyobserved from the beginning to the end of their lifetimes(see Fig. 1). However, this procedure presents several draw-backs. First, each experiment involves dozens of sequenceswith thousands of frames each, making manual analysis im-possible in large image sequences. Second, manual markingof isolated events is somewhat subjective. Third, the use ofonly isolated events leads to a biased sample, i.e. one thatincludes smaller and shorter events with higher probability.The evaluation of this bias is not possible without the as-sumption of a stochastic model about the (random) mecha-nism generating the locations, times of occurrence and du-rations of the events. In this paper, we define a stochasticmodel, the cylindrical temporal Boolean model, which willallow us to assess the bias introduced when excluding non-isolated events. We will propose an estimator of the cumula-tive distribution function of the duration of the events basedon the spatial-temporal covariance of random sets.

The procedure will be applied to fluorescent-tagged en-docytic proteins image sequences in order to estimate theduration of endocytosis, a cellular mechanism by whichcells carry traffic from the plasma membrane to internalcompartments. Specialized microscopy techniques such asTotal Internal Reflection Fluorescence Microscopy (TIRFM)[15] joint to the development of synthetic fluorescent probeshave made a strong contribution in Cell Biology, in partic-ular in live-cell imaging. This technique illuminates a thinsection near the cell-coverslip interface and gives a very highsignal-to-noise ratio, thus facilitating visualization of cellu-lar processes near the plasma membrane. Under TIRFM, theassembly of fluorescently-labeled proteins at a site of ongo-ing endocytosis results in the appearance and steady growthof a diffraction-limited spot, in such a way that the areas offluorescence of endocytic events overlap, forming randomclumps in space and time (see Fig. 3 and Video Cell1 insupplementary material).

The outline of this paper is as follows: Sect. 2 containsthe definition of the cylindrical temporal Boolean model andsome theoretical results on the statistical analysis of isolatedevents. In Sect. 3, a semi-parametric estimator of the distrib-ution function of the duration based on the spatial-temporalcovariance of random sets is introduced. Section 4 is devotedto the results. Finally, conclusions and further developmentsare summarized in Sect. 5.

Fig. 3 Several snapshots of afluorescent-tagged proteinimage sequence

J Math Imaging Vis (2010) 38: 83–94 85

2 The Model

First, let us introduce some notation and definitions. We willdenote B(0, r) a disc centered at the origin with radius r ,S0(r) its boundary and νd the Lebesgue measure in R

d (ν2 isthe area in R

2 and ν3 the volume in R3). Given two sets

A and B , the Minkowski addition is defined as A ⊕ B ={a + b : a ∈ A,b ∈ B}.

Definition 1 (Cylindrical Temporal Boolean Model) Let� = {(xi, ti )}i≥1 be a stationary Poisson point process inR

2 × R+ with intensity λ. Let {(Ri,Di)}i≥1 be a sequenceof independent and identically distributed non-negative ran-dom vectors (as (R,D)) with joint density f . We as-sume that � and {(Ri,Di)}i≥1 are independent and thatEν3(B(0,R) × [0,D] ⊕ K) < +∞ for any compact sub-set K of R

2 × R+. The cylindrical temporal Boolean modelis the random set defined as

� =⋃

i≥1

B(0,Ri) + xi × [ti , ti + Di], (1)

where B(0,Ri) + xi is the translation of B(0,Ri) at xi . Theset B(0,Ri)+xi ×[ti , ti +Di] is a cylinder in R

2 ×R+, thei-th event. This model is a particular case of the temporalBoolean model [1, 12]. Figure 1 displays some consecutivetemporal cross-sections of a simulated model. The union ofthese cylinders is a realization of the model (see Fig. 2).

The probability that a compact subset K of R2 × R+ is

contained in the complement of � is given by

Q(K) = P(K ⊂ �c)

= exp( − λEν3

[(B(0,R) × [0,D]) ⊕ K

]), (2)

where K is the symmetric of K with respect to the origin. Ifwe take K = {0} then

p = P(0 ∈ �) = 1 − Q({0})= 1 − exp{−λπE(R2D)}. (3)

This probability p is called the volume fraction of � and itcorresponds to the mean volume covered by the random setper unit area and unit time.

2.1 The Duration Distribution of Isolated Events

Let us assume that the joint distribution of (R,D), the ran-dom radius and duration of a typical grain, has a densityf (r, d; θ) where θ are the parameters within a parametricspace �. However, due to the overlapping among shapes, wecannot observe all episodes. We can only segment the iso-lated events, i.e those connected components with a smoothboundary, which are either not overlapped by any other

event or are hit by another one (or other ones) in such a waythat the hitting events lie within it. Such as isolated eventscan be completely observed from the beginning to the endof their lifetimes as a cylindrical shape.

Let us denote by (R∗,D∗) the random vector associatedwith an arbitrary isolated event and f ∗ be its correspondingdensity function. In order to study the bias in the estimationof the durations using a sample composed of isolated events,it is necessary to relate the densities f and f ∗. Note thatf corresponds to the observable and non-observable eventsand f ∗ to the isolated (and therefore observable) events.The case of the static two-dimensional cylindrical Booleanmodel was studied in [3].

Theorem 1 Let � be a cylindrical temporal Boolean modelwith intensity λ. Let (R,D) (respectively (R∗,D∗)) be therandom radius and duration of the typical grain (the typicalisolated grain) of �. If (R,D) has density f (r, d; θ) thenthe density function of (R∗,D∗) verifies

f ∗(r, d;λ, θ) ∝ f (r, d; θ)p(r, d;λ, θ), (4)

where

p(r, d;λ, θ)

= exp

(− λ

∫ +∞

0

∫ +∞

0C(u, v; r, d)f (u, v; θ)dudv

).

(5)

The function C is defined as C(u, v; r, d) = ν3(B(0, u)×[0, v] ⊕ S0(r) × [0, d]) and it is equal to

C(u, v; r, d) =

⎧⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎩

π(u + r)2(v + d)

if u ≥ r or v ≥ d,

2π(r − u)2v + 4πur(v + d),

if u < r and v < d.

(6)

The function p(r, d;λ, θ) gives the probability that a cylin-der of radius r and duration d remains as an isolated eventin the observed pattern. This is equal to the probability thatits boundary is not hit by any other grain, i.e. it belongs tothe complement of �.

Proof If (R,D) = (r, d), the probability p(r, d;λ, θ) thatthe boundary of the cylinder S0(r)×[0, d] is not covered byany other event can be obtained from (2) as

p(r, d;λ, θ)

= exp{−λEθ

[ν3

(B(0,R) × [0,D] ⊕ S0(r) × [0, d])]}.

(7)

When considering hits by a cylinder of radius u ≥ r or du-ration v ≥ d , it holds that

B(0, u)×[0, v]⊕ S0(r)×[0, d] = B(0, u+ r)×[0, v + d].

86 J Math Imaging Vis (2010) 38: 83–94

Fig. 4 Estimated densities f ∗ of the random duration D∗ for different remaining sample fractions. From left to right: uniform distribution U(4,8),Gamma distribution Ga(18,0.3333) and normal distribution N(6,1). Durations are in unit time

If u < r and v < d , we have

B(0, u) × [0, v] ⊕ S0(r) × [0, d]= [(

B(0, u + r)\B(0, u − r)) × [0, v + d]]

∪ [B(0, u + r) × ([0, v + d] ∪ [d − v, d])] (8)

and the proof is completed. �

Although we have assumed cylindrical shapes for thegrains, the respective expressions for other kind of randomshapes can be likewise derived.

The remaining sample fraction, defined as the ratio of iso-lated events with respect to all the grains, is given by

δ(λ, θ) =∫ +∞

0

∫ +∞

0p(u, v;λ, θ)f (u, v; θ)dudv. (9)

We have no closed expression for the density f ∗(·, ·;λ, θ).However, Theorem 1 opens a simple way to generate ran-dom samples of f ∗. First, we generate (R,D) = (r, d) withdensity f (·, ·;λ, θ). Second, every generated pair (r, d) isaccepted with probability p(r, d;λ, θ) or rejected with prob-ability 1 − p(r, d;λ, θ). The accepted (r, d) pairs have den-sity function f ∗(·, ·;λ, θ). By taking into account that

p(r, d;λ, θ) = exp(−λE[C(R,D; r, d)]),we can obtain an estimate of p(r, d;λ, θ) by computingE[C(R,D, r, d)].

Let (R1,D1), . . . , (Rm,Dm) be a random sample withdensity f , then the Monte Carlo estimator of E[C(R,D, r, d)]is given by

E[C(R,D; r, d)] =m∑

i=1

C(Ri,Di; r, d)

m.

In summary, we will consider the estimator

p(r, d;λ, θ) = exp

(− λ

m∑

i=1

C(Ri,Di; r, d)

m

). (10)

Given a model with parameters λ and θ , we can generatesamples of isolated episodes with durations D∗ for fixed λ

and θ , say d∗1 , . . . , d∗

m, and we can compute the error in theestimation of the distribution function of the duration.

2.2 Evaluating the Marginal Duration Distributionof Isolated Events

In this subsection, the original density of the durationsconsidering all the events f and the density of the iso-lated episodes f ∗ are compared. Three cylindrical tempo-ral Boolean models were simulated corresponding to threeduration distributions with mean 6: uniform distributionU(4,8), Gamma distribution Ga(18,0.3333) and normaldistribution N(6,1). The radii of the grains were uniformlydistributed in the interval [4,8] pixels. The durations and theradii were independently generated.

Figure 4 shows the estimates of f ∗ for the three models.The solid line corresponds to f , the dashed line represents aremaining sample fraction δ of around 58%, the dotted lineof approximately 20% and the dashed-dotted line of around10%. Note how the densities tend to be concentrated in theleft side of the interval. The higher the remaining samplefraction is, the smaller the bias is.

2.3 Error in the Estimation of the Mean Duration UsingIsolated Events

The relative error of the mean duration, defined as the dif-ference between the estimated and theoretical values withrespect to the theoretical value, were calculated as a func-tion of the remaining sample fraction, δ. We used the samemodels as in the previous subsection. We considered valuesof the intensity of the germ process ranging in the interval[0.0001,0.001] with a step of 0.0001 resulting in 10 valuesof δ ranging from 0 to 0.8. A total number of 40 replicatesfor each value of δ and model were simulated. Figure 5 dis-plays the 0.05 and 0.95 percentiles (dotted lines) and the

J Math Imaging Vis (2010) 38: 83–94 87

Fig. 5 Relative errors in the estimation of the mean duration as a function of δ. From left to right: uniform distribution U(4,8); Gamma distributionGa(18,0.3333); and normal distribution N(6,1)

median (solid line) of the relative error (in percentage) com-puted from the 40 realizations. When the remaining samplefraction is low, i.e. there are many grains overlapped, the rel-ative errors are high. For the uniform distribution the erroris lesser than 9%, for the Gamma distribution is approxi-mately 12% and for the normal distribution is around 6%.These values could be unacceptable in many real applica-tions. In Sect. 3, we propose an estimator of the cumulativedistribution function of the duration of a cylindrical tempo-ral Boolean model using all the events, i.e the observableand non-observable episodes.

3 The Covariance of a Cylindrical Temporal BooleanModel

In this section we present an estimator of the cumulative dis-tribution function of the duration based on the covariance ofa cylindrical temporal Boolean model. We define the spatial-temporal covariance of a (spatial-temporal) stationary ran-dom set as

C(h, t) = P(0 ∈ �(0),h ∈ �(t)). (11)

For a given h ∈ R2 and a given time t ∈ R+, this function

gives the probability that an arbitrary point at an arbitrarytime and the translated one by a vector h and a time t belongto the random set. The spatial origin 0 denotes an arbitrarylocation and the temporal origin 0 denotes an arbitrary time.Figure 6 provides a geometrical interpretation. If the distri-bution of � is invariant against spatial translations then C

only depends on the modulus h, h = ‖h‖.

Proposition 1 Let � be a cylindrical temporal Booleanmodel � with intensity λ and primary grain �0 = B(0,R)×[0,D], where the radii and durations are independent. LetfD be the density of the durations. The spatial-temporal co-variance of � is

C(h, t) = (2p − 1) + (1 − p)2 exp(λCB(h)L(t)

), (12)

Fig. 6 The spatial-temporal covariance of a temporal random set

where CB(h) = Eν2(B(0,R)∩B(0,R)+ h), L(t) = ED −∫ t

0 uFD(du) − tP (D ≥ t) and FD is the cumulative distrib-ution function of the duration.

Proof A cylindrical temporal Boolean model is a particularcase of a non-isotropic three-dimensional Boolean model.By taking into account the general expression of the covari-ance [6, 13, 14], we have

C(h, t) = 2p − 1

+ (1 − p)2 exp{λEν3

(�0 ∩ �0 + (h, t)

)}. (13)

Let us denote A0 = B(0,R), then �0 ∩ �0 + (h, t) = (A0 ∩A0 + h) × ([0,D] ∩ [t, t + D]) if t < D and ∅ otherwise. Itholds that

Eν3(�0 ∩ �0 + (h, t)

)

= EA0,D

(ν2

(A0 ∩ A0 + h

) × ν([0,D] ∩ [t,D + t]))

= EA0ν2(A0 ∩ A0 + h

)EDν

([0,D] ∩ [t,D + t]), (14)

because A0 and D are independent. Finally,

L(t) = EDν([0,D] ∩ [t,D + t])

=∫ +∞

t

(u − t)FD(du)

= ED −∫ t

0uFD(du) − tP (D ≥ t). (15)

�

88 J Math Imaging Vis (2010) 38: 83–94

If we assume that fD exists, the derivative of FD , thenfrom (15) it follows that

L′(t) = −tfD(t) − (1 − FD(t)) + tfD(t).

Therefore,

FD(t) = 1 + L′(t) (16)

and

fD(t) = L′′(t).

3.1 Estimation of the Distribution Function of the Duration

Our data consist of a discrete set of temporal cross-sectionscorresponding to the observation times, t ≥ 0 : � =⋃

t≥0 �(t), as shown in Fig. 1.For a temporal Boolean model it holds that γ = λED [1],

where γ is the mean number of germs per unit area inany frame, i.e. the intensity of an arbitrary temporal cross-section. From (13), we have

G(h, t) = log

(C(h, t) − 2p + 1

(1 − p)2

)= λCA0(h)L(t), (17)

where CA0(h) = EA0ν2(A0 ∩ A0 + h). We define

H(t) = G(0, t) = λa0L(t), (18)

where a0 = Eν2(A0). Note that L(0) = ED andL′(0) = −1. Therefore, H ′(0) = −λa0. Then it holds that

L(t) = − H(t)

H ′(0), (19)

and from (16), we calculated the cumulative distributionfunction of the duration as

FD(t) = 1 − H ′(t)H ′(0)

. (20)

3.2 Estimators

We replace the (unknown) continuous version of the covari-ance function by the corresponding discrete version. In or-der to simplify notation, we assume that the frames observedcorrespond to times t = 0, τ,2τ, . . . , nτ , i.e. n + 1 equallyspaced frames. Our sampling information consists of the se-quence of sets {�(kτ) ∩ W }k=0,...,n where τ is the temporaldelay between two consecutive frames and W is the sam-pling window.

The spatial-temporal covariance is estimated as

C(h, rτ )

= 1

n − r

n−r∑

k=0

ν2((�(kτ) ∩ W) ∩ (�((k + r)τ ) ∩ W)−h)

ν2(W ∩ W − h).

(21)

The volume fraction p is estimated as the average of the areafractions observed at each frame and it is given by

p = 1

n + 1

n∑

k=0

ν2(�(kτ) ∩ W)

ν2(W). (22)

The function H(t) is estimated at the observation points{t1, . . . , tn}, a discrete set of values, giving {H (t1), . . . ,

H (tn)}. We have used an approach based on functional dataanalysis in order to obtain a precise estimation of the firstderivative from this discrete set. A functional datum is a setof discrete measured values {(tj , H (tj ))}j=1,...,n. First, it isnecessary to convert these values into a function which iscomputable for any value. We did not use an interpolationprocess because we assumed that the discrete values may in-clude some observational error. Instead, we used a smooth-ing technique to transform the raw data {(tj , H (tj ))}j=1,...,n

to a function x(t) possessing a certain number of deriva-tives. We obtained them by means of a least squares fittingmethod which determines the best choice of the coefficientsck , k = 1,2, . . . ,K , in the representation of the function as alinear combination of K basis functions. We have chosen apolynomial spline basis {�k(t)}k=1,...,K , where each �k(t)

is a piecewise cubic function. We computed the coefficientsck of the expression x(t) = ∑K

k=1 ck�k(t) by minimizingthe least squares criterion

SMSSE(x/c) =n∑

j=1

(xj −

K∑

k=1

ck�k(tj )

)2

.

A detailed presentation of this method can be found in [10].

4 Results

Section 4.1 contains a simulation study to assess the perfor-mance of the proposed estimator of the distribution functionof the duration from the covariance. In Sect. 4.2 we apply themethod to images with random noise. Section 4.3 describesa biological application.

Six cylindrical temporal Boolean models were gener-ated, corresponding to three distributions for the duration(uniform, Gamma and normal) and two mean durations(6 and 10 seconds), see Table 1. The radii were uniformin the interval [4,8] pixels. Durations and radii were in-dependent. For each model, three values of the intensity

J Math Imaging Vis (2010) 38: 83–94 89

λ = {0.0001,0.0002,0.0003} were simulated that corre-sponded to low, medium and high volume fractions (seeTable 2). The volume fractions were derived using the equa-tion p = 1 − exp{−λπE(R2D)}. Image sequences were256×256 pixels, 100 seconds long and sampled at 4 framesper second. Fifty replica for each model and value of λ weregenerated, thereby analyzing a total of 6 × 3 × 50 = 900sequences. Figure 7 displays the first frame of the six mod-els simulated for the uniform distribution. In supplemen-tary material, Video1 corresponds to the uniform distribu-tion with λ = 0.0001, Video2 to the Gamma distributionwith λ = 0.0002 and Video3 to the normal distribution withλ = 0.0003 with a mean duration of 6 seconds.

Table 1 Description of the models simulated

Durations distrib. Parameters Model

Uniform [4,8] Model 1

[8,12] Model 2

Gamma [18,0.3333] Model 3

[50,0.2] Model 4

Normal [6,1] Model 5

[10,1] Model 6

Table 2 Volume fractions simulated

λ = 0.0001 λ = 0.0002 λ = 0.0003

ED = 6 0.06 0.13 0.18

ED = 10 0.10 0.20 0.29

4.1 Estimating FD , the Cumulative Distribution Function

First, we analyzed the absolute error in the estimation ofthe cumulative distribution function of the duration, FD , foreach model and intensity λ. Figure 8 displays the maxi-mum and the minimum absolute error (solid lines) and themedian (dashed line) of FD for 50 replica. The first rowcorresponds to ED = 6 and the second row to ED = 10.Columns correspond to the uniform, Gamma and normaldistributions, respectively. The intensity λ is 0.0003. Errorsare quite small, lesser than 0.06 except for the uniform dis-tribution U(8,12). The highest values are observed next tothe median of the distribution. Similar results were obtainedfor λ = 0.0001 and λ = 0.0002.

Second, we computed the maximum and the minimumdeviations of the 50 realizations from the theoretical dis-tribution function FD . Figure 9 plots the upper and lowerenvelopes (solid lines) as a function of the theoretical dis-tribution function FD . The theoretical value is within theenvelopes even for very small values of FD . The intensity λ

is 0.0003. Similar results were obtained for λ = 0.0001 andλ = 0.0002.

Finally, we have calculated the relative error of the me-dian of the duration as the difference between the esti-mated and the theoretical values with respect to the the-oretical value. The minimum, median and maximum val-ues are given in Table 3. The relative errors were verysmall, even for high volume fractions. They were lesserthan 0.8% except for the uniform [4,8], that was 1.33%.In our opinion, this error would be small enough in manyapplications. Note that the error in the case ED = 10

Fig. 7 The first rowcorresponds to ED = 6 and thesecond row to ED = 10. Fromleft to right, the columnscorrespond toλ = 0.0001,0.0002,0.0003.Uniform distribution for thedurations

90 J Math Imaging Vis (2010) 38: 83–94

Fig. 8 The median (dashed line), the maximum and the minimum (solid lines) of the absolute error in the estimation of FD

Table 3 Relative errors of themedian of the duration (%) λ = 0.0001 λ = 0.0002 λ = 0.0003

Param. min/median/max min/median/max min/median/max

Uniform [4,8] 0.44/1.33/4.00 0.44/0.44/4.00 0.44/0.44/3.11

[8,12] 0.27/0.80/2.93 0.27/0.27/1.87 0.27/0.27/1.87

Gamma [18,0.3333] 0.38/0.52/3.24 0.38/0.52/3.10 0.38/0.52/2.19

[50,0.2] 0.14/0.67/2.28 0.14/0.67/2.28 0.14/0.54/1.21

Normal [6,1] 0.44/0.44/2.22 0.44/0.44/2.22 0.44/0.44/2.22

[10,1] 0.27/0.27/1.33 0.27/0.27/1.33 0.27/0.27/0.80

is smaller than in the case ED = 6, because the dis-cretization effect of the acquisition rate is lesser in relativeterms.

Table 4 shows the relative error of the median of the dura-tion using a sample composed of isolated events. The errorswere quite high for the uniform and Gamma distributions,

J Math Imaging Vis (2010) 38: 83–94 91

Fig. 9 The minimum and maximum deviations of the estimated cumulative distribution function FD from the theoretical function

Table 4 Relative errors of themedian of the duration usingisolated clumps (%)

λ = 0.0001 λ = 0.0002 λ = 0.0003Param. min/median/max min/median/max min/median/max

Uniform [4,8] 0.00/0.00/4.17 0.00/4.17/8.33 0.00/4.17/8.33

[8,12] 0.00/1.25/5.00 0.00/2.50/5.00 0.00/2.50/7.50

Gamma [18,0.3333] 1.97/2.28/6.53 1.97/2.28/6.53 2.27/5.46/10.77

[50,0.2] 0.58/1.84/1.84 0.67/1.84/6.87 0.58/1.84/6.88

Normal [6,1] 0.00/0.00/4.17 0.00/0.00/4.17 0.00/3.12/4.17

[10,1] 0.00/0.00/2.5 0.00/0.00/2.5 0.00/1.87/5.00

reaching a maximum value of 10.77% for the Gamma dis-tribution. Higher intensities lead to a higher degree of over-lapping and therefore greater errors. The minimum and the

median of the errors are null for the case of the normal dis-tribution. This is due to the small variance we used in thesimulation of the durations (a value of 1). Almost all iso-

92 J Math Imaging Vis (2010) 38: 83–94

Table 5 Relative errors in theestimation of the median of theduration in noisy images (%)

Distribution λ D = 0.000 D = 0.005 D = 0.01 D = 0.02min/med/max min/med/max min/med/max min/med/max

N(6,1) 0.0001 0.44/0.44/2.22 0.44/1.33/2.22 0.44/1.78/3.11 2.22/3.11/4.00

0.0002 0.44/0.44/2.22 0.44/0.89/2.22 0.44/1.78/3.11 2.22/3.11/4.00

0.0003 0.44/0.44/2.22 0.44/1.33/2.22 1.33/2.22/3.11 2.22/3.56/4.00

N(10,1) 0.0001 0.27/0.27/1.33 0.27/0.53/1.33 0.80/1.33/1.87 1.87/2.67/3.47

0.0002 0.27/0.27/1.33 0.27/0.80/1.33 0.27/1.33/1.87 1.87/2.93/3.47

0.0003 0.27/0.27/0.80 0.27/0.80/1.33 0.80/1.60/2.40 2.40/2.93/4.00

Fig. 10 Several consecutivesnapshots of afluorescent-tagged proteinimage sequence. The binaryshapes correspond to thesegmented areas of fluorescence

lated grains segmented from the images last a duration equalto the median.

The spatial-temporal covariance was estimated using (21)in the interval [0,16] seconds with a step of one frame. Tocompute the first derivative of H(t), we used 15 basis func-tions to transform the raw data into a piece-wise function,which is appropriate for functions re-sampled at 65 values.The R [9, 11] package fda developed by J.O. Ramsay hasbeen used.

4.2 Noisy Images

In some applications, the image acquisition method intro-duces random noise resulting in the appearance of corruptedpixels, edge blurring and other spurious effects. In this sub-section, we analyze the performance of the estimator of themedian of the distribution function of the duration in noisyimages. We introduced salt-and-pepper noise in the randomshapes, independently in each frame. Three ratios of noisewere simulated, 0.5%, 1%, 2% (see Videos Noise1, Noise2and Noise3 in supplementary material). Ten replica weresimulated for each case. Table 5 shows the relative errorsin the estimation of the median of the duration. For eachratio of noise, the minimum, maximum and median valuesof the errors were calculated. The median of the relative er-rors was lesser than 2.2% for low and medium noise density.The maximum error was 4% for the highest ratio of noise.We can conclude that the proposed estimator performs wellin the presence of this kind of noise.

4.3 Application to Biological Image Sequences

Clathrin-mediated endocytosis is one of the main endocyticroutes. Endocytosis has recently been imaged in real time,

Table 6 Description of biological image sequences

Seq. Protein # of frames Size Video

1 Clathrin-RFP 300 120 × 67 Cell 1

2 Clathrin-RFP 300 77 × 60 Cell 2

3 Clathrin-RFP 300 71 × 54 Cell 3

by conjugating clathrin proteins to fluorescent moleculesand imaging cells with specialized microscopy techniquessuch as Total Internal Reflection Fluorescence Microscopy(TIRFM) [15]. TIRFM allows selective illumination ofthe cellular plasma membrane, thereby providing superiorsignal-to-noise. Under TIRFM, the assembly of fluorescent-labelled clathrin at a site of ongoing endocytosis results inthe appearance and steady growth of a diffraction-limitedspot. The time elapsed between the appearance and the dis-appearance of a fluorescent clathrin spot is defined as theduration of a discrete endocytic event. The areas of fluores-cence associated with the different endocytic spots overlap,forming random clumps of different sizes, shapes and dura-tions (see Fig. 10).

To obtain the duration distribution of the endocytic eventsis important in the analysis of the influence of different celltreatments on this process. Traditionally, collecting this kindof data has been very time-consuming where much of thework had to be done manually, making it virtually impossi-ble in large image sequences.

Our biological data comprise three sequences of clathrinprotein, described in Table 6 (see Videos Cell1, Cell2and Cell3 in supplementary material). RFP stands for RedFluorescent Protein. Time-lapse images were acquired byTIRFM at 0.25 frames per second. In this technique, a laser

J Math Imaging Vis (2010) 38: 83–94 93

Fig. 11 Estimated cumulative distribution function of the duration ofendocytic events

beam is sent to the sample at a critical angle. As it encoun-ters the interface between two media with a different refrac-tive index (i.e. water and glass), the beam undergoes total in-ternal reflection. As a consequence, a small excitation waveis generated. The evanescent field is only 100–200 nm thick,and it decays exponentially as it moves away from the cover-slip. Therefore, only objects which are within 100–200 nmof the bottom plasma membrane of the cell are illuminated,while the nucleus, inner cytosol and upper plasma mem-branes are left in the dark. In this way, it is possible to imagemembrane-associated events with superior signal-to-noise.The setup employed for this study was an objective-basedTIRFM (63X magnification) implemented on an invertedIX70 microscope (Olympus) and coupled to a 488-nm laserline (Melles Griot). The laser power was between 80 and100 mW. The image processing to extract the binary im-ages was based on the application of: a top-hat transform tosubtract the background and extract peaks of fluorescence, atemplate matching with a Gaussian kernel to remove even-tual noise, followed by a region growing technique in orderto delineate each marked object.

We have performed the analysis for the three clathrinsequences. The medians of the duration were 84, 74 and86 seconds, respectively. The cumulative distribution func-tions of the duration for these three sequences are shown inFig. 11. The estimated median using the sample composedof isolated episodes was 16 seconds for the three cases.Again, note how the use of a biased sample results in theunderestimation of the median of the duration.

5 Conclusions and Further Developments

We have shown that the use of a biased sample composedof isolated events can lead to a very high bias in the esti-mation of the mean of the random duration. The higher theoverlapping among grains is the higher the bias is. This canbe unacceptable in many real applications. It is necessary

to develop procedures based on stochastic models wherethe spatial temporal overlapping is explicitly assumed. Wehave shown that the temporal germ-grain models are a goodchoice and, in particular, the cylindrical temporal Booleanmodel.

We have proposed an estimator of the cumulative distri-bution function of the duration of spatially and temporallyoverlapping events based on the covariance of temporal ran-dom closed sets. Results from the simulation study showedthat the absolute errors in the estimation of the distributionfunction of the duration are small. Indeed, the relative errorin the estimation of the median of the distribution was lesserthan 1% in almost all cases. The estimator also performsquite well in images with a certain degree of random noisesuch as salt-and-pepper.

Several points remain open for future research. First, thedevelopment of new temporal germ-grain models with othergerm models. In particular, models with higher aggregationthan the Poisson point process which will lead to greateroverlapping probability. The estimation will be harder but,possibly, semi-parametric models could be proposed. Impor-tantly, the experimenters should provide information aboutthe mechanisms to be modelled in such a way that thisinformation can be incorporated in the definition of thesenew semi-parametric models. Second, the estimation of thesize distribution for the grains. Last, the method describeddepends on the assumption of stationarity. Potential exten-sions will include the use of other models, such as the Coxmodel.

We have implemented a Matlab toolbox for simulation oftemporal Boolean models and estimation. It is available athttp://www.uv.es/tracs/. Simulated videos and fluorescent-tagged proteins image sequences can be also downloaded,as well as the R scripts of the functional data analysis ap-plied in this study.

References

1. Ayala, G., Sebastian, R., Diaz, M.E., Diaz, E., Zoncu, R., Toomre,D.: Analysis of spatially and temporally overlapping events withapplication to image sequences. IEEE Trans. Pattern Anal. Mach.Intell. 28(10), 1707–1712 (2006)

2. Diggle, P.: Binary mosaics and the spatial pattern of heather. Bio-metrics 37, 531–539 (1981)

3. Dupac, V.: Parameter estimation in Poisson field of discs. Bio-metrika 67, 187–190 (1980)

4. Epifanio, I., Ayala, G.: A random set view of texture classification.IEEE Trans. Image Process. 11(8), 859–867 (2002)

5. García, P., Petrou, M., Kamata, S.: The use of boolean modelfor texture analysis of grey images. Comput. Vis. Image Underst.74(3), 227–235 (1999)

6. Matheron, G.: Eléments Pour une Théorie des Milieux Poreux.Masson, Paris (1967)

7. Matheron, G.: Random Sets and Integral Geometry. Wiley, Lon-don (1975)

94 J Math Imaging Vis (2010) 38: 83–94

8. Molchanov, I.: Theory of Random Sets (Probability and its Appli-cations). Springer, Berlin (2005)

9. R Development Core Team: R: A language and environment forstatistical computing. R Foundation for Statistical Computing, Vi-enna, Austria (2008). http://www.R-project.org. ISBN 3-900051-07-0

10. Ramsay, J., Silverman, B.: Functional Data Analysis, 1st edn.Springer Series in Statistics. Springer, Berlin (1997)

11. Ramsay, J.O., Wickham, H., Graves, S.: FDA: Functional DataAnalysis (2007). http://www.functionaldata.org. R package ver-sion 1.2.3

12. Sebastian, R., Diaz, E., Ayala, G., Diaz, M.E., Zoncu, R., Toomre,D.: Studying endocytosis in space and time by means of temporalboolean models. Pattern Recognit. 39(11), 2775–2785 (2006)

13. Serra, J.: Image Analysis and Mathematical Morphology. Acad-emic Press, New York (1982)

14. Stoyan, D., Kendall, W., Mecke, J.: Stochastic Geometry and ItsApplications, 2nd edn. Wiley, Berlin (1995)

15. Toomre, D., Manstein, D.: Lighting up the cell surface withevanescent wave microscopy. Trends Cell Biol. 11, 298–303(2001)

María Elena Díaz was born in Al-bacete (Spain) in 1969. She receivedher M.Sc.D. and Ph.D. in Physicsfrom University of Valencia in 1992and 1997, respectively. She is cur-rently Lecturer in the Computer Sci-ence Department at University ofValencia (Spain). In 2000 she helda postdoctoral fellowship at the Eu-ropean Molecular Biology Labora-tory in Heidelberg (Germany). Shewas supported by a grant of HumanFrontier Science Organization fordeveloping new models on molecu-lar Biology in collaboration with the

Yale University (USA), and CNRS (France). Her research interests in-clude biomedical image processing, motion analysis, feature extrac-tion, mathematical morphology and applications of stochastic geome-try in computer vision.

Guillermo Ayala was born in Carta-gena (Spain) in 1962. He graduatedin Mathematics (1985) and receivedhis Ph.D. in Statistics (1988), bothat the University of Valencia. Hewas a scholarship holder at CMUS. Juan de Ribera (Burjasot, Spain)from 1979 to 1987. He is currentlyProfessor at the Department of Sta-tistics and Operations Research inthe University of Valencia, Spain.His research interests are in the ar-eas of medical image analysis andapplications of stochastic geometryin computer vision.

Ester Díaz received her M.S. De-gree in Telecommunications Engi-neering from Polytechnic Univer-sity of Valencia (Spain) in 1996. Sheis currently doing her Ph.D. in In-formatics and Computational Math-ematics in Valencia University. Herresearch is focused in biomedicalimage analysis and modelling.