Hill-manipulation: An effective algorithm for color image segmentation

Zaher Al Aghbari *, Ruba Al-Haj

Department of Computer Science, University of Sharjah, P.O. Box 27272, Sharjah, United Arab Emirates

Received 21 August 2005; received in revised form 7 February 2006; accepted 16 February 2006

Abstract

In cluster-based image segmentation techniques, clusters may be viewed as hills in a histogram. These techniques may suffer from large hills

that dominate the smaller hills and thus they result in a loss of image details in the segmentation process. In this paper, we propose a novel

approach, called Hill-Manipulation algorithm, to solve the problems of the traditional cluster-based image segmentation methods. It starts by

segmenting the 3D color histogram into hills according to the number of local maxima found, and then each hill is checked against defined criteria

for possible splitting into more homogenous smaller hills. As a result, details of an image are distinguished and the details are captured in the

segmentation. Finally, the resulting hills undergo a post-processing task that filters out the small non-significant regions.

q 2006 Elsevier B.V. All rights reserved.

Keywords: Image segmentation; Hill-manipulation; Color segmentation

1. Introduction

The explosion in the amount of images being generated in

many different fields and the growth of the WWW emphasizes

the need for retrieving images based on their contents.

Therefore, content-based image retrieval (CBIR) has attracted

researchers from different fields like computer vision, image

processing, database, information retrieval, etc. Furthermore,

CBIR has demonstrated its wide usefulness in many fields such

as biochemistry, medical diagnosis, education, etc. [28].

Many researches have proposed different algorithms to

represent and retrieve images based on global features, such as

the color histogram of the whole image. However, retrieving

images by global features is too crude an approach to produce

satisfactory results.

Thus, it is essential to segment (divide an image into smaller

feature-coherent regions) and then represent the image by the

local features of its composing regions. As a consequence,

CBIR from the vast collection of images will be more effective.

Therefore, image segmentation aims at dividing an image into

several segments where each segment is visually coherent.

However, it is not an easy task to slice the image into its

0262-8856/$ - see front matter q 2006 Elsevier B.V. All rights reserved.

doi:10.1016/j.imavis.2006.02.013

* Corresponding author.

E-mail addresses: zaher@sharjah.ac.ae (Z.A. Aghbari), ruba@sharjah.ac.

ae (R. Al-Haj).

composing segments, because this depends on many features

like intensity, blurring, contrast, and even number of segments.

All of these factors will affect the quality of segmentation.

Many efforts towards image segmentation have lead to

many different techniques, and it is difficult to mention them

all. Most existing automatic image segmentation methods can

be classified into these approaches:

Threshold techniques are based on selected thresholds from

image histograms. All pixels whose values (gray intensity,

color, or other) are between two thresholds belong to one

region [12]. This technique can obtain good segmentation of

images that include two opposite components, but they are

ineffective for images that blur at the boundaries, or for

multiple component image segmentation [24,29]. A

threshold approach based on relative entropy is presented

in [4]. Other pioneer work on color segmentation are the

histogram based approaches such as [30,34].

Boundary-based techniques are based on the assumption

that pixel values change rapidly at the boundary between

two regions [8,19]. The output of most existing edge

detectors can only provide candidates for the region

boundaries, because these obtained color edges are in

general discontinuous or hard to be detected. Still, the actual

region boundary should be a closed curve. Therefore, post-

processing tasks, such as edge tracking, gap filling,

smoothing, and thinning, should be performed [9,15,22].

On the other hand, all of these post-processing tasks are very

expensive in terms of time.

Image and Vision Computing 24 (2006) 894–903

www.elsevier.com/locate/imavis

Z.A. Aghbari, R. Al-Haj / Image and Vision Computing 24 (2006) 894–903 895

Region-based techniques rely on the assumption that

adjacent pixels in the same region have similar visual

features; these features can be color or texture. The

performance of this approach mostly depends on the

selected homogeneity measure used to match colors like

in [10,13,20], the accuracy of different texture matching

algorithm like in [2,21,33,35], or both color and texture like

in [6,31]. The two most popular methods that use region-

based techniques are: (1) Seeded Region Growing (SRG)

that depends on selecting some pixels (seeds) to represent

the initial regions [19]. Then, other pixels are associated to

their most similar regions. (2) Split and merge that is based

on dividing the image into arbitrary set (say, four) of

disjointed regions and then merges or splits the region

according to some conditions [12,25,36]. An important

work in region growing is presented in [37].

Clustering techniques search for pixel values in the image

that occur frequently and close to each other. Also, these

values are separated from those of other pixels. Then these

pixels are grouped together into clusters, which are viewed as

hills in a histogram. A peak (local maximum) of each hill is

taken to represent the corresponding cluster. If these peaks

are then separated by someminimum distance, the image can

be segmented using the threshold techniques [12,18]. These

techniques suffer from the large hill problems, which are

detailed in Section 3. A statistically principled approach is

presented in [11]. Other 3D clustering-based segmentation

methods such as the work in [26] is based on amorphological

approach, [17] is based on a fuzzy c-means approach, and [5]

is based on mean shift approach.

Mixture techniques integrate the results of more than one

technique to providemore accurate segmentation results. For

example, the systems in [8,16,22,32] merge region growing

with edge detection in different ways, and [7] integrates

image labeling based on texture with region merging.

As mentioned above, the clustering techniques are easy to

implement and produce satisfactory segmentation results; how-

ever, they suffer from the large hill problems (see Section 3). Thus,

this paper proposes a novel approach, called Hill-Manipulation

algorithm, to solve the problem of the traditional cluster-based

image segmentation methods. The problem is the generation of

large hills that dominate the small hills, which have been

submerged into those large ones, and thus they result in a loss of

some image details in the segmentation process.

The rest of this paper is organized as follows: Section 2

introduces a background to the proposed algorithm. Then,

Section 3 presents the Hill-Manipulation algorithm and

discusses how it solves the problems of previous algorithms.

Section 4 presents the experimental results. Finally, we

conclude the paper in Section 5.

2. Background

In this section, we introduce briefly the related algorithms

to the proposed hill-manipulation algorithm. Further, we

introduce the HSV color space that is used to represent the

content of images in this paper.

2.1. Seeded region growing (SRG)

The SRG method [14] depends on selecting some

representative pixels (seeds) from the image. There is more

than one way for seed selection, for example, manually

according to the user’s regions of interest, or automatically by

using the peaks of the histogram. After selecting the seeds, the

algorithm labels each seed so that it represents a segment

(initial cluster), Ai, where iZ1,2,.,n. Then, for each segment

Ai, all its neighboring pixels are examined recursively. If the

distance (i.e. Euclidean distance) between the segment mean

and the examined pixel is less than a threshold, then the

examined pixel is associated with Ai and the running mean of Ai

is updated to reflect the addition of the recently examined pixel.

Each of the final segments would represent a group of

connected pixels that are feature-coherent.

2.2. Clustering

Most clustering techniques are nonparametric and lead to

satisfactory results. The two main types of clustering

techniques are:

† Distance based clustering. This technique joins the pixels

of an image based on the distance (i.e. Euclidean distance)

between their feature vectors without considering the global

distribution of a feature. As a result, artifacts are likely to

occur in the segmentation [12,28].

† Density estimation clustering. This technique examines the

feature space as a density function, and then selects the

dense feature spaces as peaks of clusters. When a peak is

determined, the cluster associated with it is established

based on the local structure of the feature space [1,23].

2.3. Hill-climbing algorithm

Traditional hill-climbing segmentation is a nonparametric

algorithm that clusters the colors of an image. The idea is that

each cluster is represented by a hill in the histogram, where the

hill consists of adjacent colors. As our hill-manipulation

algorithm utilizes the hill-climbing idea in its initial stages, we

discuss the steps of the hill-climbing algorithm below. It is

worth mentioning that we assume that input image to the hill-

climbing algorithm is a non-quantized image. However, we

preprocess the input image by a median filter to remove any

noise or meaningless details that may affect the result.

Algorithm: hill-climbing

Input: an image

Output: a set of visually coherent segments

1. Compute the 3D HSV color histogram of the image.

2. Choose a non-zero bin of the color histogram and make

uphill moves until reaching a peak as follows:

Fig. 1. The HSV color cone.

Z.A. Aghbari, R. Al-Haj / Image and Vision Computing 24 (2006) 894–903896

2.1 Compare the number of pixels in the current bin with

its immediate neighbors, in the 3D histogram there are

3dK1Z26 neighbors, where d is the dimensionality of

the feature space. Note that since the hue component is

represented as a color wheels, the far left and far right

bins in the hue axis are neighbors.

2.2 If the neighboring bins have different numbers of

pixels, the algorithm makes an uphill move towards the

neighboring bin with larger number of pixels.

2.3 If the immediate neighboringbinshave the samenumbers

of pixels, the algorithm checks the next neighboring bins,

and so on, until two neighboring bins with different

numbers of pixels are found. Then, an uphill move is

made towards the bin with larger number of pixels.

2.4 The uphill climbing is continued (repeat steps 2.1–2.3)

until reaching a bin fromwhere there is no possible uphill

movement. This is when the neighboring bins have

smaller numbers of pixels than the current bin.

2.5 If no uphill move is done, the stopping bin is identified as

a peak of a hill, and all bins leading to this peak are

associated with it.

3. Select another unclimbedbin as a startingbin andperformstep

2 to find another peak. This step is continued until all non-zero

bins of the histogram are climbed (associated with a peak).

4. The identified peaks represent the initial number of clusters of

the input image; thus these peaks are saved.

5. Neighboring pixels that lead to the same peak are grouped

together.

Although, we used a global color histogram to find the

peaks, step 5 takes into account the spatial information of the

pixels when forming the segments, i.e. only spatially close

(neighboring) pixels that lead to the same peak are grouped into

one segment [23].

2.4. HSV color space

The HSV color wheel or cone are common representation of

the color space, which consists of three components: Hue (the

color type), Saturation (the amount of color), and value (the

brightness of color). The HSV color space is chosen because of

two reasons: (1) it is easier for users to select a color from theHue

range rather then giving a combination of three components R,G,

and B, and (2) the second reason is that HSV encapsulates

information about a color that aremore familiar to humans [3,27].

Fig. 1 shows the cone, where the hue is the circle side of the cone.

Each hue value forms an angle between 0 and 3608 relative to the

0 line, which is usually assigned to the red color.

3. Hill-manipulation algorithm

The SRG, distance based clustering, and density based

clustering algorithms, which are mentioned in Section 2, are

based on clustering similar to our proposed hill-manipulation

algorithm. However, each of these algorithms has its own

approach of clustering.

The hill-climbing algorithm gives satisfactory results for a

wide range of images, where the color regions in the image are

well distinguished in terms of their colors. But its segmentation

quality deteriorates in the following scenario cases:

(Note that due to the difficulty of representing all three

components versus the number of pixels, for simplicity in this

paper, we will only show the 1D histogram representation of

the hue component, as shown in Fig. 2(b). However, the

representations of the other components are similar to the hue

representation).

(I). Scenario cases 1

Although the histogram contains several hills, a large

portion of an image falls inside one wide hill, see Fig. 2. The

wide hill may contain non-similar colors. This scenario case

usually happens with blurred or high contrast images.

(II). Scenario cases 2

The histogram contains a wide spectrum of colors in the

image including the two extreme colors in the hue axis. In such

a case, there is a possibility that a hill although appears to be

slim in the histogram space it may contain non-similar colors

that represent non-similar regions. Such a hill needs to be

dismantled, see Fig. 3.

We propose the hill-manipulation algorithm to solve the two

cases mentioned above and consequently improve the image

segmentation. The hill-manipulation algorithm is outlined below:

Algorithm: hill-manipulation

Input: an image

Output: a set of visually coherent segments

1. Quantizing the HSV color space.

2. Creating a 3D HSV color histogram.

3. Call the traditional hill-climbing algorithm.

4. Detect and correct the scenario cases.

5. Filtering out unnecessary small regions.

Fig. 2. (a) Original image, (b) the corresponding color histogram, and (c) image segmented into one large region (most of the background and part of the bird’s body)

and several other small regions.

Fig. 3. (a) Original image, (b) the corresponding color histogram, and (c) image segmented into one large blue region, the black region, and the white background.

Z.A. Aghbari, R. Al-Haj / Image and Vision Computing 24 (2006) 894–903 897

Below we will discuss these steps in details.

3.1. Quantizing HSV color space

Normally, the HSV components values are quantized to the

following ranges: 0–3608 for hue and 0–100 for saturation and

value. These divisions are large in number and it is not

practical to use these numbers to represent the colors of the

image segments since they represent very similar shades of

colors that are normally distinguished by the human eye. By

carefully examining the hue circle, we found that the major

colors come at multiples of 108, i.e. red 08, blue 2408, and

green 1208. So, the range size must be divisible by 10. Further,

we noticed that a maximum of 108 of hue contain one

homogenous color. However, a larger range may include more

than one homogenous color. Thus, we divide the hue into 36

intervals, where each interval is of size 108 and we call this

number the hue division factor, Hdf. These divisions are shown

in Fig. 4(a).

As a result of our extensive experiments and by carefully

examining the saturation and value components, which have

the same range (0–100), we concluded that dividing these

component into intervals of size 10 gives us distinguishable

intervals and yet a small number of them. This keeps us in a

consistent state with the hue divisions. From Fig. 4(b), shows

the saturation division factor and value division factors (Sdf,

Vdf). These quantization factors can be changed by the user if

necessary to adjust to specific image collection.

Fig. 4. Quantization of HSV color space to (36, 10, and 10): (a) hue quantized to

36 intervals, (b) saturation and value quantized each to 10 intervals.

3.2. Creating 3D HSV histogram

The 3D HSV histogram can be visualized as a cube of cells.

Each cell in the cube contains the number of pixels that

correspond to the three components associated with that cell.

We refer to each cell as a bin. The axis of each component is

adaptively adjusted to the available component values. That is

each axis runs from the minimum to the maximum value found.

This is a crucial step to stretch and distinguish neighboring

colors that would have been clustered together. Thus, the range

of colors that each histogram bin represents is computed as

follows:

Cbr Z ðCmaxKCminÞ=Cdf (1)

where, Cbr is the number of distinct values that are represented

by each range in component H, S or V. Also, Cmax, Cmin are the

maximum and minimum color values of a component. Cdf is a

division factor (Hdf, Sdf, Vdf) of a component. Fig. 5(a) shows

the histogram bins of similar colors that were considered as one

hill in Fig. 5(a). Then by adaptively running the histogram from

the minimum to the maximum of the available colors using Eq.

(1), the hill’s bins were stretched and thus three sub-hills were

distinguished as shown in Fig. 5(b).

3.3. Calling traditional hill-climbing algorithm

The traditional hill-climbing algorithm finds the image

clusters (histogram hills) as explained in Section 2.3. However,

some of these hills may be large and contain non-similar colors

(see the two scenario cases explained above).

Fig. 5. (a) Histogram consist of one hill, (b) histogram consist of more than one hill after stretching the original hill by applying Eq. (1).

Z.A. Aghbari, R. Al-Haj / Image and Vision Computing 24 (2006) 894–903898

3.4. Detecting and correcting scenario cases

The 3D color histogram topology is affected by many

factors, like blurring, low contrast, and color distribution. All

of these factors can change the shape of the histogram hills,

leading to large hills and thus to unexpected segmentation

result. The effects of these factors can be classified into two

scenario cases. Hence, we developed algorithms that detect and

correct the two scenario cases as follows.

Fig. 6. The eight corners of RGB cube.

Table 1

RGB cube corners values and there corresponding HSV color values

Color HSV values RGB values

Hue (8) Saturation Value Red Green Blue

Red 0 or 360 100 100 255 0 0

Yellow 60 100 100 255 255 0

Green 120 100 100 0 255 0

Cyan 180 100 100 0 255 255

Blue 240 100 100 0 0 255

Megenta 300 100 100 255 0 255

3.4.1. Solving scenario case 1

To provide a solution to this scenario case, we have to

analyze the HSV color space and define a threshold beyond

which a hill is considered wide (contains non-similar colors).

Consider the RGB cube, shown in Fig. 6, where each corner

represents one of the eight different colors: red, yellow, green,

cyan, blue, magenta, white, and black. These colors are very

different from each other because they are generated by taking

the extreme values of the RGB components. If we convert these

eight colors into their corresponding HSV colors, the result will

be as shown in Table 1.

It is clear from the RGB-to-HSV mapping that the distance,

Dhsv (c1, c2), between any two adjacent colors is 608, which is

the maximum range that can be considered homogenous in

color. Where Dhsv (c1, c2) is simply the number of degrees

between the two hue values of c1 and c2. Since we quantized

the hue into 36 intervals, the bin width, Bw, is equal to

360/36Z108. Thus, we can define a threshold, beyond which a

hill can be considered wide, called slimming threshold (St) as

follows:

St ZDhsvðc1; c2Þ

Bw

(2)

Since, Dhsv (c1, c2)Z60 and BwZ10, StZ6 bins. Thus, in

our experiments, a hill containing more than six bins in any

component (H, S, or V) is considered wide and needs to be

slimmed. In this paper, we define the hill width (Lw) as the

number of histogram bins in a hill. A wide hill is slimmed by

splitting it into homogenous sub-hills using the hill-slimming

algorithm, which is outlined below:

Algorithm: hill-slimming

Input: a wide hill

Output: set of homogenous sub-hills

1. Double the number of quantization levels of the H, S and V

component from 36, 10, and 10–72, 20 and 20, respectively.

2. For each wide hill in the original histogram create new

temporary 3D histogram called temp_hist using the new

divisions.

a. Call the hill-climbing algorithm to find the created sub-

hills in temp_hist

b. Give each sub-hill in temp_hist a unique region

number.

c. Replace the wide hill with the corresponding sub-hills

in temp_hist.

The hill-slimming algorithm creates a temporary histogram,

temp_hist, using the doubled intervals 72, 20, and 20 for H, S,

Fig. 7. (Top) original histogram with four hills, (middle) the hill slimming

algorithm divides the wide hill into four sub-hills in temp_hist, and (bottom) the

sub-hills are mapped into the original histogram.

Z.A. Aghbari, R. Al-Haj / Image and Vision Computing 24 (2006) 894–903 899

and V, respectively. As a result, the wide hills of the original

histogram are stretched out in the temp-hist and broken into

several homogenous sub-hills. Then, the algorithm calls the

hill-climbing algorithm to find the newly created sub-hills in

temp_hist. Each sub-hill is given a unique number. We reached

the choice of doubling (not any other ratio) after performing

extensive testing on different ratios. Simply doubling the

values gave the best result as lower ratios did not stretch the

histogram enough to detect the details that are hidden within

the slim hills and they resulted in under-segmented images; on

the other hand, larger ratios resulted in over-stretching the

histogram and they resulted in over-segmented images. As

illustrated in Fig. 7, the original histogram Fig. 7 (top) has 4 hill

among which hill # 2 is considered wide. Then, after applying

Fig. 8. Comparison between the result of hill-climbing and hill-slimming: (a) origin

segmentation after applying the hill-slimming algorithm.

the hill-slimming algorithm on the wide hill, it is divided into

four sub-hills Fig. 7 (middle) and mapped into the original

histogram Fig. 7 (bottom).

Actually, one round of applying the hill-slimming algorithm

on a wide hill is enough (as we found out from our extensive

experiments on the system) to stretch out the wide hill and

produce several homogenous non-wide sub-hills that show the

significant details of images. Iterative doubling of the

histogram will stretch out a sub-hill further and it will result

in showing all the un-important details of the image and thus

we end up with over-segmented images.

Fig. 8 shows a comparison between the segmentation results

before and after applying the hill-slimming algorithm for a

difficult image due to both the blurred background and its

similar color to parts of the body of the bird. Notice that using

the hill-climbing algorithm, the blurred background and part of

the bird’s body were segmented as one region (see Fig. 8(b)).

However, the different parts of the background and the bird’s

body were distinguished after applying the hill-slimming

algorithm (see Fig. 8(c)).

3.4.2. Solving scenario cases 2

The hill in this case is slim as it consists of six or less bins;

however, it represents most of the image area (see Fig. 3).

Therefore, there is a possibility that such a hill although

appears to be slim in the histogram space it may contain non-

similar colors that represent non-similar regions. Note that the

hill width is defined as the number of histogram bins in the hill.

Such a hill is dominant and detected as shown in the Detect-

Dominant-Hill algorithm:

Algorithm: Detect-Dominant-Hill

Input: HSV histogram

Output: set of dominant hills

1. Compute the size of each hill in terms of the number of

pixels.

2. Find the average size of all hills (Lsize_av).

3. Compare the size of each hill Lsize_i with Lsize_av

a. If Lsize_iOLsize_av and Lw%St then the current hill is

dominant, where Lw is the hill width.

b. Else hill Lsize_i is not dominant.

To segment the smaller regions that are lost within the

dominant hill region, we will dismantle the hill so that each bin

becomes a separate sub-hill as shown in the hill-dismantling

algorithm. Fig. 3 illustrates the idea of hill dismantling, where

Fig. 3(b) shows a slim hill that represents the blue oval and all

al image of a bird, (b) result of segmentation using hill-climbing, (c) result of

Z.A. Aghbari, R. Al-Haj / Image and Vision Computing 24 (2006) 894–903900

the circles, which are shades of blue, inside it (see Fig. 3(a)).

Then, by running the hill-dismantling algorithm, each bin

becomes a separate hill and thus each circle was recognized as

a separate segment (see Fig. 3(c)).

Algorithm: hill-dismantling

Input: dominant hill

Output: set of sub-hills

1. For each bin associated with the dominant hill

a. Represent the bin as a separate sub-hill.

b. Increment the number of regions by one

3.5. Filtering out unnecessary regions

Due to the application of the hill-slimming and hill-

dismantling algorithms to solve the two scenario casesmentioned

in Section 3.4, many unnecessary small regions may be

generated. Further, their might exist some small noise regions.

Normally, these regions are not similar in color their surrounding.

Thus, we chose to cover that area with the color of the largest

surrounding region. Thus, a post-processing step to filter out such

regions is necessary. The filtering process determines whether a

region is small or not based on two conditions:

1. If the region size Lsize_i is less than the filtering factor Ff,

which is the average size-difference between any two regions

of distinct sizes

Ff ZmaxsizeKminsize

Ndsize

(3)

wheremaxsize andminsize are themaximumandminimumdistinct

sizes of regions, and Ndsize is the number of distinct sizes.

2. If the number of occurrences No of the distinct size is more

than the average number of occurrencesNav-o of distinct sizes,

where No and Nav-o are computed from a list of produced

regions and their sizes, which our system keeps after applying

the hill-slimming and hill-dismantling algorithms.

Fig. 9. Results of HSV quantization: (a) original image, (b) segmentation using quant

and removes them by changing their colors to their neighboring

majority color. The filtering algorithm is as follows:

Algorithm: region-filtering

Input: a set of all generated regions (hills)

Output: set of significant regions (hills)

1. Determine a filtering factor Ff and No of regions as shown

above.

2. For every small region, where its Lsize_i!Ff and NoONav-o

a. For each pixel (starting from the boundary)

i Check all its neighbors and change the color of the

pixel to the color of majority surrounding the

processed pixel.

4. Experimental results

In our experiment, we used 100 real images from seven

different categories to evaluate the performance of the

proposed algorithm. These categories are: people, animals,

nature, buildings, flowers, graphics and sea. The images are

selected randomly from these categories. Before segmenting

any of the images, we preprocessed the images by a median

filter to remove any noise or meaningless details that may

affect the result. A median filter is chosen with a mask of size

3!3 as it gave us the best results for our collection of images,

but a user can choose other sizes. The segmented image is

represented by assigning the majority color (the bin with the

maximum number of pixels in a hill) to the corresponding

regions.

The images for the experiment were selected randomly

from these categories. The HSV color space is quantized to

(H, S, V)Z(36, 10, 10). Other quantization values were tried

but the segmentation either resulted in the loss of some details

(under-segmented) or in producing too many unnecessary

segments (over-segmented). Fig. 9 shows a comparison

between two sets of values: (16, 8, 8) versus (36, 10, 10).

Note that using the first set of values resulted in a loss of some

important details.

ization values (16, 8, 8), (c) segmentation using quantization values (36, 10, 10).

Fig. 10. Segmentation comparison: (a) original image, (b) using the traditional hill-climbing, (c) using our proposed algorithm.

Z.A. Aghbari, R. Al-Haj / Image and Vision Computing 24 (2006) 894–903 901

4.1. Segmentation comparison

We compared the proposed hill-manipulation algorithm with

the traditional hill-climbing algorithm as shown in Fig. 10.

Notice that most of the details that were lost using the traditional

hill-climbing algorithm (see Fig. 10(b)) were found and

distinguished by the hill-manipulation algorithm (see

Fig. 10(c)). For example, in the bear image, the traditional

hill-climbing algorithm merged part of the bear body with the

brown weeds forming one big region. Then, the hill-slimming

algorithm divided that wide region into several sub-hills and as a

result the bear body was separated from the background.

We also compared the hill-manipulation algorithm with the

SRG. We decided to compare with SRG because it is the basic

and most referenced segmentation algorithm. As discussed in

Section 2.1 the SRG algorithm requires a user to enter a seed

(threshold), which is the main drawback of SRG. Thus,

we conducted the comparison using different seeds as shown

Fig. 11. Comparison between SRG and our proposed algorithm: (a) original image,

(d) result of hill-manipulation algorithm.

in Fig. 11. Another drawback of the SRG is the large number of

regions it produces when the threshold is small. On the other

hand, if we increase the threshold, the number of regions

decreases and the segmentation quality decrease too. Table 2

shows the number of regions produced by SRG using different

threshold values and the number of regions produced by the hill-

manipulation algorithm.

From both Fig. 11 and Table 2, we notice that for the SRG

method, as the threshold increases the number of regions

decreases, however, the segmentation result deteriorates

significantly (see that the background details and the water are

one region. Further, the tiger is divided into several regions and

some parts of it are combined with the background). On the

other hand, our proposed algorithm gives a reasonable number

of regions, while extracting the whole tiger as one region. Also,

the water was separate from the other details of the background.

The numbers given in Table 2 are representative for all other

images in our image database.

(b) result of SRG using thresholdZ10, (c) result of SRG using thresholdZ100,

Table 3

Illustrates the number of colors in the HSV histogram, the number of hills found

after hill-manipulation, and the number of separate regions after hill-

manipulation of the images in Fig. 12

Image in Fig. 12 # Colors # Hills after

hill-manipulation

# Regions after

hill-manipulation

Top image 707 73 468

Second from top 980 106 320

Third from top 1497 135 289

Fourth from top 470 29 153

Fifth from top 1295 103 243

Table 2

Comparison between SRG and hill-manipulation algorithm in terms of the

number of produced regions for image shown in Fig. 11

Segmentation method Number of regions

SRG ThresholdZ10 10698

ThresholdZ30 3774

ThresholdZ50 1823

ThresholdZ100 483

Our proposed algorithm 194

Z.A. Aghbari, R. Al-Haj / Image and Vision Computing 24 (2006) 894–903902

Fig. 12 shows the segmentation results of our proposed

algorithm, where column (a) contains the original images and

column (b) contains the segmentation results. Obviously, these

images prove that our proposed algorithm produces high quality

segmentation results. Table 3 illustrates the number of colors

found in the HSV histogram of each of the images in Fig. 12.

Also, Table 3 shows the number of hills and the number of

separate regions found in the HSV histogram after applying the

hill-manipulation algorithm. Obviously, the number of regions

Fig. 12. Segmentation results of hill-manipulation algorithm: (a) original

image, (b) segmented image.

are more than the number of hills because some of hills may map

to more than one separate region if these regions are not

adjacent.

5. Conclusion

We have presented a novel algorithm, called hill-manipu-

lation, which solves the problems of cluster-based image

segmentation algorithms, such as the hill-climbing algorithm.

The algorithm has prevented non-similar colors from forming

one region by apply the hill-slimming and/or hill-dismantling

algorithms on large hills. The experiments show the effective-

ness of the proposed algorithm as compared against other

segmentation algorithms. The proposed segmentation method

works well on all image categories images even on low contrast

images. As a future work, we are building a system that groups

segments hierarchically to produce larger regions that represent

semantic content (objects) of an image.

References

[1] S. Belongie, C. Carson, H. Greenspan, J. Malik. Color and Texture Based

Image Segmentation Using EM and Its Application to Content-Based

Image Retrieval, International Conference On Computer Vision (ICCV)

1998 pp. 675–682.

[2] S. Bhagavathy, S. Newsam, B. Manjunath, ‘Modeling Object Classes in

Aerial Images Using Texture Motifs’, Proceedings of the IEEE

International Conference on Pattern Recognition (ICPR), Quebec City

August 2002.

[3] D. Cardani, Adventures in HSV Space, The advanced Developers Hands on

Conference July 2001.

[4] C. Chang, K. Chen, J. Wang, M.LG Althouse, A relative entropy approach

to image thresholding, Pattern Recognition 27 (9) (1994) 1275–1289.

[5] D. Comaniciu, P. Meer, Mean shift: a robust approach toward feature space

analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence

24 (5) (2002) 603–619.

[6] Y. Deng, B. Manjunath, ‘Unsupervised segmentation of color-texture

regions in images and video’, IEEE Transactions on Pattern Analysis and

Machine Intelligence (PAMI ’01) August 2001.

[7] Y. Deng, B.Manjunath, H. Shin, Color image segmentation, Proceedings of

the IEEE Computer Society Conference on Computer Vision and Pattern

Recognition, CVPR ’99, Fort Collins, CO, 2 1999 pp.446–51, June.

[8] A. Elmagarmid, D. Ya, J. Fanu, G. Walid, Automatic image segmentation

by integrating color-edge extraction and seeded region growing, IEEE

Transactions on Image Processing 10 (10) (2001) October.

[9] M. El, A. Manjunath, Video region segmentation by spatio-temporal

watersheds, IEEE International Conference on Image Processing (ICIP),

Barcelona, Spain 2003 September.

Z.A. Aghbari, R. Al-Haj / Image and Vision Computing 24 (2006) 894–903 903

[10] I. Grinias, Y. Mavrikakis, G. Tziritas, Region growing colour image

segmentation applied to face detection, International Workshop on Very

Low Bit Rate Coding 2001.

[11] G.Frederix, E.J.Pauwels, A Statistically Principled Approach Toward

Feature Space Analysis, Technical Report, CWI 2004.

[12] J.Freixenet, X.Munoz, D.Raba, J.Marti, X.Cufi. Yet another survey on

image segmentation: region and boundary information integration,

Proceedings of the Seventh European Conference on Computer Vision.

Part III 2002.

[13] L. Lui, Y.H. Yang, Multiresolution color image segmentation, IEEE

Transactions on PAMI 16 (7) (1994) 689–700.

[14] A. Mehnert, P. Jackway, An improved seeded region growing algorithm,

Pattern Recognition Letters 18 (1997) 1065–1071.

[15] H. Ishikawa, I. Jermyn, Region extraction from multiple images

Proceedings of the Eighth IEEE International Conference on Computer

Vision, Vancouver, Canada 2001.

[16] I. Jermyn, H. Ishikawa, Globally optimal regions and boundaries

Proceedings of the Seventh IEEE International Conference on Computer

Vision, Kerkyra, Greece 1999 pp. 904-910, September.

[17] A.W.C. Liew, S.H. Leung, W.H. Lau, Fuzzy image clustering incorporating

spatial continuity, IEE Proceedings Vision Image & Signal Processing 147

(2) (2000) 185–192.

[18] L. Lucchese, S.Mitra, Unsupervised segmentation of color images based on

k-means clustering in the chromaticity plane. Proceedings of the IEEE

Workshop on Content-based Access of Image and Video Libraries

(CBAIVL’99), Fort Collins, CO 1999 pp. 74–78, June.

[19] L. Lucchese, S. Mitra, Advances in color image segmentation, Seamless

Interconnection for Universal Services. Global Telecommunications

Conference. GLOBECOM’99, Rio de Janeireo, Brazil, 4.6 1999 5–9

December, pp. 2038–44.

[20] B. Manjunath, J. Ohm, V. Vinod, A. Yamada, ‘Color and texture

descriptors,’ IEEE Trans. Circuits and Systems for Video Technology,

Special Issue on MPEG-7, 2001.

[21] B. Manjunath, P. Wu, S. Newsam, H. Shin, A texture descriptor for

browsing and similarity retrieval, Journal of Signal Processing: Image

Communication 16 (1–2) (2000) 33–43 September.

[22] S. Newsam, S. Bhagavathy, L. Fonseca, C. Kenney, B.S. Manjunath,

Object based representations of spatial images, Acta Astronautica 48 (5–12)

(2001) 567–577 June.

[23] T. Ohashi, Z. Aghbari, A. Makinouchi, Hill-climbing algorithm for efficient

color-based image segmentation IASTED International Conference on

Signal Processing, Pattern Recognition and Applications (SPPRA Rhodes,

Greece), June 2003, p. 2003.

[24] N.R. Pal, S.K. Pal, A review on image segmentation techniques, Pattern

Recognition 26 (9) (1993) 1277–1294.

[25] H. Park, J. Ra, Homogeneous region merging approach for image

segmentation preserving semantic object contours, Proceedings of

International Workshop on Very Low Bitrate Video Coding, Chicago, IL,

Oct. 8–9 1998, pp. 149-152.

[26] S.H. Park, I.D. Yun, S.U. Lee, Color image segmentation based on 3D

clustering: morphological approach, Pattern Recognition 31 (8) (1998)

1061–1076.

[27] S. Pramanik, S. Sural, Q. Gang, Segmentation and histogram generation

using the HSV color space for image retrieval, International Conference on

Image Processing, Rochester, NewYork, September 22, 2002.

[28] Y. Rui, T. Huang, S. Chang, Image retrieval: current techniques, promising

directions and open issues, Journal of Visual Communication and Image

Representation 10 (4) (1999) 39–62.

[29] P.K. Sahoo, S. Soltani, A.K.C. Wong, Y.C. Chen, A Survey of

Thresholding Techniques, Academic Press Professional, Inc., London,

1988. ISSN 0734-189X.

[30] R. Schettini, A segmentation algorithm for color images, Pattern

Recognition Letters 14 (1993) 499–506.

[31] C. Shaffrey, N. Kingsbury, I. Jermyn, Unsupervised image segmentation

via markov trees and complex wavelets, Proceedings of the IEEE

International Conference on Image Processing (ICIP), Rochester, USA,

September 2002.

[32] B. Sumengen, B. Manjunath, C. Kenney, Image segmentation using multi-

region stability and edge strength, Proceedings of IEEE International

Conference on Image Processing (ICIP), Barcelona, Spain, September

2003.

[33] B. Sumengen, B. Manjunath, C. Kenney, Image segmentation using curve

evolution and region stability, Proceedings of IEEE International

Conference on Pattern Recognition (ICPR), Quebec City, Canada, August

2002.

[34] S. Tominaga, Color classification of natural color images, Color Research

& Application 17 (4) (1992) 230–239.

[35] L. Wang, B. Manjunath, A semantic representation for image retrieval,

IEEE International Conference on Image Processing (ICIP), Barcelona,

Spain, September 2003.

[36] J. Winkeler, B. Manjunath, Genetic programming for object detection,

Proceedings of Genetic Programming 1997 Conference, Stanford, CA,

USA, 13–16 July 1997, pp.330–335.

[37] S.C. Zhu, A. Yuille, Region Competition: Unifying Snakes, Region

Growing, and Bays/mdl for Multiband Image Segmentation, IEEE

Transactions on Pattern Analysis and Machine Intelligence 18 (9) (1996)

884–900.