Using Wavelet Extraction for Haptic Texture

Classification

Waskito Adi1, Suziah Sulaiman1

1 Computer and Information Sciences Department, Universiti Teknologi PETRONAS

Bandar Seri Iskandar, 31750 Tronoh, Perak, Malaysia

{waskito@gmail.com, suziah@petronas.com.my}

Abstract. While visual texture classification is a widely-researched topic in

image analysis, little is known on its counterpart i.e. the haptic (touch) texture.

This paper examines the visual texture classification in order to investigate how

well it could be used for haptic texture search engine. In classifying the visual

textures, feature extraction for a given image involving wavelet decomposition

is used to obtain the transformation coefficients. Feature vectors are formed

using energy signature from each wavelet sub-band coefficient. We conducted

an experiment to investigate the extent in which wavelet decomposition could

be used in haptic texture search engine. The experimental result, based on

different testing data, shows that feature extraction using wavelet

decomposition achieve accuracy rate more than 96%. This demonstrates that

wavelet decomposition and energy signature is effective in extracting

information from a visual texture. Based on this finding, we discuss on the

suitability of wavelet decomposition for haptic texture searching, in terms of

extracting information from image and haptic information.

Keywords: Texture recognition, supervised learning, machine learning, haptic

texture search engine, wavelet decomposition.

1 Introduction

Lately, computer vision has become one of the most popular research subjects.

Computer vision contributes in various fields that include medical, engineering, and

robotics. One of the most interesting topics in computer vision is texture recognition.

Being the smallest entity, textures could be used as a parameter to recognize a

particular object. There are many different types of applications involving texture

analysis, including medical imaging, industrial inspection, remote sensing, document

segmentation, and computer based image retrieval [1].

Feature extraction plays an important role in a classification process. The

effectiveness of such classification relies greatly on the choice of this feature. In this

case, a suitable extraction algorithm influences the process end result. Parallel to

computer vision, computer haptic is also another area which is gaining its popularity

among researchers. Computer haptic enable user to touch and interact with the virtual

objects. Similar to computer vision, haptic texture is a growing research topics and its

application has been noted in medical field, arts, and textile, to name a few.

Even though computer vision and computer haptic are among the favorite topics of

late, no attempt has been made to integrate both texture classifications into the same

system such as a search engine. Current practices have seen visual and haptic textures

been dealt with separately; and the trend is that visual aspect has been the dominant

topics of the two texture types. The absence of such integration may hinder one from

getting a complete set of surface information of a particular object. This is because

visual textures could inform us on the macro-geometric information while haptic

textures on the micro-geometric details [15]. The lacking in combining both visual

and haptic textures needs addressing.

In this paper, we attempt to address the issue of lacking the effort in integrating

visual and haptic textures. We propose a way to analyze and classify haptic textures

based on an approach used from computer vision. The focus of our work here is on

visual textures and examines how such result could inform those researchers working

in haptic texture classifications. This involves conducting an experiment to test the

visual texture classification and feature extraction performance. In the classification

process, supervised learning has been used as a testing method.

The outline of this paper is organized as follows: in section 2, we review some

related works pertaining to visual and haptic textures analysis. This is followed by

section 3 in which we describe the method involved in our experiment that include

wavelet transformation, and statistical classification. In section 4, we present our

scheme in extracting texture information, and also how the experiments were

conducted. Finally, in section 5 we analyze the experimental results and discuss the

findings.

2 Related Works

2.1 On Visual Textures

Visual texture analysis has been studied over the past three decades. Most of the early

research focused on the analysis of the first or second order statistics of textures and

stochastic model such as Gaussian Markov random fields and auto regression [2].

Recently, the development of texture classification and analysis has been using

frequency and spatial analysis, such as Fourier transform, Gabor filter, and Wavelet

transform. Since a texture image is in 2D representation, measuring its characteristics

could be done by employing a 2D transformation or filtering. For example, to extract

texture information, a 2D Fourier transform can be employed. Azencott et al [3]

define a distance between textures for texture classification from texture features

based on windowed Fourier filters. In the research, Fourier transformation is used as a

method to extract information from textures.

In employing a more advanced technique, Fountain [4] has conducted an

experiment using Gabor filter for texture extraction. Also, in [7], Miaohong

conducted an experiment using Gabor filter to extract features at different orientation

from hyperspectral images. In that particular research, it is said that accuracy of the

classification is over 90%. Using a different technique, Shankar conducted an

experiment [5] by using wavelet transform to extract information from finger print.

Similarly, Chi-Man Pun [6] has also conducted an experiment using wavelet to extract

information from textures. It is reported in his research that wavelet gives a better

result than Gabor filter. In this particular case, although Gabor filter is a multi-

resolution filter, wavelet transform still outperforms.

2.2 On Haptic Textures

Texture is the smallest independent quantity in an object. Research works that involve

texture are mostly conducted in image and signal processing. When including texture

in haptics field, image and signal processing algorithm will be involved. To create a

realistic object the simplest way is by employing sine wave to generate the texture

surfaces. In [16] Seungmoon Choi and Hong Z. Tan employed sine wave to render the

texture surface in order to analyze the stability during haptic texture rendering.

In order to simulate the surface of haptic texture, there are some algorithms that

can be used [16][17]. The simplest one is by using sine wave. By adjusting amplitude

and frequency of the sine wave the texture of surfaces could be obtained. In [14] Wall

developed a haptic texture rendering technique using Fourier series. The basic idea of

this rendering technique is similar with rendering using sine wave because Fourier

series using sine wave as basis to create new wave. In order to create realistic texture

for haptic rendering, Juan Wu introduced a new rendering method based on image

processing [17].

3 Methodology

3.1 Equating Haptic Texture to Visual Texture

To obtain a texture sensation, a simple technique such as sinusoidal waves could be

used to render the force feedback. Different texture pattern or profile is obtained by

adjusting frequency, and amplitude of the sine waves which could result in different

haptic sensation. For example, Wall [14] conducted an experiment that simulates

texture surface by employing Fourier series to achieve different types of haptic

texture surface.

The texture profile used in haptic rendering and its 2D image counterpart share a

similar characteristic. When rendering a property of an object, such as its haptic

texture using sinusoidal waves, the (haptic) surface will have surface properties

similar to (visual) sine waves, but in 3D plane. However, when these haptic properties

are plotted onto a 2D plane they will appear similar to 2D images. Fig. 1 shows a 2D

representation of sine and square haptic waves (Fig 1.1 & 1.2) and the actual visual

images (Fig 1.3 & 1.4).

Fig 1.1. sine wave (haptic)

texture

Fig 1.2. square wave (haptic)

texture

Fig 1.3 cloth texture (visual)

Fig 1.4 cloth texture (visual)

Fig. 1. Image visualization from haptic texture

Comparing the four images in Fig. 1, it could be assumed that haptic texture

images generated using both sine square waves have similar patterns with those non

artificial visual textures (Fig. 1.3 & 1.4). Based on the similarity of their appearances

on a 2D plane, we could claim that computer vision (CV) algorithm could also be

employed to extract information from haptic rendering data.

3.2 Determining a Method to Represent Haptic Texture

Transformation is one of the most powerful tools to analyze signal information. With

transformation, signal can be analyze and manipulated. The most common method to

analyze signal is by using Fourier transform. Fourier transforms use sine wave as

basis of the transformation, by using Fourier transform assumption, it can be inferred

that every signal is constructed by sine waves superposition. Fourier transform is

denoted as:

∫∞

∞−

− ∂= xexfwFxiωπ2)()(

(1)

The inverse of Fourier transform, which allows going from spectrum to the signal is:

∫∞

∞−

∂= ωωπ xiexFxf

2)()(

(2)

Consider a square wave, (Fig. 2), )(xf of length L2 .The range ]2,0[ L , could be written:

112)( −

−−

=

L

xH

L

xHxf

(3)

By using Fourier transform, information of square signal can be retrieved. In this

case, square signal can be decomposed into different sine wave frequency and

amplitude. Referring to the Fourier transform, if a square wave equation is

transformed, it will produce a Fourier series as follows:

∫∞

=

=

,...5,3,1

sin14

)(n

L

xn

nxf

π

π

(4)

Unlike Fourier series that uses sine wave, wavelet is a transformation that uses

transient signal as its basis. In other word wavelet transform is better than Fourier for

analyzing transient signal. Wavelet transform of )(xf is denoted as:

∫ ∂=R

ba xxfxbafW )()(),( ,ψψ (5)

Consider a signal shown in (Fig. 2), in wavelet analysis.

Fig. 2. Decomposition of square wave using Fourier Transform

The signal can be decomposed into different small basis transient signal. Otherwise if

Fourier transform is employed, the signal will be decomposed into different amplitude

and frequency of sine wave.

Fig 3. Illustration of 1-D wavelet transforms. Unlike Fourier transform that used sine wave as

basis, wavelet transform use small transient signal to transform or decompose the source signal.

Wavelet multiresolution analysis has been used as texture characterization. It provides

spatial or frequency information of an image. There are two method extractions in

wavelet multiresolution: (i) wavelet decomposition (ii) wavelet packet decomposition.

In our experiment we use 2D wavelet decomposition to extract texture images. The

decomposition offer results for image analysis. Although it is not as rich as 2D

wavelet packet (2D-WPT), it is adequate to do analysis to an image with 2D wavelet

decomposition. In 2D wavelet decomposition transform (2D-WDT), an image x is

split into an approximation and three detail image (Fig. 3). 2D-WDT can be described

as a pair of filter L and H [9], H is low pass filter, and L is high pass filter. The 2D-

WDT of a discrete N x M image can be recursively defined as:

∑∑ +++ ==

m n

p

jnimk

p

jik CnhmhCA )2,2(,

1

),(,41 )()( (6)

∑∑ ++++ ==

m n

p

jnimk

p

jik CnhmhCH )2,2(,

1

),(,141 )()( (7)

∑∑ ++++ ==

m n

p

jnimk

p

jik CnhmhCV )2,2(,

1

),(,241 )()( (8)

∑∑ ++++ ==

m n

p

jnimk

p

jik CnhmhCD )2,2(,

1

),(,341 )()( (9)

A1 is approximation coefficient, H1 is horizontal detail, V1 is vertical detail, and

D1 is diagonal detail, illustrated in (Fig. 4). In practice, the image X only has finite

number of pixels, for boundary handling different technique such as symmetric,

periodic could be used. The difference between 2D-WDT and 2D-WPT is in the

extraction process. In 2D-WDT only approximation coefficient will be transformed

whereas in 2D-WPT all of the coefficient will be recursively transformed. In the

proposed method, an image is decomposed into 4-levels. 4-level decomposition

results in 3n+1 sub-band, it means that decomposition results in 13 sub-bands.

The general transformation processes compress information from data. In wavelet

transform, if the basis signal for transformation has correlation with transformed

signal it will produce homogenous transformed coefficient. Referring to those

characteristics, characteristics of each sub-band can be used as information to form

feature vector. In order to form a feature vector, f=(x1, x2, x3,. . ., xn), the energy or

characteristic each sub-band of images have been calculated using formula (Table 1).

Table 1. Energy Calculation Formula

Energy Type Formula

Norm 2

∑∑= =

=M

m

N

n

NMCNM

e1 1

2

,1

1

Norm 1

∑∑= =

=M

m

N

n

NMCNM

e1 1

,2

1

Standard Deviation

∑∑= =

−=M

m

N

n

NMCNM

e1 1

2

,3 )(1

µ

Average Residual ∑∑ −=M N

NMCe µ,4

Entropy

∑∑−=M

NM

N

NM CCNM

e2

,

2

,5 log1

It is a common issue that data stored in feature vector have redundancy. To reduce

redundancy form the feature vector, PCA (Principle Component analysis) algorithm

should be employed [7][12][1]. With PCA algorithm, classification process could be

more effective and efficient

3.3 Testing the Classification Performance

In our research, we will employ the supervised learning to classify the texture. In

our study, in order to test the classification performance, we employ linear, quadratic

and mahalanobis classifiers. Bayesian rule has been used as a fundamental concept for

the entire classifiers. Bayesian rule is denoted as:

)(

)()|()|(

xp

PxpxP

jj

j

ωωω = (10)

Let say there are two classes of data ω1 and ω2, P(ωj) is prior probability and p(x)

is probability density function (PDF). Referring to the maximum likelihood theorem,

by maximizing the conditional probabilities P(ωj|x) decision of the class could be

obtained. In this case if P(ω1|x)>P(ω2|x) the data will belong to class ω1, otherwise if

ω2 data belong to ω2. Referring to Bayesian rule, if it assumes PDF is a normal

distribution or Gaussian, linear, quadratic and mahalamobis classifier could be

obtained. Quadratic, linear and mahalanobis could be denoted as:

Quadratic classifier:

||ln)(ln)()(2

1|max 1

jjjj

T

jPxx ∑−+−∑−− − ϖµµ (11)

Linear Classifier:

)(ln)()(2

1|max 1

jjj

T

j Pxx ϖµµ +−∑−− − (12)

Mahalanobis Classifier:

)()(|max1

jj

T

j xx µµ −∑−− − (13)

To obtain the data classes (refer to equation 11, 12, 13), there is a need to

maximize the conditional probability. In equations 11 and 12, all data are calculated

based on the relative distance between the data and data distribution, and also using

prior probability of the class as weight parameter. In mahalanobis classifier algorithm,

(equation 13), it calculates relative distance between the data and the data distribution.

4 Proposed Framework and Experiment

To this end, we have claimed that computer haptic texture classification could

replicate similar technique from computer vision. In the previous section, we have

described how wavelet and its feature extraction could be a potential candidate to

address this situation. Fig. 4 presents a proposed framework for a system that can

adopt both visual and haptic search. The framework combines both visual and haptic

texture extractions and classifications.

=

nx

x

x

x

f

...

3

2

1

Fig. 4. Proposed Framework involving the extraction and classification processes

The visual information as shown in Fig. 4 consists of the texture image data; this

information is passed to the extraction process. The process involves the wavelet

decomposition (using db-8) to get the characteristics from each image. At this stage

the energy calculation formula will be used. The feature vector f(x) is formed as a

result of this process.

In the classification stage, we employ the linear, quadratic and mahalanobis

classifiers to examine the most suitable classifier in terms of its performance. Using

the chosen classifier, the system will suggest some possible matching results.

4.1 Experiment

The effectiveness of wavelet decomposition for texture analysis and classification

has been tested in an experiment, using 76 samples of texture images. The texture set

consists of natural, artificial, and semi natural textures. The objectives of the

experiments are:

• To determine the texture classification performance for each classifier

• To identify the most effective and efficient variable to be used in a

classification process.

During the performance test, we use three different types of classifiers. We employ

linear, quadratic, and mahalanobis classifiers. Daubechies wavelet family has been

employed as an algorithm to extract the textures. Table 2 shows 16 tap of daubechies

wavelet transform filter coefficient.

Table 2. 16 Tap of Daubechies-8 Wavelet Transform Filter Coefficient

h(0) 0.0385 h(8) -0.0123

h(1) 0.2212 h(9) -0.0312

h(2) 0.4777 h(10) 0.0099

h(3) 0.4139 h(11) 0.0062

h(4) -0.0112 h(12) -0.0034

h(5) -0.2008 h(13) -0.0003

h(6) 0.0003 h(14) 0.0005

h(7) 0.0910 h(15) -0.0001

The textures have been decomposed into 4-level. It means that decomposition

processes results in 13 coefficient sub-band. To conduct the experiment we prepare a

learning sample and testing data sets. We create a random 512x512 pixel image from

the texture image. For each 512x512 pixel image we partition this image into 4

265x265 without overlapping each other. We generate 100 random images 128x128

from each 256x256 images. In other words, for each sample we have 400 samples.

We use 200 samples for learning process and the other 200 samples for testing.

To test the performance of the classification, we conduct three experiments with

different feature vectors and different classification methods. We employed the linear,

quadratic, and mahalanobis classifiers in the experiments, consecutively. For the first

experiment, we use the entire feature of vector element, meaning that all the extracted

information was used. The second experiment was to test the effect of the element

number with classification process. We start by using 1 feature vector element up to

the entire elements. In the third experiment, we use only one type of energy

calculation algorithm.

5 Findings and Analysis

As described earlier, in the experiment we have used the three classifiers to measure

the accuracy of classification performance. In each classification process we measure

the error rate of each method. The classification testing involves:

1) Measuring the number of feature vector element versus error rate

2) Measuring the effect using different type of feature vector

3) Comparing the classification method

4) Finding the optimum number of feature vector element

The first experiment was conducted by measuring the error rate for each feature

vector that have different element. Since we have Level-4 wavelet decomposition, it

means that we have 13 different sub-bands. For each sub-band we calculate the

energy using the formula in (Table 1), which means we have 65 different values to

form a feature vector. In this experiment we form a feature vector started using 1

element to 65 elements, f(x1) to f(x1, x1, x1,. . ., x65). In this test, quadratic classifier

gives the most accurate results when compared with linear and mahalanobis

classifiers. The classification results are summarized in Figure 5 and Table 3.

Table 3. 16 Tap of Daubechies-8 Wavelet Transform Filter Coefficient

Energy Norm1 Norm2 Std dev Ave. residual Entropy

1 0.8661 0.8656 0.8584 0.9212 0.8666

2 0.6899 0.8379 0.7063 0.5719 0.6389

3 0.4606 0.8200 0.5540 0.3503 0.5241

4 0.3198 0.7755 0.5337 0.2409 0.4041 5 0.3068 0.7460 0.3248 0.2226 0.3936

6 0.1672 0.7065 0.1607 0.1461 0.2459

7 0.1072 0.6691 0.1057 0.1022 0.1727 8 0.0917 0.6361 0.0744 0.0605 0.1594

9 0.0527 0.6205 0.0610 0.0311 0.1113

10 0.0348 0.6057 0.0562 0.0228 0.0840 11 0.0210 0.5903 0.0327 0.0179 0.0683

12 0.0126 0.5580 0.0199 0.0140 0.0524

13 0.0068 0.5432 0.1066 0.0119 0.0411

We carried out a second experiment similar to the first but for this time we only

use one type of energy to form the feature vector. This means that the feature vector

only formed from norm1, norm2, standard deviation, average residual and entropy

(Table 3). The experimental results show that the best performance was achieved with

average residual energy calculation. In this experiment quadratic classifier has been

employed and overall error rate is 1.19%. This suggests that average residual is more

effective than other energy calculation method.

From the experimental result, it shows that the number of element of feature

vector, and the number of energy used are affecting the accuracy of classification

process. Using the data of the experiment, optimization of the classification system

could be done by reducing the element of the feature vector. In Figure 5.1, it is

indicated that the optimum number element is 20. The visual representation shows

that the error rate starts to convergence when the number of feature vector element is

20. This means that if we add more elements in, more energy will be used. This does

not show any significant improvement to Figure 5.1.

The experiment has resulted in the quadratic classifier being the most accurate

method. Error rate for quadratic classifier is 0.18%. This result shows that feature

extraction using wavelet is suitable for texture classification. Based on the second

experiment it shows that the most effective and efficient energy calculation algorithm

is using the average residual, (Figure 5.2). Overall error rate for 13 number of energy,

i.e. element of the feature vector, is 1.19%. Overall accuracy is about 98%. From this

result we can suggest that wavelet feature extraction should be suitable for haptic

texture searching that deals not only with visual representation, but also the touch

sensation.

Fig 5.1 Fig 5.2

Fig 5. Classification Accuracy experiment by using different type of energy. In this particular

experiment, quadratic classifier has been employed. In this experiment, average residual gives

most effective feature vector element for classification. 1). error propagation for quadratic,

linear and mahalanobis classifier. 2). Propagation error for quadratic classifier for each type of

energy.

6 Conclusion and Future Work

In this paper, we have addressed the problem in haptic searching method and

proposed a framework that integrates both visual and haptic in a texture search

engine. We claimed that techniques in computer vision could be replicated for

computer haptic. We predicted that wavelet decomposition could be the best

candidate. In justifying our claim, we reviewed the standard 2D wavelet

decomposition. Then we defined an algorithm to extract the texture. The feature

extraction involves wavelet transform and five different energy measurements. The

construction of feature vector is done using different techniques. Firstly, we used the

entire energy from entire sub-band; secondly, we used only one type of energy for

entire sub-band.

The several series of experiments to find out the suitable method for extraction

and searching process show that quadratic classifier outperforms two other

classification methods. Also, based on the experiment, average residual is the most

effective and efficient energy measurements method, beside norm 2. The overall error

rate for average residual energy calculation in quadratic classification was 1.19%.

Since haptic rendering data and 2D images have similarity, based on this

experiment, we can conclude that 2D wavelet decomposition and wavelet energy

signature could be employed in haptic data extraction. With this experiment results

we could suggest that this method could be suitable for our proposed haptic texture

searching. Future work will include implementation of this method in our haptic

texture search engine.

References

1. Datta, R.: Image Retrieval: Ideas, Influences, and Trends of New Age, ACM Computing

surveys, vol. 40, No. 2, Article 5, (2008)

2. Conners, R., Wand C.A., Harlow, A.: Theoretical Comparison of Texture Algorithms, IEEE

Trans. Pattern Analysis and Machine Intelligence. 2, 204-222, (1980)

3. Azencott R.: Texture Classification Using Windowed Fourier Filters, IEEE Trans. Pattern

Analysis and Machine Intelligence. 19, 148--153, (1997)

4. Fountain S. R.: Rotation Invariant Texture Features from Gabor Filters, Computer Vision

ACCV'98, vol. 1352 (1997)

5. Nikam, S.B., Agarwal, S.: Wavelet Energy Signature and GLCM Features-based Fingerprint

Anti-spoofing. In Conference on Wavelet Analysis and Pattern Recognition. ICWAPR '08.

International, (2008)

6. Pun, C.: Log-Polar Wavelet Energy Signatures for Rotation and Scale Invariant Texture

Classification IEEE Transactions on Pattern Analysis and Machine Intelligence. 25(5),

(2003)

7. Shi, M.: Hyperspectral Texture Recognition Using a Multiscale Opponent Representation.

IEEE Transaction on Geoscience and Remote Sensing. 41(5), (2003)

8. Yektaii, M.: Cumulative Global Distance for Dimension Reduction in Hand written Digits

Database. Lecture Note in Computer Science, vol. 4781, pp. 216--222. (2007)

9. Mallat S.: A Theory for Multiresolution Signal Decomposition: The Wavelet

Representation, IEEE Trans on Pattern Analysis and Machine Intelligence. 11(7), 674--693,

(1989)

10.Daubechies, I.: Orthonormal Bases of Compactly Supported Wavelets. Comm. Pure and

Applied Math. 41, 909--996, (1988)

11.Cédric, V.: Generalized Daubechies Wavelet Families, IEEE Transactions on Signal

Processing. 55(9), (2007)

12. Farrell, M.D.: On The Impact of PCA Dimension Reduction for Hyperspectral Detection of

Difficult Targets, IEEE Transaction on Geoscience and Remote Sensing Letters. l.2(2).

(2005)

13.Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality Reduction by Learning an Invariant

Mapping. In IEEE Conference in Computer Vision and Pattern Recognition, pp. 1735--

1742. IEEE Computer Society Press, Los Alamitos (2006)

14. Wall, S.: An Investigation of Temporal and Spatial Limitation of Haptic Interfaces. PhD

Thesis, Department of Cybernetics, University of Reading (2004)

15. Klatzky, R.L. and Lederman, S.J. Touch, In A.F. Healy & R.W. Proctor (Eds.),

Experimental Psychology, vol. 4. In I.B. Weiner (Editor-in-Chief), Handbook of

Psychology, New York: Wiley. (2004)

16. Seungmoon, C., Tan, H.Z.: An Analysis of Perceptual Instability During Haptic Texture

Rendering. In 10th Symposium on Haptic Interfaces for Virtual Environment and

Teleoperator Systems, (2002)

17. Wu, J., Song, A., Zou, C.: A Novel Haptic Texture Display Based on Image Processing. In:

IEEE International Conference on Robotics and Biomimetics, pp. 1315--1320. (2007)