Ordering of diagnostic information in encoded medical images. Accuracy progression

��

��

A. PRZELASKOWSKI*, R. JÓ�WIAK, T. KRZY¯EWSKI, and A. WRÓBLEWSKA

Institute of Radioelectronics, Warsaw University of Technology, 15/19 Nowowiejska Str.,

00-665 Warsaw, Poland

A concept of diagnostic accuracy progression for embedded coding of medical images was presented. Implementation of

JPEG2000 encoder with a modified PCRD optimization algorithm was realized and initially verified as a tool for accurate

medical image streaming. Mean square error as a distortion measure was replaced by other numerical measures to revise

quality progression according to diagnostic importance of successively encoded image information. A faster increment of im-

age diagnostic importance during reconstruction of initial packets of code stream was reached. Modified Jasper code was

initially tested on a set of mammograms containing clusters of microcalcifications and malignant masses, and other radio-

grams. Teleradiologic applications were considered as the first area of interests.

Keywords: rate distortion theory, JPEG2000, medical image compression, diagnostic accuracy measure.

��

Teleradiology mostly understood as a possibility of tele-

consultation or even telediagnosis requires efficient tools

of diagnostic information management. Limitations of

fast and reliable data access in overcrowded networks,

huge database environments, developing multicast medi-

cal enterprises cause necessity to improve encoding prin-

ciples resulting in effective image flow management

tools. The most required features of data stream forming

are quality progression and scalability of embedded im-

age representation.

Progressive transmission is highly desirable when re-

ceiving imagery over slow communication links or more

generally slow access points. It enables increasing image

quality as more bytes are decoded sequentially from the

beginning of an encoded code-stream. Quality of the ana-

lyzed image is consequently improved as more data are

successively accessed. Depending on initially accessed

image information, farther rules of data availability may

be interactively changed, preliminary pattern recognition

or information routing may be arranged, etc. Increasing

image quality for teleradiology applications means the

data ordering according to its diagnostic content, depend-

ing on diagnostically important image features extracted

and assessed as objectively and reliably as possible. Such

concept of the data encoding is called diagnostic accuracy

progression. Natural scalability of data in wavelet domain

allows such successive refinement of diagnostic quality in

sequentially decoded images. Truncated peaces of en-

coded data stream are ordered according to the possibly

highest diagnostic content improvement, related to the

length of the data stream pieces.

Embedded bit-stream has the property that every pre-

fix of an encoded stream is itself an efficient representa-

tion of the image source which does not depend on a tar-

get compression rate. The desired level of decoded (or re-

constructed, received) image accuracy may be determined

interactively by the observer. Remote browsing, retrieval

of reference image databases, radiological chat based on

on-line image processing, interactive fitting of visualiza-

tion mode and many other teleradiological operations

highly require embedded image representation.

Among compression standards, JPEG2000 seems to

be especially well fitted to teleradiology applications

(Fig. 1). It contains embedding, scalability, and different

modes of progression of encoded information [1]. Effi-

cient data ordering, independent coding of small code-

blocks and data stream forming procedure was based on

embedded block coding with optimal truncation (EBCOT)

paradigm [2]. Post-compression rate-distortion algorithm

(PCRD) was used to optimize quality of embedded and

progressively coded image data. Moreover, the mecha-

nisms of error resilience, interactive protocols and API,

wireless applications, security aspects, and ISO Base Me-

dia File Format were developed [3]. JPEG2000's flexibil-

ity with respect to random access, code-stream reordering

and incremental decoding is highly exploitable in a net-

worked environment.

JPEG2000 was designed also considering medical appli-

cations [4], and accepted for DICOM [5]. Code-stream orga-

nization, especially with progression by quality, seems to be

Opto-Electron. Rev., 16, no. 1, 2008 A. Przelaskowski 87

OPTO-ELECTRONICS REVIEW 16(1), 0–0

DOI: 10.2478/s11772-007-0039-z

*e-mail: [email protected]

useful platform for diagnostic information management in

teleradiology. However, reliable measures and descriptors of

image diagnostic accuracy should be used in PCRD proce-

dure considering specificity of medical image data. It is be-

cause rate-optimization of encoded data stream according to

distortion measure was considered as realization of the pro-

posed concept of diagnostic accuracy progression. Typically

used distortion measure is a mean square error (MSE) but

diversified class of medical images with complex semantics

were often insufficiently characterized by MSE in com-

puter-aided radiological applications [6]. Therefore more re-

liable numerical quality measure is searched that correlate

more accurately than MSE with real diagnostic accuracy of

compressed and reconstructed images.

The proposed solution is a diagnostic quality measure

(DQM) of diagnostically important information that al-

lows getting progressive reconstruction of image content

with maximal increment of diagnostic significance in ini-

tial stream packets (with relatively small bit rate). Finally,

we get more diagnostically representative form of decom-

pressed images. As mentioned, a standard PCRD optimi-

zation procedure of compression enhancement was imple-

mented basing on MSE as a distortion D measure. Ac-

cording to the decreasing ÄD/ÄR factor, the data of suc-

cessive code blocks (i.e. independently encoded parts

of successive subbands and scales) are ordered progres-

sively by coding smaller and smaller peaces of image in-

formation. Study directed to farther optimization of infor-

mation progression through alternative selection of distor-

tion metrics was proposed to improve PCRD optimization

according to diagnostic criteria affecting distortion mea-

sure. Alternative D measures were designed and imple-

mented in a Jasper code according to PCRD rules and

C-based implementation procedures. Numerical assess-

ment with a help of vector measure of compressed

mammogram quality and subjective, perceptual assess-

ment of other, more reliable than MSE, diagnostic distor-

tion metrics approximating DQM was used to verify the

efficiency of accuracy progression. Maximum point dis-

tortion and average difference complemented the applied

spectrum of distortion measures. Possible diagnostic ac-

curacy improvement in progressively compressed test im-

ages was reported.

��

Encoding with diagnostic accuracy progression assumes re-

liable numerical models of diagnostic image content. Even

if such modelling is possible, it is strongly dependent on

pathology, imaging modality, patient conditioning, etc.

However, in many cases, local image features and small,

poorly distinguishable details are important for diagnosis.

Mammograms containing clusters of microcalcifications

were used as a good subject of teleconsultations with ne-

cessity of fast manipulation of huge data sets. To make our

consideration more universal, a set of radiograms (bones,

soft tissue, contrast, angiograms) with diagnostically im-

portant low contrast areas and details were tested addition-

ally. We tried to realize accuracy progression of encoded

and transmitted images by modification of PCRD algo-

rithm, i.e., replacement of MSE with complementary dis-

tortion measures based on local characteristics of data con-

tent. Enhanced codec was verified in order to test the abil-

ity of diagnostic quality improvement in successively re-

ceived images, and on-line reconstructions from communi-

cated JPEG2000 code stream packets.

Ordering of diagnostic information in encoded medical images. Accuracy progression

88 Opto-Electron. Rev., 16, no. 1, 2008 © 2008 SEP, Warsaw

Fig. 1. Modern paradigm of flexible image compression according to JPEG2000 standard that is useful for teleradiology.

��

One of the method to define the relationship between the

length of a code representation R and the distortion level of

information source D is the rate distortion function D(R).

Distortion measure should determine the usefulness of ap-

proximately reconstructed code representation at a given

specified length. Respectively, the defined distortion mea-

sure for specified data source model is a crucial element of

this theory.

Progressive data representation takes advantage of opti-

mal, maximum increment of reconstruction quality in an

initial encoding stage for fast identification of overall infor-

mation content. In this case, the used distortion measure

serves a function of image quality criterion. Information or-

dered by quality is presented in Fig. 2. Even at small bit

rate 0.01 bpp (bits per pixel) in initial reconstruction stage,

first signatures of diagnostically important information

(i.e., larger microcalcifications signs – bright spots of the

cluster located in the centre part of the image) were slightly

visible. A meaningful part of microcalcifications cluster

became visible at a rate of 0.02 bpp. However, single

microcalcifications were blurred and unclear. As bit rate

and detailed information increased, microcalcifications

were progressively sharpened, reaching satisfactory quality

at 0.5 bpp.

Quality progression was based on the optimized infor-

mation ordering (in a sense of R-D theory) in the embedded

bit stream. Maximum quality reconstruction of important

image features is searched at each embedding stage (Fig.

3). Each of the sub-block (truncated part of code-block) is

characterized by particular influence on the distortion de-

crease and length increment of coded bit stream basing on

initially estimated encoding efficiency. Mutual relation be-

tween these two dimensions defines R-D slope, which de-

termines encoding cost [symbolical slope of D(R) curve in

Fig. 3]. Encoding order of successive bit stream pieces

(sub-blocks) is from symbols with the highest encoding

slope of D(R) (what means violent information quality

growth in the first packets of data stream) to symbols with

the lowest D(R) slope.

Due to PCRD, after the whole set of wavelet coeffi-

cients was encoded and the bit stream was formed, optimal

R-D curves for all code blocks are determined. Two-tier

coding structure is created, the independent code block data

coding and the final coding of block contributions to each

quality layer (each sub-blocks in each quality layers).


Fig. 2. Quality progression of mammogram (selected ROI with microcalcifications) in JPEG2000 encoding. As bit rate was increased,

remaining details are progressively filled in gradual image reconstruction of encoded stream at specified length (bpp): (a) 0.01, (b) 0.02,

(c) 0.1, and (d) 0.5, respectively.

��

A class of medical images is one of the most demanding.

Considerable requirements are related especially to image

reconstruction fidelity together with important diagnostic

details preserving. It is worthy to notice that sometimes

very small or slightly visible details can determine the pres-

ence of pathology. Therefore they are meaningful and sig-

nificant. Ensuring diagnostic accuracy of a reconstructed

image becomes both necessary and superior condition, es-

pecially in determined by application flow condition, ac-

cessibility and existing bit rate limitations.

Quality progression assumes maximally large recon-

struction quality growth, defined by a predetermined dis-

tortion measure, even after initial, a small part of code

stream is read and processed. In order to optimize a decod-

ing process, with respect to diagnostic accuracy of recon-

structed images, it is indispensable to complete natural

quality criteria with required factors of diagnostic content

accuracy. This image accuracy is often defined by features

which are not strictly described by traditional quality mea-

sures. Realization of accuracy progression boils down to

searching most reliable, numerical diagnostic content mea-

sures that provide optimal selection and emphasis of diag-

nostically important information. Then, a modified R-D op-

timization procedure is able to order image information ac-

cording to its diagnostic significance. The idea of accu-

racy-driven R-D optimization that modifies classical qual-

ity-driven procedure was presented schematically in Fig. 4.

The presented design of proper DQM estimate was

based on earlier experiences with numerical diagnostic

quality measures of mammograms [6]. In particular, effec-

tive DQM should be searched in the category of vector

quality measures with scalar equivalent that generally char-

acterize image disturbances and describe diagnostically im-

portant local image features. The results were adapted to

JPEG2000 compression with quality progression. Approxi-

mates of DQM were implemented in reference software of

JPEG2000 standard. To verify the usefulness of DQM in

PCRD optimization, highly reliable assessment of recon-

structed image accuracy is required.

��

Commonly available JPEG2000 code (e.g. Jasper [7],

JJ2000 [8], Kakadu [9]) adapted the rate-based distortion

minimization encoding approach based on mean squared

error (MSE) metric. The encoder tries to meet the rates

while minimizing the distortion according to the following

formula

D G y j y jiz

b i ij

i

( )( � [ ] [ ]) ,� ��

2(1)

for pieces of data separated by the code-block truncation

points z. Here Diz( )

is the distortion contribution from the

code block Bi, the subband is identified by bi, and Gbi is the

squared norm of the wavelet synthesis base for this sub-

band (according to the notation of Ref. 1). The distortion is

calculated in wavelet domain and Gbi is used as the energy

gain factor because wavelet transform used in most

JPEG2000 applications is not unitary. Jasper [10,11] as a

free reference implementation of the codec specified in the



Fig. 3. R-D compression optimization: (a) graphically described

code sub-blocks to be encoded – coding cost is represented by

D(R) curve slope, (b) optimal sub-block order according to

maximal information growth (or distortion decrease) per

transmitted data sub-block, and (c) resulting optimal coding

presented in D-R plane.

Fig. 4. R-D based block data ordering for two different distortion

measures, MSE (red solid line) and DQM (green dashed line):

(a) the elements to be encoded (coding cost is represented by R-D

curve slope), (b) optimal encoding order resulting in maximal

information growth per transmitted data, and (c) optimal coding

effect presented for two measures in R-D plane.

JPEG-2000 Part-1 standard [12] was used for implementa-

tion of accuracy progression concept. Jasper library version

1.701.0 was used.

Perception of image information and observer interpre-

tation characteristics is required to optimize image accu-

racy progression. For visual information, the human eye is

the ultimate receiver. When considering image quality, one

definitely should take the human visual system (HVS) into

consideration. Most solutions used for perceptual improve-

ment in JPEG2000 compression were based on a weighted

MSE, i.e., CSF(contrast sensitivity function) weighting or

VIP (visual progressive) weighting that allows the weight-

ing to change as a function of embedded encoding rate.

Moreover, different visual masking schemes were applied

to utilize quantization artifacts. The modified equation ex-

tends distortion measure as

D w Gy j y j

m jiz

b bi i

iji i

( ) ( � [ ] [ ])

[ ]�

�

�

2

(2)

with the HVS weighting factor w and the masking factor m.

However, masking-sensitive distortion measures are

largely ineffective for small images, for images containing

many small regions with differing statistics. Moreover,

other ideas like perceptual distortion metrics were used to

control perceptual quality of JPEG2000 compressed im-

ages [13]. One step forward is including the meaning (i.e.

semantics) of ordered image information dependent on user

requirements. In case of teleradiology it is diagnostic con-

tent of image examinations.

�� !��

R-D optimization was based on the hierarchical structure of

code blocks (typically of size 64×64). After each data

block is independently bit-plane encoded from the most

significant bit-plane (MSB) to the least significant

bit-plane (LSB) using three coding passes, an embedded

bit-stream is then generated. The distortion reduction ÄD

and rate increase ÄR associated with each coding pass is

cumulated. A rate control procedure, which in JPEG2000 is

realized by PCRD-optimization stage, uses this information

to determine each coding block’s contributions to the final

bit-stream according to flexible truncation points and es-

tablished length constraint. The distortion is controlled as

follows

D Gy j y j

m jiz

bi

p ji

iji

iz

( )

( [ ])( � [ ] [ ])

[ ],

( )

�

�

�

2

(3)

where p jiz( )

[ ] refers to the index of the bit plane to which

sample y ji[ ] has been coded prior to the truncation point z,

� [ ]( )

y jip

is the reconstructed value of y ji[ ] from the quanti-

zation index with j as the counter of binary samples in code

stream.

If noticed that according to PCRD-opt algorithm all

samples in coding pass P pk( , )are refined from the bit-plane

p � 1 to the bit-plane p, distortion reduction may be esti-

mated according to the following formula

�D

G y j y j y j y j

ip k

bi ip

i ip

i

( , )

( ) ( )( � [ ] [ ]) ( � [ ] [ ]

�

� � � �

�1 2

� �) ,

( , )

2

j Pip k

�

�

(4)

where the bit-plane p and type of the coding pass k (propa-

gation, cleanup or magnitude refinement) determining trun-

cation points.

Furthermore, dequantization rule of mid-point recon-

struction implies that

� [ ] ( [ ])( [ ] ) [ ]( )

( ) ( )

y j sign y jv j if v j

ip

i

pi i

pip

�

� �2 01

2�

0 0if v jip( )

[ ]

,

�

�

��

(5)

with the quantization index v and the step size Ä. Then, the

distortion assigned to the sample y ji[ ] when reconstructed

down to the bit-plane p is given by

D G y j y j

Gy j v j

ip k

bip

bii

pi i

p

( , ) ( )

( )

( � [ ] [ ])

[ ] ( [

� � �

�

�

2

2 � �

�

] ) [ ]

[ ] [ ]

( )

( )

� �

�

�

�

�

1

2

2

2

0

0

if v j

y j if v j

ip

i ip

(6)

Let the fraction part of y jip

i[ ] 2 � is denoted by~

[ ]( )

v jip

,

i.e.

~[ ]

[ ][ ]

[ ] [ ]( ) ( )

v jy j

v jy j y j

ip i

pi

ip i

pi

i

pi

� � � �

�

�2 2 2� � �

�

�

�

�

�

�

� [ , )01 . (7)

After substituting Eq. (6) and Eq. (7), Eq. (4) may be trans-

formed into

� �

�

�

D G

v j v j

ip k

bip

i

ip

ip

( , )

( ) ( )~[ ]

~[ ]

�

� �

�

�

�

�

�

�

2

2

2

2

12

1

2

2

�

��

� � � �

�

�

�

�

j P

v j

ip

ip

ip k

ip

v j v j

( , )

( )

,

[ ]

( ) ( )~[ ]

~[ ]

1

12

2 1 �1

2

2

1

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

j P

v ji

p k

ip

( , )

( )

,

[ ]

, (8)

with remark that v jip( )

[ ] � 1 implies v jip( )

[ ]�

�

10, and

v jip( )

[ ] � 1 implies v jip( )

[ ]�

�

10. Furthermore, it is worth

observing that~

[ ]( )

v jip

is a function of~

[ ]( )

v jip�1

� �~

[ ]~

[ ]~

[ ]( ) ( ) ( )

v j v j v jip

ip

ip

� �

� �

2 21 1

, (9)

and when a sample first becomes significant in the

bit-plane p, v jip( )

[ ] � 1 and 2 11~

[ ]( )

v jip�

� . Using this fact,

the distortion reduction may be denoted as


� �

�

�

D G

Ts v j Tm v j

ip k

bip

i ip

ip

j Pi

( , )

( ) ( )~[ ]

~[ ]

(

�

�

� �

�

22

2 1 1

p k

ip

ip k

ip

v j

j P

v j

, )

( )

( , )

( )

,

[ ]

,

[ ] �

�

�

��

�

�

�

�

�

�

�

�

�

�

�

11 �

, (10)

where the operators Ts and Tm are defined as

Ts v v v(~) (

~) (

~)� � �2 2

2 3

2

2, (11a)

� � �Tm v v v v j(~) (

~)

~ ~[ ]� � � � �2 1 2 2

2 1

2

2

, (11b)

with simplification for lossless coding of the final bit plane

Ts v v(~) (

~)� 22and Tm v v(

~) (

~)� �2 12. The operators may be

implemented as small look-up-tables (LUTs) with initially

determined values for all coding passes (Ts for propagation

or cleanup, and Tm magnitude refinement). It is possible to

index LUTs by the first few bits without impact on the per-

formance of the coder. For example, in Jasper, all LUTs are

matrices containing 128 elements.

�� !��

To modify distortion measure, defined by Eq. (3), or to im-

plement more complex measure of distortion we need to ei-

ther add or transform previously deduced relationships.

The following distortion measures (between f and �f ) en-

hancing local data characteristics were selected and imple-

mented additionally to realize diagnostic accuracy progres-

sion

ADMN

f m n f m n

m n

� ��

1�( , ) ( , )

,

, (12)

MD f m n f m nm n

� �max �( , ) ( , ),

. (13)

These two metrics were positively verified as the distor-

tion measures in wavelet compression of medical images

[6]. Implementations of AD and MD were based on a mod-

ified Jasper code by adding modified LUTs and some new

procedures in function determining PCRD optimization.

Following the previous reasoning, distortion reduction

may be redefined basing on AD as follows

�D G

y j y j y j y j

ip k

bi

ip

i ip

i

( , )

( ) ( )� [ ] [ ] � [ ] [ ]

�

� � ��

��

�

�

�1

�

�

�

j Pip k( , )

, (14)

where Gbi is a square root from previously used energy

factor and rest of indexes are defined in the same way as

for MSE case. Using again mid-point dequantization rule,

Eq. (5) and Eq. (7), Eq. (14) may be transformed analo-

gously to a formula

�

�

D G

v j v j

ip k

bip

i

ip

ip

j

( , )

( ) ( )~[ ]

~[ ]

�

� ��

��

�

��

�

�

�

2

21 1

2

P

v j

ip

ip

ip k

ip

v j v j

( , )

( )

,

[ ]

( ) ( )~[ ]

~[ ]

�

�

�

� � ��

��

�

1

1 1

22 1

��

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

j P

v ji

p k

ip

( , )

( )

,

[ ] 1

, (15)

and following earlier reasoning we achieve

�

�

D G

Ts v j Tm v j

ip k

bip

i ip

ip

j P

( , )

( ) ( )(~

[ ]) (~

[ ])

�

�

� �

�

2

1 1

ip k

ip

ip k

ip

v j

j P

v j

( , )

( )

( , )

( )

,

[ ]

,

[ ] �

�

�

��

�

�

�

�

�

�

�

�

�

11 �

�

�

, (16)

with the operators Ts and Tm defined as follows

Ts v v v(~)

~ ~� � �2 2

3

2(17a)

� �Tm v v v v j(~)

~ ~ ~[ ]� � � � �2 1 2 2

1

2(17b)

and Ts v v(~)

~� 2 , Tm v v(

~)

~� �2 1 for the last losslessly en-

coded bit plane.

In case of MD, maximum value of absolute sample dif-

ferences for each truncated coding pass P pk( , )is used to

characterize the reduction of distortion

� �D G

Ts v j if v

ip k

bip

i

j P

ip

ip

ip k

( , )

( ) ( )

max(~

[ ])

( , )

�

�

�

2

1[ ]

(~

[ ]) [ ]( ) ( )

j

Tm v j if v jip

ip

�

�

�

��

�

�

�

��

�

1

11

, (18)

where Ts and Tm were defined in Eq. (17a) and Eq. (17b)

and the following, respectively. Implementation of more

complex distortion metrics in Jasper code increases codec

complexity and has limited flexibility because of code

structure limitations.

�� !

A design and optimization of reliable DQM presents many

difficulties, mainly due to necessity of objective diagnostic

content description and assessment criteria standardization.

However, our previous study concerning vector quality

measures of compressed medical images effected in hybrid

vector measure (HVM) experimentally verified on a large

group of mammograms. HVM was highly correlated to ra-

diologists’ subjective rating and interpretation [6]. Realiza-

tion of PCRD-DQM scheme basing on six HVM factors

could be really flexible and efficient tool for different pa-

thology description and numerical assessment. However,

including human visual system with complex perception

models increased Jasper codec complexity too much. Addi-

tionally, implementation of such models in multilayer, hier-



archical and packetized structure of embedded code stream

required significant metric simplifications with loosing es-

sential advantages. Thus, we proposed a preliminary, sim-

plified form of DQM to verify the rightness of investigated

concept of diagnostic accuracy progression in JPEG2000

compression. Basing on discussed implementation issues

of MSE, MD and AD, the following form of DQM was

proposed and verified

DQM MSE MD AD� � �� 1 2 3, (19)

where �i are fitted depending on image characteristics, pa-

thology, and other test data conditions.

Different forms of DQM were verified experimentally

with various combinations of { } ,...,�1 1 3i� and numerical and

subjective assessment of reconstructed mammograms con-

taining clearly defined pathologies. According to initial

consultations and result discussions, the important role of

point distortions for diagnostic quality assessment was as-

sured by increased weight of MD in comparison to appro-

priate mean image characteristics based on MSE or AD.

The images reconstructed after PCRD-AD compression

(with �1 0� , � 2 0� , �3 1� ) achieved generally lower

numerical and subjective scores of quality assessment in

comparison to standard PCRD-MSE optimization (�1 1� ,

� 2 0� , �3 0� ). Hence, to make DQM extremely simple

the value of �3 factor is suggested to be reduced or even re-

set for the presented applications. Thus, the proposed form

of DQM is DQM MSE MD� � 2 but for low bit rates of

pathological mammogram progression only MD measure

seems to be the most useful for PCRD.

"� #$��

The compression process was carried out using mammo-

grams from reference database digital database for screen-

ing mammography (DDSM) including the clusters of ma-

lignant microcalcifications and in some cases masses.

Moreover, several radiograms were encoded to extend a set

of test images to other modalities mostly used in teleradio-

logy. Often used numerical objective peak signal to noise

ratio (PSNR) was applied to compare the efficiency of

JPEG2000 compression with the modified PCRD-optimi-

zation. Additionally, a subjectively optimized picture qual-

ity scale (PQS) [9] with implemented models of human vi-

sual system and HVM deigned for diagnostic accuracy as-

sessment of medical images completed computational cri-

teria of progressive compression optimization.

Especially, HVM provides a good accuracy approxima-

tion for radiograms. It was designed according to gold stan-

dard describing local image features important for diagno-

sis. Adaptive version of HVM (A-HVM) was proposed to

increase its flexibility for different pathologies and imaging

modalities. A-HVM assures comparable influence of each

quality factor (i.e. element of HVM vector) to HVM equiv-

alent (impact of each distortion estimate depends on mean

level of measured distortions for given class of test images

and verified codec). Additionally, diagnostic A-HVM (DA-

HVM) enhances diagnostically important factors (e.g. MD

for small bright spots of microcalcifications) according to

radiological reasons. Experimental verification of Jasper

codec with improved PCRD optimization was done in the

following stages:

• initial numerical PSNR, PQS, and HVM-based (in

adaptive and diagnostic mode) quality assessment,

• subjective, psychovisual assessment of depicted diag-

nostic features (visibility of grouped bright spots in

mammograms) or general image quality perception in

accuracy progression.

The observers carried out visual quality testing by rat-

ing the images encoded with the modified PCRD in com-

parison to the standard, MSE-based PCRD. Relative scale

from –3 to 3 (“–3” indicated definite worse quality; “–2”

indicated worse quality; “–1” indicated slightly worse qual-

ity; “0” means images equal in quality; “1” indicated

slightly better quality, “2” indicated better quality, “3” in-

dicated definite better quality) was used. Seven observers

experienced in medical image processing, analysis and rec-

ognition evaluated the examinations independently. The as-

sist of radiologist confirmed the preferences of realized ac-

curacy progression.

The results of numerical assessment for 13 representa-

tive test mammograms containing local clusters of patho-

logical bright spots and 13 other radiograms were pre-

sented in Table 1. The assessment was made after compres-

sion with the following rates, 0.001, 0.002, 0.005, 0.01,

0.05, and 0.1 simulating progressive transmission reality.

The images were grouped into the following sets:

• mammograms with clearly defined pathology rated ac-

cording to microcalcifications quality,

• radiograms with bright objects and dark background

rated according to general quality criteria,

• radiograms with dark objects in bright background

rated according to general quality criteria (see Fig. 5).

Mean subjective scores were collected in Table 2. The

selected examples of compression results were shown

in Figs. 6, 7, and 8.

Numerical assessment does not indicate the best

PCRD version evidently. According to PSNR, standard

PCRD-MSE (accidentally PCRD-AD) scheme tends to be

the best. It confirmed MSE-based quality criteria of data

ordering implemented in PCRD optimization algorithm

but PSNR results suffered from lack of fidelity to ob-

server’s criteria. Moreover, it was noticed that PSNR re-

sults for simple DQM in PCRD were very close to PCRD-

MSE results. This tendency was confirmed by PQS-based

image quality assessment but for lower bit rates, PCRD-

DQM efficiency was indicated to be even comparable to

PCRD-MSE. The numerical results based on HVM mea-

sure were more straightforward and convincing argument

for DQM-based improvements in PCRD. AHVM indi-

cated suggestively better quality of DQM-driven progres-

sion for mammograms and radiograms with dominant

bright information but the recommendation was unclear

for reversely oriented radiograms. Diagnostic HVM, ap-




Table. 1. Mean numerical scores of 13 compressed mammograms in comparison to originals. Lower HVM means better

quality, and higher PSNR or PQS means better quality. The best results were bolded.

Image quality

measure

Rate

Pathological mammograms Radiograms

with bright diagnostic details

Radiograms

with dark diagnostic details

PCRD with

MSE MD AD DQM MSE MD AD DQM MSE MD AD DQM

Mean PSNR

(dB)

0.001 39.93 39.15 39.88 39.82 29.31 27.77 29.28 29.33 22.48 20.78 22.36 21.83

0.002 41.21 40.56 41.13 41.08 32.56 31.82 32.55 32.42 26.26 25.9 26.01 26.22

0.005 42.64 41.87 42.61 42.57 35.66 34.95 35.58 35.51 29.75 29.38 29.76 29.64

0.01 43.87 43.07 43.86 43.82 37.65 36.93 37.6 37.48 32.33 31.59 32.24 32.07

0.05 48.13 47.61 48.03 48.07 42.27 41.63 42.25 42.21 38.9 38.27 38.88 38.85

0.1 51.7 51.17 51.54 51.67 45.23 44.75 45.19 45.19 41.92 41.34 41.83 41.88

Mean PQS 0.001 4.39 4.31 4.38 4.37 –20.4 –21.5 –21.1 –20.0 –46.2 –42.8 –44.8 –44.4

0.002 4.5 4.44 4.49 4.49 –13.3 –13.6 –13.6 –13.4 –29.2 –29.7 –31.1 –29.5

0.005 4.64 4.56 4.63 4.63 –6.62 –8.03 –6.74 –6.76 –15.9 –17.1 –15.8 –16.2

0.01 4.77 4.67 4.76 4.76 –2.56 –4.04 –2.67 –3.0 –8.72 –11.6 –8.89 –9.52

0.05 5.08 5.03 5.07 5.08 2.26 1.78 2.47 2.27 0.65 –0.24 0.81 0.53

0.1 5.20 5.18 5.2 5.20 3.43 3.35 3.43 3.42 2.33 1.81 2.6 2.34

Mean adaptive

HVM

0.001 34.14 36.34 34.15 33.54 21.87 23.82 23.24 20.13 13.08 18.15 13.42 16.09

0.002 27.59 26.7 27.62 25.27 12.7 12.59 12.72 12.35 8.61 9.04 8.79 8.71

0.005 20.8 20.7 20.83 18.68 8.33 8.44 8.4 7.89 5.55 5.84 5.49 5.55

0.01 16.37 16.65 16.64 14.7 6.26 6.31 6.37 6.03 4.08 4.52 4.14 4.24

0.05 8.14 8.46 8.3 7.9 3.34 3.36 3.25 3.11 1.67 1.85 1.66 1.67

0.1 5.4 5.67 5.5 5.34 2.26 2.34 2.26 2.22 1.18 1.29 1.16 1.16

Mean diagnostic

HVM

0.001 16.19 14.82 16.07 15.04

0.002 13.55 10.75 13.38 10.86

0.005 10.1 8.19 10.05 7.76

0.01 7.76 6.58 8.02 6.03

0.05 3.54 3.44 3.61 3.23

0.1 2.27 2.32 2.31 2.2

Fig. 5. The example of test radiogram (left) with dark diagnostic structures in bright background, inversed version of the original, more

susceptible to JPEG2000-based accuracy progression (right).

plied only for mammograms, indicated PCRD-MD as the

most effective optimization for lower, and PCRD-DQM

for higher bit rates, both significantly better than PCRD-

MSE. HVM-based numerical assessment in adaptive and

diagnostic mode has had full convergence with subjective

rating, respectively.

According to Table 2, mean scores of 13 test mam-

mograms indicated MD and DQM = MSE + 2MD used in

PCRD optimization as a better tool of diagnostic accu-

racy progression for the rate 0.001 (Figs. 6 and 7).

Higher accuracy means undoubtedly better visibility of

pathology grounded in clinical opinions (DDSM). Clus-

ter of microcalcifications was practically invisible for


Table 2. Subjective test scores. 26 radiograms compressed to the rate 0.001 were scored according to clear diagnostic cri-

teria or general image overview. According to radiological practice, two classes of radiograms were used, with bright and

dark dominant of diagnostically important information.

Subjective rating No. of

images

Mean, relative score of range from –3 to 3

MD versus MSE DQM versus MSE

Diagnostic quality assessment (clearly defined pathology – bright spots

of microcalcifications)

13 1.47 1.08

General quality assessment (bright objects with dark background

– radiograms, angiograms)

7 –1.08 0.57

General quality assessment (dark objects with bright background

– contrast radiograms, angiograms)

6 –1.91 –0.83

Fig. 6. Evaluation of progression efficiency with the rate 0.001 for

different PCDR distortion measures; mammogram ROI with

cluster of malignant microcalcifications in version of: (a) original,

(b) compressed with PCRD-MSE, (c) compressed with PCRD-

MD, (d) compressed with PCRD-AD, and (e) compressed with

PCRD-DQM (MSE+2MD).

Fig. 7. Evaluation of progression efficiency with the rate 0.001

for different PCDR distortion measures, two ROIs of mam-

mograms with cluster of malignant microcalcifications in

version of: (a) original, (b) compressed with PCRD-MSE, (c)

compressed with PCRD-MD, and (d) compressed with PCRD-

DQM (MSE+2MD).

PCRD–MSE with the rate 0.001, while modified PCRD

schemes have already demonstrated broad outlines. For 9

cases, all observers agreed that modified PCRD is defi-

nitely more useful than standard PCRD-MSE, higher di-

agnostic accuracy was evident after decoding of the first

part of bit stream. Diverse opinions were noticed for

other 4 cases what means comparable accuracy of test

images.

MD used as a distortion measure tends to preserve

small image details, note that even contours of particular

microcalcifications were slightly visible. However, PCRD-

MD optimization has a general ability to sharpen near de-

tail context and smooth image background too much. De-

fined simple approximation of DQM for PCRD seems to be

a “golden mean” compromise between details preservation

in PCRD-MD and general image fidelity reconstruction ac-

cording to MSE-based criterion.

Subjective quality assessment was extended to other

radiograms with scoring a general psychovisual percep-

tion of image quality. Diagnostic interpretation was

omitted for those cases. According to readers’ opinions,

PCRD-DQM was slightly superior to PCRD-MSE what

means preserving of image quality according to typical

criteria and slight enhancement of small details, subtle

edge extraction, etc. Bright details extraction in PCRD-

MD procedure was not useful for this kind of images. Re-

versed orientation of image information with dark struc-

tures of interests in bright background is not susceptible

to JPEG2000-based accuracy progression because of as-

sumed data content ordering according to decreasing sig-

nal energy (i.e., bright spots first, dark spots last). Mean

subjective score for 0.001 rate of PCRD-DQM represen-

tation of test image presented in Fig. 5 was –1.29 while

its greyscale inversion was rated to 0.8. Good advice for

those cases is inversion of greyscale for images commu-

nicated during teleconsultations.

Subjective opinion confirmed usefulness of HVM

as numerical accuracy estimate what is really important for

further JPEG2000 codec optimization in teleradiology ap-

plications. Generally, improvement in rate-distortion opti-

mization procedure applied for JPEG2000 compression of

medical images was noticed. Replacement or rather com-

plement of MSE as a base metric for distortion measure-

ment was proposed to make information ordering more

flexible and matched to perception or semantic require-

ments. Experimental subjective scores and numerical as-

sessment results depicted faster, more reliable support of

image accuracy for the initial, low bit rate image progres-

sion during transmission of the first image data packets ac-

cording to JPEG2000 paradigm.



Fig. 8. Evaluation of progression efficiency with the rate 0.001 for different PCDR distortion measures. Selected ROI from radiograms

with important structures in version of: (a) original, (b) compressed with PCRD-MD, (c) compressed with modified PCRD-DQM

(MSE+2MD), and (d) compressed with basic PCRD-MSE. Brighter details and structure edges were better visible for MD and DQM but

darker areas (wedges) were more reliably reconstructed by DQM and MSE driven progression.

%� ��

The primary objective of our study comprises image com-

pression optimization with information ordering according

to a semantic content. This topic incorporates into several

up-to-date research areas carried out in many research cen-

tres around the world: semantic technologies with image

content analysis, a more reliable observer, implied as an in-

formation receiver, characteristics, and diagnostic content

descriptors within a framework of evidence based medi-

cine. Finally, a powerful tool for efficient design of an im-

age representation according to radiological criteria and re-

quirements was sought. The compression paradigm of

JPEG2000 standard with PCRD optimization was selected

as a platform of integrated solution design for medical im-

age exchange, transmission, storing, analysis and automatic

understanding.

A new approach to semantic JPEG2000 stream optimi-

zation was proposed. Existing methods concern mainly

modification of the wavelet distributions across scales and

subbands by setting the weights based on the human visual

perception. The presented method concerns modification of

standard progressive image coding, optimized according to

the R-D theory. The modification is based on extension of

MSE-based distortion metric that describes image content

and its semantic characteristic more precisely, taking into

account more complex receiver characteristic. This charac-

teristic is both significant and complicated for the sake of a

practical application to the medical image compression.

Taking an advantage of our experience in mammogra-

phic image analysis, processing and understanding as well

as previous results concerning diagnostic quality measure

construction, the new accuracy progression concept was

proposed. This method seems to be very attractive if ap-

plied to teleradiological systems.

Accuracy progression results in monotonically decreas-

ing diagnostic value increment of the reconstructed image,

which boils to obtain as much as possible diagnostically

important information, progressively completed at succes-

sive reconstruction stages. The obtained preliminary results

confirm the rightness of the presented conceptions. Future

experiments will concern diagnostic quality measure exten-

sions what requires more deep improvements in the codec

structure. Clinical tests with pathology detection that verify

usefulness of accuracy progression for real applications in

teleradiology are planned.

The method of diagnostic information ordering within a

framework of JPEG2000 standard presented here exists as

a practical implementation of diagnostic features extrac-

tion. These features, predominant part of which was lo-

cated at the beginning of a communicated code stream and

advance image processing allow for content based image

retrieval and pathology searching. Because of the enhanced

data representation, functioning of computer aided diagno-

sis (CAD) systems and objective description of stored im-

age information can be potentially improved.

��

1. D.S. Taubman, and M.W. Marcellin, JPEG2000. Image

Compression Fundamentals, Standards and Practice,

Kluwer Academic Publishers, 2002.

2. D.S. Taubman, “High performance scalable image compres-

sion with EBCOT”, IEEE T. Image Process. 9, 1158–1170

(2000).

3. www.jpeg.org/jpeg2000

4. D.H. Foos, E. Muka, R.M. Slone, B.J. Erickson, M.J. Flynn,

D.A. Clunie, L. Hildebrand, K.S. Kohn, and S.S. Young,

“JPEG 2000 compression of medical imagery”, Proc. SPIE

3980, 85–96 (2000).

5. dicom.nema.org/

6. A. Przelaskowski, “Vector quality measure of lossy com-

pressed medical image”, Comput. Biol. Med. 34, 193–207

(2004).

7. Jasper: http://www.ece.uvic.ca/~mdadams/jasper/

8. JJ2000: http://jj2000.epfl.ch/

9. Kakadu: http://www.kakadusoftware.com/

10. M.D. Adams and F. Kossenini, “JasPer: a software-based

JPEG-2000 codec implementation”, in Proc. IEEE Int.

Conf. Image Process. 2, 53–56 (2001).

11. ISO/IEC 15444-5, “Information technology – JPEG 2000

image coding system – Part 5: Reference software”, ISO/

IEC JTC 1/SC 29/WG1, Dec. 2001.

12. ISO/IEC 15444-1, “Information technology – JPEG 2000

image coding system – Part 1: Core coding system”, ISO/

IEC JTC 1/SC 29/WG1, Jan. 2001.

13. Z. Liu, L.J. Karam, and A.B. Watson, “JPEG2000 encoding

with perceptual distortion control”, IEEE T. Image Process.

15, 1763–1778 (2006).

14. http://marathon.csee.usf.edu/Mammography/Database.html

15. �PSNR MSE� 10 40952

log .

16. M. Miyahara, K. Kotani, and V.R. Algazi, “Objective pic-

ture quality scale (PQS) for image coding”, IEEE T.

Commun. 46, 1215–1226 (1998).


Ordering of diagnostic information in encoded medical images. Accuracy progression

Documents

Transcript of Ordering of diagnostic information in encoded medical images. Accuracy progression