Watermarking via Optimization Algorithms forQuantizing Randomized Statistics of Image Regions

M. Kıvanc Mıhcak

Microsoft Researchkivancm@microsoft.com

Ramarathnam Venkatesan

Microsoft Researchvenkie@microsoft.com

Mustafa Kesal∗

University of Illinois, Urbana-Champaignkesal@comm.csl.uiuc.edu

Abstract

We introduce a novel approach for blind signal watermarking and apply it toimages. We derive randomized robust semi-global features in a suitable transformdomain (wavelets in case of images) and quantize them in order to embed the wa-termark. Quantization is carried out by adding an embedding sequence to the host;this sequence is computed by solving an optimization problem whose parametersare known to the information hider, but unknown to the attacker. We experimen-tally identify some conditions (our randomizations are aimed at achieving them)satisfied by our parameters, which formally and experimentally imply robustnessof our algorithm against malicious optimal estimation attacks. We also tested itsrobustness against many generic (i.e. non-malicious benchmark attacks ) attacks.

1 Introduction

Watermarking (wm) aims to embed a message (i.e., “watermark”) reliably in host data,where the message can be later detected and used by a decoder for identification or copy-right information purposes. The embedding is done by an encoder using some secret key,which results in an output that is required to be perceptually (approximately) the sameas the original. The watermarked image may then undergo many possible changes byusers and attackers: unintentional modifications and malicious attacks that particularlyaim to disable the detection of the watermark. Ideally, the system must resist modifi-cations and attacks as long as they result in images that are of perceptually the samequality.

We distinguish here between data embedding (for communications purposes) and wa-termarking (for security purposes) problems, both of which fall into the category ofinformation hiding. The latter involves a malicious attacker, and our approach empha-sizes this aspect and presumes the attacker to be resource-bounded, and our approachmay be suitable data embedding by considering a very mild adversary.

There are fundamental difficulties in modelling such attacks and providing a formalsecurity analysis of a given algorithm. Firstly, we do not have a reasonable model ofperceptually non-distortive attacks. The standard models measuring some Euclidean-like norm in some canonically associated vector space falls short when one considers

∗Work was performed while M. Kesal was with Microsoft Research.

malicious attacks. Second, the security arguments must be carried out in a model thatexplicitly recognizes that the attacker is resource bounded and this is the case we areinterested in.For a discussion related to these issues we refer to [?] As there, our blindwatermarking method has two main generic steps:

(1) Given an image, derivation of robust image characteristics (a vector) that are notlocal (i.e. in the domain where the watermarking takes place) and are in fact semi-globalin nature.

(2) Quantization of these characteristics for information hiding.It is not obvious that why such characteristics may exist or how to derive them.

The motivation for the semi-global nature of the characteristics is that experiments indi-cate that simple adversarial attacks can change local characteristics quite dramatically.Randomness is used to implicitly hide the characteristics or the metric the algorithmis using, by making it not easy for an adversary (without the key) to compute them.Finally the characteristics are so chosen such that they will stay approximately invariantif one introduces non-noticeable visual distortions. Note that by hiding information insuch statistics, we are implicitly imposing or assuming a perceptual model on images.The robustness of similar statistics has previously been studied for the image hashingproblem [?], where no information embedding was considered.

In our scheme, we use random linear statistics of random regions (i.e., these statisticsare given by pseudo-randomly weighted linear combinations of data in randomly chosenregions, where the weights are chosen randomly). These regions can potentially overlapand need not be connected in general. In [?], we hide information in the statistics of non-overlapping regions. Confining to non-overlapping regions not only brings limitations tothe rate of the watermark to be embedded, but also in case of strong watermarks blockingartifacts would be visible due to non-overlapping nature of randomly chosen regions. Theproposed scheme in this paper gets rid of these limitations by designing an embeddingsequence for random overlapping regions using minimum-norm criterion. The existenceis guaranteed under some mild assumptions. As a natural byproduct of our quantization-based approach, we can perform decoding and detection jointly at the receiver side (unlikespread-spectrum based schemes).

Randomized steps of our algorithm are crucial in terms of security and constitutean essential contribution of this work (for other works using explicit randomization asa defense see [?] and references in [?]). One can view this as a way of minimizingthe chances that an adversary can foil certain security related conditions specified later(which imply robustness against estimation attacks), but it is only part of the picture.We need to formalize this last remark and the quantizations studied in this work needto be extended to higher dimensions. Note that, when randomization is mentioned inthis paper, it should be understood that the randomization is carried out by means of arandom number generator whose seed is the secret key, where this key is known to boththe encoder and the decoder, however unknown to the attacker.We experimentally showthe robustness of our algorithm against a particular type of malicious estimation attacks,where the attacker knows the unmarked host data statistics and partial information aboutthe randomized steps of our algorithm. However, we stress that robustness against sucha particular type of malicious attack (where the attacker has clearly the upper handdue to the information he has), by no means, implies absolute security for the proposedapproach. Furthermore, we present successful experimental results relating to robustnessagainst bench mark attacks that include a wide range JPEG compression and independentadditive white Gaussian noise (AWGN) addition type attacks.

2 Watermarking algorithm

Our algorithm will use a secret K and we shall assume that a cryptographically securepseudo-random generator is used to suitably randomize our steps below. We write B ={0, 1} representing the domain of each WM element. We represent the signals as vectors,which does not describe the geometry of a 2-D picture, but is sufficient to describe ouralgorithm. In the notation below, subscripts denote individual elements of vectors, unlessotherwise stated.

s ∈RN : The host signal.w ∈BM : Watermark to be embedded, an encoded version of some given message.Ri: A pseudo-randomly chosen region denoting a subset of {1, 2, . . . N}, i ≤ M .ci∈RN :Pseudo-random host-dependent weights, one ci for each bit wi of w, 1 ≤ i ≤ M .

m ∈ RM : Random linear statistics of host, mi4=

∑j∈Ri

cijsj, 1 ≤ i ≤ M .

Define f(c1, ..., cM , s) = m, for a fixed choice of regions.C: A discrete set of pseudo-randomly specified points in RM (codebook).QK : BM × RM → C: A keyed quantization algorithm.

We drop K and write Q(w,m) = m. Given these definitions the watermarkingalgorithm is a map (dependent on K) that produces the watermarked signal x ∈RN :

s 7→ x, so that

f(c1, . . . , cM ,x) = Q(w,m) =m.This will have many solutions but we pick our x to minimize certain distortion metric

described below. In general distortion metrics are not helpful (and can be a main mod-eling weak-point) but our metric is randomized, and as such is not efficiently computableby an adversary who does not know K.

Note that the above transformation f (·) is non-linear if ci are host-dependent in anon-linear fashion. In this paper we mainly consider ci to be independent of the input,and our limited experiments produced similar results when the dependencies were non-linear (which is important from security view point, and not explored here).

This work extends [?] in a crucial way, where the regions are not overlapping. Themotivation is that of security concerns (as it enables increasing the dimensionality ofthe problem) as well as to minimize the perceptual distortions introduced by the water-marking procedure. The non-trivial problem is that the watermarking criteria which isspecified globally can pose contradictory conditions on data in the intersection of regionsRi and Rj, where weighted average in Ri may have to be increased while that of Rj hasto be decreased.The point in using an optimization algorithm is to resolve this conflict,and each choice of such algorithm’s effect must be formally cryptanalyzed, and this iswhat this work does in part. Quantization has been used in [?, ?, ?] but our use (andmotivation) is fundamentally different, as we mentioned earlier.

In this paper, we make our quantizations local (i.e. scalar) and study the effectsof attacks on it. For our approach, it is essential that the quantization occurs in largeenough dimensions to exploit the combinatorial hardness of some underlying problems,and this is being addressed in an ongoing work [?].

We use scalar quantizers Q0 and Q1, which are derived from the codebooks Ci ={(−1)i ∆/4 + j∆ + kij| j ∈ Z}, i = 0, 1. Here ∆ is the step size of a base quantizer and

{kij} are random quantities and chosen independently uniformly from [−γ∆/2, γ∆/2].

Here γ∈ (0, 1)is an input parameter to ensure sufficient randomization of the codebookdesign while maintaining desired amount of robustness.

In order to enhance robustness properties, we use error-correction codes (ECC) atthe encoder to produce w from a given message. For simplicity, we use block repetitioncodes at the moment. Let m0 (resp. m1) be the quantized version of m using Q0 (resp.Q1). Then watermark embedding is carried out by finding m, where

mi =

{m0i if wi = 0,m1i if wi = 1,

1 ≤ i ≤ M.

More explicitly the above criteria specifies the following task: given s, {Ri}, {ci}, m,and m, to find x, such that

∑j∈Ri

(xjcij) = mi, 1 ≤ i ≤ M. (2.1)

We provide a solution to this problem using the minimum-norm criterion [?]; this isdiscussed in detail in Sec. ??.

At the receiver, the initial task is to find the random statistics of the input data giventhe secret key. Let y be the receiver input and m be the random statistics of y, i.e.,mi =

∑j∈Ri

(xjcij), 1 ≤ i ≤ M . Then, given m, we find the most likely codewords m0

and m1 from C0 and C1 respectively via nearest neighbor decoding:

mij4= argmint∈Ci

||mj − t||, 0 ≤ i ≤ 1, 1 ≤ j ≤ M.

The next step is soft decoding, where thresholding is applied to the log-likelihood ratios(where AWGN attack channel is assumed on a possible codeword, m0 or m1). Considera block repetition code of rate 1/L applied at the encoder. In that case, the first decoded

bit at the receiver is 0 if logPL

j=1(mj−m0j)2

PLj=1(mj−m1j)

2 < 0; 1 otherwise. Similar procedure is applied

for the next decoded bits. Note that this decoding technique is similar to the one in [?]and a simpler variant of the one in [?]. Thus, we refer the reader to [?, ?] for furtherdetails on decoding. In this paper, we consider only the “decoding” problem (i.e., receiverfinds an estimate of m with the goal of no-error under attacks). However, it is possibleto use this framework for “verification” purposes as well (where detection is done atthe receiver and the output is a binary decision on the existence of a possibly embeddedwm). If this is the case, then the decision on the existence of the watermark is carried outby thresholding the Hamming distance between the decoded vector and the embeddedwatermark vector (similar to the method used in [?]).

3 Design of Embedding Sequences

We now present two methods to design x s.t. (??) is satisfied. First, we design an“additive” embedding sequence such that ||x− s|| is minimized and secondly we design“multiplicative” embedding sequence to minimize the distance between x./s (where ./denotes element-wise division, si 6= 0, 1 ≤ i ≤ N) and unity vector is minimized. Now,consider the following definitions:

d ∈ RM : d4= m−m.

T ∈RM×N :Tij4=

{cij if j ∈ Ri

0 else1∈ RN : A vector with all entries equal to one.

Note that T is chosen such that Ts = m and our goal is to find x such that Tx = m.Furthermore, we assume that M < N and T is full-rank. Note that this is a mildassumption when M ¿ N which can be satisfied by keeping the watermarking rate M

N

small enough.

3.1 Design of Additive Embedding Sequences

In this section, we use the definition of n4= x− s.

Lemma 3.1 The solution to

minx||x− s|| s.t. Tx = m. (3.2)

is given by x = s + TT(TTT

)−1(m−m) .

Proof : The problem (??) can be rewritten as minn ||n|| s.t. Tn = d, whose solu-

tion is given by ([?]) nMN = TT(TTT

)−1d. The result follows.

Remark 1: Lemma ?? provides the solution that is optimal in the sense of Euclideannorm. It is, of course, questionable if Euclidean norm is a good measure of perceptual

quality relative to perceptual distortion issues. For instance, the solution TT(TTT

)−1(m−m)

may have some spikes, which would degrade the perceptual quality. Our experiments sug-gest such artifacts become more likely as ∆ increases. For security concerns, we recallthat our metric is randomized and not easily computable without the key. If the solu-tion given by Lemma ?? is visually annoying, it is possible to use some heuristics. Suchheuristics can be viewed as analogous to having regularization terms in inverse problems.In that case, the formulation is still useful, provided that the heuristics can be expressedas extra linear constraints in (??). For instance, we may be interested in finding the op-timal additive embedding sequence that both satisfies the condition Tx = m ⇔ Tn = dand is approximately band-limited (to provide smoothness). Hence, one possibility toanalytically express the smoothness constraint could be to impose Dn = 0 where Dis a submatrix of the conventional dct matrix, which ensures that n is approximatelyband-limited to a low frequency range. Then, (??) can be rewritten as

minn||n|| s.t. T′n = d′, (3.3)

where T′ 4= [T ; D ] and d′4= [d ; 0 ]; here 0 is a vector of appropriate size that consists

of 0’s everywhere. Therefore, the solution to (??) is given by applying Lemma ??, whereT and d are replaced by T′ and d′ respectively.

Remark 2: Our non-singularity assumption on TTT , while it is valid for most practicalsituations, imposes constraints on the choice of random regions and the random weights.In some extreme cases, where the average region size and the number of such regions islarge, it may happen that even though T is still full-rank, the condition number of TTT

is large, which may lead to numeric problems. Furthermore, if the condition number islarge, then there is a potential possibility of information leakage to the attacker, due tothe unbalanced nature of computing the statistics. Hence, parameters must be chosento prevent these cases as our experiments suggest (possibly involving a search in secretkey space).

3.2 Design of Multiplicative Embedding Sequences

Now we embed via a multiplicative embedding sequence n, i.e., xi = sini, 1 ≤ i ≤ Nand our measure of quality is the distance between n and 1, i.e., solve the optimizationproblem:

minx||n− 1|| s.t. Tx = m. (3.4)

Lemma 3.2 The solution to (??) is given by xi = nisi, 1 ≤ i ≤ N where

n = 1 + STT(TS2TT

)−1(m−m) ,

and S =diag(s1, ..., sN).

Proof : The constraint in (??) can be rewritten as

m−m = d = T (x− s) = TS (n− 1) ,

where S is as defined in the statement of Lemma ??. Thus the problem (??) is equivalentto minn ||n− 1|| s.t. T (n− 1) = d. Applying the standard minimum-norm result, we

get (with a diagonal S), (n− 1)MN = (TS)T[TS (TS)T

]−1

d = STT(TS2TT

)−1d.

Remark 1: The difference between the multiplicative and the additive embeddingschemes is in the visual quality while they are comparable in robustness aspects, asour tests suggest. The distribution of the resulting watermarking distortion is quitedifferent since they use different metrics. The superior method often depends on theinput image.Remark 2: We can use techniques similar to the ones mentioned in Remark 1 of thelast section to achieve smoothing and avoiding visual artifacts.

4 Robustness Against Estimation Attacks

Attacks on watermarking schemes can be either generic (i.e., benchmark type [?, ?]) oralgorithm-specific that take into account the structure of the targeted algorithm. Suchattacks would be cryptanalytic in spirit. See [?], for a latter type of an attack on analgorithm that is quite robust against generic attacks. There, it is suggested that the useof stochastic models for attackers is realistic while a similar assumption on attacked signalby a detector can be inappropriate. In this section, we derive an optimal estimationattack on the additive embedding scheme using a particular stochastic model on the hostdata and the embedding sequence. Moreover, we allow some host data statistics andsome partial information about T be known by the attacker. Consequently, the attackerapplies minimum mean-squared error (MMSE) estimation on the watermarked data asan attack. We consider additive embedding sequences in this section. We now list a fewassumptions:

1. For each i ≤ M, di is uniform in [−∆/2, ∆/2] .

2. {di} are uncorrelated, i.e., E [didj] = ∆2

12δij.

3. d and s are uncorrelated i.e., E(disj) = E(di)E(sj).

−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.50

0.2

0.4

0.6

0.8

1

1.2

normalized quantization error

p.d.

f.

(a)

Estimated autocorrelation matrix of quantization errors

Rectangle index

Rec

tang

le in

dex

(b)

Figure 1: (a) Solid : Histogram of {di/∆} for image Lena; dashed : probability densityfunction of uniform distribution in the corresponding region. (b) Empirical estimate ofE

[ddT

]over 80 different images.

4. n is Gaussian.

First, we discuss only the experimental aspects of verifying these conditions. Aninspection of the histogram of {di/∆} in Fig. ?? for various values of ∆ of interestverifies assumption (1). Next, we repeated the same experiment for 80 different imagesfixing the secret key (i.e., same T and same codebook is used in all these images).This yields empirical estimates of E [didj] and thus an estimate of the autocorrelation

matrix Rd4= E

[ddT

]which is shown in Fig. ?? (b). The experimental estimate is nearly

diagonal, which validates (2). In these experiments we worked in the 3rd dc subband ofthe input image; this is as in our algorithm in section Sec. ??. As M and the densityof non-zero entries of T increases, assumption (4) becomes more valid by central limittheorem.

By (2), Rd = ∆2

12IM , where IM is the identity matrix. Now, using the full rank

assumption on T, we let singular value decomposition (SVD) of T be

T = UΣTV

where U ∈ RM×M is unitary, ΣT ∈ RM×M is a non-singular diagonal matrix and V∈ RM×Nwith orthonormal columns. From section 3.1 , n = TT (TTT )−1d = VΣ−1

T UTd.

From this we get Rn4= ∆2

12VΣ−2

T VT .Our current attack analysis uses Gaussian models on the host data s; however our al-

gorithmic construction is motivated by the desire to derive similar results while avoidingor minimizing such stochastic assumptions, but rather by the randomization steps thatresult from random secret key. This issue shall be explored in our future research. Fol-lowing [?], we now model s as an independent, but not necessarily identically distributed0-mean Gaussian vector. Furthermore, we assume that the variance field of s is smoothlyvarying, based on which we use locally approximately i.i.d. assumption on s. Using thisapproximation, we estimate the underlying variance field; see [?] for further details. LetRs denote the autocovariance matrix of s. We now make the following assumption:

• The attacker knows Rs and Rn

Now, since Rn =∆2

12VΣ−2

T VT , the attacker gets some information about T for free.As our experiments reveal. even then the attacker does not succeed: Indeed, in thisGaussian setup, the attacker can estimate s, given x = s + n optimally using Wienerfiltering:

s = Rs (Rs + Rn)−1 x.

Due to the Gaussian nature of the setup, this estimate coincides with both mmse and mapestimates of s given x [?]. Consequently, we apply approximate Wiener filtering attackon x; the estimation method of Rs given s in the locally Gaussian model is explainedin detail in [?], estimation of Rs given x is however non-trivial. Nevertheless, we allowthe adversary to know the estimate of Rs and it only strengthens our result. In all ofour experiments, we observed that all the embedded bits are recovered without an errorunder Wiener filtering attack.

5 Application to Image Watermarking

We used many standard test images of size 512 × 512 as inputs to our algorithm ap-plied to dc subband coefficients for a 3-level discrete Wavelet transform dwt withDaubechies length-8 wavelets. The transformation matrix T is chosen such that mrepresents weighted averages of randomly-chosen rectangles in this subband. Weightsare chosen as i.i.d. realizations of a Gaussian distribution, whose mean is the reciprocalof the area of the particular rectangle and variance is an algorithm parameter (we alsotested when these are input dependent). Typically, we chose around 200 rectangles toavoid singularity of T; their locations and sizes are chosen randomly from uniform distri-butions such that the expected area of each rectangle is 1

16-th of the area of the subband.

The exact parameters of these distributions are user-dependent algorithm parameters.The base quantization step size ∆ is chosen in an image-dependent manner, such thatthe distortion introduced at the encoder is just noticeable. We applied both additiveand multiplicative designs for embedding sequences. We used rate 1

5block repetition

codes at the encoder where the repetitions are scattered randomly. At the receiver, wefirst resized the input image to size 512 × 512 by bicubic interpolation. Then, we usednearest-neighbor decoding followed by soft decoding based on log-likelihood ratios to per-form decoding. In general, it turns out that soft decoding produces better results thanhard decoding.

We applied several attacks in our tests. In all our experiments, we observed that allembedded bits are recovered correctly (for both additive and multiplicative designs ofembedding sequences) in case of Wiener filtering attack (explained in Sec. ??), JPEGcompression attacks (with quality factor as low as 10%), AWGN attacks (with noisestandard deviation as high as 40) and several smoothing attacks with linear shift-invariantfiltering. Furthermore, we can withstand (i.e., recover all the bits correctly) geometricattacks mentioned in [?] when we use lower-rate codes (such as rate 1/40) as long as theseattacks do not create severe perceptual distortion; for instance we are robust againstall scaling attacks, rotation attacks up to ±1 degree and cropping attacks up to 2%.Watermarked image Lena and some attack examples are shown in Fig. ??.

6 Discussion

We proposed a novel signal watermarking scheme, where message bits are embedded inrandomly-chosen semi-global statistics of the input via quantization. The embedding se-quence is found by solving an optimization problem, whose parameters are known to theencoder and decoder, but unknown to the attacker. Our approach stresses randomization

(a) (b)

(c) (d)

Figure 2: (a) Watermarked Lena with ∆ = 100. Attacked versions of watermarked Lenaare shown in (b) (where Wiener filtering is applied as explained in Sec. ??), (c) (whereJPEG compression with quality factor 10% is applied) and (d) (where AWGN attack isapplied with noise standard deviation is 40). In all the attack examples, we recover allthe embedded bits.

in watermarking applications. In this paper, we applied this approach to images and ex-perimentally demonstrated robustness against not only non-malicious JPEG compressionand AWGN attacks, but also malicious estimation attacks. Our scheme is generic andcan be applied to audio signals as well; those results shall be presented elsewhere.

Our future research includes design of better codes and better transforms (possiblyrandomized) to enhance robustness. Furthermore, we would like to apply this approachin the fingerprinting problem as an embedding layer.Acknowledgments : We thank Mariusz Jakubowski (Microsoft Research) and PierreMoulin (University of Illinois) for useful discussions.

References[1] A. J. Menezes, P. C. van Oorschot and S. A. Vanstone, Handbook of applied cryptog-

raphy, CRC Press, Boca Raton, FL, 1997.

[2] R. Venkatesan, S.-M. Koon, M. Jakubowski and P. Moulin, “Robust image hashing,”Proc. IEEE ICIP 2000, Vancouver, Canada, September 2000.

[3] M. K. Mıhcak and R. Venkatesan, “Blind Image Watermarking Via Derivation andQuantization of Robust Semi–Global Statistics,” Proc. IEEE ICASSP 2002, Orlando,FL, May 2002.

[4] R. Venkatesan and M. H. Jakubowski, “Image watermarking with better resilience,”Proc. IEEE ICIP 2000, Vancouver, Canada, September 2000.

[5] B. Chen and G. W. Wornell, “Quantization index modulation: A class of provablygood methods for digital watermarking and information embedding,” IEEE Trans.on Information Theory, vol. 47, no. 4, pp. 1423-1443, May 2001.

[6] M. K. Mıhcak and P. Moulin, “Information Embedding Codes Matched to LocallyStationary Gaussian Image Models,” Proc. IEEE ICIP 2002, Rochester, NY, Sep.2002.

[7] M. Kesal, M. K. Mıhcak, R. Kotter and P. Moulin, “Iteratively Decodable Codesfor Watermarking Applications,” Proc. 2nd Symposium on Turbo Codes and TheirApplications, Brest, France, Sep. 2000.

[8] K. Jain, M. K. Mıhcak and R. Venkatesan, “Lattice Rounding Approach to Water-marking and Hashing,” preprint, 2002.

[9] R. A. Horn and C. R. Johnson, Matrix Analysis, Cambridge University Press, NY,1985.

[10] F. A. P. Petitcolas and M. G. Kuhn: StirMark software, available fromwww.cl.cam.ac.uk/ ˜fapp2/watermarking/image watermarking/stirmark/.

[11] S. Pereira, S. Voloshynovskiy, M. Madueno, S. Marchand-Maillet and T. Pun: Check-mark software, available from watermarking.unige.ch/Checkmark/.

[12] M. K. Mıhcak, R. Venkatesan and M. Kesal, “Cryptanalysis of Discrete-SequenceSpread Spectrum Watermarks,” Proceedings of 5th Information Hiding Workshop,Holland, Oct. 2002.

[13] H. V. Poor, An Introduction to Signal Detection and Estimaton, 2nd Ed., Springer-Verlag, 1994.