Post on 16-Jan-2023
University of Ioannina - Department of Computer Science and Engineering
Filtering in the Frequency Domain (Circulant Matrices and Convolution)
Digital Image Processing
Christophoros Nikou
cnikou@cs.uoi.gr
2
C. Nikou – Digital Image Processing (E12)
Toeplitz matrices
• Elements with constant value along the main
diagonal and sub-diagonals.
• For a NxN matrix, its elements are determined by
a (2N-1)-length sequence ( 1) 1|n N n Nt
1
( 1)0 1 2
1 0 1
2 2
1
2 1 0N
N
N N
t t t t
t t t
t t
t
t t t t
T
( , ) m nm n t T
3
C. Nikou – Digital Image Processing (E12)
Toeplitz matrices (cont.)
• Each row (column) is generated by a shift of the
previous row (column).
− The last element disappears.
− A new element appears.
1
( 1)0 1 2
1 0 1
2 2
1
2 1 0N
N
N N
t t t t
t t t
t t
t
t t t t
T
( , ) m nm n t T
4
C. Nikou – Digital Image Processing (E12)
Circulant matrices
• Elements with constant value along the main
diagonal and sub-diagonals.
• For a NxN matrix, its elements are determined by
a N-length sequence 0 1|n n Nc
1
1
2 1
1
0 1 2
0 1
0 2
1
1 2 0
N N
N
N
N
N N N N
c c c c
c c c
c c c c
c
c c c c
C
( )mod( , ) m n Nm n c C
5
C. Nikou – Digital Image Processing (E12)
Circulant matrices (cont.)
• Special case of a Toeplitz matrix.
• Each row (column) is generated by a circular shift
(modulo N) of the previous row (column).
( )mod( , ) m n Nm n c C
1
1
2 1
1
0 1 2
0 1
0 2
1
1 2 0
N N
N
N
N
N N N N
c c c c
c c c
c c c c
c
c c c c
C
6
C. Nikou – Digital Image Processing (E12)
Convolution by matrix-vector
operations
• 1-D linear convolution between two discrete
signals may be expressed as the product of a
Toeplitz matrix constructed by the elements of
one of the signals and a vector constructed by the
elements of the other signal.
• 1-D circular convolution between two discrete
signals may be expressed as the product of a
circulant matrix constructed by the elements of
one of the signals and a vector constructed by the
elements of the other signal.
• Extension to 2D signals.
7
C. Nikou – Digital Image Processing (E12)
1D linear convolution using Toeplitz
matrices
• The linear convolution g[n]=f [n]*h [n] will be of
length N=N1+N2-1=3+2-1=4.
• We create a Toeplitz matrix H from the
elements of h [n] (zero-padded if needed) with
− N=4 lines (the length of the result).
− N1=3 columns (the length of f [n]).
− The two signals may be interchanged.
1 2[ ] {1, 2, 2}, [ ] {1, 1}, 3, 2f n h n N N
8
C. Nikou – Digital Image Processing (E12)
1D linear convolution using Toeplitz
matrices (cont.)
1 2[ ] {1, 2, 2}, [ ] {1, 1}, 3, 2f n h n N N
4 3
1 0 0
1 1 0
0 1 1
0 0 1
H
Length of f [n] = 3
Length of the result =4
Notice that H is not
circulant (e.g. a -1
appears in the second line
which is not present in the
first line.Zero-padded h[n] in the first column
9
C. Nikou – Digital Image Processing (E12)
1D linear convolution using Toeplitz
matrices (cont.)
1 2[ ] {1, 2, 2}, [ ] {1, 1}, 3, 2f n h n N N
1 0 0 11
1 1 0 12
0 1 1 02
0 0 1 2
g = Hf
[ ] {1, 1, 0, 2}g n
10
C. Nikou – Digital Image Processing (E12)
1D circular convolution using
circulant matrices
• The circular convolution g[n]=f [n]h [n] will
be of length N=max{N1, N2}=3.
• We create a circulant matrix H from the
elements of h [n] (zero-padded if needed)
of size NxN.
− The two signals may be interchanged.
1 2[ ] {1, 2, 2}, [ ] {1, 1}, 3, 2f n h n N N
11
C. Nikou – Digital Image Processing (E12)
1D circular convolution using
circulant matrices (cont.)
1 2[ ] {1, 2, 2}, [ ] {1, 1}, 3, 2f n h n N N
3 3
1 0 1
1 1 0
0 1 1
H
Zero-padded h[n] in the
first column
12
C. Nikou – Digital Image Processing (E12)
1D circular convolution using
circulant matrices (cont.)
1 2[ ] {1, 2, 2}, [ ] {1, 1}, 3, 2f n h n N N
1 0 1 1 1
1 1 0 2 1
0 1 1 2 0
g = Hf
[ ] { 1, 1, 0}g n
13
C. Nikou – Digital Image Processing (E12)
Block matrices
• Aij are matrices.
• If the structure of A, with respect to its sub-matrices, is
Toeplitz (circulant) then matrix A is called block-Toeplitz
(block-circulant).
• If each individual Aij is also a Toeplitz (circulant) matrix then
A is called doubly block-Toeplitz (doubly block-circulant).
11 12 1
21 22 2
1 2
N
N
M M MN
A A A
A A AA
A A A
14
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices
m
n
1 4 1
2 5 3
f [m,n]
m
n
1 11 -1
h [m,n]
M1=2, N1=3 M2=2, N2=2
The result will be of size (M1+M2-1) x (N1+N2-1) = 3 x 4
15
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices (cont.)
• At first, h[m,n] is zero-padded
to 3 x 4 (the size of the
result).
• Then, for each row of h[m,n],
a Toeplitz matrix with 3
columns (the number of
columns of f [m,n]) is
constructed.
m
n
1 1 0
1 -1 0
h[m,n]
00
0 0 0 0
m
n
1 4 1
2 5 3
f [m,n]
m
n
1 11 -1
h [m,n]
16
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices (cont.)
• For each row of h[m,n], a
Toeplitz matrix with 3 columns
(the number of columns of f
[m,n]) is constructed.
m
n
1 1 0
1 -1 0
h[m,n]
00
0 0 0 0
1
1 0 0
1 1 0
0 1 1
0 0 1
H2
1 0 0
1 1 0
0 1 1
0 0 1
H 3
0 0 0
0 0 0
0 0 0
0 0 0
H
17
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices (cont.)
• Using matrices H1, H2
and H3 as elements, a
doubly block Toeplitz
matrix H is then
constructed with 2
columns (the number of
rows of f [m,n]).
m
n
1 4 1
2 5 3
f [m,n]
m
n
1 11 -1
h [m,n]
1 3
2 1
3 2 12 6
H H
H H H
H H
18
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices (cont.)
• We now construct
a vector from the
elements of f [m,n].
m
n
1 4 1
2 5 3
f [m,n]
m
n
1 11 -1
h [m,n]
2
5
(2 5 3)3
(1 4 1)1
4
1
T
T
f
19
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices (cont.) m
n
1 4 1
2 5 3
f [m,n]
m
n
1 11 -1
h [m,n]
1 3
2 1
3 2
2
5
3
1
4
1
H H
g = Hf H H
H H
20
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices (cont.) 1 0 0 0 0 0 2
1 1 0 0 0 0 3
0 1 1 0 0 0 2
0 0 1 0 0 0 2 3
1 0 0 1 0 0 5 3(
1 1 0 1 1 0 3 10
0 1 1 0 1 1 1 5
0 0 1 0 0 1 4 2
0 0 0 1 0 0 1 1
0 0 0 1 1 0 5
0 0 0 0 1 1 5
0 0 0 0 0 1 1
g = Hf
2 3 2 3)
(3 10 5 2)
(1 5 5 1)
T
T
T
21
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices (cont.)
*
m
n
1 4 1
2 5 3
f [m,n]
m
n
1 11 -1
h [m,n]
=3
2
m
n
3 10 5
2 3 -2
g [m,n]
1 5 5 1 (2 3 2 3)
(3 10 5 2)
(1 5 5 1)
T
T
T
g =
22
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices (cont.)
m
n
3 4
1 2
f [m,n]
m
n1 -1
h [m,n]
M1=2, N1=2 M2=1, N2=2
The result will be of size (M1+M2-1) x (N1+N2-1) = 2 x 3
Another example
23
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices (cont.)
• At first, h[m,n] is zero-padded
to 2 x 3 (the size of the
result).
• Then, for each line of h[m,n],
a Toeplitz matrix with 2
columns (the number of
columns of f [m,n]) is
constructed.
m
n
0 0 0
1 -1 0
h[m,n]
m
n
3 4
1 2
f [m,n]
m
n1 -1
h [m,n]
24
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices (cont.)
• For each row of h[m,n], a Toeplitz
matrix with 2 columns (the
number of columns of f [m,n]) is
constructed.
m
n
0 0 0
1 -1 0
h[m,n]
1
1 0
1 1
0 1
H2
0 0
0 0
0 0
H
25
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices (cont.)
• Using matrices H1 and
H2 as elements, a
doubly block Toeplitz
matrix H is then
constructed with 2
columns (the number of
rows of f [m,n]).
1 2
2 1 6 4
H HH
H H
m
n
3 4
1 2
f [m,n]
m
n1 -1
h [m,n]
26
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices (cont.)
• We now construct a
vector from the elements
of f [m,n].
1
(1 2)2
(3 4)3
4
T
T
f
m
n
3 4
1 2
f [m,n]
m
n1 -1
h [m,n]
27
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices (cont.)
m
n
3 4
1 2
f [m,n]
m
n1 -1
h [m,n]
1 2
2 1
1
2
3
4
H Hg = Hf
H H
28
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices (cont.)
1 0 0 0 1
1 1 0 0 1 1
(1 1 2)0 1 0 0 2 2
(3 1 4)0 0 1 0 3 3
0 0 1 1 4 1
0 0 0 1 4
T
T
g = Hf
29
C. Nikou – Digital Image Processing (E12)
2D linear convolution using doubly
block Toeplitz matrices (cont.)
*
=
m
n
3 1 4
1 1 -2
g [m,n]
m
n
3 4
1 2
f [m,n]
m
n1 -1
h [m,n]
(1 1 2)
(3 1 4)
T
T
g =
30
C. Nikou – Digital Image Processing (E12)
2D circular convolution using doubly
block circulant matrices
where H is a doubly block circulant matrix
generated by h [m,n] and f is a vectorized form
of f [m,n].
0 1, 0 1,m M n N
The circular convolution g[m,n]=f [m,n]h [m,n]
may be expressed in matrix-vector form as:
g = Hf
with
31
C. Nikou – Digital Image Processing (E12)
2D circular convolution using doubly
block circulant matrices (cont.)
Each Hj, for j=1,..M, is a circulant matrix with N
columns (the number of columns of f [m,n])
generated from the elements of the j-th row of
h [m,n].
0 1 2 1
1 0 1 2
2 1 0 3
1 2 3 0
M M
M
M M M
H H H H
H H H H
H H H H H
H H H H
32
C. Nikou – Digital Image Processing (E12)
2D circular convolution using doubly
block circulant matrices (cont.)
[ ,0] [ , 1] [ ,1]
[ ,1] [ ,0] [ , 2]
[ , 1] [ , 2] [ ,0]
j
N N
h j h j N h j
h j h j h j
h j N h j N h j
H
Each Hj, for j=1,..M, is a NxN circulant matrix
generated from the elements of the j-th row of
h [m,n].
33
C. Nikou – Digital Image Processing (E12)
2D circular convolution using doubly
block circulant matrices (cont.)
m
n
1 3 -1
1 2 1
f [m,n]
m
n
1 0
1 -1
h [m,n]
0 1 0
0
0
0 0 0
0
1 0 1
1 1 0
0 1 1
H 1
1 0 0
0 1 0
0 0 1
H 2
0 0 0
0 0 0
0 0 0
H
34
C. Nikou – Digital Image Processing (E12)
2D circular convolution using doubly
block circulant matrices (cont.)
m
n
1 3 -1
1 2 1
f [m,n]
m
n
1 0
1 -1
h [m,n]
0 1 0
0
0
0 0 0
0 2 1
1 0 2
2 1 0
1 2 1
1 3 1
0 1 0
T
T
T
H H H
g = Hf H H H
H H H
35
C. Nikou – Digital Image Processing (E12)
2D circular convolution using doubly
block circulant matrices (cont.)
1 0 1 0 0 0 1 0 0 1 0
1 1 1 0 0 0 0 1 0 2 2
0 1 0 0 0 0 0 0 1 1 1
1 0 0 1 0 1 0 0 0 1 3
0 1 0 1 1 1 0 0 0 3 4
0 0 1 0 1 0 0 0 0 1 3
0 0 0 1 0 0 1 0 1 0 1
0 0 0 0 1 0 1 1 1 1 4
0 0 0 0 0 1 0 1 0 0 2
g = Hf
36
C. Nikou – Digital Image Processing (E12)
2D circular convolution using doubly
block circulant matrices (cont.)
m
n
1 0
1 -1
h [m,n]
m
n
3 4 -3
0 2 -1
g [m,n]
1 4 -2
0
0
0 0 0
=
m
n
1 3 -1
1 2 1
f [m,n]
0 1 0
0 2 1
3 4 3
1 4 2
T
T
T
g =
37
C. Nikou – Digital Image Processing (E12)
Diagonalization of circulant matrices
Theorem: The columns of the inverse DFT matrix
are eigenvectors of any circulant matrix. The
corresponding eigenvalues are the DFT values of
the signal generating the circulant matrix.
Proof: Let
2 2 nj j k
nk
N Nw e w e
be the DFT basis elements of length N with:
0 1, 0 1,k N n N
38
C. Nikou – Digital Image Processing (E12)
Diagonalization of circulant matrices
(cont.)
We know that the DFT F [k] of a 1D signal f [n]
may be expressed in matrix-vector form:
where
F = Af
[0], [1],..., [ 1] , [0], [1],..., [ 1]T T
f f f N F F F N f = F =
0 1 2 10 0 0 0
0 1 2 11 1 1 1
0 1 2 11 1 1 1
N
N N N N
N
N N N N
NN N N N
N N N N
w w w w
w w w w
w w w w
A
39
C. Nikou – Digital Image Processing (E12)
Diagonalization of circulant matrices
(cont.)
The inverse DFT is then expressed by:
where
-1f = A F
*0 1 2 1
0 0 0 0
0 1 2 11 1 1 1
1 *
0 1 2 11 1 1 1
1 1
TN
N N N N
NT
N N N N
NN N N N
N N N N
w w w w
w w w w
N N
w w w w
A A
The theorem implies that any circulant matrix has
eigenvectors the columns of A-1.
40
C. Nikou – Digital Image Processing (E12)
Diagonalization of circulant matrices
(cont.)
Let H be a NxN circulant matrix generated by the 1D
N-length signal h[n], that is:
mod( , ) ( ) [ ]N Nm n h m n h m n H
Let also αk be the k-th column of the inverse DFT
matrix A-1.
We will prove that αk, for any k, is an eigenvector of H.
The m-th element of the vector Hαk, denoted by
is the result of the circular convolution of the signal
h[n] with αk.
k mHα
41
C. Nikou – Digital Image Processing (E12)
Diagonalization of circulant matrices
(cont.)
1
0
[ ] [ ]N
k N kmn
h m n n
Hα1
0
1[ ]
Nk n
N N
n
h m n wN
( 1)( )1
[ ]m Nl m n
k m l
N N
l m
h l wN
( 1)1
[ ]m N
k m k l
N N N
l m
w h l wN
1
( 1) 0
1[ ] [ ]
mk m k l k l
N N N N N
l m N l
w h l w h l wN
We will break it into two parts
42
C. Nikou – Digital Image Processing (E12)
Diagonalization of circulant matrices
(cont.)
1 1 1
( 1) 0 1
1[ ] [ ] [ ]
N Nk m k l k l k l
N N N N N N N
l m N l l m
w h l w h l w h l wN
Periodic with respect to N.
1 1 1
( 1) 0 1
1[ ] [ ] [ ]
N N Nk m k l k l k l
N N N N N N N
l N m N l l m
w h l w h l w h l wN
1 1 1
1 0 1
1[ ] [ ] [ ]
N N Nk m k l k l k l
N N N N N N N
l m l l m
w h l w h l w h l wN
43
C. Nikou – Digital Image Processing (E12)
Diagonalization of circulant matrices
(cont.)
1
0
1[ ]
Nk m k l
k N N Nml
w h l wN
Hα [ ] k m
H k α
DFT of h[n] at k.
This holds for any value of m. Therefore:
[ ]k kH kHα α
which means that αk, for any k, is an eigenvector of
H with corresponding eigenvalue the k-th element
of H[k], the DFT of the signal generating H .
44
C. Nikou – Digital Image Processing (E12)
Diagonalization of circulant matrices
(cont.)
The above expression may be written in terms of
the inverse DFT matrix:
1 1 1 HA A Λ H A ΛA
Based on this diagonalization, we can prove the
property between circular convolution and DFT.
=diag [0], [1],..., [ 1]H H H N Λ
or equivalently: 1Λ ΑHA
45
C. Nikou – Digital Image Processing (E12)
Diagonalization of circulant matrices
(cont.)
g = Hf
DFT of g[n]
-1g = HA Af -1Ag = AHA Af G = ΛF
DFT of f [n]DFT of h [n]
[0] [0] 0 0 [0]
[1] 0 [1] 0 [1]
[ 1] 0 0 [ 1] [ 1]
G H F
G H F
G N H N F N
=
46
C. Nikou – Digital Image Processing (E12)
Diagonalization of doubly block
circulant matrices
• These properties may be generalized in 2D.
• We need to define the Kronecker product:
11 12 1
21 22 2
1 2
N
N
M M MN MK NL
a a a
a a a
a a a
B B B
B B BA B
B B B
,M N K L A B
47
C. Nikou – Digital Image Processing (E12)
Diagonalization of doubly block
circulant matrices (cont.)
• The 2D signal f [m,n],
may be vectorized in lexicographic ordering
(stacking one column after the other) to a
vector:1MNf
0 1, 0 1,m M n N
• The DFT of f [m,n], may be obtained in
matrix-vector form:
F A A f
48
C. Nikou – Digital Image Processing (E12)
Diagonalization of doubly block
circulant matrices (cont.)
1
Λ A A H A A
Theorem: The columns of the inverse 2D DFT
matrix
are eigenvectors of any doubly block circulant
matrix. The corresponding eigenvalues are the 2D
DFT values of the 2D signal generating the doubly
block circulant matrix:
1
A A
Doubly block circulantDiagonal, containing the 2D DFT
of h[m,n] generating H