Implementing Algorithms; Graphs

Implementing Algorithms

1 Data Structuresimplementing algorithmsarrays and linked lists

2 Implementing the Gale-Shapley algorithmselecting data structuresoverview of the selected data structures

3 Graphsdefinition and terminologyadjacency matricesgraph traversals: breadth first and depth first

CS 401/MCS 401 Lecture 3Computer Algorithms I

Jan Verschelde, 22 June 2018

Computer Algorithms I (CS 401/MCS 401) Implementing Algorithms; Graphs L-3 22 June 2018 1 / 43

implementing algorithms

Niklaus Wirth: algorithms + data structures = programs.

What do we mean by implementing an algorithm?

DefinitionImplementing an algorithm is selecting the best data structures thatwill make the algorithm run efficiently.

Example: we proved that in the worst case, the Gale-Shapleyalgorithm for n men and n women does require no more than n2

iterations of the loop.

This proof does not imply that the running time of the Gale-Shapleyalgorithm is O(n2) because we have not proven yet that every step ineach iteration of the loop has constant time.

We denote constant time by O(1).


arrays

An array A is a sequence of consecutive data elementsof fixed size, size(A), andthe cost of accessing the i-th element in A is O(1).

Example: an array which stores 3, 5, 4:

3 5 4

Exercise 1: Given is an array of numbers A, sorted in increasing order,and some number x . Prove that finding the position of x in A can bedone in sublinear time, in the size of A.


two dimensional arrays

A matrix is stored as an array of arrays:

A =

3 2 9 15 4 2 71 2 9 66 1 8 3

A

s - 6 1 8 3

s - 1 2 9 6

s - 5 4 2 7

s - 3 2 9 1

Double indexing works as follows:

A2,1 = 5 = A[2][1] = A[2,1]

Given indices i and j , the cost of accessing Ai,j = A[i , j] is O(1).


linked lists

Example: a linked list L to store 3, 5, 4:

Ls- Node

next: sdata: 3

- Node

next: sdata: 5

- Node

next: sdata: 4

- NULL

A node in a linked list consists of a data element and a pointer.A linked list consists of

a head node, linked to the next node,which is linked to the next node, etc,linked to the last node in the list,which has NULL as the pointer to its next node.


circular lists

In a linked list, one often needs a separate pointer to the last elementof the list, to append an element to the end of the list.

When the order among the stored data does not matter,distinguishing between first and last does not matter either,and the list is a pointer to some element in the list.

Example: a circular linked list L stores 3, 5, 4:

Ls- Node

next: sdata: 3

- Node

next: sdata: 5

- Node

next: sdata: 4

6


doubly linked lists

Example: A doubly linked list L to store 3, 5, 4:

Ls- Node

next: sprev: sdata: 3

?NULL

- Node


� - Node


� - NULL

At the expense of one extra pointer per node,reverse transversal becomes efficient.

Doubly linked list can also made to be circular.


properties of linked lists

The cost to locate an element in a list of n elements is O(n),because to reach the i-th element in a list,all previous i − 1 nodes need to be visited.

Linked lists are dynamic data structures:Insertion of a new node costs O(1) time.Given the pointer to a node in a doubly linked list,the deletion of the node costs O(1) time.

Exercise 2: Given is a doubly linked list with n elements.Prove the above statements: inserting and deleting a node is O(1),independent of the size n of the list.


input data of the Gale-Shapley algorithm

Input data: every man and woman has a preference list with no ties.

Example: Consider 4 men {1,2,3,4} and 4 women {a,b, c,d}.

Preferences for men Preferences for women1 : a c b d a : 2 3 4 12 : b a d c b : 1 4 2 13 : a c b d c : 1 3 4 24 : d a c b d : 2 1 3 4

Read the first line as 1 prefers a to c, 1 prefers c to b, 1 prefers b to d ,a prefers 2 to 3, a prefers 3 to 4, a prefers 4 to 1.

The number of men and women is fixed at the start of the algorithm.


storing the input data

Order the n men and n women into a sequence 1,2, . . . ,n.Because of the fixed size of constants, we use a two dimensional array:

Preferences for men1 : 1 3 2 42 : 2 1 4 33 : 1 3 2 44 : 4 1 3 2

ManPref

s - 4 1 3 2

s - 1 3 2 4

s - 2 1 4 3

s - 1 3 2 4

For a man m:ManPref[m, i] stores the i-th woman on the preference list of m.

Similarly for the women, the two dimensional array WomanPref storestheir preferences. For a woman w :WomanPref[w , i] stores the i-th man on the preference list of w .


the first statement in the Gale-Shapley algorithm

Consider the first statement in the Gale-Shapley algorithm:1 Let m ∈ M: isfree(m) and

there is a w ∈W : hasproposed(m,w) is false.We need to be able to locate the next free man.

The set of free men changes:A man is no longer free if his proposal is accepted.A man becomes free if an engagement is broken.

A linked list FreeMen will store the set of free men:The next free men m is at the front of the list: O(1).If his proposal is accepted, then we delete the first element: O(1).If the woman who accepted the proposal of m breaks up with m′,then m′ is inserted to the front of the list of free men: O(1).

However, it takes O(n) time to initialize FreeMen.


proposing to the highest ranked

Consider the second statement in the Gale-Shapley algorithm:2 Let w ∈W : w is the highest ranked in the preference list of m

for whom hasproposed(m,w) is false.Observe:

1 Proposals are made in the order of preference.2 Preferences are stored in decreasing order: highest ranked first.

• Consider the preferences of m.• Assume m proposed to 2 and 1.• Next time m is free, he proposes to 4.• next is an array which stores for

each m his next highest ranked:m proposes to ManPref[m,next[m]]

• Initially, next[m] = 1 for all m.

ManPref

s -s -m s - 2 1 4 3

6next[m]

s -


storing the engagementsConsider the beginning of the third statement in the loop:

3 hasproposed(m,w) := trueIf isfree(w) then

S := S ∪ (m,w)

The hasproposed(m,w) := true is implemented by incrementingnext[m], as: next[m] := next[m] + 1.

Need to know: isfree(w) and if not, who is w engaged to?

Introduce an array current of size n:If isfree(w), then current[w ] = 0.If not isfree(w), then current[w ] = m, w is engaged to m.

The current implements the set of engaged pairs S.

Because accessing an array element is O(1),checking if w is free and to whom w is engaged is O(1).


ranking the men along preferences of the women

Consider the breaking of an engagement:

3 if ...else

there is an m′ ∈ M: (m′,w) ∈ S (w is engaged to m′)if w prefers m to m′ then

S := S \ (m′,w) (m′ and w are no longer engaged)

If there is an m′ ∈ M: (m′,w) ∈ S, then current[w ] = m′.

How to determine if w prefers m to m′?Let there be a two dimensional array Ranking such that

Ranking[w ,m] < Ranking[w ,m′] if w prefers m to m′.

Ranking[w ,m] is the position of m in the preference list of w ,e.g.: if Ranking[w ,m] = 2, then m is the second best choice of w .


building the auxiliary data structure Ranking

The cost of consulting Ranking[w ,m] is O(1),but what about the cost of constructing Ranking?

Preferences for women1 : 2 3 4 12 : 1 4 2 33 : 1 3 4 24 : 2 1 3 4

WomanPref

s - 2 1 3 4

s - 1 3 4 2

s - 1 4 2 3

s - 2 3 4 1

for all woman w = 1,2, . . . ,n:for all ranks r = 1,2, . . . ,n:

Ranking[w ,WomanPref[w ,m]] := r

The loop makes n2 assignments, so the cost is O(n2).


overview of the selected data structures

1 ManPref is a two dimensional array for the preferences of n men.2 WoManPref is a two dimensional array

for the preferences of n women.3 FreeMen is a linked list of all free men.4 next is an array of size n for the next woman to propose to.5 current is an array of size n to store the engaged pairs.6 Ranking is a two dimensional array to rank men.

Exercise 3: The construction of Ranking is O(n2). It is the most timeconsuming operation. In which scenario is Ranking not needed?Explain.


running the Gale-Shapley algorithm once again

Consider 4 men {1,2,3,4} and 4 women {1,2,3,4}.

Preferences for men Preferences for women1 : 1 3 2 4 1 : 2 3 4 12 : 2 1 4 3 2 : 1 4 2 33 : 1 3 2 4 3 : 1 3 4 24 : 4 1 3 2 4 : 2 1 3 4

Read the first line asman 1 prefers woman 1 to 3, prefers 3 to 2, prefers 2 to 4;woman 1 prefers man 2 to 3, prefers 3 to 4, prefers 4 to 1.

Exercise 4: Run the Gale-Shapley algorithm on the above input data,show the evolution of all data structures in each step.


the Gale-Shapley algorithm is an efficient algorithm

TheoremFor n men and n women,the running time of the Gale-Shapley algorithm is O(n2).

Proof. Consider the costs of the data structures defined above.We have two n-by-n arrays: ManPref and WomanPref, given oninput. The other n-by-n array Ranking is constructed only oncewhich takes O(n2) time. Accessing arrays is O(1).The linked list FreeMen takes O(n) time to initialize.Both deleting a free man and inserting a free man is O(1).The initialization of the arrays next and current takes O(n)time. Accessing arrays is O(1).

After initialization, all steps in the algorithm take O(1) time.As proven in Lecture 1, the number of iterations is bounded by n2,so the Gale-Shapley algorithm runs in O(n2) time. Q.E.D.


graphs, terminology and definition

A node in a graph is called a vertex.A connection between two vertices is an edge.

A graph G is defined by a tuple (V ,E):V is the set of vertices, andE is the set of edges.

An example:��A��B

��C��D

@@@ �A

A

V = {A,B,C,D}

E = {(A,B), (A,C), (A,D), (C,D), (D,D)}

Two vertices are adjacent if there is an edge between them.A path is a sequence of successively adjacent vertices.For example, a path from B to D is B,A,D.


directed and undirected graphsIn an undirected graph (which is the default), the edge (A,B) means:there is an edge from A to B and there is an edge from B to A.��A��B

��C��D

@@@ �A

A

V = {A,B,C,D}

E = {(A,B), (A,C), (A,D), (C,D), (D,D)}

��A��B

��C��D

-�

6@@@R

��AA�

V = {A,B,C,D}

E = {(A,B), (B,A), (C,A), (A,D), (D,C), (D,D)}

In a directed graph or digraph, (A,B) means there is an edge from Ato B and (B,A) means there is an edge from B to A.


weighted graphs

In an weighted graph, a value (a weight) is associated to every edge.��A��B

��C��D

5

8@@@

3

7

�AA4

��A��B

��C��D

-5�

668@@@R

3

�7

�AA�

�AA4

�

Edges in a weighted graphs are triplets (u, v ,w),with w the weight of the edge which connects u to v .

A cycle is a path where the first vertex equals the last one.For example: A, D, C, A is a cycle.

A graph without cycles can be viewed as a tree.

The degree of a vertex is the number of edges that contain that vertex.


adjacency matricesFor a graph G = (V ,E), the adjacency matrix M

has as many rows and columns as the number of vertices in V ;for all vi , vj ∈ V : if (vi , vj) 6∈ E : Mi,j = 0;for all vi , vj ∈ V : if (vi , vj) ∈ E : Mi,j = 1.��A��B

��C��D

@@@ �A

A

A B C D0 1 1 11 0 0 01 0 0 11 0 1 1

ABCD��A

��B

��C��D

-�

6@@@R

��AA�

A B C D0 1 0 11 0 0 01 0 0 00 0 1 1

ABCD


adjacency matrices for weighted graphsFor a weighted graph G = (V ,E), the adjacency matrix M

has as many rows and columns as the number of vertices in V ;for all vi , vj ∈ V : if (vi , vj) 6∈ E : Mi,j = 0;for all vi , vj ∈ V : if (vi , vj) ∈ E : Mi,j = w , w is the weight of (vi , vj).��A��B

��C��D

5

8@@@

3

7

�AA4

A B C D0 5 8 35 0 0 08 0 0 73 0 7 4

ABCD��A

��B

��C��D

-5�

668@@@R

3

�7

�AA4

�

A B C D0 5 0 36 0 0 08 0 0 00 0 7 4

ABCD


visiting each vertex in a graph

Consider the graph: ��0 ��1 ��2��3 ��4 ��5��6 ��7 ��8

@@@

@@@��

��

HHHHH

HH

��

��

��

Problem: visit each vertex in a systematic order.


breadth-first searchWe start at vertex 0.��0 ��1 ��2��3 ��4 ��5��6 ��7 ��8

@@@

@@@��

��

HHHH

HHH

��

��

��

��0 ��1��3 ��4@@@

at 0: visit 1, 3, 4��0 ��1��3 ��4 ��5��6

@@@

@@@

��

at 1: visit 5 and 6


breadth-first search continued

��0 ��1 ��2��3 ��4 ��5��6 ��7 ��8

@@@

@@@�

��

HHHHHHH

��

��

��

��0 ��1 ��2��3 ��4 ��5��6 ��8

@@@

@@@�

��

HHHHHHH

��

at 3: visit 2 and 8

Every vertex is visited only once.Although there is an edge from 4 to 5, the breadth-first search will notvisit this edge because 5 has already been visited.


breadth-first search continued

��0 ��1 ��2��3 ��4 ��5��6 ��7 ��8

@@@

@@@��

��

HHHHH

HH

��

��

��

��0 ��1 ��2��3 ��4 ��5��6 ��7 ��8

@@@

@@@��

��

HHHHH

HH��

at 4: visit 7

Observe that the graph on the right is a tree.


the breadth-first tree

At the left is the result of the breadth-first search.

At the right is the breadth-first tree.

��0 ��1 ��2��3 ��4 ��5��6 ��7 ��8

@@@

@@@��

��

HHHHH

HH��

��0�

��

@@@��1 ��3 ��4��5 ��6 ��2 ��8 ��7

@@@

��

@@@

The breadth-first tree of a graph contains all the vertices of the graphand the edges visited in the breadth-first search.


the breadth-first search algorithm

Input: G = (V ,E).

Select the start vertex v ∈ V .

Initialize the queue Q with v : Q.push(v).

while not Q.empty() dou = Q.pop();visit u;for all v adjacent to u do

if v is not visited thenif v 6∈ Q thenQ.push(v).


running the breadth-first search algorithm

V = {0,1,2,3,4,5,6,7,8}E = {(0,1), (0,3), (0,4), (1,5), (1,6), (2,3), (2,5), (3,8),

(4,5), (4,6), (4,7), (5,7), (5,8), (7,8)}

Q = 0pop 0, visit 0, Q = 1,3,4pop 1, visit 1, Q = 3,4,5,6pop 3, visit 3, Q = 4,5,6,2,8pop 4, visit 4, Q = 5,6,2,8,7pop 5, visit 5, Q = 6,2,8,7pop 6, visit 6, Q = 2,8,7pop 2, visit 2, Q = 8,7pop 8, visit 8, Q = 8pop 7, visit 7, Q = ∅


depth-first searchWe start at vertex 0.��0 ��1 ��2��3 ��4 ��5��6 ��7 ��8

@@@

@@@��

��

HHHH

HHH

��

��

��

��0 ��1��0 ��1 ��5@@@

��0 ��1 ��2��5@@@


depth-first search continued

��0 ��1 ��2��3 ��4 ��5��6 ��7 ��8

@@@

@@@��

��

HHHH

HHH

��

��

��

��0 ��1 ��2��3 ��5@@@��

��

��

��0 ��1 ��2��3 ��5��8

@@@��

��

��

HHHHH

HH


depth-first search continued

��0 ��1 ��2��3 ��5��7 ��8

@@@�

��

HHHHHHH

��0 ��1 ��2��3 ��4 ��5��7 ��8

@@@�

��

HHHHHHH


the depth-first search tree

��0 ��1 ��2��3 ��4 ��5��6 ��7 ��8

@@@��

��

HHHHH

HH

��

��0A��1A��5A��2A��3A��8A��7A��4A��6

The depth-first search tree of a graphcontains all the vertices of the graph andthe edges visited in the depth-first search.


the depth-first search algorithm

algorithm VISIT(V ,E ,u,Visited)// Visits all vertices in the graph (V ,E), starting at u.// Visited collects the vertices visited.

visit u;Visited.push(u);for all v ∈ V , v is adjacent to u do

if v 6∈ Visited thenVISIT(V ,E , v ,Visited).

We call algorithm VISIT with vertex 0 for uand an empty list for Visited.


running the depth-first search algorithm

V = {0,1,2,3,4,5,6,7,8}E = {(0,1), (0,3), (0,4), (1,5), (1,6), (2,3), (2,5), (3,8),

(4,5), (4,6), (4,7), (5,7), (5,8), (7,8)}

visit 0, VISIT(V ,E ,1, {0})visit 1, VISIT(V ,E ,5, {0,1})

visit 5, VISIT(V ,E ,2, {0,1,5})visit 2, VISIT(V ,E ,3, {0,1,5,2})

visit 3, VISIT(V ,E ,8, {0,1,5,2,3})visit 8, VISIT(V ,E ,7, {0,1,5,2,3,8})

visit 7, VISIT(V ,E ,4, {0,1,5,2,3,8,7})visit 4, Visited = {0,1,5,2,3,8,7,4}

visit 6, Visited = {0,1,5,2,3,8,7,4,6}

The tree of recursive calls is the depth-first search tree.


Implementing Algorithms; Graphs

Documents

Transcript of Implementing Algorithms; Graphs