Distributed Algorithms (CAS 769) - Week 2: Graph ... - MSU CSE

Distributed Algorithms (CAS 769)Week 2: Graph Algorithms

Dr. Borzoo Bonakdarpour

Department of Computing and SoftwareMcMaster University

Dr. Borzoo Bonakdarpour Distributed Algorithms (CAS 769) - McMaster University 1/67

Presentation outline

Graph Traversal

Minimum Spanning Tree Algorithm

Shortest Path (Routing) Algorithms


Acknowledgments

Most of the contents of these slides are obtained from thefollowing books:

I Distributed Algorithms: An Intuitive Approach - Wan FokkinkI Distributed Systems: An Algorithmic Approach - Sukumar

Ghosh


Wave Algorithms

In distributed computing, a process often needs to gather information from allother processes in the network.

Wave AlgorithmsIn such cases, the process initiates a wave algorithm, in which eachcomputation, called a wave, satisfies the following:

I It is finite.I It contains one or more decide eventsI For each decide event e and process p, f ≺ e for some event f at p.

Question: What if a process refuses to participate?


Graph Traversal

Traversal AlgorithmA traversal algorithm is a centralized wave algorithm; i.e., there is oneinitiator, which sends around a token.

I In each computation, the token first visits all processes.I Finally, a decide event happens at the initiator, who at that time holds the

token.

Spanning TreeA traversal algorithm builds a spanning tree:

I the initiator is the root, andI each non-initiator has as parent the neighbor from which it received the

token first.


Graph Traversal

Question 1: What is a simple algorithm for a ring topology?

Question 2: In the solution to Question 1, for each process, which event iscausally before the decide event?


Tarry’s Algorithm (1895!)

AssumptionAn undirected network of processes.

The AlgorithmInitially, an initiator sends out a token.

I Rule 1: A process never forwards the token through the same channeltwice.

I Rule 2: A process only forwards the token to its parent when there is noother option.

The token travels through each channel both ways, and finally ends up at theinitiator.


Tarry’s Algorithm - Example

s

p

t

q r

12

3

4

5

6

7

8 9

10

1112

Solid arrows establish a parent-child relation (in the opposite direction).



s

p

t

q r

1

2

3

4

5

6

7

8 9

10

1112




s

p

t

q r

12

3

4

5

6

7

8 9

10

1112




s

p

t

q r

12

3

4

5

6

7

8

9

10

1112




s

p

t

q r

12

3

4

5

6

7

8 9

10

1112




s

p

t

q r

12

3

4

5

6

7

8 9

10

11

12




s

p

t

q r

12

3

4

5

6

7

8 9

10

1112




The parent-child relation is the reversal of the solid arrows.

s

p

t

q r

Tree edges, which are part of the spanning tree, are solid.

Frond edges, which aren’t part of the spanning tree, are dashed.



s

p

t

q r

Question: Does Tarry’s algorithm generate a DFS tree?

In the spanning tree of a depth-first search, all frond edges connect anancestor with one of its descendants in the spanning tree.


Tarry’s Algorithm

ComplexityMessage complexity: 2E messages

Time complexity: 2E time units


Depth-first Search Algorithm

Depth-first search is obtained by adding to Tarry’s algorithm a third rule:

Rule 3: When a process receives the token, it immediately sends it backthrough the same channel if this is allowed by Rules 1 and 2.

s

p

t

q r

12

3

4 5

6

7

8

910

11

12





s

p

t

q r

1

2

3

4 5

6

7

8

910

11

12





s

p

t

q r

12

3

4 5

6

7

8

910

11

12





s

p

t

q r

12

3

4

5

6

7

8

910

11

12





s

p

t

q r

12

3

4 5

6

7

8

910

11

12





s

p

t

q r

12

3

4 5

6

7

8

9

10

11

12





s

p

t

q r

12

3

4 5

6

7

8

910

11

12


DFS with Neighbor Knowledge

To prevent transmission of the token through a frond edge (e.g., steps 3, 4and 9, 10), visited processes are included in the token.

The token is not forwarded to processes in this list (except when a processsends the token back to its parent).

Message complexity: 2N − 2 messages.

Each tree edge carries 2 tokens.

Space complexity: O(N log N) bits per message.

Time complexity: 2N − 2 time units


Tree Algorithm

AssumptionsAn undirected acyclic network.

The tree algorithm is a decentralized wave algorithm.

The AlgorithmThe local algorithm at a process p:

I Process p waits until it received messages from all neighbors exceptone, which becomes its parent. Then it sends a message to its parent.

I If p receives a message from its parent, it decides. It sends the decisionto all neighbors except its parent.

I If p receives a decision from its parent, it passes it on to all otherneighbors.

Always two processes decide (Proof in homework!).

ComplexityMessage complexity: 2N − 2 messages.


Tree Algorithm - Example

s

p

t

q r

u



s

p

t

q r

u



s

p

t

q r

u

q decides



s

p

t

q r

u

r decides



s

p

t

q r

u


Chang’s Echo Algorithm

It generates a BFS spanning tree. It is the basis of many other algorithms,e.g., Dijkstra-Scholten termination detection and Bellman-Ford shortest path.

AssumptionsAn undirected network.

The AlgorithmI The initiator sends a message to all neighbors.I When a non-initiator receives a message for the first time, it makes the

sender its parent.I If p receives a decision from its parent, it passes it on to all other

neighbors. Then it sends a message to all neighbors except its parent.I When a non-initiator has received a message from all neighbors, it

sends a message to its parent.I When the initiator has received a message from all neighbors, it decides.

ComplexityMessage complexity: 2E messages.


Echo Algorithm - Example

s

p

t

q r



s

p

t

q r



Graph Traversal




Minimum Spanning Tree

A graph can have many different spanning trees.

The weight of a spanning tree is sum of the weights of all edges.

Of all possible spanning trees of a graph, the spanning tree with the smallestweight is called the minimum spanning tree (MST), which has manyapplications.


Prim’s Sequential Algorithm

InputA non-empty connected weighted graph with vertices V and edges E (theweights can be negative).

The AlgorithmI Vn = {x}, where x is an arbitrary node in V , En = {}I Repeat until Vn = V :

I Choose an edge (u, v) with minimum weight such that u ∈ Vn and v 6∈ Vn, ifthere are multiple edges with the same weight, any of them may be picked)

I Vn := Vn ∪ {v}I En := En ∪ {(u, v)}

OutputVn and En (i.e., the MST)


Prim’s Sequential Algorithm

Question: Why is it difficult to make Prim’s algorithm distributed?

Prim’s algorithm requires:I processing one node at a timeI processes to know the state of the whole graph


The GHS Distributed Algorithm

InputA weighted graph G = (V ,E). The weight of an edge e is denoted by w(e).

AssumptionsI G must be connected.I Weights are distinct. Hence, a unique MST should be constructed.I Each node initially knows the weight for each edge incident to that node.I Channels are FIFO.

Overall Strategy

Consider two MSTs T1 = (V1,E1) and T2 = (V2,E2), called fragments, whereV1,V2 ⊆ V and V1 ∩ V2 = ∅. The algorithm finds the edge e with minimumweight that connects a vertex in V1 to a vertex in V2. Then the subgraphconsisting T1, T2, and e is the MST covering the nodes V1 ∪ V2.


Fragment 1 Fragment 2

6

10

3


GHS Algorithm

ChallengesI Challenge 1: How will the nodes in a given fragment identify the

minimum weight edge to be used for connecting with a differentfragment?

I Challenge 2: How will a node in a fragment T1 determine if a given edgeconnects to a different fragment T2 or the same tree T1?

SolutionsI Answer 1: Each fragment designates a coordinator (or root) to initiate

the search. The root will coordinate the task of choosing the edge withminimum weight and connecting it to a different fragment.

I Answer 2: All nodes in the fragment should adopt the same name.


GHS Algorithm

Each fragment belongs to a level, which is a non-negative integer.

Initially each node is a fragment at level 0.

Joining FragmentsI Merge: Fragments at the same level L connect. The level of the new

fragment is L + 1 and is named after the edge joining them. Why is thename unique?

I Absorb: A fragment at level L joins a fragment at level L′ where L′ > L.The level of the new fragment becomes L′ and the name of the newfragment is the name of the fragment at level L′.

Question: How many nodes a fragment at level L has?

At least 2L.


GHS Algorithm - Example

6

0

4

1

3

2

53

2

1

4

9

8

7

5

6

merge merge

merge



6

0

4

1

3

2

53

2

1

4

9

8

7

5

6

merge

absorb



6

0

4

1

3

2

53

2

1

4

9

8

7

5

6

absorb



6

0

4

1

3

2

53

2

1

4

9

8

7

5

6


GHS Algorithm

Algorithm SketchI Initially, every singleton node is a fragment, which is the root of the

fragment.I Each node then looks for the least weight edge connecting to a neighbor.

If both nodes pick each other, then a fragment of two nodes is formed atlevel 0.

I When two fragments join, the node with higher id across the least weightoutgoing edge serves as the new root. However, during absorb, the rootof the higher level fragment continues to serve as the new root.

I The notification about the change of root is sent out using thechangeroot message.

I Communication within a fragment takes place via the edges of thespanning tree. Thus, each nodes keeps track of its parent and children.


GHS Algorithm - Details

Detecting Least Weight Outgoing Edge (lwoe)Root broadcasts “initiate” in its own fragment, collects the report from othernodes about eligible edges using a convergecast, and determines the leastweight outgoing edge.

To test if an edge is outgoing, each node sends a test message through acandidate edge. The receiving node may send

I a reject message (if when it belongs to same fragment) orI an accept message (when it belongs to a different fragment).



Detecting Least Weight Outgoing Edge

6

0

4

1

3

2

53

2

1

4

9

8

7

5

6

0test

test

accept

reject




6

0

4

1

3

2

53

2

1

4

9

8

7

5

6

0

test

test

accept

reject




6

0

4

1

3

2

53

2

1

4

9

8

7

5

6

0test

test

accept

reject



Accept/RejectLet process i send “test” to process j :

I Case 1. If name(i) = name(j) then send “reject”I Case 2. If name(i) 6= name(j) ∧ level(i) ≤ level(j), then node j sends

“accept”I Case 3. If name(i) 6= name(j) ∧ level(i) > level(j), then wait until

level(j) = level(i) and then send “accept/reject”.

Question: Explain the reason for Case 3.

It may be the case that the responding node belongs a different fragmentwhen it received the test message, but it is also trying to merge with thesending fragment.

Question: Is there possibility of deadlock?



The Major Stepsrepeat

1. Test edges as outgoing or not

2. Determine least weight outgoing edge - it becomes a tree edge

3. Send join (or respond to join)

4. Update level & name & identify new coordinator/root

until there are no outgoing edges

Classification of EdgesBasic: initially all branches are basic

Branch: all tree edges

Rejected: not a tree edge

Branch and rejected are stable properties. Why?



Once the lwoe has been found, the root broadcasts a changeroot message toits fragment. This is a go ahead approval from the coordinator. The node thatfound the lwoe receives a changeroot message, it sends a join messagealong that edge indicating willingness to join.

Scenario 1: Merge

If i sends a join message to j , where level(i) = level(j) = L and receives ajoin message from j , then:

I Edge (i , j) becomes a branch message.I Between i and j whoever has the larger id becomes the new root.I The name of the fragment will be w(i , j).I Its level will be L + 1.I The root will broadcast (initiate, L + 1, name) message to the entire

combine fragment.



Scenario 2: AbsorbI Scenario 2.1: If i in fragment T at level L sends a join message to j in

fragment T ′ at level L′, where (1) L < L′, and (2) T ′ has not yet foundlwoe, then T ′ absorbs T :

I Process j sends a join message to i , indicating that T has been absorbed.I The root of T ′ will be the root of the combined fragment.I T changes its level to L′ and adopts the name of T ′.I The collectively search for a lwoe.I Edge (i, j) becomes a branch message.

I Scenario 2.2: If T ′ has already chosen a lwoe, then the lwoe cannot be(i , j). That means T ′ is joining another T ′′. Once this join is complete,the combined fragment of T ′ and T ′′ starts a new search and will sendan initiate message to T to signal that T has been absorbed.



Graph Traversal




Routing

Routing means guiding a packet in a network to its destination.

A routing table at node u stores for each v 6= u, a neighbor w of u: Eachpacket with destination v that arrives at u is passed on to w .

Criteria for good routing algorithms:

I use of optimal pathsI computing routing tables is cheapI small routing tablesI robust with respect to topology changes in the networkI table adaptation to avoid busy edgesI all nodes are served in the same degree


Chandy-Misra Algorithm

A centralized algorithm to compute all shortest paths to an initiator i .

VariablesEach process p maintains variables:

I distp is the length of the shortest known path from p to i . Initially,dist i = 0 and distp =∞, for all p, where p 6= i .

I parentp is the process after p on the currently known shortest knownpath from p to i . Initially, parentp⊥, for all processes.


Chandy-Misra Algorithm

The AlgorithmI i sends the message 〈0〉 to its neighbors.I When a process p receives 〈d〉 from a neighbor q, and if

d + w(p, q) < distp, then:I distp ← d + w(p, q)I parentp ← qI p sends 〈distp〉 to its neighbors (except q)

Chandy-Misra algorithm requires termination detection.


Chandy-Misra Algorithm - Example

x

v

i

w

1

1

1

61 4

dist i ← 0 parent i ← ⊥

distw ← 6 parentw ← idistv ← 7 parentv ← wdistx ← 8 parentx ← vdistx ← 7 parentx ← wdistv ← 4 parentv ← idistw ← 5 parentw ← vdistx ← 6 parentx ← wdistx ← 5 parentx ← vdistx ← 1 parentx ← idistw ← 2 parentw ← xdistv ← 3 parentv ← wdistv ← 2 parentv ← x



x

v

i

w

1

1

1

61 4

dist i ← 0 parent i ← ⊥distw ← 6 parentw ← i

distv ← 7 parentv ← wdistx ← 8 parentx ← vdistx ← 7 parentx ← wdistv ← 4 parentv ← idistw ← 5 parentw ← vdistx ← 6 parentx ← wdistx ← 5 parentx ← vdistx ← 1 parentx ← idistw ← 2 parentw ← xdistv ← 3 parentv ← wdistv ← 2 parentv ← x



x

v

i

w

1

1

1

61 4

dist i ← 0 parent i ← ⊥distw ← 6 parentw ← idistv ← 7 parentv ← w

distx ← 8 parentx ← vdistx ← 7 parentx ← wdistv ← 4 parentv ← idistw ← 5 parentw ← vdistx ← 6 parentx ← wdistx ← 5 parentx ← vdistx ← 1 parentx ← idistw ← 2 parentw ← xdistv ← 3 parentv ← wdistv ← 2 parentv ← x



x

v

i

w

1

1

1

61 4

dist i ← 0 parent i ← ⊥distw ← 6 parentw ← idistv ← 7 parentv ← wdistx ← 8 parentx ← v

distx ← 7 parentx ← wdistv ← 4 parentv ← idistw ← 5 parentw ← vdistx ← 6 parentx ← wdistx ← 5 parentx ← vdistx ← 1 parentx ← idistw ← 2 parentw ← xdistv ← 3 parentv ← wdistv ← 2 parentv ← x



x

v

i

w

1

1

1

61 4

dist i ← 0 parent i ← ⊥distw ← 6 parentw ← idistv ← 7 parentv ← wdistx ← 8 parentx ← vdistx ← 7 parentx ← w

distv ← 4 parentv ← idistw ← 5 parentw ← vdistx ← 6 parentx ← wdistx ← 5 parentx ← vdistx ← 1 parentx ← idistw ← 2 parentw ← xdistv ← 3 parentv ← wdistv ← 2 parentv ← x



x

v

i

w

1

1

1

61 4

dist i ← 0 parent i ← ⊥distw ← 6 parentw ← idistv ← 7 parentv ← wdistx ← 8 parentx ← vdistx ← 7 parentx ← wdistv ← 4 parentv ← i

distw ← 5 parentw ← vdistx ← 6 parentx ← wdistx ← 5 parentx ← vdistx ← 1 parentx ← idistw ← 2 parentw ← xdistv ← 3 parentv ← wdistv ← 2 parentv ← x



x

v

i

w

1

1

1

61 4

dist i ← 0 parent i ← ⊥distw ← 6 parentw ← idistv ← 7 parentv ← wdistx ← 8 parentx ← vdistx ← 7 parentx ← wdistv ← 4 parentv ← idistw ← 5 parentw ← v

distx ← 6 parentx ← wdistx ← 5 parentx ← vdistx ← 1 parentx ← idistw ← 2 parentw ← xdistv ← 3 parentv ← wdistv ← 2 parentv ← x



x

v

i

w

1

1

1

61 4

dist i ← 0 parent i ← ⊥distw ← 6 parentw ← idistv ← 7 parentv ← wdistx ← 8 parentx ← vdistx ← 7 parentx ← wdistv ← 4 parentv ← idistw ← 5 parentw ← vdistx ← 6 parentx ← w

distx ← 5 parentx ← vdistx ← 1 parentx ← idistw ← 2 parentw ← xdistv ← 3 parentv ← wdistv ← 2 parentv ← x



x

v

i

w

1

1

1

61 4

dist i ← 0 parent i ← ⊥distw ← 6 parentw ← idistv ← 7 parentv ← wdistx ← 8 parentx ← vdistx ← 7 parentx ← wdistv ← 4 parentv ← idistw ← 5 parentw ← vdistx ← 6 parentx ← wdistx ← 5 parentx ← v

distx ← 1 parentx ← idistw ← 2 parentw ← xdistv ← 3 parentv ← wdistv ← 2 parentv ← x



x

v

i

w

1

1

1

61 4

dist i ← 0 parent i ← ⊥distw ← 6 parentw ← idistv ← 7 parentv ← wdistx ← 8 parentx ← vdistx ← 7 parentx ← wdistv ← 4 parentv ← idistw ← 5 parentw ← vdistx ← 6 parentx ← wdistx ← 5 parentx ← vdistx ← 1 parentx ← i

distw ← 2 parentw ← xdistv ← 3 parentv ← wdistv ← 2 parentv ← x



x

v

i

w

1

1

1

61 4

dist i ← 0 parent i ← ⊥distw ← 6 parentw ← idistv ← 7 parentv ← wdistx ← 8 parentx ← vdistx ← 7 parentx ← wdistv ← 4 parentv ← idistw ← 5 parentw ← vdistx ← 6 parentx ← wdistx ← 5 parentx ← vdistx ← 1 parentx ← idistw ← 2 parentw ← x

distv ← 3 parentv ← wdistv ← 2 parentv ← x



x

v

i

w

1

1

1

61 4

dist i ← 0 parent i ← ⊥distw ← 6 parentw ← idistv ← 7 parentv ← wdistx ← 8 parentx ← vdistx ← 7 parentx ← wdistv ← 4 parentv ← idistw ← 5 parentw ← vdistx ← 6 parentx ← wdistx ← 5 parentx ← vdistx ← 1 parentx ← idistw ← 2 parentw ← xdistv ← 3 parentv ← w

distv ← 2 parentv ← x



x

v

i

w

1

1

1

61 4

dist i ← 0 parent i ← ⊥distw ← 6 parentw ← idistv ← 7 parentv ← wdistx ← 8 parentx ← vdistx ← 7 parentx ← wdistv ← 4 parentv ← idistw ← 5 parentw ← vdistx ← 6 parentx ← wdistx ← 5 parentx ← vdistx ← 1 parentx ← idistw ← 2 parentw ← xdistv ← 3 parentv ← wdistv ← 2 parentv ← x


Merlin-Segall Algorithm

A centralized algorithm to compute all shortest paths to an initiator i . Itcombines the idea of Chandy-Misra algorithm and Chang’s Esho Algorithm.

VariablesEach process p maintains variables:

I distp is the length of the shortest known path from p to i . Initially,dist i = 0 and distp =∞, for all p, where p 6= i .

I parentp values form a sink tree with root at i .


Merlin-Segall Algorithm

The AlgorithmI i sends the message 〈0〉 to its neighbors.I When a process p receives 〈d〉 from a neighbor q.

If d + w(p, q) < distp, then:I distp ← d + w(p, q)I p stores w as future value for parentp

If q = parentp, then p sends 〈distp〉 to its neighbors except parentp

I When a p 6= i has received a message from all neighbors, it sends〈distp〉 toparentp, and updates parentp.

I i starts a new round after receiving a message from all neighbors.

Question: Does this algorithm need termination detection?

The algorithm can terminate after N − 1 rounds.


Merlin-Segall Algorithm - Example (Round 1)


Toueg’s Algorithm

A decentralized algorithm to compute all shortest paths to all pairs of verticesin a graph G = (V ,E). It generalizes the well-known Floys-Warshallalgorithm:

For a set of processes S, it computes a distance function dS(p, q),denoting the length of a shortest path between p and q with allintermediate processes in S.

Algorithm’s ConditionsThe following equations hold:

I dS(p, p) = 0I d∅(p, q) = w(p, q) if p 6= q and (p, q) ∈ EI d∅(p, q) =∞ if p 6= q and (p, q) 6∈ EI dS∪{r} = min{dS(p, r) + dS(r , q), dS(p, q)} for each r 6∈ E


Floyd-Warshall Algorithm

Initially, S = ∅; the first three equations define d∅.

While S doesn’t contain all nodes, a pivot r 6∈ S is selected:I dS∪{r} is computed from dS using the fourth equation.I r is added to S.

When S contains all nodes, dS is the standard distance function.

Time complexity: Θ(N3)


Floyd-Warshall Algorithm

Floyd-Warshall Algorithm in a Distributed SettingChallenge 1: All nodes must pick the pivots in the same order.

Challenge 2: Each round, nodes need the distance values of the pivot tocompute their own routing table.


Toueg’s Algorithm

AssumptionEach node knows from the start the id’s of all nodes. (Because pivots mustbe picked uniformly at all nodes.)

InitializationInitially, at each process p:

I Sp = ∅I distp(p) = 0 and parentp(p) = ⊥I For each q 6= p, either

I distp(q) = w(p, q) and parentp(q) = q, if (p, q) ∈ EI distp(q) =∞ and parentp(q) = ⊥, otherwise.


Toueg’s Algorithm

At the r -pivot round, r broadcasts its values dist r (q), for all nodes q.

If parentp(r) = ⊥ for a node p 6= r at the r -pivot round, then distp(r) =∞, sodistp(r) + dist r (q) ≥ distp(q), for all nodes q.

Hence the sink tree of r can be used to broadcast dist r .

Each node p sends 〈request, r〉 to parentp(r), if it isn’t ⊥, to let it pass ondist r to p.

If p isn’t in the sink tree of r , it proceeds to the next pivot round, withSp ← Sp ∪ {r}.


Toueg’s Algorithm

Suppose p is in the sink tree of r .

If p 6= r , it waits for the values dist r (p) from parentp(r).

p forwards the values dist r (q) to neighbors that send 〈request, r〉.

If p 6= r , it checks for each node q whether

distp(r) + dist r (v) < distp(q)

If so, distp(q)← distp(r) + dist r (p) and parentp(q)← parentp(r).

Finally, p proceeds to the next pivot round, with Sp ← Sp ∪ {r}.


Toueg’s Algorithm - Example

s

p

r

q

1

1

4

1

pivot p dists(q)← 5 distq(s)← 5parents(q)← p parentq(s)← p

pivot q distp(r)← 5 dist r (p)← 5parentp(r)← q parent r (p)← q

pivot r dists(q)← 2 distq(s)← 2parents(q)← r parentq(s)← r

pivot s distp(r)← 2 dist r (p)← 2parentp(r)← s parent r (p)← s

distp(q)← 3 distq(p)← 3parentp(q)← s parentq(p)← r


Distributed Algorithms (CAS 769) - Week 2: Graph ... - MSU CSE

Documents

Transcript of Distributed Algorithms (CAS 769) - Week 2: Graph ... - MSU CSE