Algorithmique DistribuéM2 Recherche

Cours 3: Algorithms pour les réseaux à passage de messages

Shantanu DAS (shantanu.das@univ-amu.fr)

Message Passing Network Model

• Graph G : Network of n processors• Each processor has its own Data• Edge (u,v) : Communication link between u and v

Local vision

Initial KnowledgeEach vertex of G knows …• Its identity (ID)• The degree d• The incident edges are numbered 1,2,…d

(port numbering)

21

d

Additional Knowledge :• Topology of G• Number of vertices n, Number of edges m, Max degree D , Diameter(G), etc.

Assumptions

• Each node has a UNIQUE identifier (ID).

• G is connected ( n-1 <= m <= n(n-1)/2 )• Each edge is bidirectional link (G = undirected graph)• A node can send/receive messages to/from any neighbor.

• No faults : Each message M is received at destination 1. Synchronous : M is received after 1 unit of time.2. Asynchronous : M is received after arbitrary delay.

Distributed Algorithms

An algorithm has the following steps:1. Wakeup (how?)2. While ( Not “Termination Condition” )3. 4. Send messages5. Receive Messages6. Perform local computations7. 8. Output Results

Event-driven Programming!

- Instantaneously (External event)- By Clock (alarm)- On receiving a message

Fundamental Problems

1. Broadcast (Diffusion de message)2. Wakeup All (Reveiller tous le monde)3. Leader Election4. Consensus5. Spanning Tree Construction (Arbre Couvrant de G)6. Token Passing7. …

Correctness and Complexity

• Specify pre-conditions: assumptions on G, initial knowledge etc.• Specify post-conditions: on set of outputs by all nodes• Proof of Correctness: Show that -

• The algorithm terminates (on each node)• On termination, the set of output states satisfy the problem definition.

Complexity Analysis• Communication : Number of messages transmitted during the execution.• Time : Number of rounds of communication until all nodes terminate.(Local computations are instantaneous!)

Broadcast Problem (Diffusion)

• Pre-condition: Exactly one node u contains M• Post-condition: All nodes in G must obtain M

Algorithm (FLOOD)Node u : Send M to all neighbors;All Nodes : Repeat

If (received M) send M to all neighbors; until Termination;

When does a node terminate?

Broadcast Problem (Diffusion)

• Pre-condition: Exactly one node u contains M• Post-condition: All nodes in G must obtain M

Algorithm (FLOOD-once)Node u : Send M to all neighbors;All Nodes : Done = 0;

If (received M and Done ==0) send M to all neighbors; Done = 1;

When does the algorithm terminate? (Global Termination)

Broadcast (Correctness/Complexity)Algorithm (FLOOD-once)Node u : Send M to all neighbors;All Nodes : Done = 0;

If (received M and Done ==0) send M to all neighbors;

Done = 1;

Correctness: (Synchronous Networks)• After 1 round, all nodes at distance 1 (from u) have received M• After D rounds, all nodes at distance D (from u) have received M

Complexity: (Synchronous Networks)• Communication : O(m) messages - if G has m edges. • Time : O(D) rounds (time steps)

Effect of Topology of G

The complexity of algorithm FLOOD depends on G• If G has O(n^2) edges, Communication complexity = O(n^2)• If G has O(n log n) edges, Communication complexity = O(n log n)• If G has O(n) edges, Communication complexity = O(n)• …

If G is a tree, only O(n) messages would be sent.

If G is not a tree, construct a spanning tree T of G and execute algorithm on T=> Spanning Tree Construction

Spanning Tree Construction

Ø Pre-condition: Unique initiator node uØ Post-condition: Each node has one incident edge marked “Parent”.

The set of Parent links form a Spanning Tree of G.

AlgorithmExecute algorithm FLOOD-once with the following addition:• When a node v receives M first time, mark the sender as Parent.

Complexity: O(m) messages.

Spanning Tree Construction

Ø What happens if there are more than one initiator ?

LEADER ElectionØ Pre-condition: For each node, state = Undefined Ø Post-condition: Exactly one node has state=LEADER;

All other nodes have state=Non-Leader;

LEADER ElectionØ How to elect a leader?Recall: Each node has a unique ID

LEADER ElectionØ How to elect a leader?Recall: Each node has a unique ID• Node with smallest ID can be leader!• Each node knows its own ID, but not other’s ID …

LEADER ElectionØ How to elect a leader?Recall: Each node has a unique ID• Node with smallest ID can be leader!• Each node knows its own ID, but not other’s ID …

Trivial Algorithm:• Each node broadcast its ID • Each node compares its ID with all received ID (Assumption: n is known)

If (smallest ID) state = LEADER;Else state = Non-Leader;

Complexity ?

LEADER ElectionØ How to elect a leader?Recall: Each node has a unique ID• Node with smallest ID can be leader!• Each node knows its own ID, but not other’s ID …

Trivial Algorithm:• Each node broadcast its ID • Each node compares its ID with all received ID (Assumption: n is known)

If (smallest ID) state = LEADER;Else state = Non-Leader;

Complexity = O(n m) messages.More efficient algorithms ?

LEADER Election in TreeConverge-cast Technique• Leaves of T start computation (send ID to neighbor)• Node computes min(received IDs) => send to other neighbor

LEADER Election in TreeConverge-cast Technique• Leaves of T start computation (send ID to neighbor)• Node computes min(received IDs) => send to other neighbor• If a node receives the minimum from all neighbors, it can

determine the Leader.

LEADER Election in TreeComplexity = O(n) messages.Time = O(D) rounds.

LEADER Election in Ring

Algorithm LCR(Unidirectional Ring)• Send my ID to right neighbor• Repeat on receiving a message

If (x= received ID) < my IDSend x to right neighbor;Become Non-Leader ;

If (received ID = my ID)Become Leader ;

LEADER Election in Ring

Algorithm LCR(Unidirectional Ring)• Send my ID to right neighbor• Repeat on receiving a message

If (x= received ID) < my IDSend x to right neighbor;Become Non-Leader ;

If (received ID = my ID)Become Leader ;

Complexity = O(n^2) messages.

1

2

3

n

LEADER Election in Ring

Algorithm Phases(Bidirectional Ring)• X = 1;• While (state==Undecided)

Send ID to distance X;Receive messages;If (received ID > my ID)

state = Non-Leader ;If (received ID = my ID)

state = Leader ;X = 2*X

LEADER Election in Ring

Algorithm Phases(Bidirectional Ring)• X = 1;• While (state==Undecided)

Send ID to distance X;Receive messages;If (received ID > my ID)

state = Non-Leader ;If (received ID = my ID)

state = Leader ;X = 2*X

Complexity = O(n log n) messages.

Lower Bound on complexityTheorem (Burn 1980, Bodlaender 1987)Leader Election in ring networks requires Ω (n log n) messages

2 3

Lower Bound on complexityTheorem Leader Election in arbitrary graph requires Ω (m + n log n) messages

G

Lower Bound on complexityTheorem Leader Election in arbitrary graph requires Ω (m + n log n) messages

G

Lemma Any Election algorithm must send a message on each edge of G

Lower Bound on complexityTheorem Leader Election in arbitrary graph requires Ω (m + n log n) messages

G G

An algorithm that does not send a message on each edge e cannot solve Leader Election.

Election in Arbitrary GraphsAlgorithm GHS (Gallager Humblet Spira ‘83)• Each node is a tree with Level = 1, ID=node ID• Repeat

Each tree chooses a min outgoing edge e;Send merge request on edge e=(u,v); If (Level(u)== Level(v) and both send message on e)

Tree(u) is merged with Tree(v) using e;Level(u)=Level(u)+1; ID(u)=min(ID(u),ID(v))

If (Level(v) > Level(u))Tree(u) is absorbed in Tree(v) using e;ID(u)=min(ID(u),ID(v));

Complexity:O(m + n log n)

Algorithm GHS

Complexity Analysis• A tree with Level=i, has at least 2^i nodes.• Max number of Levels = log(n)• Trees at Level i together send O(n) messages.• On any non-tree edge at most 2 messages are sent.

Theorem:During the execution of algorithm GHS at most O(m + n log n) messages are sent. Thus the algorithm is optimal for message complexity.

Algorithm Yo-Yo

Algorithm Yo-Yo

Algorithm Yo-Yo

Algorithm Yo-Yo

Correctness:• There is always at least one source and one sink node.• The number of source nodes decreases after every round of

communication between sources and sinks.• There is eventually only one source which become LEADER.

Complexity:• There are log(n) phases.• In each phases, O(1) messages are sent over each edge of G.

Algorithm Yo-Yo has a complexity of O(m log n) messages.

Algorithm KKM (Korach, Kutten, Moran)

Modular Approach: • Use an exploration algorithm to obtain an algorithm for Election

Exploration Algorithm for G:A node u can send a token which will visit all nodes of G in some specific order and return to node u. Complexity = f(n)

Algorithm KKM :• Each node sends a token containing its ID.• The token containing the max ID survives, all other tokens are eliminated.Complexity = (n + f(n))(1+log n) messages

Leader Election (Specific G)

Complexity of Algorithms for Leader Election• Trees : O(n) messages• Rings : O(n log n) messages• Complete Graph K(n) : O(n^2) messages• Arbitrary Graphs : O(m + n log n) messages.

Exercice:Find an efficient algorithm for Leader Election in Grids (Grille)

Exercice:Find an efficient algorithm for Leader Election in Grids (Grille)

Leader Election using edge labels

Leader Election in Chordal Graphs• Complexity = O(n/k log n/k)

Leader Election using edge labels

Leader Election in Chordal Graphs• Complexity = O(n/k log n/k)

LE in Complete Graphs with chordal labelling• Complexity = O(n)

Leader Election using edge labels

Leader Election in Chordal Graphs• Complexity = O(n/k log n/k)

LE in Complete Graphs with chordal labelling• Complexity = O(n)

Leader Election in Hypercubes• Complexity = O(n)

Leader Election in HypercubesAlgorithm:v = ID;For d=1 to log(n)

If (state == Undecided)• Send v on edge of dimension d

If (received ID < v)• State = non-leader;• v = received ID;If (state == Undecided)

State = Leader;Complexity = O(n)

Synchronous Networks

• On Synchronous networks, each message takes one time step.• The absence of message carries information.

Leader Election (using time)• Wait for (n*ID) time steps• If (received a message) state= Non-leader;• Else: state=Leader; Broadcast message “I’m Leader!”Complexity = O(m) messages.

Synchronization

• On asynchronous networks, no bounds on message delays…• How to execute algorithms designed for synchronous networks?

Synchronizers:• Synchronizer

• Send “ok” to all after each round

• Synchronizer Beta• Use a spanning tree and send “ok” by converge-cast

• Synchronizer Gamma• Use a combination of the two synchronizers

Fault Tolerance

What happens if messages can be lost?• Omission Errors : A message is send, nothing is received• Corruption Error : A message is send, another message is received.

Broadcast with message omissions …• If the max number of message omissions per round is k, and• If the network is (k+1)-edge-connected,• then Broadcast is possible.