BGP Routing for Large Scale Data Centers

2

3

Design Requirements

4

100’s thousands of servers 10G NICs

Aware of the network Explicit parallelism Example: Web Index computation

5

Query Background

Query

Background

6

REQ1:

REQ2:

REQ3:

REQ4:

7

Network Design

8

Folded on diagram

ECMP Based – implements Valiant Load Balancing

9

10

11

Otherwise: Route Black-Holing on link failure!

12

Otherwise: Route Black-Holing on link failure!

13

Why BGP over IGP

14

Better vendor interoperability Less state-machines, data-structures etc

This way we can inject prefixes at any layer Used for unequal-cost Anycast load-balancing solution

15

BGP RIB structure is simpler compared to link-state LSDB Clear picture of what sent where (RIBIn, RIBOut)

E.g. link failures have limited propagation scope More stability due to reduced event “flooding” domains Hard to achieve the same using areas in IGPs

16

Not a problem with automated configuration generation

Is not our primary goal anyways, few seconds are OK Practical convergence in less than a second (Fast Fallover)

17

Details and Design Choices

18

We rely on ECMP for routing Needed for Anycast prefixes

Simplifies path hiding at WAN edge (remove private AS) Simplifies route-filtering at WAN edge (single regexp)

19

Allow AS in

20

ECMP programming

AS_PATH Multipath Relax

Allow AS In

Fast eBGP Fall-over

Remove Private AS

Unequal-cost load-balancing

32-bit Private ASNs

21

Questions?