BRKOPT-2006.pdf - Cisco Live

#CLUS

Errol Roberts – Distinguished Systems EngineerMark Nowell – Cisco Fellow, EngineeringBRKOPT-2006

Get up to Speed -Preparing for 400 GbE

© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS

400 GbE implementations will be affected by cabling, new form factors and system architecture requirements. The session will explore some of the technology implications, challenges and requirements as well as provide deployment insights to help with the adoption of 100 GbE to 400 GbE in current and future architectures. The session in targeted to everyone interested in learning more about this technology space.

Abstract

BRKOPT-2006 3


Agenda

• 100 GbE-400 GbE Industry

• Technology Considerations

• Infrastructure Considerations

• Architectural and Deployment Implications

• Conclusion

BRKOPT-2006 4

100G-400G Industry


Summary

Demand for 400 GbE is here

Uniquely for 400 GbE, multiple solutions will

exist in a common QSFP-DD pluggable

form factor

Industry is broadly engaged to deliver 400

GbE now

Cisco is a leader in all aspects

BRKOPT-2006 6


Ethernet Roadmap6 new speeds in 1st 35 years of Ethernet

6 new speeds in a 2 year span

• 2.5 GbE – 2016

• 5 GbE – 2016

• 25 GbE – 2016

• 50 GbE – late 2018

• 200 GbE – 2017

• 400 GbE – 2017

• Proliferation of MSAs/Consortia

• QSFP-DD, OSFP

• 100G Lambda

• COBO

https://ethernetalliance.org/the-2018-ethernet-roadmap/

BRKOPT-2006 7


Ethernet Port Speed Transitions

Source:

•

•

Millions

70

0

1 GbE

10 GbE 25 GbE

40 GbE

50 GbE

100-200 GbE

400 GbE

BRKOPT-2006 8


Optical Module Market Forecast slides100 GbE Optical Modules by reach 100 GbE & 400 GbE 2 & 10 km SMF Modules

• 100 GbE optics market still in strong growth phase• 400 GbE at start of its ramp but expected to be fast (possibly used for dense 100 GbE)

Courtesy Dale Murray, Light CountingCourtesy Dale Murray, Light Counting

100-300 m

2 km

10-20 km

500 m

40 km

5.6 million

modules

in 2018

BRKOPT-2006 9


400 GbE Optical Module market projections

-

5

10

15

20

1 2 3 4 5Years after initial adoption

Normalized Optical Module Shipping

Volumes

Total 100 GbE

Total 400 GbE

Courtesy: LightCounting

Year 5 of 400 GbE is ~20x larger than Year 5 of 100

GbE

Driven by multiple markets looking to adopt at same time.

This provides an exciting challenge for equipment and component manufacturers

BRKOPT-2006 10


400 GbE development impacts 100 GbE

11BRKOPT-2006

The technology development for 400 GbE (and 200 GbE) has driven new technologies which will impact 100 GbE deployments

Technology considerations:

Client and Line/Transport optics


Client Optics Technology summary

Increase baud rate

(e.g. 10G to 25G)SR, LR, etc.

Increase number of

fibers

(Parallel)SR4, PSM4, DR4,

Increase number of

wavelengths

(WDM)

LR4, ER4, BiDi, CWDM4, FR4

Change modulation

format

(e.g. NRZ to PAM4)

100G-FR, 100G BiDi, 400G-DR4

Enhance Bit Error Rate

with Forward Error

Correction (FEC)

Everything above 40G

(except 100G-LR4)

Block of data Protected data

BRKOPT-2006 13

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwiOxY2u9I7TAhWZ0YMKHfrND1cQjRwIBw&url=http://www.fiber-optic-transceiver-module.com/tag/40gbase-sr4&bvm=bv.151426398,d.amc&psig=AFQjCNG7wbSN5kvC2OmaWYsgB4Cy2cOqWA&ust=1491536315661363


Anatomy of a pluggable module

PHY/CDR n:m Gearbox

TIA

LD

Host-Module SERDESinterface

High Speed optical side SERDES

interfaces

• Ethernet’s architecture allows these sets of interfaces to evolve independently. • Enables continual optimization of cost to occur.

Optimum when n = m

n m

BRKOPT-2006 14


Anatomy of a linecard with pluggable modules

15BRKOPT-2006

ASIC

SystemModule

Interface

Interfaces: • Point of interoperability• 400G-DR4, 400G-FR4,

400GBASE-LR8, 40G-BiDi etc

Modules:• System design choice• Density, portfolio• Does not affect interoperability

System:• Responsible for integration of

solution. Power, Cooling etc

Eth

ern

et

Cisco linecard

Host (linecard) Connector

Pluggable ModuleStandards - IEEE

Non-Standard

MSA


Building an optical link

Implementations can vary but as long as the necessary blocks are identical, interop is guaranteed.

Ethernet architecture allows different implementations to happen as technology matures

ASIC opticsEth

ern

et

RT ASICoptics

Imple

menta

tion

Eth

ern

et

BRKOPT-2006 16

Must be identical


ASIC optics

Eth

ern

et

RT ASICoptics

Imple

menta

tion

Eth

ern

et

Building an Ethernet optical link

MD

I

PC

S

FEC

PM

A

PM

D

MED

IUM

MA

C/R

S MII M

DI

PC

S

FEC

PM

A

PM

D

MED

IUM

MA

C/R

S

MII

Must be identical

Eth

ern

et

Same picture but looking at it from an Ethernet perspective

The Ethernet blocks can exist and be

duplicated anywhere

BRKOPT-2006 17


ASIC SerDes: need to know

As ASIC bandwidths increase the max SerDes speed has to increase with it

ASICs Maximum Ethernet

MAC Rate

Most likely maximum SerDes

speed

Interface width

400 GbE50 Gb/s

(100 Gb/s coming)8

(4)

100 GbE 25 Gb/s 4

40 GbE 10 Gb/s 4

25 GbE 25 Gb/s 1

10 GbE 10 Gb/s 1

SerDes are generally multi-rate:

e.g. 50 Gb/s SerDes can also run at:• 25 Gb/s & 10 Gb/s

This info becomes relevant when interconnecting different

generation systems using breakout optics

BRKOPT-2006 18


Pluggable Form Factors: Need to know

SFP QSFP

10 GbE SFP -

25 GbE SFP28 -

40 GbE - QSFP+

50 GbE SFP56 -

100 GbE SFP-DD QSFP28

200 GbE - QSFP56

400 GbE - QSFP-DD

SFP & QSFP pluggables are THE industry standard• Multi-rate backwards compatibility

SFP – 1 high-speed interfaceQSFP – 4 high-speed interfacedDD (Double Density) – twice the high-speed interfaces

Other legacy form factors exist (XFP, CFP) or challenger form factors (OSFP)

BRKOPT-2006 19


Interconnection evolution

Duplex Fiber

Parallel Fiber

WDM FEC

10 GbE

40 GbE

100 GbE

200/400 GbE

25/50 GbE

Enables Breakout

Faster optics required more techniques

Provides more deployment options but also awareness when interconnecting

BRKOPT-2006 20

FEC was first introduced for use in Ethernet optics for most of the 100 GbE specifications but not the earliest ones.


Coherent Optics (Transport/Line)Technology summary

21BRKOPT-2006

Increase baud rate

(e.g. 32 GBaud to 72 GBaud)

Advanced Modulation

(and flexible/hybrid)

Dual Polarization Modulation

Tunable Lasers

Advanced Equalization and

Digital Signal Processing

Enhance Bit Error Rate with

Forward Error Correction

(FEC)Block of data Protected data

BPSK QPSK 8QAM 16QAM 32QAM 64QAM


ASIC optics

Eth

ern

et

DSP ASICoptics

Must be identicalImple

menta

tion

Eth

ern

et

Building a DWDM optical link (with pluggable optical modules)

Must be identical

DW

DM

Same picture but looking at it from an DWDM functionality perspective.Typically this means single-vendor book-ended links, but some recent DWDM standards enabling multi-vendor interop (400ZR)

The DSP can exist in the optical

module

DSP Line Side

Optics DSP

Line Side

Optics

BRKOPT-2006 22


Coherent Optics Integration Innovation

CFP2-ACO

5x7 inches

3x6 inches

CFP2

2011 2014 2016 2018 2020

QSFP-DDDCO*

CFP2 DCO* DCO* - Digital Coherent OpticsHas DSP + Coherent Optics

More

inte

gra

tion

BRKOPT-2006 23


Introducing QSFP DD• QSFP-DD MSA has very broad industry support

• MSA has 66 member companies

• Port is backward compatible to QSFP+, QSFP28, QSFP56

• Leverages industry cost structure and production capability of

QSFP

• Nearly 30M QSFP modules deployed to date

• More than 66M QSFP ports will be deployed by end of ‘19

• Support 400 GbE and 2/4x 100 GbE designs

• QSFP-DD will support up to 20W of power dissipation

• Broad product offering from copper cable to 400G-ZR

• Supports ASIC interfaces - 400G AUI-8 (8x 50G PAM4)

• Support network requirements for system density: 32 & 36 ports

• Support necessary thermal/SI for implementations (all optical and copper reaches

BRKOPT-2006 24


QSFP-DD Enhanced Heatsink Performance• Improved heatsink raises the power

dissipation performance to over 20W

• MSA compliant form factor

• Will fit all QSFP-DD ports

• Chart shows performance in 1RU fixed box

• Module temp rise well within limit for Tcase

• 30C case temp limit allows for 45C ambient and maintain Tcase below 75C

15W

18W

21W

15.0

20.0

25.0

30.0

35.0

14 16 18 20 22

Tem

pera

ture

ris

e (

°C

)

Power (W)

Module Temperature Rise with Power

Dissipation

QSFP-DD 20W? No Problem!

30C temp rise limit

Data as shown at OFC2019

BRKOPT-2006 25

• Cisco N3K-C3432D-S Ethernet Switch with 32 Ports of QSFP-DD

• 8 thermal modules each dissipating 20W

• Average Power: 21.6W

• Average temperature rise: 18.3C

• Cisco SP360 blog on the demo• https://blogs.cisco.com/sp/cisco-demonstrates-20w

-power-dissipation-of-qsfp-dd-at-ofc-2019

OFC2019 QSFP-DD Thermal Demo

2 0 W

2 0 W

20W 20W

20W 20W

Nexus N3K-C3432D-SEthernet Switch

32 Ports QSFP-DD

Thermal Modules21.7W each

Infrastructure Considerations


100G

1km

10km

40G

Multimode Fiber

Shorter link distances with higher data rates on OM4 multimode fiber.

Network fiber migration with increase in data rates

2km

2km

500m

10G

1G

400m

150m

100m

10km

10km

10km

Single Mode Fiber

Short reach optics for single mode fiber at higher data rates.

400G100m 2km500m

10km


Optics Codes (Cheat Sheet)

wG-xRy.z e.g. 400G-DR4

(w) Data Rate:10 Gb/s25 Gb/s50 Gb/s100 Gb/s200 Gb/s 400 Gb/s

(x) Reach:MMF

S = 100 mSMF

D = 500 mF = 2 kmL = 10 kmE = 40 kmZ = 80 km

(y.z) Lane Count:(fiber or Wavelength)y = “ “ single fiber/wavelengthy = “4”

F/L = 4 wavelengthsD = 4 fibers

y = “4.2”e.g. SR4.2MMF: 4 fibers

2 wavelengths per fiberBRKOPT-2006 29

Narrower Lower Cost


400 GbE modules and use cases

3+ m

400G-CR88x 50G-CR400G-AOC(30m)

100 m

400G-SR8400G-SR4.2400G-DR4

500m-2km

400G-DR4400G-FR4

10 km

400G-LR4400G-LR8

100+ km

400ZR400ZR+

Copper Cables /

AOC(Active Optical

Cable)

MMF / SMF

SMF SMF SMF

Dis

tance

Optics

Media

BRKOPT-2006 30


10 GbE 25 GbE 40 GbE 50 GbE 100 GbE 200 GbE 400 GbE

SMF

Duplex 10G-LR25G-LR25G-ER

40G-LR450G-FR50G-LR

100G-DR100G-FR100G-LR100G-LR4100G-ER4CWDM4

200G-FR4200G-LR4

400G-FR4400G-LR4400G-FR8400G-LR8

400ZR

Parallel (x4)

- - - - 100G-PSM4 200G-DR4 400G-DR4

MMF

Duplex 10G-SR 25G-SR 40G-BiDi 50G-SR 100G-BiDi - (future?)

Parallel (x2/x4)

- -40G-SR4

40G-CSR4-

100G-SR4100G-CSR4100G-SR2

200G-SR4400G-SR4.2

(BiDi)

Parallel (x8/x10/x16)

- - - - 100G-SR10 -400G-SR8

400G-SR16

Fiber Infrastructure – 10 GbE to 400 GbEUpgrade paths for different fiber options

BRKOPT-2006 31

Broadest long term support

Largest installed base of MMF

Lower cost enabled by SiPhotonics

Parallel needed to support higher data rates


Multimode fiber options

OM3

OM5

0

1000

2000

3000

4000

5000

850nm 880nm 910nm 940nm

Not characterized

OM3/OM4• Optimized for 850nm

OM5• Wideband MMF

specified @ 850 nm and 953nm

• Created to enable SWDM interfaces

(e.g. 100G-SWDM4)

**

* Illustrative – not specified

Cisco’s 40G/100G BiDi uses 850nm and 910nm but can operate over all MMF types

Fiber Modal Bandwidth

MHz*km

BRKOPT-2006 32


Two dominant ferrule/connector types

Optical Connector Considerations

MPO/MTP –multiple fiber ferrule

LC – single fiber ferruleNew Dual ferrule

connectorsMDC

SN

New

Used for all duplex fiber applications – SFP, QSFP,

QSFP-DD etc

Traditionally used for all parallel fiber applications –

QSFP, QSFP-DD etc

New high-density connectors for breakout applications

(market adoption pending…)

CourtesyCourtesy

BRKOPT-2006 33

Uni-boot designs are common


Connector and optical lane mapping

The mapping of Tx/Rx to specific fibers locations is defined by standards

Any duplex interface(eg 100G-CWDM4)

Dual duplex interface(eg 2x 100G-LR4)

Quad duplex interface(eg 400G-DR4 used

as 4x100G-DR) SN or MDC

Quad interface(eg 100G-SR4)

Quad BiDi interface(eg 400G-SR4.2)

Octal interface on MPO12(eg 400G-SR8)

Octal interface on MPO16(eg 400G-SR8)

BRKOPT-2006 34


Connector Considerations

• LC connector –widely used, good performance, robust, many suppliers

• MPO connector. Highest density connector. Used for MMF and SMF. Disadvantages, higher loss, difficult to clean, less robust than single ferruled connectors

• CS connector – single supplier currently, in the process of being qualified. Training re: deployment adoption a consideration – cleaning, inspection etc.

• SN or MDC – not compatible today, yet another new connector. Training and deployment considerations.

• Advantage – LC Ferruled connectors are robust, low insertion loss relative to MPO, consistent fiber to fiber connector. Early in development for high density application, break out in the module

• Disadvantage – Not as dense as MPO. Small size harder to insert/de-insert manually.

• DC fiber management- bridge between LC and high density MPO

BRKOPT-2006 35


2X100G QSFP-DD to 100G QSFP28 BreakoutCS Connector application – breakout in connector

• QSFP-DD 2X100G Modules provide the ability to connect legacy 100G modules to QSFP-DD ports

• Available in 2X 100G-LR4, 2X 100G-CWDM4, 2X 100G-PSM4 or 2X 100G-SR4 modules

Optical breakout cable or breakout thru a patch panel

2 X 100G-LR4QSFP-DD Module

QSFP-100G-LR4

QSFP-100G-LR4

Standard Duplex LC SMF Connectors

Dual CS SMF Connectors

2X 100G-LR4 Breakout Example

Module Type Optical Connector

2X 100G-LR4 Dual Duplex CS Connector

2X 100G-CWDM4 Dual Duplex CS Connector

2X 100G-PSM4 SMF MPO-24 Connector

2X 100G-SR4 MMF MPO-24 Connector

BRKOPT-2006 36


400GBASE-DR4 to 100GBASE-DR Breakout (SN / MPO)• 400GBASE-DR4 provides 4 lanes of 1λ-100G-PAM4 (100GBASE-DR) optical

signal with a reach of 500m • 4x 100G-FR provides 4 lanes of 1λ-100G-PAM4 (100G-FR) optical signal extended to 2km

• Module will use an MPO-12 SMF connector

• 400GBASE-DR4 can be used for high density 100G interface with breakout QSFP-100G-DR/FR module

• QSFP-100G-DR/FR will provide low cost 100G to any QSFP28 port

• QSFP-DD provides the industry’s highest density 100G interface

Optical breakout cable or breakout thru a patch panel

400GBASE-DR4 or 4x100G-FR

Module(4x100G PAM4, 1-λ)

QSFP-DD

QSFP-100-DR/FRModule

(100G PAM4, 1-λ)QSFP28





100-DR/FRModule

(100G PAM4, 1-λ)QSFP28 QSFP 100G-DR

Duplex LC SMF Connector 1 lane of 1λ-100G-PAM4

signals

QSFP-DD 400GBASE-DR4 or QSFP-DD 4x100G-FRMPO-12 SMF Connector w/ 4 parallel lanes of 1λ-

100G-PAM4 signals


400ZR and 400ZR+

• OIF / OpenROADM / ITU driven interface DWDM specification - Multivendor

• Pluggable DCO (Digital Coherent Optics) module

• Tunable laser

• Configurable speeds: 100G, 200G, 300G or 400G

• Maximum 20W power for 400ZR+

• Supported by QSFP-DD (demonstrated at OFC 2019)

• Provides the same dense port density as other optics –mix and match possible

• Expected to deliver significant reduction in cost/bit

• Multiple options expected (different reaches and power)

BRKOPT-2006 38


400ZR/ZR+ Approximate Reaches

Specification Framing Data Rate Modulation Ref. Reach Form Factor

400ZR OIF ZR

400G DP-16QAM 120 km QSFP-DD


100G QPSK >7000 km QSFP-DD

400ZR+(400G FlexO-LR)

FlexO


300G DP-8QAM 1500 km QSFP-DD / CFP2

200G DP-QPSK 4000 km QSFP-DD / CFP2

100G DP-QPSK 8000 km QSFP-DD / CFP2

39BRKOPT-2005

Vendor differences exist on performance for non-standardized modesCFP2 enables higher launch power

Standard Vendor Specific


400ZR modules and use cases

BRKOPT-2002

QDD-400G-ZR-S - Point to Point ~80-120 km amplified~40km unamplified

QDD-400G-ZR-S - DWDM ~80-120 km

QSFP-DD 400ZROIF 400G-ZR compliant

• Mapping to 400GZR Frame• 400G 16QAM • C-FEC 15%• 2400ps/nm (80-120 km)• C-Band tunability• Pout: -10dBm min• <15W


400GZR+ modules and use cases

41BRKOPT-2006

~1500kmQDD-400G Metro

…

…

…

…

~450kmQDD-400G-ZRP - WDMQSFP-DD 400ZR+ (Extended Reach)

• Mapping to 400G ZR Frame• 400G 16QAM

O-FEC / SD-FEC<36,000ps/nm (<1500km@400G)

• C-Band tunability• Pout: -10dBm min• <18W• Other speed options:

200G QPSK/300G 8QAM

QSFP-DD-400G-Metro• OpenROADM Compliant

• Map to ODU4 & ODUflex• 400G 16QAM

O-FEC / SD-FEC<36,000ps/nm (<1500km@400G)

• C-Band tunability

• Pout: -10dBm min• <21W• Other speed options:

200G QPSK/300G 8QAM/100G QPSK w/ SC-FEC


Increasing Baud Rates – Broader Spectrum usage

Finding the correct fit for WDM (line) filters and interface speed

75 GHz50 GHz

32Gbd

• 75 GHz Spectrum required for 400GbE

• Fixed grid 100 GHz/75 GHz/FlexGrid

BRKOPT-2006 42

64Gbd


400G ZR/ZR+ Considerations

• Channel Bandwidth for 400 GbE >60GBd

• Mux/Demux channel width considerations – 75 GHz/100 GHz Mux, FlexSpectrum

• Modules can be configured to operate in multiple modes (Data rates, FECs)

• Network planning required – brown field considerations, assess gain, … amp/ gain module, etc…

• Choosing the module for right applications – OIF, Ethernet, OpenROADM, ITU, & proprietary modes. Interop depends on

• QSFP-DD Advantage – Higher density

BRKOPT-2006 43


100G – 400G Multi-Rate WDM Pluggable Details Trade Distance for Capacity

• Supported Trunk Rate:• 200G, 300G and 400G - OFEC• 100G - OTU4 with Staircase FEC• 400G CFEC

• Channel Spacing:• 75 GHz & 50 GHz

• TX Power Range: • Metro: 0dBM• ZR/ZR+: -10 dBm

• RX Sensitivity:• -19dBm• OTU4: -22dBm

• CD Compensation:• 200G: +/-50,000ps/nm• 300G: +/-30,000ps/nm• 400G: +/-16,000ps/nm

• DGD Compensation:• 200G, 300G, 400G: 60ps• OTU4: 90ps

Brownfield Multi-Haul

CFP-2

Greenfield Multi-Haul

QSFP-56DD

BRKOPT-2006 44


CFP2 DCO Considerations

• Immediately Available @ 100G, 200G

• Brownfield deployment friendly

• Higher output power

• Applications

• Transponder

• Router Platforms e.g. NCS 5500, ASR9K

BRKOPT-2006 45

IP+DWDM IP + DCO

1 x 100G per fiber pair

“Grey”

• Regular Optics• Simpler management

(No Two Networks)

• Distance Limitations (<10Kms)

• No distance limitations (Amplifiers required in case of large distances)

• Two disparate networks: Additional

Capex + Opex• Inter-operability (IP + Optical)• 100-150W Power Dissipation per port

• Multiple Point-to-Point links on DCO Pluggable

Optics to carry IP traffic over DWDM

• 15-21W Power Dissipation per port

• Deployment flexibility (PAY as you Grow (100GE to 200GE transition over time)

• Add services by adding Pluggable Optics – turn

on new services

• Interop with ROADM, maximize use of existing

infrastructures

Architectural and Deployment Considerations


Data Center transition from 100G to 400G

Connectivity Typical Distance 100G Interface 400G Interface

Server to ToR Switch Up to 3 m100G-CR44 x 25G-CR

400G-CR88 x 50G-CR

ToR Switch to Leaf Up to ~20 m100G-AOC(Active Cable)

400G-AOC(Active Cable)

Leaf to Spine Up to 500 m100G-PSM4100G-CWDM4

400G-SR8400G-DR4/FR4

Campus Up to 1 km100G-LR4100G-CWDM4

400G-FR4400G-LR4/LR8

Metro or Long Haul Data Center Interconnect

40 to 120kmMore than 100km

100G-DCO 400G-ZR/ZR+

Reaching 4x capacity increase over the same fiber infrastructure

BRKOPT-2006 47

Note: 400G modules can support 4x 100 GbE, 2x 200 GbE and 1x 400 GbE


DC Topology – Scale

400G-CR88x50G-CR

400G-AOC400G-SR8

400G-SR4.2

400G-FR4400G-DR4

400G-LR4400G-FR4

400G-ZR/ZR+

100G-CR44x25G-CR

100G-AOC100G-BiDi

100G-CWDM4

100G-PSM4

100G-LR4

100G-DCO

CNF

ToR

Servers

Leaf

Spine#1

Data Hall /Building spine

<3m

10’s m

100’s m

< 10km

100’s km

BRKOPT-2006 48


400 GbE to 100 GbE backwards compatibility

QSFP28

QSFP-DD

400 GbE QSFP-DD ports are backwards compatible with 100 GbE QSFP28 modules

• 400 GbE is based on 8x 50Gb/s interfaces• ASICs with 50 Gb/s interfaces can down-rate to 25 Gb/s (or 10 Gb/s)

• Compatible with QSFP28 (4x 25 Gb/s)

BRKOPT-2006 49

• Ease of migration to next higher speed• Connecting to legacy equipment• Mix and match optics for cost reasons


Leaf-Spine Deployment considerationsSome examples

SPINE

LEAF

QS

FP

-D

DQ

SFP

-D

D

QS

FP

-D

D

QS

FP

28

QS

FP

28

400 GbE QSFP-DD ports


100GbE

4x 100GbE

400GbE

QS

FP

-D

D

QS

FP

28

QS

FP

28

QS

FP

28


100 GbE QSFP28 ports

100GbE

4x 100 GbE

QS

FP

28

QS

FP

28

QS

FP

28

QSFP-DD

QSFP28

QSFP28

Any 400 GbE optics

Any 100 GbE optics

QSFP-DD400 GbE breakout capable optics

Advanced 100 GbE

400G-DR4, 400G-SR-BD

100G-DR, 100G-FR, 100G-BiDi

Full port bandwidth

Reduced port bandwidth

QS

FP

-D

D

BRKOPT-2006 50

QS

FP

-D

DQ

SFP

28

100GbE


400 GbE Leaf-Server Deployment considerationsSome examples

LEAF

SERVER

QS

FP

-D

D

SFP

56


NICs supporting 50 Gb/s Serdes

4x

100G-CR2

QSFP-DD

QSFP28

CR Breakout

2x 100GQSFP-DD

SR4, CWDM4

50G-CR

2x 100G-SR4, 2x 100G-CWDM4

SFP

56

8x 50G-CR

QS

FP

-D

D

QS

FP

56

QS

FP

56


NICs supporting 25 Gb/s Serdes

8x25G-CR

QS

FP

-D

D

SFP

28

SFP

28

SFP56

SFP28 25G-CR

Previous slide’s optical options all possible for server IO tooFull switch port bandwidth

Reduced switch port bandwidth

QSFP56

2x 100G(SR4 or

CWDM4)

QS

FP

28

QS

FP

28

QS

FP

-D

D4x 50G-CR, 2x 100G-CR2

BRKOPT-2006 51


Architecture Deployment -

• Utilizing 400G breakout for dense 100 GbE

QS

FP

-D

D



Wider Fabric w/ 100 GbE (Similar

with 2x 200 GbE)

4x

100 GbE

4x 100

GbE400

GbE

400GE-

DR4 4x100GE in 18mm

BRKOPT-2006 52

QS

FP

-D

D

QS

FP

-D

D

QS

FP

-D

D

QS

FP

-D

D

QS

FP

-D

D

QS

FP

-D

D

QS

FP

-D

D

QS

FP

-D

DQ

SFP

28

QS

FP

28

QS

FP

28

QS

FP

28


100 GbE QSFP28 ports

Dense 100G Spine w/ 100

GbE Fabric

Different Data Center Architectures can chose to utilize 400G modules in different ways.


Leaf-Spine Deployment considerationsSome examples

SPINE

LEAF

QS

FP

-D

D

QS

FP

28

QS

FP

28



100GbE

4x 100GbE

QS

FP

-D

DQ

SFP

-D

D

400GbE

Full switch port bandwidth

Reduced switch port bandwidth

QS

FP

-D

D

400 GbE- Better hash efficiency- Less ports manage vs n x 100 GbE links- Improved system performance

BRKOPT-2006 53


400ZR and 400ZR+ Use CasesBuilt in DCO capabilities enables new applications

• Data Center Interconnect

• Amplified or Unamplified

• Metro, Regional and Long Haul

• High capacity aggregation networks

• Wireline (ex. XGS-PON)

• Mobile (ex. 5G)

• Cable (ex. R-PHY)

BRKOPT-2006 54


Coherent deployment considerationsReaches and form factors

55BRKOPT-2006

Short distance~80-120 km

Metro~1000 km

Long Haul3000+ km

Sub-sea >8000 km

100G200G400ZR

DCO PluggableQSFP-DD

200G300G400ZR+

Transponder orDCO Pluggable (QSFP-DD, CFP2)

200G300G400ZR+600G

Transponder orDCO Pluggable (QSFP-DD, CFP2)

600GTransponder

Despite simplicity of pluggable modules, network planning considerations still required


400ZR/ZR+ Enabling architectural shift

Router + TXPs + ROADM

SubWavelength

Services

OTN ‘Wave’

Services

Packet Services

Router w/ 400ZR

TDM Services

OTN ‘Wave’

Services

Packet Services

Advantages:• Simplified Network Lifecycle• Lower TCO with highest fiber utilization • Reduced number of network devices - more

secure, less power and smaller footprint• Multi or single layer visibility

Simplification of Network:• Removal of optical complexity while maintaining

optical advantages• Manage traffic at a single layer, including

protection• Maximize Network utilization• Optical network life-cycle simplification• Manage optical network one span at a time

Architectural shift Hop to Hop Benefits

BRKOPT-2006 56


Present Mode Operation – ROADM - 100G Links

Example• Full mesh router topology• 6 Lambdas Used• 100G/l

• 24 Interfaces• 200G Max traffic between

any 2 routers (without TE)

ROADMNetwork

PacketNetwork

BRKOPT-2006 57


Simplified network architecture - 400G Links

P2PDWDMNetwork

PacketNetwork

Example• 2 wavelengths used per

direction• 400G/l• 16 Interfaces• 1.6T Max traffic between any 2

routers• No complex ROADMs required

BRKOPT-2006 58


400ZR/ZR+ Application Benefits

• Standardized modes which will be interoperable and multivendor

• Revolutionary price-performance

• Low power – 400G with 1000 km reach - 20W

• No trade-offs in port density for DCO optics (QSFP-DD)

• Eliminates special system designs

• Enables simpler, more cost effective networks

• Eliminates transponder hardware with DCO directly in routers/switches

59BRKOPT-2006

Summary


Summary

Demand for 400 GbE is here

Uniquely for 400 GbE, multiple solutions will

exist in a common QSFP-DD pluggable

form factor

Industry is broadly engaged to deliver 400

GbE now

Cisco is a leader in all aspects

BRKOPT-2006 61

Questions? Use Cisco Webex Teams to chat with the speaker after the session

Find this session in the Cisco Live Mobile App

Click “Join the Discussion”

Install Webex Teams or go directly to the team space

Enter messages/questions in the team space

How

Webex Teams will be moderated by the speaker until June 16, 2019.

1

2

3

4


Cisco Webex Teams

cs.co/ciscolivebot#

62

BRKOPT-2006

Complete your online session evaluation

• Please complete your session survey after each session. Your feedback is very important.

• Complete a minimum of 4 session surveys and the Overall Conference survey (starting on Thursday) to receive your Cisco Live water bottle.

• All surveys can be taken in the Cisco Live Mobile App or by logging in to the Session Catalog on ciscolive.cisco.com/us.

Cisco Live sessions will be available for viewing on demand after the event at ciscolive.cisco.com.

© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS BRKOPT-2006 63

http://ciscolive.cisco.com/us

https://ciscolive.cisco.com/


Continue your education

64BRKOPT-2006

Related sessions

Walk-in labsDemos in the Cisco campus

Meet the engineer 1:1 meetings

Thank you

#CLUS

BRKOPT-2006.pdf - Cisco Live

Documents

Transcript of BRKOPT-2006.pdf - Cisco Live