Please read - Cisco Live

© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CiscoLiveLA

This presentation template uses the CiscoSans TT Light font. If the text in these two columns does not match, please take a moment to install the font. Otherwise, your presentation will not display correctly.

Please download the fonts from Brand Exchange here. The font can also be found in the zipped folder. Double-click the font file and click “Install” in the window that appears.

Please read

1BRKACI-2117

#CiscoLiveLA

#CiscoLiveLA

Lionel Hercot, Technical Marketing Engineer, DCN@LHercotBRKACI-2117

Journey from single DC to Multi-CloudCisco ACI Anywhere


Legal DISCLAIMER

4BRKACI-2117

Any information provided in this document regarding future functionalities is for informational purposes only and is subject to change including ceasing any further development of such functionality. Many of these future functionalities remain in varying stages of development and will be offered on a when-and-if available basis, and Cisco makes no commitment as to the final delivery of any of such future functionalities. Cisco will have no liability for Cisco’s failure to deliver any or all future functionalities and any such failure would not in any way imply the right to return any previously purchased Cisco products.


Virtual ACI ACI Cloud ACIIP WAN IP WAN

Edge / Remote MulticloudCore Data Centers

ACI Anywhere

ACI 2.0 ACI 3.0 ACI 3.1 ACI 4.0 ACI 4.1

ACIMulti-POD

ACIMultisite

ACIRemote Leaf

VirtualACI

CloudACI

5BRKACI-2117


ACI: Turnkey integrated solution

• Zero-touch provisioning

• Auto deployment of the Underlay and the Overlay

• Managed like a single large switch

• Single management point• Underlay and Overlay• Monitoring• Troubleshooting


Network Admin

Application Admin

PHYSICALSERVER

VLANVXLAN

VLANNVGRE

VLANVXLAN

VLAN

ApplicationManagement

Penalty Free OverlayAPIC

VMwareMicrosoft

Red Hat Docker

ESXVMware

Hyper-VMicrosoft

KVMRed Hat

ContainerDocker

VLANVXLAN

Any workloadVirtual / Bare Metal / Container APIC

ACI : Any Type of Workload – Anywhere

• Integrated gateway for VLAN and VXLAN networks from virtual to physical to container

• Normalization for VXLAN, and VLAN networks

• Customer not restricted by a choice of hypervisor

• Fabric is ready for ANY workload


Any VLAN anywhere

Outside

Outside_VLANs(Pools)

Outside_Fabric

UCS_VLANs (Static Pool)

Bare Metal Servers

vDS-01(Dynamic Pool)Virtual Machines

Linux_VLANs(Static Pool)

Physical Servers

Windows_VLANs(Static Pool)

Physical Servers

with Integrated DCI solution (Multi-Pod)


Cisco ACI : Secure Multi-Tenant FabricAuthentication, Authorization, and RBAC

• Multi-Tenancy

• Any type of workload anywhere

• “Availability" zones structured with loose coupling


ACI Components

Spines ControllersLeafs

APIC-CLUSTER-M3APIC-CLUSTER-L3

(> 1250 Edge Ports)

Nexus 9300 Nexus 9300 Nexus 9500


ACI Architecture

Leafs

Spines

Controllers


ACI Architecture

Modular Switch

Dev

VRF-1

ACI Object ModelTenant (ex: Dev, Prod, …)

VRF (L3)BD (L2) BD (L2)

Subnet Subnet

BD

10.10.10.254

BD

10.10.20.254 10.10.21.254

VRF-2ANP

EPG EPG EPG

C CANP

Web App DBC C


Service Insertion in ACI

Users EPG Web EPG

• Managed or unmanaged• Can copy or redirect traffic• L2 or L3

Service Graph can be:

How to extend ACI outside one DC?


Data Center Interconnect SolutionsACI Simplifies the Deployment of DCI

Pod ‘A’

MP-BGP - EVPN

…

IPNPod ‘n’

ACI Multi-Pod Fabric

APIC Cluster

Fabric ‘A’

MP-BGP - EVPN

…

IPFabric ‘n’

ACI Multi-Site

ACI Physical Remote Leaf ACI Virtual Remote Leaf (vPod)

• Common Control/Data Plane options used across different architectures• Consistent security policies end-to-end

TECACI-2009 16

Multi-Pod

© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CiscoLiveLA 18

Pod ‘A’

MP-BGP - EVPN

Pod ‘n’

…

IS-IS, COOP, MP-BGP IS-IS, COOP, MP-BGP

Inter-Pod Network

APIC Cluster

Availability Zone

For More Information on ACI Multi-Pod:BRKACI-2003

VXLAN

ACI Multi-PodThe Ideal Architecture for Active/Active DC Deployments

§ Forwarding control plane (IS-IS, COOP) fault isolation

§ Data Plane VXLAN encapsulation between Pods

§ End-to-end policy enforcement

§ Multiple ACI Pods connected by an IP Inter-Pod L3 network, each Pod consists of leaf and spine nodes

§ Managed by a single APIC Cluster

§ Single Management and Policy Domain

50 msec RTT

TECACI-2009 18


POD 1 POD n

Web/AppDB Web/App

…

Intra-DC Two DC sites directly connected

POD 1 POD 2

Web/AppDB Web/App

Dark fiber/DWDM (up to 50 msec RTT)

Multiple sites interconnected by a generic L3 network

POD 1 POD 2

POD 3

3 DC Sites directly connected

Dark fiber/DWDM (up to 50 msec RTT)

L3(up to 50msec RTT)

40G/100G 40G/100G

10G/40G/100G40G/100G 40G/100G

40G/100G 40G/100G

40G/100G

10G/40G/100G

40G/100G

40G/100G

40G/100G

40G/100G

APIC Cluster APIC Cluster

ACI Multi-PodSupported Topologies

BRKACI-2117 19

Multi-Site Orchestrator


Scale-Up Model to Build a Large Intra-DC Network

Data Center Interconnect (DCI)

ACI Multi-SiteUse Cases

21BRKACI-2117


ACI MultisiteShipping

VMVMVM

Site A

Site B

Site C

Site D

VMVMVM

Multisite Orchestrator

VMVMVM

VMVMVM

Policy Consistency

Single Point Of Orchestration

Availability Fault Isolation

Scale

Consistent Policy across sites

Single Point of Orchestration

Fault Isolation

Scale

BRKACI-2117 22


ACI Multi-SiteOverview

§ Separate ACI Fabrics with independent APIC clusters§ ACI Multi-Site Orchestrator pushes cross-fabric

configuration to multiple APIC clusters providing scoping of all configuration changes

§ MP-BGP EVPN control plane between sites§ Data Plane VXLAN encapsulation across sites§ End-to-end policy definition and enforcement

MP-BGP - EVPN

Availability Zone ‘A’ Availability Zone ‘B’

IP Network

ACI 3.0 Release

VXLAN

Site 1 Site 2RESTAPI GUI


BRKACI-2117 23

For More Information on ACI Multi-Site:BRKACI-2125LABACI-2000


ACI Multi-SiteSoftware and Hardware Requirements

• Support all ACI leaf switches (1st Gen, EX, FX, FX2)

• Modular Spine with EX/FX line card to connect to the inter-site network

• 9364c or 9332c fixed spine supported for Multi-Site from ACI 3.1 release (shipping)

• 1st generation spines (including 9336PQ) not supported

• Can still leverage those for intra-site leaf to leaf communication

1st Gen

Inter-Site Network

-EX -EX

Can have only a subset of spines connecting to

the IP network

1st Gen

BRKACI-2117 24


RESTAPI GUI

ACI Multi-Site Orchestrator

…..Site 1 Site 2 Site n

• Three MSO nodes are clustered and run concurrently (active/active)

§ Typical database redundancy considerations (minority/majority rules)

§ Up to 150 msec RTT latency supported between MSO nodes

§ vSphere VM only form factor initially, physical appliance planned for a future ACI release

• OOB Mgmt connectivity to the APIC clusters deployed in separate sites

§ Up to 1 sec RTT latency between MSO and APIC nodes

• Main functions offered by MSO:

§ Monitoring the health-state of the different ACI Sites

§ Provisioning of day-0 infrastructure configuration to establish inter-site EVPN control plane and VXLAN data plane

§ Defining and provisioning tenant policies across sites

§ Day-2 operation functionalities

VM

ACI Multi-SiteMulti-Site Orchestrator (MSO)

VM VM

150 msec RTT (max)

Hypervisor

1 sec RTT (max)

TECACI-2009 25


Layer 3 only across sites

§ Bridge Domains and subnets not extended across Sites

§ Layer 3 Intra-VRF or Inter-VRF communication (shared services across VRFs/Tenants)

ISN

Site 1

Site 2

AWS

1IP Mobility without BUM flooding

§ Same IP subnet defined in separate Sites

§ Support for IP Mobility (‘cold’ and ‘live’* VM migration) and intra-subnet communication across sites

§ No Layer 2 BUM flooding across sites

Site 2

ISN

Site 1

Site 2

AWS

2Layer 2 adjacency across Sites

§ Interconnecting separate sites for fault containment and scalability reasons

§ Layer 2 domains stretched across Sites, support for ‘live’* VM migration and application clustering

§ Layer 2 BUM flooding across sites

ISN

Site 1

Site 2

AWS

3

ACI Multi-Site Networking OptionsPer Bridge Domain Behavior

BRKACI-2117 26


ACI Multi-Pod and Multi-SiteConnectivity between Pods and Sites

Pod ‘A’ Pod ‘B’

IPN

IP WAN

Site 2

Site 1 Site 2

1st Gen 1st Gen

APIC Cluster

§ Only 2nd generation spines must be connected to the external network• Need to add 2nd gen spines in each Pod (at least two per Pod) and migrate connections to the IPN from 1st gen

spines to 2nd gen spines

§ Single ‘infra’ L3Out and set of uplinks to carry both Multi-Pod and Multi-Site East-West traffic

BRKACI-2117 27


§ Back-2-back connections are ONLY supported for 2 sites § Multi-Site + Multi-Pod not supported

APIC Cluster APIC Cluster

Intersite E-W (Direct Cable or Dark Fiber)

Multi-Site Back-2-Back Spine

BRKACI-2117 28


Multi-Site and External Layer 3 Connectivity

29BRKACI-2117

Monitoring & Troubleshooting

Common Governance

Operational Consistency

Single Point Of Orchestration

Discovery & Visibility

Policy Translation

IP Network

VMVMVM

Multi-Site

ACI 4.2

29Presentation ID

Endpoint in Site-A Using L3Out in Site-B Endpoint Behind L3Out in Site-A Using Site-B L3out

VMVMVM

Site A Site B

L3 Peering L3 OUT

(Mainframe)

L3 OUT


ACI Multi-Site L4-L7 Services Support

• ACI Multisite + L3 PBR + L4-L7 Services• 1 node (firewall) service graph shipping in ACI 3.2• 2 node (firewall and load-balancer) service graphs supported in ACI 4.0

• N-S and E-W service graphs support

• ACI Multisite + L1/L2 PBR + L4-L7 Services • 1 node (IPS) service-graph supported in ACI 4.1

• N-S and E-W service-graphs supported

BRKACI-2117 30


MP-BGP EVPN

VXLAN

• Multi-Site Infra: Unicast, Multicast, BGP TEPs and Tunnel state

• Multi-Site Tenant and EPG granularity: § Inspect and validate full-stack programming:

MSC, APICs and Spine translations§ Validate the consistency of local and remote

inter-site EPGs, BD, VRF, External EPG, policies, etc.

§ Root cause configuration programming issues without calling TAC

• GUI and APIs supported Spines Spines

ACI 3.2 Release ACI Multi-Site

Day-2 Operations: Full-Stack Consistency Checker

BRKACI-2117 31


ACI Multi-Site API(Swagger)

• Swagger benefits• Allow end developers to effortlessly interact and try out every single operation your API exposes

for easy consumption.• Swagger UI can auto import the Authorization token from MSC UI giving seamless access to the

APIs.

• Types of endpoints: API GET, POST, PUT, PATCH, DELETE

BRKACI-2117 32

How to use them?


Fabric ‘A’ (AZ 1)

Fabric ‘B’ (AZ 2)

Application workloads deployed

across availability zones

Typical RequirementCreation of Two Independent Fabrics/AZs

TECACI-2009 34


Pod ‘1.A’ Pod ‘2.A’

Pod ‘1.B’ Pod ‘2.B’

‘Classic’ Active/Active

Multi-Pod Fabric ‘A’ (AZ 1)

Multi-Pod Fabric ‘B’ (AZ 2)

‘Classic’ Active/Active

ACI Multi-Site

Typical RequirementCreation of Two Independent Fabrics/AZs

Application workloads deployed

across availability zones

TECACI-2009 35

Remote Leaf

© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public#CiscoLiveLA 37BRKACI-2117

IP Network

(WAN Core – IPv4, MPLS, SR, etc …)

ACI: Physical Remote Leaf Extend ACI to Satellite Data Centers

On-Prem DC

Remote Locations

Zero Touch Auto Discovery of Remote Leaf

Two Remote Leaf vPC Pair Up To 32 Remote Locations

Multi-site SupportStretch Tenant, EPG, etc

All benefits of ACI visibility Health Scores, Stats

VMVMVM VMVMVMVM VMVMVM VMVMVMVM

Shipping


ACI Remote LeafUse Cases

Satellite DC

Brownfield

Co-location

Remote Location A

VM

ACI Main Data Center

VMVMVM VMVMVMVM

IP Network

Telco 5G

VMVMVM VMVMVMVM

Remote Location B

VMVMVM VMVMVMVM

Remote Location C

VMVMVM VMVMVMVM

Remote Location D

VMVMVM VMVMVMVM

BRKACI-2117 38


Supported SpinesFixed• N9K-C9364C• N9K-C9332C• N9K-C9316D-GXModular• N9K-X9732C-EX• N9K-X9736C-FX

Supported Leaf• N9K-C93180YC-EX• N9K-C93108TC-EX• N9K-C93180LC-EX• N9K-C93180YC-FX• N9K-C93108TC-FX• N9K-C9348GC-FXP• N9K-C9336C-FX2• N9K-C93240YC-FX2• N9K-C93600CD-GX

On-Premise Data Center Remote Site

ACI: Physical Remote Leaf Hardware Support


ACI Remote LeafLocal Traffic Forwarding for vPC Endpoints

IP Network (WAN Core – IPv4, MPLS, SR, etc …)

Main DCRemote Location

Switches are in vPCdomain EP info synch over vPC control plane

Po1 Po2

• “Greedy Forwarding” vPCPo1 to vPC Po2 on RL

EP3 EP1 EP2

ACI 3.1

BRKACI-2117 40


ACI Remote LeafLocal Traffic Forwarding for Orphan Endpoints


Main DC Remote Location

Switches are in vPCdomain EP info synch over vPC control plane

EP3 EP1 EP2

ACI 3.2

BRKACI-2117 41


ACI Remote LeafPBR



EP1 EP2

EP1EPG1

EP2EPG2

ContractPBR to Service

Node at RL

L4-L7Service Node

BRKACI-2117 42


ACI Remote LeafPBR



EP3EP1 EP2

EP1EPG1

EP2EPG2

ContractPBR to Service

Node at RL

L4-L7Service Node

ACI 4.0

BRKACI-2117 43


ACI Remote LeafInter-VRF Traffic



EP3EP1

VRF1EP2

VRF2

BRKACI-2117 44


ACI Remote LeafInter-VRF Traffic



EP3EP1

VRF1EP2

VRF2

ACI 4.0

BRKACI-2117 45


Remote Leaf : Direct Switching over IPN

Pod 1 Pod 2

ACI 4.1.2

Inter-Pod IP Network

Remote Leaf (Location A, Pod 1)

Remote Leaf (Location B, Pod 1)

Remote Leaf (Location X, Pod 2 )

Remote Leaf (Location Y, Pod 2)

RL to RL Forwarding Within Pod RL to RL Forwarding Across Pod


Remote Leaf Multisite Support

Site 1 Site 2

ACI 4.1.2

Inter-site IP Network

Remote Leaf (Location A, Site 1)

Remote Leaf (Location B, Site 1)

Remote Leaf (Location X, Site 2 )

Remote Leaf (Location Y, Site 2)

Consistency Policy Stretched between On-Prem and Remote Locations


Cisco ACI Virtual Edge


ACI Virtual Edge

Maintain Existing Operational Models

Policy Consistency Across Multiple Hypervisors

VMVMVM VMVMVMVM

ACI Virtual Edge (AVE)

Cisco ACI Virtual Edge

Hypervisor Agnostic

ACI Virtual EdgeVM VM VM

Hypervisor

Bare Metal Server

Native Switch

Shipping

BRKACI-2117 49

Virtual Pod (vPod)


Virtual ACI: Virtual PodExtend ACI to Bare Metal Clouds and Remote Data Centers

51BRKACI-2117

Shipping

IP Network

Bare Metal Clouds (IBM, OVH, etc.)

Remote Data Centers

Co-location Facilities

(Equinix, CoreSite etc.)

Brownfield Deployments

Remote location On-premises ACI Data Center

VMVMVM VMVMVMVM

VMVMVM VMVMVMVM

Hypervisor

Policy extension from On-premise DC


ACI Virtual Pod (vPod)

Management Cluster (vSpine + vLeaf)• vSpine and vLeaf: Run ACI control plane function

• vLeaf: Distribute APIC policies to ACI Virtual Edge

ACI Virtual Edge (vPod Mode)• Implements ACI data plane function and policy

enforcement data plane

• iVXLAN for communication within vPod and across Pods

vSpine

vLeafvLeaf

ACI Virtual Edge

Virtual Pod

vSpine

BRKACI-2117 52


ACI vPod Use Cases

Bare Metal Cloud

Brownfield

Co-location/Remote DC

Data Center A

Data Center B

Data Center C

VM VM VM VM

ACI Main Data Center

VMVMVM VMVMVMVM

IP Network

BRKACI-2117 53


ACI vPod RequirementsHardware & Software Components

54BRKACI-2117

Supported SpinesFixed Spine• N9364C• N9332C

Modular Spine (C9504/C9508/C9516)• N9732C-EX with N9K-C950x-FM-E(2)• N9736C-FX with N9K-C950x-FM-E(2)

APIC Controller Software• ACI 4.0+ onward release

ü VMware vCenter running 6.0 or laterü 2 hosts for Management cluster

recommended• Management & Payload Can Co-exist

ü ESXi 6.0 or 6.5

• Each vSpine (x2) & vLeaf(x2) VM consumes 4vCPU, 16 GB RAM and 80 GB storage

• Each AVE (one per ESXi host) VM consumes 2vCPU, 8 GB RAM and 8 GB storage

vPod Data CenterOn-Premises Data Center


Virtual Pod Scaling

55BRKACI-2117

Cisco ACI Virtual Edge (vPod Mode - per Workload Server)

ACI Virtual Edge

Management Cluster – per vPod

AVE (vPod Mode) – per Server


32 Hosts

Up To 6 vPods Up to 32 AVE per vPod



ACI Infrastructure EnhancementsACI 4.0

Deployment

Networking

Operations QOS Enhancements

FC NPV Inter-VRF Multicast

Host Route On Border LeafRoCE v2

Mini ACI


ACI: Mini ACI Fabric

58BRKACI-2117

VMLeaf 1 – 48 ports

Leaf 2 – 48 ports

Spine 1

Spine 2

APIC

No. of EPGs

No. of Tenants

No. of Spines

No. of Leafs

Cloud

Co-Location DC | SMB DC | SP Micro-DC

ACI Fabric For Small Scale Deployments – 5RU System

VM

No. of BDs

No. of EPs

No. of VRFs

1000

25

2

2-4

1000

20,000

25

Virtual APIC

Physical APIC

2

1

Shipping Since ACI 4.0

Multi-Tier


ACI: Multi-Tier Architecture

60BRKACI-2117

Seamless Migration From Legacy 3-Tier Architectures

Three Tier ACI Fabric

Vertical Expansion Of ACI Pol icy Domain

Investment Protection: Reuse Existing Cable Plan

Replace FEX Architecture With 2nd Tier Leaf: Better Visibi l i ty & Pol icy Enforcement

1

2

3VMVMVM VMVMVMVM

2nd Tier Leaf

1st Tier Leaf

Spine

Simplify N2/N5/N7k Migration to ACI4

ACI 4.1


ACI: Multi-Tier Architecture

61BRKACI-2117

Seamless Migration From Legacy 3-Tier Architectures

Three Tier ACI Fabric

Tier-2 Leaf can connect to mult iple Tier-1 Leafs (advantage over tradit ional VPC)

APIC control ler can be connected to Tier-2 Leaf or to Tier-1 leaf

L3out can be connected to Tier-2 Leaf or to Tier-1 leaf

VMVMVM VMVMVMVM

2nd Tier Leaf

1st Tier Leaf

Spine

ACI 4.1


Supported Platforms in ACI 4.1

• Spine: Any EX/FX/C spines (9332C, 9364C)

• Tier-1 Leaf: Any EX/FX/FX2 except N9K-C93180LC-EX

• Tier-2 Leaf: Any EX/FX/FX2

• 1st gen is not supported

• Max number of Tier-1-leaf + Tier-2-leaf is equal to the max number of Leaf in the fabric (200 per pod. 400 per Multi-Pod)

• Max number of Tier-2-leaf per Leaf is 48.

VMVMVM VMVMVMVM

VMVM

Spine

Tier-1 Leaf

Tier-2 Leaf

BRKACI-2117 62


Connectivity requirement to 2nd Tier Leaf

• 2nd Tier Leaf fabric port connects to 1st

Tier Leaf’s fabric port.

• All ports of 1st Tier Leaf can be converted to fabric port using port profile feature

• 2nd Tier Leaf can connect to multiple 1st

Tier Leaf. It could be an advantage for ACI design where customer can connect to more than 2 upstream switches in comparison to traditional double sided vPC design with only 2 upstream switches.

VMVMVM VMVMVMVM

VMVM

Spine

Tier-1 Leaf

Tier-2 Leaf

BRKACI-2117 63


1G support on leaf downlink to Tier-2-leaf uplink

• Use case: Long OM2 fibers from 93180YC Leaf to 9348 Tier-2-leaf.

• 10G range is shorter on OM2 than 1G• 10G OM2 (10GBASE-SR. 82 m)• 1G OM2 (1000BASE-SX. 550m)

Spine

Leaf (93180)

Tier-2-leaf (9348)

QSA on 9348 40/100G uplink port and use it as 1G

1G downlink from leaf to Tier-2-leaf

BRKACI-2117 64

Cloud ACI


Challenges in building a Multi Cloud environment

66

• Maintain consistent policy, security and analytics for workloads deployed across on-premises and cloud locations

• Building an automated and secure interconnect between On Premises and Cloud datacenters with ease of provisioning and monitoring at scale

• Requires a single pane of glass to manage policies across on-premise and cloud locations


Cloud ACI

67

On-Premises


VMVMVM

Cloud Region(s)

VMVMVM

Cloud Region(s)

VMVMVM


EPG Web

EPG APPContract Contract EPG

DB

SG Web

SG APPSG Rule SG Rule SG

DB

ACI Extensions to Cloud

68

IP Network

AWS Region

On-Premises DC

VMVMVM

Public Cloud

Multi-Site

Automated Inter-connect provisioning

Simplified Operations with end-to-end visibility

Consistent Policy Enforcement on-Premises & Public Cloud

ASG Web

ASG APPNSG NSG ASG

DB

Azure Region

IP Network


Why does this matter?

Use Cases

© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Public

APIC Cloud APIC

Tenant

VRFBD1/Subnet1Web-EPG1

BD3/Subnet3

App-EPG1

CIDR 2Web-EPG2

CIDR 4App-EPG2

Application Stretch

On-Premises Public Cloud


• Stretch tenant/VRF across on-premises and cloud sites

• During peak times easily deploy application tiers and resources in the cloud site

• Consistent segmentation policy and enforcement within and across on-premises and cloud sites

• Application stack failover between sites (active/disaster recovery)

Supported ACI 4.1

HTTPs HTTPs

72


Stretched EPG with Consistent Segmentation

• Web Tier and App Tier are stretched and securely segmented across on-premise and public cloud sites

• Consistent segmentation policy and enforcement for endpoints of Web/App Tier are independent of location

APIC Cloud APIC

Tenant

VRFBD/Subnet1

BD3/Subnet3

CIDR 2

CIDR 4



EPG - Web

EPG - App

HTTPs, redis

Supported ACI 4.1

73


Shared Services for Hybrid-Cloud

• Provides a capability to deploy shared service across hybrid cloud

• Shared Service deployed in 1 Site can be consumed by endpoints across other sites

• Contract will leak subnet between VRFs for reachability

APIC Cloud APIC

Tenant 1

VRF1

BD/Subnet1

DNS-EPG



CIDR 3App-EPG

CIDR 2Web-EPG

HTTPs

Tenant 2

VRF2

DNS

Route Leaking

CIDR 5App-EPG

CIDR 4Web-EPG

Tenant 3

VRF3

HTTPs, redis

74

Supported ACI 4.1


Cloud and On-Prem L3outs

On-PremiseMulti-Site Orchestrator (MSO)

Public CloudSite B

Infra VPC

AZ-1 AZ-2

Region 1

CSR CSR

Site A

User VPC -2

VGW

User VPC - 1

VGWIPSec Tunnel IPSec Tunnel

EPG-1 EPG-3EPG-2EPG-1

SG-1 SG-1 SG-3SG-2

Instance 01 Instance 02 Instance 03 Instance 04

IGWL3outL3out

L3out

• Cloud local L3out via IGW

• On-Prem local L3out

• On-Prem site endpoints cannot use Cloud L3out

• Shared On-Prem L3out for Cloud VPCs *

Supported ACI 4.1

* Depends on QA Validation Completion by FCS

75

IGW


Cloud First

76

• Cloud APIC only without on-premises ACI

• Optional MSO

• Abstract AWS networking constructs from user that is familiar with ACI, delivering ACI-consistent policy and operational model

• Deploy EPG and contracts on top of AWS public cloud

Supported ACI 4.2

MSO

Architecture


Cloud APIC Architecture

78

API (AWS, Azure...) NetConf (CSR1000v)

Cloud Policy Element

Policy Manager (PM)

Policy Distributor (PD)

Web Server (NGINX)

Connector

….

• Virtual Form Factor of APIC

• Automates / Manages Cloud Routers

• Translates ACI Policy to cloud native constructs

• Deploys cloud resources and infrastructure components

• Intuitive GUI and Similar ACI UI look and feel

• REST API North Bound Interface

• cAPIC manages 1 or more regions

Cloud Policy Element

Connector


Topology Health

79

• Network connectivity and Health


Endpoints in an EPGs

80


Policy Mapping - AWS

81

For your info &

reference

Security Group

Virtual Private Cloud

Security Group Rule

Outbound rule

Inbound rule

User Account

Source/Destination: Subnet or IP or Any or ‘Internet’ProtocolPort

Network Adapter

Tenant

VRF

BD Subnet

EP to EPG Mapping

Contracts, Filters

Consumed contracts

Provided contracts

EC2 Instance

VPC subnet

EPG

Tag / Label

End Point (fvCEp)

Network Access List Taboo


Application Security Group (ASG)

Virtual Network

Subnet

Network Security Group (NSG)

Outbound rule

Inbound rule

Resource Group

Source/Destination: ASG or Subnet or IP or Any or ‘Internet’ProtocolPort

Network Adapter

Tenant

VRFBD Subnet

EPG

Filters

Consumed contracts

Provided contractsVirtual Machine

Policy Mapping - AzureFor your info &

reference


Cloud Infra – AWS

83


Region - 1

ACI DC

VMVMVM


Infra VPC

User VPC 2User VPC 1

IPSec Tunnel

VGW VGW

CSR1kv CSR1kv


Cloud Infra – Azure


Region - 1

ACI DC

VMVMVM


Infra VNET

User VNET 2User VNET 1

IPSec Tunnel

VNG VNG

CSR1kv CSR1kv


Cloud EPGMapping Endpoints by Tags / Region / AZ / IP

Site B

US-East-1 US-West-1

Subnet-S1 – 10.1.1.0/24

Subnet-S2 – 10.1.2.0/24

Subnet-S3 – 10.1.3.0/24

Subnet-S4 – 10.1.4.0/24

WEB EPG DB EPG

Deploying Cloud APIC


Cloud APIC in AWS Marketplace http://cs.co/capic-aws


Cloud APIC in Azure Marketplace http://cs.co/capic-azure

Virtual Network Integration


ACI Virtual Networking Integrations

90BRKACI-2117

CCP


Container Platforms with ACI-CNI integration

Baremetal ESXi KVM/OpenStack

Open Source Kubernetes 1.6-1.13 Future

Cisco Container Platform Future

Docker EE 2.1 (Kubernetes) Future

OpenShift 3.6, 3.9, 3.11

ACI Hardware


Nexus 9000 & APIC Hardware

93BRKACI-2117

Nexus Foundation: CloudScale Platforms

* No Support for copper NICs

Nexus Foundation: CloudScale Platforms

Nexus 9300

Nexus 9500

Nexus 9316D-GX Fixed Spine 16p 400G QSFP-DD

ACI4.0

APIC-CLUSTER-M3*(< 1200 Leaf Ports)

ACI4.0Nexus 9332C – Fixed Spine

32p 40/100G QSFP28, 2p 10G

APIC-CLUSTER-L3*(>= 1200 Leaf Ports)

ACI4.0

Nexus 9716D-GXModular Spine Future

Q2CY19

Nexus C93360YC-FX296p 25G SFP2812p 100G QSFP28

ACI4.1(2)

Nexus C93216TC-FX2 96p 10GT12p 100G QSFP28

ACI4.1(2)

Nexus 93600CD-GX 28p 100G QSFP288p 400G QSFP-DD

ACI4.2(2)

Nexus 9336C-FX236p 40/100G

ACI3.1(2)

ACI4.2(2)


ACI software simulator as a VM

• Experience ACI without hardware• Full-featured APIC controller with a

simulated fabric• Native APIC, uses the same APIs that are

published for third parties• Use cases – Training, Lab, Test, etc., • Control plane only, no data plane • Support offered through Cisco

Communities, no TAC support

| x86 hardware | 24GB RAM | 100GB hard drive |

Available starting 4.2 on CCO as a software download

Leaf 2

Leaf 1

1 x Spine

1 x APIC

Offe

red

as a

sin

gle

VM

New in 4.2

Automation


Automation Tools

96TECACI-2009


ACI Policy Driven vRealize Automation Blueprints to Accelerate Application Deployment

Cloud Automation with vRealize

vRealize Automation vRealize Orchestrator

üFabric Bring-up ü Infrastructure provisioning üSecurity Domains

üShared Services Plans üVirtual Private Cloud üNetworks, Subnets, SecurityTenant 1 App WebDB

ESX Hypervisor

Day Zero Operations

Day 1/ Day 2 Operations

Deploy Tenant

Deploy Load

Balancer

Deploy App

Deploy Firewall

Day 2 Operations


How Cisco Network Assurance Engine

• How it Works

Capture DC Wide Intent, Policy, Control/State across

Forwarding & Security

Precise Mathematical Models that codify Cisco’s 30+ Years of Networking and Cross Customer Domain Knowledge

Data Collection Formal Modeling of Network Continuous Analysis

Models verify that Network operates per Intent and accurately tell what is

wrong, where, why, impact and how to fix

Reasoning you do after the fact, the Engine does before the fact, continuously, network wide


Smart Events & Compliance Score for Compliance

100BRKACI-2117

COMPLIANCE VIOLATED SMART EVENT

• Identify compliant policy

• Identify requirements

satisfied

• Identify compliant EPGs

• Identify non compliant policy

• Identify requirements violated

• Identify non-compliant EPGs

COMPLIANCE SATISFIED SMART EVENT

COMPLIANCE SCORE


Epoch Delta AnalysisCorrelated Ad hoc Analysis Workflow

101BRKACI-2117

4 Qs, correlated answers…• What changed?• Who was impacted? • Was it due to config changes? • What happened as a result?

Use Cases• Change Management• Root-cause analysis• Migration• Maintenance Upgrades• Capacity Management

Before / Baseline

After / Current


Health Delta - SummaryChange in the health of the Fabric

102BRKACI-2117


Network Insight Telemetry Applications On APICProviding Network Health Visibility & Enabling Proactive Insights

103BRKACI-2117

Network Availability Network Health

New Apps

Network Insights Advisor

Proactive Software Recommendations/Notifications Issue Vulnerability Detection & Remediation

Network Insights Resources

Enhance Availability, Uptime & Network Wide Visibility

Physical/Logical Network Capacity & Utilization Data & Control Plane & Environmental Health

NIA NIR


ACI: Network Insights-ResourcesUnderstand What’s Running In Your Network

104BRKACI-2117

ResourceAnalytics

Data Collection

Anomaly Detection

Remediation

Event Analytics Dashboard Displays Faults, Events, And Audit Logs In A Time Series Fashion.

Event Analytics Dashboard


Resource Analysis – Flow AnalyticsProactive Anomaly Detection for ACI Deployments

Targeted Flow Monitoring Use Cases –• Application Performance Issues:

• Forwarding/policy Drops indicating congestion• High end to end application latency

• Application Downtime Event –• Policy misconfiguration due to ACL’s

TECACI-2009 106


Network Insights-Advisor

107BRKACI-2117

NetworkInsightsAdvisor

Software/Hardware RecommendationsWorkarounds

Avoid multiple TAC calls

Significant CAPEXAnd OPEX Savings

Remove ComplexityAvoid OutagesFaster Deployment times

Anomalies

Forwarding State CheckLoops DetectionCable Checkers

Keep Network up to dateAdhere to Cisco policies Recommendations

Prevent traffic black holingAvoid downtimes

Known Issues/PSIRTsUnknown runtimeConfig anomalies

EOL/EOSField NoticesSMUs

Version Scale Limits/Hardening Check Configuration


Network Insights Advisor Targeted Use CasesProactive supportability insights

Fabric wide analysis

AdvisoriesProvides advisories based on anomalies, bugs, PSIRTs and field notices. Measure upgrade impact

Dashboard ”Give me a summary of issues”

Anomalies hardening checks, scale checks

Bugs and PSIRTsKnown bugs and vulnerabilities in the system

TECACI-2009 108

ACI Service Engine


3rd Party AppsNetworkInsights

Network Assurance Engine

ACI: Services Engine

110BRKACI-2117

Dual Boot Option | Cluster For Redundancy | APIC-L3

New Application Hosting Platform

ACI 4.2

ACI Services Engine

2.1 GHz 8 core CPU x 2

192 GB memory

2.4 TB x 2 HDD

16 GB USB Flash drive

Network Security


ACI Security Certifications

112BRKACI-2117

PCI

Certified

DoD

Certified

FIPS

Certified

Common Criteria

Certified

Vulnerability Scanners

Passed: Nessus, Fuzzing,

Port Scan

Shipping

Every Major and Minor Release We Run Our Hardening Suite


ACI 2-Factor Authentication Options

113BRKACI-2117

VMVMVM VMVMVMVM

External Authentication

via SAML and IDPs supported Okta &

MSFT ADFS

Local AuthenticationTOTP using Google Authenticator for 2nd

factor pin/barcode

RSA SecureID PingFederate SSO PingID 2-FA

Federal Common Access Card (CAC)


Shipping


ACI: L4-L7 Service Integration

114BRKACI-2117 114

Multi-site ServicesIntra-EPG contract

with PBRService EPG in preferred group

ACI 4.0

L1/L2 PBRACI Fabric, MPOD, Remote Leaf and

Multisite

ACI 4.1

Floating L3outPBR N+M standby

PBR with Service EPG in L3out

Future

Anycast IP/MACMulti-node PBR

Resilient hash PBRPBR with vzAny

ACI 3.2

PBR with Multi-Node Tracking

ACI 4.1.2

ACI Services Graph

L4-L7 Service AutomationL4-L7 Service

L1/L2/L3 PBR


ACI AnywhereEncrypted DCI Connectivity

115BRKACI-2117

Multi-Site

IP / WAN

Site A Site BVMVMVM

Site C

MACSEC MACSEC

CloudSecShipping

Future

Shipping

Cisco Integrations


UCSM Integration with VMM domain

• New ACI App to integrate UCSM to provision VLANs on-demand.

• With this integration, there is no need to pre-configure all of VLANs in VMM VLAN pool on UCS FI beforehand that consumes logical-ports (p*v).

• Requirement• APIC version 4.1 or later• UCSM version 3.2 or later


• Need to configure VLANs on FIs beforehand

• Consume logical-ports even though VLANs are not actually used.

Current operation With the integration

• No need to pre-configure VLANs on FIs

• Automate VLAN provisioning

Blade

pNIC

eth0 eth1

Blade

pNIC

eth0 eth1

UCSFabric Interconnect

ACI Leafs

ACI Spines

UCSChassis

VLAN-pool: 1000-1999EPG1: VLAN1000EPG2: VLAN1001

Need to trunk VLAN 1000-1999 on interfaces connected to each blade.

VLAN 1002-1999 are not actually used in this example

Blade

pNIC

eth0 eth1

Blade

pNIC

eth0 eth1

UCSFabric Interconnect

ACI Leafs

ACI Spines

UCSChassis

VLAN-pool: 1000-1999EPG1: VLAN1000EPG2: VLAN1001

Automatically add VLANs if VLAN is allocated for an EPG

Only VLAN1000-1001 are allowed in this example.

Multi-Domain - ACI and SDA

Cisco SDA CampusMulti-Site Orchestrator

VM

Cisco ACI Hybrid-DC

VM VM VM

DNAC ISE

Sales Finance CCW DB Finance DB Finance WebCCW Web

Group Exchange

ACI 4.0 Scale: 64K Bindings on Border Leaf (ISE Version 2.4 Patch 6, DNA Version 1.2.10)


San FranciscoData Center Multi-Site

New York Data Center

vEdge vEdgeSD-WAN Fabric

Region EastRegion West

Los AngelesBranch

Chicago Branch

vManage

MPLS Internet

FW

Web server

Appserver

DBserver

Subnet 10.1.1.0/24

FW

Web server

Appserver

DBserver

Subnet 10.121.0/24

1App Policy Determines Routing Path Between

Branch And Data Center To Meet SLA

1

2

2Optimal Path Selection

Between On-PremApps and Services

Hosted In Multi-Region AWS

ACI 4.1

ACI: SD WAN (Viptela) IntegrationExtend Operational Domain And Policy To Branch & Public Cloud

120BRKACI-2117


ACI to SD-WAN (Viptela) Integration – Phase 1

San Francisco Data Center

SD-WAN Fabric

vEdgevEdge

Los AngelesBranch

vManage

2

1

3

1) Physical Connectivity – L3Out per VPN

2) Application Policy – Export of Classification to vManage

3) Application Aware Routing -DC to Branch Ensured

App 1

User 1

ACI pushes Application Aware Policy to vManage

DSCP for signaling between L3-Out and vEdge in DC

DSCP based path selection out of 4 classes

ACI 4.1


ACI: AppDynamics IntegrationIdentify Problems Faster By Correlating Applications & Network Data

122BRKACI-2117

Network & Application Health

Correlation

VMVMVM VMVMVMVM

APPDYNAMICS

Shipping

• Map application and service components to ACI• Cross launch AppDynamics and ACI-APIC to correlate network and app data

Ecosystem Integrations


F5 ACI App in Cisco ACI App CenterExtend F5 BIG-IP and Cisco ACI Joint Solution Use Cases

124BRKACI-2117

Shipping


ACI: ServiceNow Integration Automated discovery and provisioning of ACI Fabric from ServiceNow ITOM

125BRKACI-2117

Discovery: Automatically discover ACI’s Physical & logical

entities from ServiceNow

Visibility: Accurate & up-to date CMDB

Infrastructure Visibility & Mapping

Configuration drift & rollback

Provisioning: Component

configuration, 40+ custom activity packs &

workflow automation

ServiceNow's External Credential Store

support, Compatible with latest ServiceNow releases,Incident dashboards

Discover cAPIC entities from ServiceNow ITOM

Automate ACI software update from

ServiceNow ITOM

Shipping Shipping Shipping Shipping Future

Cisco ACI Fabric

Mid server

CMDB

ACI App for ServiceNow v1.8

Discovery & Provisioning (APIC REST APIs)

ACI App v1.8 (compatible with Jakarta, Kingtson & London)

App Store

Shipping


ACI: Splunk Integration Central Proactive Monitoring, operational analytics and troubleshooting

126BRKACI-2117

Real time and historical insights into ACI fabricDrilldown into health scores, performance

metrics

Operational Analytics, Automated alerting, Root cause analysis

Audit, Risk and Compliance Analysis -Prevent unauthorized

access

Cross-tier correlation -gain visibility across the

entire data center

Splunk dashboards to monitor c-APIC,

Additional drilldown and troubleshooting, CIM Compliance, Syslog

parsing

Shipping Shipping Shipping Shipping Future

Shipping

ACI Fabric Monitoring, cross-tier correlation, Troubleshooting

VMVMVM VMVMVMVM

ACI App & Add-on for Splunk Enterprise

Published on Splunkbase Splunk App Inspect passed


üContinuous compliance and risk analysisüSupport for PCI, HYPPA, NERC, SOX,

BASEL II, ISO 2700, organizational stds

üSupport for Multi-vendor firewalls- Cisco ASA, Palo Alto, Fortinet, CheckPoint)

üPredefined workflows for automationüAbility to provision ACI contracts from

AlgoSec (New!)

Visibility and Compliance

Security Policy Automation

ShippingACI: AlgoSec Integration Multi-tenant, policy-driven, application-centric model for Security

127BRKACI-2117

üAlgoSec product release (2017.2 onwards)üOfficially Supported by AlgoSec


Cisco ACIBroad Ecosystem to Use, Customize and Extend Your IT Investments

128BRKACI-2117


Q2 CY 2017

Q3 CY 2017

Q4 CY 2017

Q1 CY 2018

Long Lived Releases ACI 3.2(x)

ACI 2.3

ACI 3.0

ACI 3.1

Maintenance Releases =>

ACI 2.2(2)

ACI 2.3(2)

ACI 3.0(2)

ACI 2.2(x)

ACI 3.1(2)

Q2 CY 2018

ACI 3.2

ACI 4.0

Q4 CY 2018

Q2 CY 2019

ACI 3.2(2)

ACI 4.0(2)

ACI 4.1

Major Releases =>

Q1 CY 2019

ACI 4.2

Q3 CY 2019

ACI 4.2(x)

ACI 4.1(2)

Q1 CY 2020

ACI 5.0

ACI 4.2(2)

Q4 CY 2019

Q3 CY 2019

ACI Software Release CadenceTarget – one release every four months


Virtual ACI ACI Cloud ACIIP WAN IP WAN

Edge / Remote MulticloudCore Data Centers

ACI Anywhere


ACIMulti-POD

ACIMultisite

ACIRemote Leaf

VirtualACI

CloudACI

130BRKACI-2117

Thank you

#CiscoLiveLA

#CiscoLiveLA

Please read - Cisco Live

Documents

Transcript of Please read - Cisco Live