Deep Learning with UCS - Cisco Live

Deep Learning with UCS

Ravi Mishra

Sr. Technical Marketing Engineer, UCS

BRKINI-2348

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

Cisco Spark

Questions? Use Cisco Spark to communicate with the speaker after the session

1. Find this session in the Cisco Live Mobile App

2. Click “Join the Discussion”

3. Install Spark or go directly to the space

4. Enter messages/questions in the space

How

cs.co/ciscolivebot#BRKINI-2348

• Introduction to Artificial Intelligence• AI Technology Refresher• Use case examples of AI• AI solution with Cisco• Deep learning/AI solution with Cisco UCS• Conclusion

Agenda


Electricity, the Internet and now Artificial Intelligence.

Andrew Ng“AI is the new Electricity. Artificial Intelligence is set to revolutionize

every sector, and will likely

eliminate whole categories of jobs

in the next few years.”

BRKINI-2348 5

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 6BRKINI-2348

What is Artificial Intelligence?


The AI race is on

Google Brain

ImageNet NVIDIA cuDNN

IBMWatsonJeopardy

Theano

Caffe

Torch Microsoft

Google

ML BeatsHumans

Google Car 1M Miles

Toyota $1B AI Lab

Facebook Torch

MS AzureML CNTK

Google TensorFlow

Amazon ML

IBMWatson

2010

2011

2012

2013

2014

2015


Healthcare Media & Entertainment Security & Defense Retail

� Cancer cell detection� Drug discovery� Medical research

� Theft detection� Auto checkout� Targeted marketing

� Face recognition� Crowd analytics� Cyber security

� Video captioning� Content based search� NLP, VR & AR

� Fraud Detection� Cryptocurrencies� Algorithmic Trading

Fintech

8BRKINI-2348

AI Across Industries


59%

25%

6% 6%

Knowledge gathering/developing strategy

Piloting Implementing Deployed/in use today

Investigating Using

The majority of organizations are still gatheringinformation to inform their AI adoption strategy

9BRKINI-2348

It Is Only The Beginning


Top 5 AI Use Cases Through 2021• Worldwide spending on AI in 2017 $12B grow to $57B by 2021

1. $6.13B – Automated Customer Service Agents

2. $5.04B – Diagnostic & Treatment Systems

3. $4.30B – Intelligent Processing Automation

4. $3.96B – Automated Threat Intelligence & Prevention

5. $3.62B - Program Advisors & Recommendation Systems Program Advisors & Recommendation Systems

10BRKINI-2348

IDC Worldwide Semiannual Cognitive/Artificial Intelligence Systems Spending Guide from IDC Customer Insights & Analysis. October 2017


Fun Fact #1

#Awkward

Joseph Weizenbaum, one of the founding fathers of Artificial Intelligence, later became one of it’s leading critics when he found his secretary getting very emotionally involved with ELIZA, a chat bot that he himself programmed

AI Technology Refresher


Traditional machine perceptionHand crafted feature extractors

S peaker ID, speech transcription, …

Raw data Feature extraction ResultClassifier/detector

SVM,shallow neuralnet,

…

HMM,shallow neuralnet,

…

Clustering, HMM,LDA, LSA

…

Topic classification, machine translation,sentimentanalysis…


Machine learning approach

Deploy:

Cat

Honey badger

Errors

DogCat

Raccoon

Dog

Train:Dog

MODEL

MODEL


Deep learning approach

Deploy:

Cat

Honey badger

Errors

DogCat

Raccoon

Dog

Train:Dog

DNN

DNN


CPU vs GPU for Deep Learning

[M] x [N ] x [K] [M] x [N] = [R] [K] x [R] = Result

S1

S2

S3

SK

W1

Error

7th Grade students

Result1

S1

S2

S3

SK

Error

W2

Result2W3

8th Grade students

[M] x [N ] x [K] [M] x [N]

[M] x [N]

[M] x [N]

[M] x [N]

S1

S2

S3

SK

Result

6th Grade students


Artificial neural network

17BRKINI-2348

Input layer Output layer

A collection of simple, trainable mathematical units that collectively learn complex functions

Hidden layers

Given sufficient training data an artificial neural network can approximate very complex functions mapping raw data to output decisions


Artificial neurons

From Stanford cs231n lecture notes

Biological neuron

w1 w2 w3

x1

y

x2 x3

y=F(w1x1+w2x2+w3x3

)

F(x)=max(0,x)

Artificial neuron


Deep neural network (DNN)

Input Result

Application components:

Task objectivee.g. Identify face

Training data10-100M images

Network architecture~10s-100s of layers 1B parameters

Learning algorithm~30 Exaflops1-30 GPU days

Raw data Low-level features Mid-level features High-level features


Training and Inference


Deep learning benefits

• Robust

• No need to design the features ahead of time – features are automatically learned to be

optimal for the task at hand

• Robustness to natural variations in the data is automatically learned

• Generalizable

• The same neural net approach can be used for many different applications and data

types

• Scalable

• Performance improves with more data, method is massively parallelizable

21BRKINI-2348


Fun Fact #2

22BRKINI-2348

#The Man vs. The Machine

In May of 1997, IBM’s Deep Blue Chess computer beat Garry Kasparov in a match. This was the first time a machine beat a chess champion.

Some Use case examples


AlphaGo

First Computer Program to Beat a Human Go Professional

Training DNNs: 3 weeks, 340 million training steps on 50

GPUs Play: Asynchronous multi-threaded search

Simulations on CPUs, policy and value DNNs in parallel on

GPUs Single machine: 40 search threads, 48 CPUs, and 8

GPUs Distributed version: 40 search threads, 1202 CPUs and

176

GPUs

Outcome: Beat both European and World Go champions in

best of 5 matches

http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html

http://deepmind.com/alpha-go.html


15

English and Mandarin speech recognition

Transition from English to Mandarin made simpler by end-to-end DL

No feature engineering or Mandarin-specifics required

More accurate than humans

Error rate 3.7% vs. 4% for human testshttp://svail.github.io/mandarin/

http://arxiv.org/abs/1512.02595

25BRKINI-2348

Baidu Deep Speech 2End-to-end Deep Learning for English and Mandarin Speech Recognition


Deep Learning for Autonomous vehicles


Automating Cardiac MRI analysisDL performance matches expert cardiologist at computing ejection fraction – a key indicator of heart disease

MRIimaging

Manualannotation

Software volume estimate


Fun Fact #3

28BRKINI-2348

#MachineOverlord

Google’s Artificial Intelligence Bot says the purpose of living is “to live forever”.

AI Solution with Cisco


IoTBig Data

Security

Network

AI

UCS

30BRKINI-2348

AI With Cisco?


DATACENTER

Trained Neural

Networks

+

ESLA

EDGE CLOUD

TRAINING

DATA

JETSON

JETSON

TESLA

TESLA

C240/P100 & V100C480/P100 & V100

C220/P4C240/P4 & P40

31BRKINI-2348

Cisco UCS – End to End AI Compute Platform

The Cyber Valet Service


• Valeo and Cisco announced a cooperation agreement to develop strategic innovations in smart mobility services.

• the driver gets out of the vehicle at the car park entrance and activates the automatic parking system using a smartphone. The vehicle continues its journey in automatic mode until it has finished parkingØ This technology combines the power of automatic parking technologies (Valeo

Park4U®Auto), Valeo on-board telematics and secure key systems (Valeo InBlue®)Ø with Cisco Parking Controller technologies, which equip car parks with Wi-Fi, video

sensors and AI/DL tracking by detection based solutions

Cyber Valet Services Project https://www.youtube.com/watch?v=z-XL1fyKHeQ

BRKINI-2348 33


Car via Wifi

Cisco UCS hosting Dual NVidia P100 DL, Data Fusion, dynamic path generation, Real time tracking by detection

IP Cam

Tracking & Path Matching

Path Generation

34BRKINI-2348

Cyber Valet Services Project


CityMind

Digital Network Architecture

Traffic Management Video Analytics helps Traffic Management

departments monitor and evaluate traffic flow data to help them more quickly and accurately detect incidents before they escalate and trigger quicker response by supporting agencies.

35BRKINI-2348

Traffic using Video Analytics Emergency Response Team

Video camera for Surveillance

Video camera for Surveillance


CityMind

Digital Network Architecture

Operations Center Cameras installed in key areas of the City, monitor

surroundings for unattended objects, person moving beyond a configured boundary / area. The system can be configured to alert or execute an event when such security violations occur. These alerts / events can then be used to automatically inform appropriate law enforcement systems / agents.

36BRKINI-2348

Safety and Security using Video AnalyticsLaw Enforcement

VA Camera

Control the Camera for investigation

Unattended object


Safety and Security: Tailgating

37BRKINI-2348

Dat

a

Insi

ghts

Service

Management

Actions/Applicat

ions

Auto

matio

n

Vide

o

Badge

ReaderCM

XISE

Cisco

Prime Infra.

Crowd

Analytics

Tailgating

Analytics

Deep

Fusion

Cont

ainer

s

on

Cisc

o

IOx

…

Webhooks

Registry Scheduler

Application

and Policy

Management

…

Digital

Ceiling

Lenel

Dashb

oard

Security

Monitoring and

Dispatch

Dynamic Building

Resourcing

…

Dynamic Building

HVAC

adjustments


Fun Fact #4

#InfinityOrcs

MASSIVE is a software package used to generated thousands of actors with their own Artificial Intelligence. It was used to generate the battle scenes in the Lord of the Rings films.

BRKINI-2348 38

Deep Learning/AI Solution with UCS


GPU-accelerated computing is the use of a graphics processing unit (GPU) together with a CPU to accelerate deep learning, analytics, and engineering applications. ... They play a huge role in accelerating applications in platforms ranging from artificial intelligence to cars, drones, and robots.

For Cisco it encompasses two categories:1. Artificial Intelligence - Deep Learning2. Accelerated Analytics - Databases

What is Accelerated Compute?

BRKINI-2348 40


Cisco GPU Accelerated Data Center

Accelerated Analytics & AppsArtificial IntelligenceGPU-Accelerated VDI

Real-Time and Location AnalyticsWindows 10Virtual Workstations

Deep LearningMachine Learning

Virtualization Accelerated Compute


Fun Fact #5

#CantFoolMe

In the Turing Test, a machine and a human both converse sight unseen with a second human, who must evaluate which of the two is the machine.

BRKINI-2348 42


UCS GPU Portfolio UCS Integrated for accelerated VDI, Deep Learning, and HPC Applications

Virtualization Accelerated Compute

UCS M6 Blade GPUFor M4 blades only

Enterprise Class NVIDIA GPU for Remote Knowledge Workers, Task Workers and Designers

UCS NVIDIA M60For M4 & M5 server only*

Ultimate choice for Remote Engineering Workstations

and Application Delivery via the Cloud

UCS NVIDIA M10For M4 and M5 servers

Accelerated Remote Desktop, Maximum User

Density per Server

UCS Tesla V100For M5 server only

World’s most advanced data center GPU ever built to accelerate AI and HPC

UCS Tesla P40For M5 server only

Remote Engineering Workstations and Fast Inferencing for Deep

Learning

UCS Tesla P4For M5 server only

Inferencing Engine for Deep Learning at the edge.

UCS P6 Blade GPUFor M5 blades only

Doubles user density for Remote Knowledge Workers, Task

Workers and Designers

UCS Tesla P100For M4 and M5

Deep Learning Training and HPC applications

UCSM 3.2(3)

*M5 support for M60 in 3.2(2c)


Cisco GPUs for AI

BRKINI-2348 44


Nvidia GPU on UCS with M5 serverHigh Level View of Integration

C240M5

2x Nvidia V100,P100,P40 2x M10,M60, 6x P4*

UCSC240 M5

S

X

NVME SSD

800 GBNVMEHWH800

2 TBHD2T7KL6GN

SATA HDD

XX

NVME SSD

800 GBNVMEHWH800

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

821 76543

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

14131211109 201918171615 24232221

6x Nvidia V100,P100, P40, 3x M10

C480M5

UCSC480 M5

X

NVME SSD

800 GBNVMEHWH800

2 TBHD2T7KL6GN

SATA HDD

XX

NVME SSD

800 GBNVMEHWH800

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X X

NVME SSD

800 GBNVMEHWH800

X

NVME SSD

800 GBNVMEHWH800

X

NVME SSD

800 GBNVMEHWH800

2 TBHD2T7KL6GN

SATA HDD

XX

NVME SSD

800 GBNVMEHWH800

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X

2 TBHD2T7KL6GN

SATA HDD

X X

NVME SSD

800 GBNVMEHWH800

X

NVME SSD

800 GBNVMEHWH800

ReW ritabl eRECORDERM U L T I

DVD+ReWritable

S

X

X

B200M5

2x Nvidia P6 GPU per blade, up to 16x per chassis

! ResetConsole

UCS-HD300G10L12G126bps 10K SAS300GB

!

UCS B200 M5

UCS-HD300G10L12G126bps 10K SAS300GB

!

B480M5

4x P6 GPU per Blade, up to 16x per chassis.

! ResetConsole

UCS-HD300G10K12G12Gbps 10K SAS300GB

! UCS-HD300G10K12G12Gbps 10K SAS300GB

!

M4 SRVN

UCS-HD300G10K12G12Gbps 10K SAS300GB

!UCS-HD300G10K12G12Gbps 10K SAS300GB

!

C220M5

2x Nvidia P4*

*3.2(3) software


Why Cisco UCS for GPUDifferentiation

Firmware, vBios, Inventory, Service Profiles

• Run at PCIe Gen 3 speed

• Supported with all high-bin CPUs

Management Performance Power & Cooling

Test up to 35c in Heat chambers with all max configs. Ensure NO GPU throttling


Fun Fact #6

#NotARobot

CAPTCHA is short for Completely Automated Public Turing test to tell Computers and Humans Apart.

BRKINI-2348 47


Market dynamics along the stack

ML Value StackProfessional services Consulting Practices Implementation Practices Learning/training Practices

ML Framework/Library

Drivers

OS, run time environ

Intel® Math Kernel Library (MKL, MKL-DNN)

Intel® NervanaTM

Graph

Data Science / ML Platform

Data Prep >> Modelling support >> Deployment

Roots from open source

Proprietary

Infra management Data mgmtInfra mgmt

ML application and API


Cisco Spark

Questions? Use Cisco Spark to communicate with the speaker after the session

1. Find this session in the Cisco Live Mobile App

2. Click “Join the Discussion”

3. Install Spark or go directly to the space

4. Enter messages/questions in the space

How

cs.co/ciscolivebot#BRKINI-2348


• Please complete your Online Session Evaluations after each session

• Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt

• All surveys can be completed via the Cisco Live Mobile App or the Communication Stations

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at www.ciscolive.com/global/on-demand-library/.

Complete Your Online Session Evaluation


Continue Your Education• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Tech Circle

• Meet the Engineer 1:1 meetings

• Related sessions

51BRKINI-2348

Thank you

Deep Learning with UCS - Cisco Live

Documents

Transcript of Deep Learning with UCS - Cisco Live