Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

Devops with git and Jenkins

Develop Execute

Commit | Clone | Build | Test |…

Logs Tests

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

Dataops med devops mot

SAS Viya med git og JenkinsDevOps Applied to SAS Viya Data Management

Litt fag og litt demo

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

DataOps for Analytics …Trusting in the data as we trust in the water we drink..

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

DataOps and DevOps

Source:medium.com

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

Analytics Life CycleAutomation, Operationalizing, Industrializing

MANAGEDATA

DEPLOYINSIGHTS

DECISIONQUESTIONANALYTICS

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

DataOPs – ComponentsThe How and the What … From the Source to the Consumer

Metadata Management

Data Integration

Data Discovery

Data Transformation

Data Preparation

Data DeliveryD

ata

Sou

rces

Data Quality Assurance

Business (Governance) Rules

Data Authoring

/ SSM

Data Governance, Usage Management, Monitoring

Data C

on

sum

ers

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

DataOPs – with DevopsGetting your ETL, data prep, data quality and other dataops tasks in operations

smooth and frequent through devops practices

Manage code and other SAS artifacts (and code running in other environments) doing these things through devops

practices with Git and Jenkins

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

Devops with git and Jenkins

Develop Execute

Commit | Clone | Build | Test |…

Logs Tests

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

Demo time

Git

• Configure git in SAS Studio

• Get your code into git – learn to trust git

• Local vs remote repository

Jenkins

• Ways of automation – freestyle and pipelines

• Integrate with gitlab and SAS Viya

• Run programs and flows from Jenkins

– (programs as batch, flows as job)

• CI / CD with automated tests

How does it look? Using Git and Jenkins with SAS Viya

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

Git CommandsResources

Introduction to Git Git guides

• Youtube: Git Integration in SAS Studio and SAS Enterprise Guide

– https://www.youtube.com/watch?v=CZwT7kxmBmw

• https://www.atlassian.com/git/tutorials/atlassian-git-cheatsheet

• https://www.git-tower.com/blog/git-cheat-sheet/

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

SAS StudioGit plug-in

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

Sample Git Workflow

• Create a Git profile

• Clone a repository

• Pull the latest changes from the remote repository

• Create a new feature branch

• Make changes and test them

• Stage the changes

• Commit the changes

• Merge the feature branch back into the master branch

• Push the changes remotely

Steps

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

Sample Git WorkflowCreate a Git profile (HTTP clone)

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

Environment Overview

Service A1

STATELESS NODE POOL

Microservices and Web Apps

Service B1

Service C1

Service A2

Service B2

Service C2

SAS Compute

SAS Compute

CAS Worker

SAS Compute

COMPUTE NODE POOL

SAS Compute Services

Web App Web App

CAS Controller

CAS Cluster

CAS Worker

CAS Worker

CAS NODE POOLSTATEFUL NODE POOL

Commodity Services

Postgres Consul RabbitMQ

OPS4VIYA

SYSTEM NODE POOL

Monitoring + Logs

Ingress

sas-viya CLI

SAS Viya sse namespace

jenkins

Jenkins Agent

GitLab Docker containerJenkins Pipelines

Linux hostJenkins Blue Ocean plug-in

Jenkins GitLab plug-in

/workshopPATH

Jenkins Docker container

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

Jenkins• Open-source, free automation software

• Used to automate software building, testing,

deployment and delivery

• Typically associated with software development CI/CD

• Can be used to automate many facets of SAS Viya

• Extensible with numerous available plug-ins

• Fast deployment available via Docker image (or k8s)

• Good documentation

• Version control integration

Automation Server

jenkins.io

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

Automation Server

Develop Execute

Clone | Load in CAS | CAS View | Tests |…

Logs Tests

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

Jenkins Project

Automation Server

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

Automation ServerJenkins Pipeline and Jenkinsfile

Company Conf idential – For Internal Use OnlyCopyr ight © SAS Inst i tute Inc. Al l r ights reserved.

Summary: Devops with SAS Viya, git and jenkins

Git

• Configure git in SAS Studio

• Get your code into git – learn to trust git

Jenkins

• freestyle and pipelines

• Integrate with gitlab and SAS Viya

• Run programs and flows from Jenkins

• CI / CD with automated tests

devops practices – through Git and Jenkins