SIT Automation Tool: Failure Use Case ... - UPCommons

1

SITAutomationTool:FailureUseCaseAutomationandDiagnosis

ADegreeThesis

SubmittedtotheFacultyofthe

EscolaTècnicad'EnginyeriadeTelecomunicaciódeBarcelona

UniversitatPolitècnicadeCatalunya

by

ArianShamsSanchez

Inpartialfulfillment

oftherequirementsforthedegreein

TelecommunicationsystemsENGINEERING

Advisor:JaumeComellasColomé

Barcelona,October2016

2

Abstract

ThescopeofthisthesisistheSIT(SystemIntegrationTesting)processwhichisthetestingprocedureexecutedincustomertestenvironmentbeforethesoftwaregoesonproductionenvironment.Themainobjective for this thesis isnoother than improving thecurrentprocessstepbysteptakingintoaccounttheautomation,efficiency,missingchecksandmuchmore.

ThisprojectisakindofIndustrialprocesstocreateapowerfultestingtoolwhichcanallowthecompanytodeliverqualityadaptorproductsefficiently,dobetterinlesstimehelpingtoreducecosts,asAdaptorsarethemostdemandedproductofMYCOMOSIportfolio.Takeintoaccountthatbusiness isnotonlygeneratedwhenanAdaptor isdeliveredforfirsttimebutalsowhenVendorsprovidewithnewreleasesandnewfunctionalitiesandoperators needs to order an upgrade of the Adaptor to be able to monitor the newfunctionalitiesdeployedontheirnetwork.

3

Resumen

El campodeaplicaciónenelqueestácentradoesta tesiseselSIT (System IntegrationTesting), proceso de testeo ejecutado en un servidor de testeo del cliente antes dedesplegarelsoftwareelmediodeproducción.Elobjetivoprincipaldeestatesisnoesotroquemejorarelprocesoactualpasoapasoteniendoencuentalaautomatización,eficiencia,lafaltadeverificaciones,entreotros.

Esteproyectoesunaespeciedeprocesoindustrialparacrearunaaplicaciónpotentedetesteoquepuedapermitiralacompañíaentregaradaptadoresdecalidadconeficiencia,quehaganmásenmenostiempoayudandoasíareducircostes.LosadaptadoressonelproductomásdemandadodelporfoliodeMYCOMOSI.Hayquetenerencuentaqueelnegocionosegenerasolamentecuandoseentregaporprimeravezeladaptadoralcliente,sinoquecuandolosproveedoreslanzannuevasversionesconnuevasfuncionalidadesylosoperadoresnecesitanencargarunamejoradeladaptadorparapodermonitorizarlasnuevasfuncionalidadesdesplegadasensured.

4

Resum

El camp d'aplicació en que es basa aquesta tesi és el SIT (SystemIntegrationTesting),procés detesteigexecutat en un servidor detesteigdel client abans de desplegar elsoftware al mitjà de producció. L'objectiu principal d'aquesta tesi no és un altre quemillorarelprocésactualpasapastenintencomptel'automatització,l'eficiència,lafaltadeverificacions,d'entrealtres.

Aquest projecte és una mena de procés industrial per crear una aplicació potentdetesteigquepuguipermetrealacompanyialliuraradaptadorsdequalitatambeficiència,quefacinmésenmenystempsajudantaixíareduircostos.ElsadaptadorssónelproductemésdemandatdelporfoliodeMYCOMOSI.Caltenirencomptequeelnegocinonoméses genera quan es lliura per primera vegada l'adaptador al client, sinó que quan elsproveïdors llancen noves versions amb noves funcionalitats i els operadors necessitenencarregar una millora de l'adaptador per podermonitoritzarles noves funcionalitatsdesplegadesalasevaxarxa.

5

Acknowledgements

Afterfourmonthsofhardwork,mydegreethesisisfinallyfinished.Withit,IconcludemyTelecommunicationSystemsEngineeringDegreeand,therefore,astageofmylive.Thistimehasbeenanamazingincrediblejourneyofself-discovery.ThereweretimeswereIfacedseveraldifficultiesthatcompletelyovertookmebutIalwayscameupwithasolution.Iwillalwaysrememberthissweetmomentwhenyourwrittencodeworksasexpected.

DoingthisthesisinaTelecombusinessinaforeigncountry,farfromfamilyandfriendsit´sbeingatoughchallenge.LivinginacitylikeLondonasstressfulasitisandworkinginasmall town like Slough (more than one hour of travel in every trip) it was also anuncomfortablesituationtodealwith.

Butnevertheless,it´sbeenagreatopportunityandaconstantlylearningprocesswhichIwouldneverforget.

IwouldliketothankmycollegesfromtheETSETBAlfonsSolerRomeo,JoanAdriàRuizdeAzuaandGerardSánchezGasullafortheirhelpandinterest.Ofcourse,thankstomytutorJaumeComellasforhisguidanceandadvicesineverystepofthethesis.

Iwouldalso like to thankall theMYCOMOSIUK team for all their supportduringmyinternshipthere,helpingmeeverytimewhenIcouldnotfindawaytocontinue.ButinspecialtoPatriciaDelgado,thepersoninchargeofmethere,whoneverstopteachingandhelpingmeeverysingleday(evenviaSkypewhenshewasnotattheoffice).

Finally, Iwould like to thankall theamazingpeople Ihavemetduring this timeat theuniversity,thatwon´tbeonlymyoldclassmatesbuttruefriends.

Thankyouall!

6

Revisionhistoryandapprovalrecord

Table0.1Revisionhistoryandapprovalrecord

Revision Date Purpose

0 27/06/2016 Documentcreation

1 Documentrevision

2 Finalrevision

3 Documentapproval

DOCUMENTDISTRIBUTIONLIST

Name e-mail

ArianShamsSanchez [email protected]

JaumeComellasColomé [email protected]

Writtenby: Reviewedandapprovedby:

Date 27/06/2016 Date

Name ArianShamsSanchez Name JaumeComellasColome

Position ProjectAuthor Position ProjectSupervisor

7

Tableofcontents

Abstract...............................................................................................................................2

Resumen..............................................................................................................................3

Resum..................................................................................................................................4

Acknowledgements.............................................................................................................5

Revisionhistoryandapprovalrecord..................................................................................6

Tableofcontents.................................................................................................................7

ListofFigures......................................................................................................................8

ListofTables........................................................................................................................8

Glossary...............................................................................................................................9

1. Introduction..............................................................................................................101.1. Projectrequirementsandspecifications...........................................................111.2. Methodsandprocedures..................................................................................111.3. WorkPlan..........................................................................................................11

2. Stateoftheartofthetechnologyusedorappliedinthisthesis:.............................162.1. MYCOMOSI.......................................................................................................162.2. PrOptima™........................................................................................................182.3. Adaptors............................................................................................................192.4. SIT......................................................................................................................19

3. Projectdevelopment.................................................................................................213.1. CurrentSITprocess...........................................................................................213.2. SIT_AA_01currentprocess...............................................................................23

4. Results.......................................................................................................................234.1. SITProcesssolutionproposal............................................................................234.2. SIT_AA_01solutionproposal............................................................................244.3. Nextstepssolutionproposal.............................................................................27

5. Budget.......................................................................................................................27

6. Conclusions...............................................................................................................28

8

ListofFigures

Figure1:MYCOMOSISoftwareDistribution....................................................................16

Figure2:AdaptorInformation..........................................................................................19

Figure3:Exampleofthecontentinausedfile.................................................................25

ListofTables

Table0.1Revisionhistoryandapprovalrecord..................................................................6

Table1.1Milestones.........................................................................................................13

Table1.2Ganttdiagram....................................................................................................15

Table3.1:CurrentStepsanddescription..........................................................................22

9

Glossary

CAPEX:CapitalExpenditures

CM:ConfigurationManagementData

CSP:CommunicationServiceProvider

EMS:ElementManagementSystems

ETL:Extract,TransformandLoad

FM:FaultManagement

GIS:GeographicalinformationSystem

GUI:GraphicaluserInterface

IK:ImportKey

IoT:InternetofThings

ISP:InternetServiceProvider

MIS:Mycom-osiInterfaceSpecification

MOC:ManagedObjectClass(inPrOptima)

NE:NetworkElement

NOC/SOC:Network/SecurityOperationalCenter

OPEX:OperatingExpense

PM:PerformanceManagementData

QoS:QualityofService

ROI:ReturnonInvestment

SI:SystemIntegrator

SIT:SystemIntegrationTesting

TELNET: Application layer protocol used on networks to provide a virtual terminalconnection

UAT:UserAcceptanceTesting

UFS:UNIXFileSystem

UI:UserInterfaces

VTI:VendorTechInterface(NameoftheAdaptor)

VTP:VendorTechnologyPlug-in

XML:ExtensibleMarkupLanguage

10

1. Introduction

Asnetworksbecomelargerandvirtualized,servicesbecomemorecomplextomanageandcustomer experience expectations for service quality rise, Communications ServiceProviders(CSPs)requirefuture-proofsolutionstoaddressthenewchallengesofthesmartdigitalworld.MYCOMOSIprovidesPerformancemanagementsolutionsforthedigitaleraandtheadventofhybrid–physicalandvirtualized–networks.

TodoitMYCOMOSIhasdevelopedPrOptima™acarriergrademulti-vendorandmulti-technology performance management software solution for Communication ServiceProviders.It’sasolutionthatmanagestheperformanceofbothmobileandfixednetworksandprovidescriticaldatafornetworkandserviceoperationscentres,networkengineeringand optimization departments, as well as capacity planning and infrastructure rolloutteams.

Thesoftwarepackagesusedon thismediationarecalledAdaptors.TheyparseVendorspecificinputfilesintostandardMYCOMOSIfileformat(MIS)sotheycanbemanipulatedbyMYCOMOSIproducts.TheparsingresultsarethenloadedintotheDataWarehouse.Itisimportanttonotethateveryadaptorisdesignedpervendor,technologyandreleasetomaximizetheefficiency.

Tocheck thisparsing software,a testprocesscalledSIT (System IntegrationTesting) isperformedonthecustomertestServerbyMYCOMOSIengineers,whichchecksthingsliketheextractionandloadingofdata,differentreleasesbehavior,updatesofthesoftware,modificationsandmuchmore.

ThepurposeofthisprojectistooptimizethecurrentSITprocess,increasingtheefficiencyand the quality in order to achieve better results in less time.This should be doneanalysing the existing steps of the Testing process and identifying the gaps andunproductivecommands,todevelopanautomationframework.

Atthemoment,theSITprocesstakesbetween3and5daystogetpassedbuteventhenthereisroomformoreimprovingbyincreasingefficiencyandaddingnewestcases.

Theprojectmaingoalsare:

1.ImprovemyknowledgeinScriptingwithUNIXandtheOSitself.

2.Analysethewholetestingprocessandbeabletoidentifygapsandunproductivesteps.

3. Develop a framework to optimize and automate the process with which the R&Ddepartmentcoulddofurtherimprovement.

4.ImprovethetimingandthequalityofTestingReports.

11

1.1. Projectrequirementsandspecifications

Projectrequirements:

v PreviousknowledgeonUNIXscriptingandUNIXOSinordertodevelopthenewinstances.

v Previous knowledge on C++ (or similar) to be able to learn quickly Perl codinglanguage.

v Theoreticalbaseofanymobiletransmittingtechnologies(2G,GRPS,3G,4G...)andtheNetworkElements.

v AlevelofEnglishthatallowstowritetherequireddocumentation.

Projectspecifications:

v Identifythegapsintheactualprocess.v Improvetheexistingtestingprocesstoreducethetesttimeandgivemoreaccurate

information.v Developanautomationframeworkforoneofthemainsteps

1.2. Methodsandprocedures

ThisprojectisbasedonaMYCOMOSIsoftwareapplicationPrOptima™,tooltomonitorany of their costumer networks. Then, there are a couple of other complementarysoftware like ICM(ImportControlManager)which isusedto Importthedatafromthecompanyservers(MISdata).

Putty,anSHHandtelnetclienttoaccesstheserversorEclipse,anintegrateddevelopmentenvironmentforcomputerprogrammingareothersoftwareusedduringtheproject.

Thisprojectdoesnotstartfromscratchsincethecurrenttesttoolsarealreadydevelopedbythecompany.Butthisisnotacontinuationofapreviousprojecteithersincethepre-existing material has been developed by the R&D department and not by any otherstudent.

1.3. WorkPlan

Tasks

WP1-Introductiontothesoftware:Gettoknowtheproductwhichtheprojectisbased.

-InternaltaskT1:PassthemainPrOptimatrainings.

-InternaltaskT2:Practiceallthetheoreticalknowledgeachievedontheactualsoftware.

12

WP2- Introduction to SIT Process: Get used to the testing process both in theory andpractice.

-InternaltaskT3:10htrainingwithanexpertengineerontheprocess.

-InternaltaskT4:StudyoftheSITprocess.

-InternaltaskT5:PerformcoupleofcompleteSITinthetestserver.

WP3-Businessdocumentation:Compilationandwritingofthecompany.

-InternaltaskT6:Findandwriteinformationaboutthecompanyandtheproductsinvolvedontheproject.

WP4-Perl:Introductiontothecodinglanguage.

-Internal task T7:Readandunderstand thedocumentationof themain structures andmodules.

-InternaltaskT8:PassanintermediatePerlcourse.

WP5- SIT process analysis and solution development: Analyze the process, design anddevelopasolution.

-InternaltaskT9:RecompilationoftheMaxPriorityProblemsduringSITprocess(12lastmonths).

-InternaltaskT10:Analysisandgapfindingontheprocess.

-InternaltaskT11:Stepanalysis.

-InternaltaskT12:Solutiondesign.

-InternaltaskT13:Solutiondevelopment.

WP6-Tooltestingandbugfixing:Testandfixanypossiblebugonthenewtool.

-InternaltaskT14:Tooltesting.

-InternaltaskT15:Bugfixing.

WP7-FinalProject:WriteandReview.

-InternaltaskT16:WriteuptheThesis.

-InternaltaskT17:ThesisReview.

13

Milestones

Table1.1Milestones

WP# Task# Shorttitle Milestone/Deliverable Date(week)

1 1 Stateofart DothemaintrainingsofPrOptima 02/05

1 2 Toolpractice FamiliarizewithPrOptima 23/05

2 3 SITTraining 10hinteractivetrainingwithanexpert 30/05

2 4 SITProcess FamiliarizewithSITprocess 30/05

2 5 SITPractice PasscoupleofSITontestserver 6/06

3 6 CompanyDocumentation

Find andWrite info fromMYCOMOSIandtheproducts

27/06

4 7 PerlDocumentation

LearningPerlfromTheperl.org 4/07

4 8 PerlCourse Passanintermediateonlinecourse

11/07

5 9 PR1Recompilation RecompilationoftheMaxPriorityProblemsduringSITprocess(12lastmonths)

01/08

5 10 SITAnalysis AnalyzetheprocessandIdentifythegaps

08/08

5 11 Stepanalysis Findasolutionfocusingonthisspecificstep

15/08

5 12 Solutiondesign Designofthefinalsolution 22/08

5 13 Solutiondevelopment

Developmentofthecode 29/08

6 14 Tooltestingandbugfinding

Testthenewtool 19/09

6 15 Bugfixing Fixthenewerrors 19/09

14

7 17 FinalReport Writethefinalreport 26/09

7 18 Review Reviewthefinalreport 03/10

Deviationsandincidences:

The original planwas to analyze thewhole testing process and build up a solution toimproveandautomateeverystepontheSIT.

ThefirstincidencecamewhenIhadtodecidethecodinglanguagetobeused.TheoriginalplanwastouseC++todevelopanyoftheneededscripts,butaftertalkingwiththeR&Ddepartment, the urgedme to use Perl. Themain reason they gaveme is that, as anagreement,everybodyhastodevelopthescriptsinthesamelanguagesothewholeteamcanworkonthesameprojectincasethesolutionit´snotcompletelyfinishedorsomethingisneedtobechangedoradded.SoasIneverusedthatcodinglanguagebeforeIhadtotakeanintermediatelevelcoursechangingtheoriginaltiming.

Asaconsequenceofthisfirstissueandafteranalyzingthecurrentprocess,I’vedecidedtofocusonthefirststepofthetestduetothelimitationoftime.ThisstepchecksthatthedatafetchedfromtheRemoteServerofthecustomerisproperlystoredonMYCOMOSIServers.Thefileshavetobethesame inboth,andalsotheyhavetoaccomplishsomepredefinedparameterslikethefilenameformat,thetimestamporextension.

Themainchangesontheworkplanarethe4weeksPerlcourseandchangingthestepbystepsolutiondesignanddevelopmentforonlyoneofthesteps.Eventhen,thecompleteanalysisof the currentprocessand the recompilationof themost importantproblemsoccurredonthelast12monthsisdoneanddeliveredtotheR&DdepartmentforfurtherimprovementsoftheSIT.

15

TimePlan

Table1.2Ganttdiagram

16

2. Stateoftheartofthetechnologyusedorappliedinthisthesis:

2.1. MYCOMOSI

MYCOMOSIisanindependentproviderofAssurance,AutomationandAnalyticssolutionsto worldwide CSPs including AT&T, Deutsche Telekom, Maxis, Sprint, STC, Telefonica,Telenor,T-Mobile,VerizonandVodafone.

Figure1:MYCOMOSISoftwareDistribution

ExperienceAssuranceandAnalytics

MYCOMOSI’s Experience Assurance and Analytics (EAA) addresses three initiatives ofCSPs:managingtheCustomerExperience,evolvingtoNetworkVirtualization(NFV)andexploitingtheDigital/IoTenvironment.

By integrating quality products and solutions for Performance Management, FaultManagement,DigitalServiceQualityManagement,CustomerExperience,AutomationandNetworkAnalytics intoabigdata-basedplatform that is tightly integratedby commonresource, mediation and visualization layers, it enables CSPs to use consolidatedinformationtoaddressanyoftheseaspects.

17

ExperienceAssuranceandAnalyticsBenefits

v Capexefficiencythroughpre-integratednetwork,service,customerassuranceandbigdataanalytics

v Multi-teamefficiency fromcollaborativeplatform for technical,marketingandcareteams

v Customerinsightsandnetworkmonetizationwithpredictiveanalytics

v Increasedemployeeproductivitybyclosed-loopautomations

v Cloud-based,scalable,secure,network-independentplatform

MYCOMOSIBenefitsforCSPs

v Useofanend-to-endindependentviewofnetwork,serviceandcustomerexperience,tofacilitatecollaborationacrossteams

v Proactivelydetectnetworkandserviceissues,toimproveend-userexperiencev Simplifyoperationsviapre-integrationandautomation,toaddresscomplexityandto

increaseoperationalefficiencyv Make decisions based on network and service/device/customer relatedmetrics, to

optimizeinfrastructureinvestmentsandtomaximizeROI

NetworkAnalytics

The company provides Analytics solutions to enable the optimal use of network andservice/device/customer data from different sources (including big data stores) at thesametimeandaspartoffamiliarworkflows/processes.

MYCOMOSIhelpsCSPsmakeintelligentuseofthewealthofdataattheirdisposaltorevealnetworkutilizationandsubscriberbehaviour insights,enablingnewdigitalservicesandincreasethereturnonnetworkinfrastructureinvestments.

Solutions

MYCOM OSI helps CSPs deploy a Next Generation NOC/SOC by building on the coreplatformsoffaultmanagement,performancemanagementandservicemanagement,andaddingusecase-basedsolutions.TheseenableadvancedoperationsinNOC/SOCthat:

v Delivercustomercentricityv Enableproactiveandpreventativeoperationsv Increaseproductivitywithautomationv Solvecomplexincidentswithguideddiagnostics

18

2.2. PrOptima™

PrOptima™isafixed/mobilenetworkperformancesolutionthatscalestomeettheneedsofCSPswithsupportformultipletechnologies,domainsandequipmentvendorsacrossaccess, backhaul, core and service networks. It processes very large volumes ofperformance management data in near real-time, has advanced correlation, analysis,reportingandvisualizationmodules,andishighlyflexibleviaautomation,workflowandconfigurationcapabilities.

Thesoftwareconsumeshighlyvaluableresourceandserviceperformancemanagementdatacontainingverydetailedandstatisticallysignificantcontrolanduserplaneevents,elementconfigurationandnetworkelementresourcesdata.Thismaybecomplementedwithotherdata,suchasfaultmanagement,troubleticketingandotherdatasources.

Thedatastorecanthenbemanagedwithautomatedandmanualmethodstoprovide24x7issuedetectionforresolutiondepartments,networkengineeringplanners,optimizersandmanagement.

Thebasicmodulesavailableontheapplication:

v Workspace:Allowingtobuildprofessionalreportsanddashboardscombiningtabularandgraphviews

v Alarm Module: Providing multiple deviation algorithms to generate performancealarmsinvariousformatsforconsumptionwithinoroutsidePrOptima™

v ProfilingModule: Providing trendingandenvelopanalysisandpossibly feeding thealarmmodulefordeviationidentification

v GeographicalInformationSystem(GIS)module:Representingstatisticalinformationon geographical and vector maps provisioned locally or through Google Mapintegration

v Decision Support System: A workflow analysis module allowing to capture youroperationalandengineeringbusinessprocesses

v Network Planning: A forecasting module providing various extrapolations andprojectionmechanismstoassessthepotentialresourcestatusinthefuture

19

2.3. Adaptors

Adaptors or Vendor Technology Interfaces are software in charge of parsing theinformation.TheyparseVendorspecificinputfilesintostandardMYCOMOSIfileformat(MIS)sotheycanbemanipulatedbyMYCOMOSIproducts,suchasPrOptima™inthiscase.Theparsing results (MIS files)are then loaded into theDataWarehousewithdifferentlifetimedependingonthegranularityandtheinformationtype.

Adaptors Functional & Technical Specifications Documents define the requiredfunctionalityoftheparserandcontainsalltheneededinformationtoinstallanddeploytheadaptorineachnetwork,makingthempersonalizedforeachvendorandtechnology.

Figure2:AdaptorInformation

2.4. SIT

EveryAdaptorgoesthroughadetailedtestingprocessbeforeiscompletelydeployed.

Firstofall,theR&Ddepartment,whichisinchargeoftheimplementationofeachadaptor,performs a QA (Quality Assurance) test to ensure that everything is working in in acontrolledenvironment.

Thenthesoftwareisdeliveredtooneofthequalifiedengineersinthecompanytopassthe SIT (System Integration Test) in a test server, as the QA it’s performed in a labenvironmentwithnorealtimedata.

Afterthat,theadaptorisdeliveredtothecustomertopasstheUAT(UserAcceptanceTest),averysimilartesttotheSITbutperformedbythecustomerengineeringteamintheirownenvironment(butstillonthetestServer).

20

Finally,theliveinstallationisdoneandthedatastartstobeparsed.

Mostofthe issuesrelatedtotheadaptorarefoundontheSIT,asthistest is themostexhaustiveoneandthat’swhythisthesisisbasedonit.

Thecurrentprocesshas20stepsandchecks:

v Extractingprocessv Multi-Releaseissues(everytimeanadaptorisupdatedanewSITisperformed)v Networkentities,countersandattributesproperlyparsedandstoredv Duplicationofdatav Previewofdatav Newdataorstructures(anymodificationshavetobechecked)

21

3. Projectdevelopment

3.1. CurrentSITprocess

As said in the previews sections, the aim of the project is to develop a framework toimproveandautomatethecurrenttestingprocessoftheparsingsoftware.

Anexhaustivestudyit’sperformedoneverysteptodeterminatethedifferentchangestoproposeineachcase.

Everycaseisnamedwiththesamestructure,SIT_AA_XX,whereXXisthenumberofstep.Then,eachofthemisorganizedbyareasandjustasthenameofthestepthestructurestartswithSIT_andaddedtothatsomespecificinformationtosortthemall.

Tounderstandthetestingprocessisimportanttounderstandfirstthatthereare2mainplatformsinvolved.Inonehand,wehavetheDatabasesfrombothcustomerandMYCOMOSI,Serversfromwherethenetworkdataisfetchedandstored(fromthecustomertothecompanydatabase).Andintheotherhand,thePerformanceManagertool,PrOptima™whichisinchargeofmanipulatingthisinformationandpresentitinmanydifferentwaysaccomplishingtheuserneeds.Dataisorganizedandshownintopalettesfromwheretheuser can choose the different options, such as NE (Network Elements), AE (AttributesElements)andCE(CountersElements).

Theprocessisshownstepbystepwithabriefdescriptionofeachoneinthenexttable:

22

Table3.1CurrentStepsanddescriptionStepname StepArea Description

SIT_AA_01 SIT_ADMIN_TEST

CM/PMfilenamecheck:

Thepurposeofthischeckistoensurethateverysinglefile

isproperlyparsedandthefilenamestrucutreofthefilespresented

matchesthefilenamingconventionexpectedbytheAdaptor,

thetimestamp(sendingandarrivaltimes)andextension.

SIT_AA_02 SIT_MULTI_RELEASE

ReleaseIdentification:

-Identifythereleasetagsavailableindata

-IdentifytheACCEPTEDReleasesconfigured.

EnsurethatonlythePM/CMdatafortheapplicablereleaseisprocessed.

SIT_AA_03 SIT_MULTI_RELEASEChecktheETLprocessingofManagedObjectInstances

andcorrectdiscriminationofreleases

SIT_AA_04 SIT_MULTI_RELEASE ChecktheETLiscorrectlydiscriminatingperformancefilesbyrelease.

SIT_AA_05 SIT_MULTI_RELEASE

CheckthatImportKeyshaveNOTchangedbetween

releasesBEFOREenablingtheAdaptor.

Thisisaprioritytestcase.Itisessentialthatunplanned

IMPORTKEYchangesareNOTappliedtoTestorProductionenvironments

toprotectagainstNEduplicationandtheexpensiveoperationtorevertthesituation.

SIT_AA_06 SIT_DUPLICATES_CHECK

Checkifduplicateshaveappearedinthesystem

afterthemulti-releasevtihasbeeninstalled.

Thisisaprioritytestcase.ItisessentialthatunplannedIMPORTKEYchanges

appliedtoTestorProductionenvironmentsaredetected.

SIT_AA_07 SIT_DUPLICATES_CHECKCheckwhethertheClientisshowingduplicatedNENAMES

normallyasaresultofIKChange.

SIT_AA_08 SIT_CALC_PALETTE_CHECK Checktoensurethatoldandnewpalettesaremerged.

SIT_AA_09 SIT_PREVIEW_NE ChecktoensureNEpreviewsarecorrectfortheinterface.

SIT_AA_10 SIT_PREVIEW_NE

ChecktoensureNEpreviewsarecorrectfortheinterface(Negative).

Historically,ithasbeenobservedthatincorrectlydefinedDATETIME

patternsreturndatabyexamplespecifyingtheDATEbutnotthetimecomponent.

SIT_AA_11 SIT_PREVIEW_CONF ChecktoensureAEpreviewsarecorrectfortheinterface.

SIT_AA_12 SIT_PREVIEW_CONF

ChecktoensureNEpreviewsarecorrectfortheinterface(Negative).



SIT_AA_13 SIT_PREVIEW_COUNTER ChecktoensureCEpreviewsarecorrectfortheinterface.

SIT_AA_14 SIT_PREVIEW_COUNTER

ChecktoensureCEpreviewsarecorrectfortheinterface(Negative).



SIT_AA_15 SIT_NE_PALETTE ChecktoensuretheNEPaletteHierarchymatchesthatdefinedinthespecification.

SIT_AA_16 SIT_CALC_PALETTE ChecktoensuretheCLIENTpalettepresentationiscorrect.

SIT_AA_17 SIT_COUNTER ChecktoensuretherawCOUNTERdataisbeingparsedcorrectly.

SIT_AA_18 SIT_CALC_PALETTE ChecktoensuretherawATTRIBUTEdataisbeingparsedcorrectly.

SIT_AA_19 SIT_NIMS_NOM ChecktoensuretheParent-ChildRelationshipsareinplace.

SIT_AA_20 SIT_FLATFILE_SIZECheckflatfilesizepostMRupgradeisreasonableandthatthereissufficent

headroomontheserver(i.e.Flatfilesshouldtakenomorethan75%ofdiskspace).

23

3.2. SIT_AA_01currentprocess

After analyzing thewhole process and due to the limitation of time, the scope of theprojectchangesandthesolutionit’sfocusedonthefirststepofthetestsolution.

As seen in Table 3.1, this first step is in charge of checking the ETL process from thecustomerRemoteServer(wherealltherawdataisstored)totheMYCOMOSIDatabases.

This case checks that the Extract Process is extracting raw file from the remote EMScorrectly,andthedatacomingintoPrOptimaTMdatabaseisconsistent.It’salsoinchargeof checking the file name format, the time stamp (sending and arrival time) and theextensionofthefilesmatchthestructuresetontheTechnicalSpecificationdocument.

Allthisprocedureisdonemanually,sothecostoftimeforthepersoninchargeofthetestingisreallyhigh,andalsointroducesthepossiblehumanerrors.

4. Results

4.1. SITProcesssolutionproposal

AsaresultoftheanalyticstudysomenotesofeverysteparebeingdeliveredtotheR&Ddepartment.Thereisnospecificsolutionforanyofthem(lessforthefirstone,AA_01)butatleastthereisthewaytofollow.

TherearetwodifferenttypesoffcheckingduringtheSIT,theoneswhichconcerntothefilesstoredonthedatabasesandtheETLprocessandothersthattestthatoncethedataisparsed,PrOptimaiscorrectlyupdatedandthisinformationoroptionsareavailablefortheuser.

Regardingonthis,2differentframeworksshouldbedevelopedtocoverallthesteps.Inthisprojecttheframeworkdevelopedcorrespondstofirsttypeexplainedinthepreviewsparagraph.

24

4.2. SIT_AA_01solutionproposal

Asaresultoftheanalysisdone,anautomationframeworkwasdevelopedtryingtosatisfytheneedsofthecompanyinthisconcretearea(SIT).It´struethatthissolutionisfocusedinthefirststepoftheprocessbutfromhere,otherstepscanbeincluded.

Firstofall,therewasacustomerServeraccessingandoccupyingproblem.OnlytheProjectManagerandsomeofthequalifiedengineersinvolvedoneveryprojecthavepermissiontoenteronthecustomerDatabaseandlikeinmostoftheTelecomcompanies,thispeopleareinvolvedinotherprojectsatthesametime.SoitisnoteasytoscheduleanSIT,takingintoaccountthatthepersoninchargeoftheprocesshastobefullyavailableforatleast3days(normalexecutionisbetween3and5days,dependingonthecomplexityofthenetworkandtheamountofdata).Intheotherhand,beinginsidetheRemoteServercouldaffecttothecorrectoperationofthesystemandotherassociatedissues.

Tosolvethis,insteadofenteringeverytimetotheRemoteServertocheckthefiles,forthisfirststepweonlyneedsomespecificinformationofthem.ThesolutionproposedwastoenteronetimetotheServer(allofthemareinUnixOS),findthedesiredfiles,listthemwiththeappropriatedcommand(ls–lhtr)andsavethislistintoatxtorcsvfile.Bydoingthis,bothproblemsarealmostsolved.

Anyonewithaccesstotheservers(MYCOMOSIorCustomerpeople)cangeneratethisfiles,andanyofthecompanyqualifiedengineerscanpasstheSIT,sothedependenceofthePMortheothersinvolvedintheprojectisnofurtheranissue.Italsosolvesthesecondproblem,theoccupationoftheServer,becausetheServerisonlyonceconsulted.Afterallthe needed files for the test are saved, there is nomore need to access again to theDatabase.

Anexampleoffileusedforthecodeisshowninthenextfigure:

25

Figure3:Exampleofthecontentinausedfile

Then,theother importantaspecttobesolvedwastheautomationoftheprocess.Thiswasbyfarthemostdifficultpartofthesolutionformanyreason.

First,theorganizationofthefilesarereallydifferentforeverycustomerandtechnologyandasthenetworksbecomemorecomplexthestructureofthefilesincreaseinlevelsandquantity.Sothefirstideaofdevelopingascriptthatcouldfetchautomaticallythefileswasdiscarded.AunifiedmethodologyofstoragewasproposedtotheR&Ddepartment forfurtherimprovementsinthisdirection.

Then,asimilarproblemhappenedforthefilenamestructure.Everycustomerchoosesthenamingofthefiles,soisreallydifficulttodevelopascriptthatcouldcompareeachofthefileswiththenameconventionagreedintheTechnicalSpecificationDocument.

In order to solve this, a unique file name standard should be used. As the solution isdeveloped in Perl and for the actual requirements (name structure, time stamp andextension)the3GPPfilenamingconventionfitsperfectly.

26

<Type><Startdate>.<Starttime>-[<Enddate>.]<Endtime>[_-<jobId>][_<UniqueId>][_-_<RC>]

v Thewholenamingstandardit´sfurtherexplainedontheannex,buthereitisabriefexplanation:The first fieldcontains the results for singleormultipleNEsand thegranularityperiods.

v TheStardatefieldindicatesdedataofthebeginningofthegranularityorthefirstgranularityperiod.

v TheStarttime,indicatesinthesamewayastheStartdatefield,thetime.v Endtime and Enddate correspond to the finishing date and time periods of

granularity

So theautomation frameworkwasdeveloped inPerlcodeand it’sbased in3differentmodules(Preview,ComparisonandReport).

Everytimetheuserselectsoneoftheoptions,theprogramasksforthepathofthefileslocateonthesamecomputerasthescriptisrunning.

Thefirstoption,PreviewMode,asksforonefileandafterenteringthecorrectpathgivesdifferentoptionstochoose.Size,Month,Day,TimeorYear,FileName,FileExtensionortheCompleteLinearethe7optionstobepreviewedonthescreenbytheuser.Whenanoptionischosen,thescriptshowsanumberedlistbyrowswiththedifferentdataandafinalcounterofthetotalnumberofrows.

OntheComparisonMode,theuserisaskedfortwofilepaths.Afterenteringthecorrectones,thescriptshowsthesame7optionstotheuser,butthistimetheoutputis,asthenameofthemodulesays,acomparisonbetweenrowsinbothfiles.Theoutputisshownonthescreenandsortedbyrows,givinginformationevenifthedataarethesameorincasetheyaredifferent,thedifferencesbetweenthem.AsinthePreviewmodule,afinalcounterwiththetotaloffilesisshown.

Thelastoption,theReportMode,isalmostthesameastheComparisonMode,butinthiscase all the options are selected automatically and only when there are differencesbetweenfilestherowisprinted.AnothercharacteristicofthismodeisthattheoutputisshownonthescreenbutitalsogeneratesacsvortxtfiletosavetheReport.Afterenteringthefilepaths,thescriptaskstheuserforapathtosavetheoutputfile.

With this frameworkanddeveloping littlemodificationson thenameoption iseasy toaccomplish the target of this Step. For now, if the company do not agree to urge thecustomers to use one unique naming standard, the solution proposed is the mostautomatedoneandalthoughthisframeworkisnotcompletelyreadytouse,itcouldbethewaytoleanonfordevelopingausefulautomatedtestingtool.

27

4.3. Nextstepssolutionproposal

Oncethenamingstandardischosen,iseasytoincludemorestepstotheactualframeworkwithnomuchprogramingeffort.

AA_02andAA_03/AA_04canbeautomatedbyaddingalways inthesamepositionthereleasetagtothefilename,makingeasytofilterandcomparewiththevalidreleasesoftheadaptor.

AA_20isinchargeofcheckingthefilesizesbytechnologieswiththatvaluebelowacertainthreshold.Withasmallchangeonthesizeoptionoftheframework,asumcountercouldbeeasilyintroducedandthisstepwouldbepassed.

5. Budget

ThefundsforthisprojecthavebeenmanyhoursofworkandthePerlprogramingcoursewhichhasallowedmetowritethecodeinthesamelanguageusedfortheprojectsinthecompany.MYCOMOSIprovidestheprograms,trainingsandaccesstothedifferentservers.Ifyouweretofinancethisprojectfromscratch,thiswouldbeitscosts:v PerlCourse60€v 720hoursx10euros/hour(estimatedpriceofajuniorengineer)=7200euros.Totalcost:7260euros.

28

6. Conclusions

Theaimoftheprojectwasalwaystodevelopanautomationframeworkforoneofthetestingprocessesofthemainsoftwareonthecompany.Afterworkingreallyhardonthissolutionwithnomuchtime,doingeverythingfromthescratchandwithlittlesupportonthecreativearea,Ithinkthatthefinalresultsarequiteconsistent.

TheframeworkdevelopedisabletotestthestepsthatinvolvetheETLprocessandtheDatabasefiles,whicharetheonesthatrequiresmoretimeandeffort.

ThiswasmyfirsttimecodingoutsideoftheUniversityandwithacompletelynewcodinglanguage,withthedifficultythatconcerns.

I’vebeenworkingwithdifferentdepartmentsfromallovertheglobe,workingaloneorinateam,learningalotfromthecompanybusinessandtheirmethodology.Allofthistotrytoreflexitlatteronthisproject.

NowtheR&Ddepartment,whichisinchargeofimprovingandadaptingthesoftwaretotheirneeds,hasaframeworktostartworkingwithandabigamountofnotesandanalysisofthecurrentprocess.

Tosummarize,it´sbeingagreatprojecttostartmycarrieroutsideoftheUniversity,whereI learned a lot of different things relatedwith the Telecom industry and gaveme thecapabilitiesandtheconfidencetoworkinanyotherplaceandinalmostanyotherproject.

SIT Automation Tool: Failure Use Case ... - UPCommons

Documents

Transcript of SIT Automation Tool: Failure Use Case ... - UPCommons