Db2 for Linux, Unix, and Windows - IBM

Db2 for Linux, Unix, and Windows Version 11+ Highlights

Actionable Insights Continuous Availability

Massive Scalability Outthink the Possible

Db2forLinux,Unix,andWindows:Version11HighlightsCopyright©2017byInternationalBusinessMachinesCorporation(IBM).Allrightsreserved.PrintedinCanada.ExceptaspermittedundertheCopyrightActof1976,nopartofthispublicationmaybereproducedordistributedinanyformorbyanymeans,orstoredinadatabaseorretrievalsystem,withoutthepriorwrittenpermissionofIBM,withtheexceptionthattheprogramlistingsmaybeentered,stored,andexecutedinacomputersystem,buttheymaynotbereproducedforpublication.Thecontentsofthisbookrepresentthosefeaturesthatmayormaynotbeavailableinthecurrentreleaseofanyproductsmentionedwithinthisbookdespitewhatthebookmaysay.IBMreservestherighttoincludeorexcludeanyfunctionalitymentionedinthisbookforthecurrentreleaseofDb211.1,orasubsequentrelease.Inaddition,anyclaimsmadeinthisbookarenotofficialcommunicationsbyIBM;rather,theyareobservedbytheauthorsinunauditedtestingandresearch.TheviewsexpressedinthisbookisthoseoftheauthorsandnotnecessarilythoseoftheIBMCorporation;botharenotliableforanyoftheclaims,assertions,orcontentsinthisbook.IBM'sstatementsregardingitsplans,directions,andintentaresubjecttochangeorwithdrawalwithoutnoticeandatIBM'ssolediscretion.Informationregardingpotentialfutureproductsisintendedtooutlineourgeneralproductdirectionanditshouldnotbereliedoninmakingapurchasingdecision.Theinformationmentionedregardingpotentialfutureproductsisnotacommitment,promise,orlegalobligationtodeliveranymaterial,codeorfunctionality.Informationaboutpotentialfutureproductsmaynotbeincorporatedintoanycontract.Thedevelopment,release,andtimingofanyfuturefeatureorfunctionalitydescribedforourproductsremainsatoursolediscretion.PerformanceisbasedonmeasurementsandprojectionsusingstandardIBMbenchmarksinacontrolledenvironment.Theactualthroughputorperformancethatanyuserwillexperiencewillvarydependinguponmanyfactors,includingconsiderationssuchastheamountofmultiprogrammingintheuser'sjobstream,theI/Oconfiguration,thestorageconfiguration,andtheworkloadprocessed.Therefore,noassurancecanbegiventhatanindividualuserwillachieveresultssimilartothosestatedhere.U.S.GovernmentUsersRestrictedRights-Use,duplicationordisclosurerestrictedbyGSAADPScheduleContractwithIBM.InformationinthiseBook(includinginformationrelatingtoproductsthathavenotyetbeenannouncedbyIBM)hasbeenreviewedforaccuracyasofthedateofinitialpublicationandcouldincludeunintentionaltechnicalortypographicalerrors.IBMshallhavenoresponsibilitytoupdatethisinformation.THISDOCUMENTISDISTRIBUTED"ASIS"WITHOUTANYWARRANTY,EITHEREXPRESSORIMPLIED.INNOEVENTSHALLIBMBELIABLEFORANYDAMAGEARISINGFROMTHEUSEOFTHISINFORMATION,INCLUDINGBUTNOTLIMITEDTO,LOSSOFDATA,BUSINESSINTERRUPTION,LOSSOFPROFITORLOSSOFOPPORTUNITY.

IBMproductsandservicesarewarrantedaccordingtothetermsandconditionsoftheagreementsunderwhichtheyareprovided.ReferencesinthisdocumenttoIBMproducts,programs,orservicesdoesnotimplythatIBMintendstomakesuchproducts,programsorservicesavailableinallcountriesinwhichIBMoperatesordoesbusiness.Informationconcerningnon-IBMproductswasobtainedfromthesuppliersofthoseproducts,theirpublishedannouncementsorotherpubliclyavailablesources.IBMhasnottestedthoseproductsinconnectionwiththispublicationandcannotconfirmtheaccuracyofperformance,compatibilityoranyotherclaimsrelatedtonon-IBMproducts.Questionsonthecapabilitiesofnon-IBMproductsshouldbeaddressedtothesuppliersofthoseproducts.IBMdoesnotwarrantthequalityofanythird-partyproducts,ortheabilityofanysuchthird-partyproductstointeroperatewithIBM'sproducts.IBMEXPRESSLYDISCLAIMSALLWARRANTIES,EXPRESSEDORIMPLIED,INCLUDINGBUTNOTLIMITEDTO,THEIMPLIEDWARRANTIESOFMERCHANTABILITYANDFITNESSFORAPARTICULARPURPOSE.Theprovisionoftheinformationcontainedhereinisnotintendedto,anddoesnot,grantanyrightorlicenseunderanyIBMpatents,copyrights,trademarksorotherintellectualpropertyright.IBM,theIBMlogo,ibm.com,Aspera®,Bluemix,BlueworksLive,CICS,Clearcase,Cognos®,DOORS®,Emptoris®,EnterpriseDocumentManagementSystem™,FASP®,FileNet®,GlobalBusinessServices®,GlobalTechnologyServices®,IBMExperienceOne™,IBMSmartCloud®,IBMSocialBusiness®,InformationonDemand,ILOG,Maximo®,MQIntegrator®,MQSeries®,Netcool®,OMEGAMON,OpenPower,PureAnalytics™,PureApplication®,pureCluster™,PureCoverage®,PureData®,PureExperience®,PureFlex®,pureQuery®,pureScale®,PureSystems®,QRadar®,Rational®,Rhapsody®,SmarterCommerce®,SoDA,SPSS,SterlingCommerce®,StoredIQ,Tealeaf®,Tivoli®,Trusteer®,Unica®,urban{code}®,Watson,WebSphere®,Worklight®,X-Force®andSystemz®Z/OS,aretrademarksofInternationalBusinessMachinesCorporation,registeredinmanyjurisdictionsworldwide.OtherproductandservicenamesmightbetrademarksofIBMorothercompanies.AcurrentlistofIBMtrademarksisavailableontheWebat"Copyrightandtrademarkinformation"at:www.ibm.com/legal/copytrade.shtml.AlltrademarksorcopyrightsmentionedhereinarethepossessionoftheirrespectiveownersandIBMmakesnoclaimofownershipbythementionofproductsthatcontainthesemarks.InitialPublication:October23,2016Updated:May16,2017Updated:June20,2017Updated:October2,2017

Revisions

InitialPublication:October23,2016

Db211.1.1.1:May16,2017SomeFixpack1(Mod1)featureswereaddedtothebook.Changesinthebookaremarkedwiththenumber�atthebeginningofthesection.Correctionswerealsomadetosometextandexamplesbutarenotmarked.Db211.1.2.2:June20,2017SomeFixpack2(Mod2)featureswereaddedtothebook.Changesinthebookmarkedwiththenumber�atthebeginningofthesection.ThisversionoftheeBookalsointroducesthenewspellingofDb2.Itisnotaspellingmistake!Db211.1.2.2:October2,2017Someadditionalinformationaddedonneweditions,packaging,andfeaturesinthelatestfixpack.

About the Authors

GeorgeBaklarz,B.Math,M.Sc.,Ph.D.Eng.,hasspent31yearsatIBMworkingonvariousaspectsofdatabasetechnology.GeorgewaspartoftheteamthathelpedmovetheOS/2ESdatabasetoTorontotobecomepartoftheDb2familyofproducts.Hehasworkedonvendorenablement,competitiveanalysis,productmarketing,productplanning,andtechnicalsalessupport.Georgehaswritten10booksonDb2andotherdatabasetechnologies.GeorgeiscurrentlypartoftheWorldwideCoreDatabaseTechnicalSalesTeam.EnzoCialini,B.Sc.,isaSeniorTechnicalStaffMemberandMasterInventorintheWorldwideCoreDatabaseTechnicalSalesTeamandformerlytheChiefQualityAssuranceArchitectforDb2&PureDataintheIBMTorontoDb2DevelopmentTeam.HeisalsoapublishedbookauthorandwrittenvariouspapersonDb2.Enzohas25yearsofexperienceindatabasetechnology,softwaredevelopment,testing,support,competitiveanalysisandproductiondeployments.

Forward

Theneweconomyischangingthewaywegatherinformation,managedata,gaininsights,reinventourbusinesses,anddosoquicklyanditeratively.Thedigitaltransformationofeverythingischangingtheinformationvaluechainnotjustfromyourownorganization,butfromstakeholdersandthird-partydataproviders.Andwiththisnewinformationandincontextfromwhichthey'regathered,wegainadeeperunderstandingofclientbehaviorsandmarketconditions,andformulatefasterresponsestochangingcompetitivelandscape.Theadventofcloudandmobilecomputingfurtherencouragesthedistributionandconsumptionofinformationatarapidpace,whichinturngeneratesmoreinformationtoanalyze.Therefore,thisneweconomyrequiresarobustdatabasesoftwarethatsitsattheheartofanenterprisetosupportitstransformationinthisneweraofdigital,cloudandcognitive.Db2isaversatileandscalabledatabasethatcansupportyourtransactionalandanalyticalworkloadsfordifferentapplicationswhetheronpremisesorintheclouduptopetabytesinvolume.Itiseasytodeploy,upgradeandmaintainandatthesametimeitsimplifiesthemovefromOracledatabasesbyleveragingtheSQLCompatibilitycomponentalongwithyourexistingskillsandassetswithminimalchangestotheapplication.Db2'spureScalecapabilityensuresthatyourbusinessdataisalwaysavailableandaccessiblewithoutanyinterruptionstoyourbusinessprocesses,ensuringthatyourconsumersarenotimpactedwherevertheymaybe.

WithBLUMPP,youleveragemultiparallelprocessingwithin-memorycolumnartechnologytoprovideextremeperformanceenhancementsforyouranalyticworkloads.Finally,Db2providestheflexibilityofdeployingonpremiseoronthecloudwithintegratedsecuritytoensurethatyournewbusinessprocessapplicationsanddataarewheretheymakesense,nearyourconsumers.SeveralotherdistributeddatabaseofferingsareavailablefromIBMinamanagedenvironment,hostedenvironment,virtualized,usingyourowndatacenter,oronanappliance:

• Db2Warehouse(LocalDockerContainer)• Db2WarehouseonCloud(ManagedWarehouseService)• Db2onCloud(ManagedOLTPService)• Db2Hosted• Informix• InformixonCloud• PureDataSystemforAnalytics• IBMIntegratedAnalyticsSystem• Cloudant• IBMDb2EventStore

Nomatterwhatyourrequirementsare,IBMhasadatabaseofferingthatwillmeetyourneeds.

Packaging,Installation,andAdministration..................................................2EndofServiceandMarketing...............................................................................2NewIncludedFeatures.........................................................................................3

Federation.........................................................................................................3Encryption.........................................................................................................3pureScaleAdministrationFeature.....................................................................4

Db2ExpressandDb2ExpressCommunityEditions..............................................6Db2WorkgroupandEnterpriseEditions...............................................................7AdvancedWorkgroupandAdvancedEnterpriseEditions....................................7AdditionalProductsforDb2..................................................................................8

AdvancedRecoveryFeature..............................................................................8Db2PerformanceManagementOffering.........................................................8

Db2DirectEditions................................................................................................9�Db2Developer-C(Community)Edition...........................................................10�Db2DeveloperCommunityEdition.................................................................11OperatingSystemSupport..................................................................................11VirtualizationSupport.........................................................................................12UpgradeSupport.................................................................................................13AdministrationImprovements............................................................................14

RangePartitionReorganization......................................................................14ADMIN_MOVE_TABLE....................................................................................14RemoteStorage..............................................................................................15Db2HistoryFileBackup..................................................................................15HardwareBackupCompression......................................................................15

Db2BLUEnhancements..............................................................................18MassivelyParallelProcessingArchitecture.........................................................19FastCommunicationManager............................................................................20FasterSQLMERGEprocessing.............................................................................21NestedLoopJoinSupport...................................................................................21SortProcessingEnhancements...........................................................................22QueryRewriteEnhancements............................................................................23Push-downofOLAPfunctions.............................................................................24SQLFunctionsOptimizedforBLU.......................................................................24AutomaticDictionaryCreation............................................................................25

ContinuousAvailability................................................................................27pureScaleArchitecture........................................................................................27Deployments.......................................................................................................28Virtualization.......................................................................................................31Installation...........................................................................................................31WorkloadBalancing............................................................................................32

Contents ix

Failoverbehavior.................................................................................................34Failbackbehavior................................................................................................35Manageability......................................................................................................35

Pre-Installation................................................................................................35Post-Installation..............................................................................................36

DisasterRecovery................................................................................................36HADR...............................................................................................................36GeographicallyDispersedDb2pureScaleCluster(GDPC)...............................37

CompatibilityFeatures................................................................................40OuterJoinOperator............................................................................................40CHARdatatypesizeincrease...............................................................................41BinaryDataTypes................................................................................................41�BooleanDataTypes.........................................................................................42SynonymDatatypes............................................................................................43FunctionNameCompatibility..............................................................................44

NetezzaCompatibility..................................................................................47SpecialOperatorCharacters...............................................................................47GROUPBYOrdinalLocation................................................................................48TRANSLATEFunction...........................................................................................49

SQLExtensions.............................................................................................52SimpleSQLwithFetchFirstClause.....................................................................52OFFSETExtension................................................................................................53FETCHFIRSTandOFFSETinSUBSELECTs.............................................................54AlternateSyntaxforFETCHFIRST.......................................................................56TableCreationwithSELECT.................................................................................57CREATEFUNCTIONforAggregateUDFs..............................................................58

DateFunctions.............................................................................................61ExtractFunction..................................................................................................61DATE_PARTFunction...........................................................................................62DATE_TRUNCFunction........................................................................................63ExtractingSpecificDaysfromaMonth...............................................................65DateAdditionFunctions......................................................................................66ExtractingWeeks,Months,Quarters,andYearsfromaDate............................67NextDayFunction...............................................................................................68BetweenDate/TimeFunctions............................................................................69MONTHS_BETWEENFunction.............................................................................70DateDurationFunctions.....................................................................................71OVERLAPSPredicate............................................................................................72UTCTimeConversions.........................................................................................73UsingUTCFunctions............................................................................................74

Contents x

HexFunctions..............................................................................................80INTNFunctions....................................................................................................80TO_HEXFunction.................................................................................................81RAWTOHEXFunction...........................................................................................81

RegularExpressions.....................................................................................83RegularExpressionExamples..............................................................................83RegularExpressionCommands...........................................................................84RegularExpressionFlagValues...........................................................................85RegularExpressionSearchPatterns....................................................................85AnchoringPatternsinaSearch...........................................................................86Matchingpatternsacrossmultiplelines.............................................................87LogicalOROperator............................................................................................88CombiningPatterns.............................................................................................89Matchingcharactertypes...................................................................................93SpecialPatterns...................................................................................................96NegatingPatterns................................................................................................97CapturingParenthesis.........................................................................................99

StatisticalFunctions...................................................................................103SamplingFunctions...........................................................................................103AdditionalAnalyticalFunctions.........................................................................104WIDTHBUCKETandHistogramExample..........................................................106

HashingFunctions......................................................................................109HASH4-Four-byteHashEncoding...................................................................109HASH8-Eight-byteHashEncoding...................................................................110HASH-VariableHashEncoding.........................................................................111

InternalDb2JSONRoutines......................................................................113Db2JSONFunctions..........................................................................................114PathStatementRequirements..........................................................................114CreatingTablesthatSupportJSONDocuments................................................115JSONDocumentRepresentation.......................................................................116JSON2BSON:InsertingaJSONDocument.........................................................116BSON2JSON:RetrievingaJSONDocument.......................................................117DeterminingDocumentSize..............................................................................118BSON_VALIDATE:CheckingtheFormatofaDocument....................................118RetrievingJSONDocuments..............................................................................119

SampleJSONTableCreation.........................................................................119AdditionalJSON_DEPTTable.........................................................................120

JSON_VAL:RetrievingDatafromaBSONDocument........................................121RetrievingAtomicValues..............................................................................122RetrievingArrayValues.................................................................................123

Contents xi

RetrievingStructuredFields..........................................................................124DetectingNULLValuesinaField...................................................................124JoiningJSONTables.......................................................................................125

JSONDataTypes................................................................................................126ExtractingFieldsUsingDifferentDataTypes....................................................128

JSONINTEGERSandBIGINT..........................................................................128JSONNUMBERSandFLOATINGPOINT.........................................................130JSONBOOLEANVALUES................................................................................132JSONDATE,TIME,andTIMESTAMPS............................................................133JSONStrings..................................................................................................134

JSON_TABLEFunction.......................................................................................135JSON_LENFunction...........................................................................................138JSON_GET_POS_ARR_INDEXFunction..............................................................139UpdatingJSONDocuments...............................................................................140IndexingJSONDocuments................................................................................142SimplifyingJSONSQLInsertsandRetrieval.......................................................143

Summary....................................................................................................146

ResourcestoBuildYourDb2Skills............................................................148IBMCertificationExams....................................................................................148IBMTraining......................................................................................................148BigDataUniversity.............................................................................................148InformationManagementBookstore...............................................................149IBMSupportforDb2.........................................................................................149IBMDataMagazine...........................................................................................149InternationalDb2UserGroup(IDUG)...............................................................149JointheConversation........................................................................................150AdditionaleBookMaterial................................................................................150

Acknowledgments

Wewanttothankthefollowingpeoplewho,inonewayoranother,contributedtothisbook:

• MichaelSpringgay,KellySchlamb,PaulBird,MattHuras,CalistoZuzarte,JohnHornibrook,GregStager,andAlanLee

Wewouldalsoliketothankallthedevelopmentteamforhelpingtodeliverthisreleasegiventhetremendousdeadlinesandconstraintsthattheyhavebeenunder.ThecontentsofthiseBookaretheresultofalotofresearchandtestingbasedonthecontentsofourDb211.1KnowledgeCenter.Theauthorsofouronlinedocumentationdeservespecialthanksforgettingthedetailstousearlyintheproductdevelopmentcyclesowecouldbuildmuchofourmaterial.Forthemostup-to-dateinformationonDb211.1features,pleaserefertotheIBMKnowledgeCenter:http://www.ibm.com/support/knowledgecenter/SSEPGGThereyoucanselecttheVersion11.1.0releasefromthedrop-downmenuandgetmoredetailsonthefeaturesweexploreinthiseBook.

Introduction

AboutThisBookTheDb211.1releasedeliversseveralsignificantenhancementsincludingDatabasePartitioningFeature(DPF)forBLUcolumnartechnology,improvedpureScaleperformanceandfurtherHighAvailabilityDisasterRecovery(HADR)support,andnumerousSQLfeatures.Thisbookwaswrittentohighlightmanyofthenewfeaturesandfunctionsthatarenowavailableinthisrelease,withoutyouhavingtosearchthroughvariousforums,blogs,andonlinemanuals.WehopethatthisbookgivesyoumoreinsightintowhatyoucannowaccomplishwithDb211.1,andincludeitonyourshortlistofdatabasestodeploy,whetheritisonpremise,inthecloud,orahybridapproach.GeorgeandEnzo

HowThisBookIsOrganizedWeorganizedthisbookinto11chaptersthatcovermanyofthehighlightsandkeyfeaturesfoundintheDb211.1release.

• Chapter1givesyouanoverviewofthenewpackagingchangesinDb211.1,alongwithpre-requisitesforinstallingtheproduct

• Chapter2discussestheenhancementstoDb2BLU,includingsupportforBLUcolumnartablesinaDPF(DatabasePartitioningFeature)environment

• Chapter3examineswhatpureScalecapabilitiesprovideyouinaproductionenvironment,alongwithalltheenhancementsthathavebeenmadetosimplifyinstallationandmanagementofacluster

• Chapter4through12arefocusedonSQLandcompatibilityenhancements

1

Packaging,

Installation, and Administration

ENHANCEDLICENSINGANDDEPLOYMENTOPTIONSFORDB2

Chapter1:Packaging,Installation,andAdministration 2

Packaging,Installation,andAdministration

Db211.1introducedseveralpricingandpackagingchangesthatwillaffectsomecustomers.Inaddition,theendofserviceofDb29.7andDb210.1wereannounced,whichmeansthatcustomerswitheitherofthosereleaseswillneedtostartmakingplanstoupgradetoanewversionofDb2overthenextyear.

EndofServiceandMarketingThreeadditionalannouncementsweremaderegardingtheendofmarketingandtheendofserviceforolderDb2releasesandfeatures.Db210.5endofmarketing,onSeptember30th,2016

• Db29.7andDb210.1endofservice,onSeptember30th,2017• DiscontinuedBusinessValueOfferings(BVO)

EndofmarketingmeansthatcustomerswillnotbeabletopurchaseDb210.5aftertheendofSeptember2016.ThismeansthatyoucanonlypurchaseDb211.1now,butyouarestillabletogetcopiesofDb210.5ifrequired.CustomerscontinuetogetserviceandsupportforDb210.5sinceendofserviceisnotbeingannouncednow.ForthosecustomersrunningeitherDb29.7orDb210.1,theywillneedtobeginplanningonupgradingtoeitherDb210.5orDb211.1.ExtendedservicecontractswillbeavailabletomaintainyourolddatabaseimagesafterSeptember2017,buttherewillbeanadditionalchargetodothat.ForthosecustomerswhopurchasedanyoftheBLUAccelerationBusinessValueOfferings,theywillbeupgradedtoanAdvancedlicenseofDb2dependingonwhichlicensetheyhadwhentheypurchasedtheBVO.TheEncryptionOfferingandtheBusinessApplicationContinuityOfferingarebundledwitheveryeditionofDb211.1,socustomersthatpurchasedthatfeaturewillnolongerhavetopayserviceandsupportonitwhenupgradingtoDb211.1.


NewIncludedFeaturesOnesignificantchangetoDb2packagingisthatallDb2Editions(Workgroup,Enterprise,AdvancedWorkgroupandAdvancedEnterprise,DigitalStandard,DigitalAdvanced)willincludeEncryption,Federation,andpureScale.WhiletherearelimitationsontheuseofFederationandpureScaleinWorkgroup,Enterprise,andDigitalStandardEditions,theEncryptionfeatureisfullysupportedacrossalleditions.

Federation

Db211.1nowincludesdatafederationandvirtualizationthrougharobustSQLinterfaceandafullrelationaldatabaseenginedesignedforglobaloptimizationofdataaccess.Thiscapabilityoffershomogeneousorheterogeneousfederation(dependingontheDb2Edition)betweenselectIBMandnon-IBMdatasources.Youcanquicklyprototypeandimplementsolutionsinvolvingdisparatedatabyvirtualizingaccessandoptimizingperformance.Datacanremaininplaceforbothtransactionalaswellasanalyticalworkloads.

Figure1:FederationSupport

Db2AdvancedServerEditionsincludeallofthewrappersthatarepartofthefederationsupport.TheDb2WorkgroupandEnterpriseEditionproductsonlyincludesupportforconnectingtoDb2andInformixdatasources.

Encryption

Db2includedencryptionatthedatabaselevelinVersion10.5.However,theencryptionfeaturerequiredtheuseofalocalkeystoretocontainthe


masterkeyforthedatabase.Thisrequiredthatthekeystoreitselfbebackedupandmanagedindependentlyfromthedatabasebackup.

Figure2:EncryptionSupport

Ratherthanmanageseveralseparatekeystoresmanually,customerscannowuseanyKMIP1.1centralizedkeymanagertomanagetheDb2keys.TheISKLMproduct(IBMSecurityKeyLifecycleManager)hasbeenvalidatedtoworkwithDb2,butanyKMIP1.1compliantmanagerissupported.ISKLMhelpsmeetregulationsandstandardssuchas:

• PaymentCardIndustryDataSecurityStandard(PCIDSS)• Sarbanes-Oxley• HealthInsurancePortabilityandAccountabilityAct(HIPAA)

UsingthePKCS#11protocol,Db2NativeEncryptionwillbeenhancedtosupportusingHardwareSecurityModules(HSM)asthekeystoreforthemasterkey.TheinitialsetofHSMproductsvalidatedwillincludeSafenetLunaSAandThalesnShieldConnect+,butanyPKCS#11complaintproductissupported.�HSMsupportisnowavailableinDb211.1.1.1.Moredetailedinformationonthesupportedenvironmentscanbefoundinthefixpacksupportinformationon:http://www.ibm.com/support/knowledgecenter/SSEPGG_11.1.0/com.ibm.db2.luw.kc.doc/welcome.html

pureScaleAdministrationFeature

Db211.1Workgroup,Enterprise,andDigitalStandardEditionincludethepureScalefeature,limitedtotwomemberclusters.CustomerswhodonotrequirethescalabilityofpureScalebutrequirethebenefitsofcontinuousavailability,candosonowatlowercosts.


Thelimitedtwo-memberpureScaleclusterconfigurationenablesonemembertoprocessapplicationworkloads,andasecondmembertoprovidecontinuousavailabilitywhenthefirstmemberisundergoingplannedorunplannedoutages.Inaddition,thesecondmembercanalsobeusedtoperformadministrativetasksandutilities,thusoff-loadingthesetasksfromtheprimarymember.Thefirstmemberisfullylicensed,whilethesecondmemberonlyrequiresminimallicensingsimilartoanHADRenvironment(licensedaswarmstandby,e.g.100PVUsor1VPC).

Figure3:pureScale2-nodeconfiguration

Allapplicationworkloadsaredirectedtotheprimaryactivemember(s),sometimesreferredtoasthe"primary"memberwhileutilitiesandadministrativetasksareallowedonthesecondarymember.Thisconfigurationisidealforoff-loadingbackupsfromprimarymembersaswellasensuringcontinuousavailabilityintheeventofamemberfailure.Administrativetasksandutilities,however,areallowedtorunonthesecondarymember.Nothingispreventingthesameadministrativeworkfromrunningontheprimarymembersaswell,butthebestpracticeisto


keepadministrativetaskstothesecondarymember-thus,freeingtheprimarymemberresourcesforprocessingtheworkload.Thisisanactive/activeclusterasworkisbeingdoneonALLmembersoftheclusterbutusingan'active/passive'licensingmodel.Thesame'active/passive'licensingmodelisavailabletobeappliedtothefullyscalableDb2pureScaleactive/activeconfigurationavailableinDb2AdvancedEditions,includingthenewDirectAdvancedEdition,allowingforoff-loadingofadministrationtasksandutilitiesinacontinuouslyavailableandscalable>2-membercluster.

Db2ExpressandDb2ExpressCommunityEditionsForthoseuserslookingtodevelop,deployanddistributeDb2atNOchargeshouldlookatDb2Express-C.Db2Express-CEditionisano-chargecommunityeditionofDb2availableforWindowsandLinuxplatforms,whichprovidesthecorefeaturesofDb2Editions.Formoreadvancedfeatures,suchasDb2pureScaleandBLUAcceleration,thesolutioncanbeupgradedwithoutrequiringmodificationtoyourapplications.Db2Express-Coffersthesefeaturesandbenefits:

• Encryptionfordataatrestandin-transittoprotectyourdata• UsesTimeTravelQuerytoprovidebi-temporaldatamanagement• DeliversanintegratedtoolsenvironmentwithIBMDataStudiofor

easiermanagementanddevelopment• ContainstheSQLcompatibilityfeaturetomoreeasilyrun

applicationswrittenforotherdatabasesonDb2• ContinuousDataIngest• Federation• pureXMLstorage• JSONNoSQL

ThelicensedDb2ExpressEditionisnolongerofferedinDb211.1.ThosecustomersthatcurrentlyhaveDb2ExpresswillbeupgradedtotheDb2WorkgroupEdition.


Db2WorkgroupandEnterpriseEditionsBothDb2WorkgroupandEnterpriseEditionshavehadseveraladditionalfeaturesincludedintothepackaging:

• pureScaleStandbyMemberOption• Tablepartitioning,Encryption• Federation(Db2&Informix)

Inaddition,Db2WorkgroupEditionhashadfurtherfeaturesaddedwhicharealreadyincludedinDb2EnterpriseEdition

• Multi-dimensionalClustering• Multi-TempStorage• QueryParallelism

TherearesomeexclusionstowhatisincludedintheWorkgroupandEnterpriseeditions.Thelistincludes(butitnotlimitedto):

• DataPartitioning• SQLWarehouse(SQW)• MaterializeQueryTables(includedinEnterpriseEdition)• BLUAcceleration,Compression

TheonebigchangeinthelicensingofDb2WarehouseEditionisthatthereisnolongeracaponthesizeofdatabase.Thecorelimitremainsthesameat16cores,buttheamountofmainmemoryhasincreasedto128GB.Therearenolimitsonthecores,memoryordatabasesizewithDb2EnterpriseEdition.TherearetwooptionalproductsthatcustomersmaywanttopurchaseforWorkgroupandEnterpriseEdition:

• Db2PerformanceManagementOffering(WLM)whichispartoftheDataServerManagerEnterpriseEdition

• AdvancedRecoveryFeature

Theseproductsaredescribedinanothersection.

AdvancedWorkgroupandAdvancedEnterpriseEditionsAswiththepriorreleaseofDb210.5,theAdvancedEditionsofWorkgroupandEnterpriseEditionincludeallfeaturesforonesingleprice.WhatisnewinDb211.1isthenewFederatedfeaturewhich


includesconnectivitytoDb2,Informix,andmanyotherrelationalandunstructureddatasources. TherearethreeproductsthatareincludedaspartoftheAdvancedEditionsthathavelimitationonusage.Theseproductsare:

• Db2ConnectincludedforusingSQWtoolingtoaccessDb2forzandDb2fori

• InfoSphereDataArchitectlimitedto10authorizedusers• CognosAnalyticslimitedto5authorizedusers

TheonlyoptionalfeaturethatcustomersmaywanttoconsideristheAdvancedRecoveryFeature.

AdditionalProductsforDb2InconjunctionwiththeDb211.1databaserelease,severaltoolswerereleasedthatareincludedintheAdvancedEditionsofDb2,andavailableforpurchasewiththeothereditions.

AdvancedRecoveryFeature

Thisfeatureconsistsofadvanceddatabasebackup,recovery,anddataextractiontoolsthatcanhelpyouimprovedataavailability,mitigaterisk,andacceleratecrucialadministrativetaskswhentimeisoftheessence.NewfeaturesofthetoolsincludesupportforDb211.1,includingusingthetoolswithDb2NativeEncryption.TheAdvancedRecoveryFeaturebundlecontainsthefollowingnewreleasesoftheincludedtools:

• Db2MergeBackupforLinux,UNIX,andWindowsV3.1• Db2RecoveryExpertforLinux,UNIX,andWindowsV5.1• OptimHighPerformanceUnloadforDb2forLinux,UNIX,and

WindowsV6.1

Db2PerformanceManagementOffering

TheDb2PerformanceManagementOffering(WLM)ispartoftheDataServerManagerEnterpriseEdition.Db2PerformanceManagementOfferinghelpsorganizationsmanagetheirDb2databasesacrossthe


enterprisethroughtoolingthathasacommonintegratedconsole,enhancedsimplicity,improvedperformanceandscalability,andsimplifiedworkflowwithactionablerecommendations.Organizationscandothefollowing:

• Simplifydatabaseadministrationandreduceerrorswithintegratedwebconsoletoviewandmonitorthehealthofdatabasesacrosstheenterprise.

• Identify,diagnose,prevent,andsolveperformanceproblemswithamorepredictabledatabaseserverexecutionenvironment.

• Trackandmanageclient/serveranddatabaseenvironmentefficientlywithcentralizedmanagement.

• Optimizedatabaseperformancewithexpertrecommendationsonquerydesign,uniqueindexes,andstatisticsquality.

Db2DirectEditionsDb211.1includesanewdeliverymechanismforlicensing.TheDb2DirectEditionsareintendedtomakeiteasiertoacquireDb2licensesforclouddeployments:

• Newlicensemetricstofacilitatehybridclouddeployments• Acquiretheproductdirectlyonline(PassportAdvantage)• Optiontodeployeitheron-premisesoroncloud

Itdoesn'tmatterwhereyouplaceyourDb2server,onpremise,oncloud,etc.YouarelicensedbythenumberofVirtualProcessorcorevCPUs(explainedonthenextpage).Thelicenseistotallyflexibleandcanbemovedbetweenplatformswithoutanychangeinlicensefees.NotethatthislicenseisanOPEX(OperatingExpense)license–youarerentingorleasingtheDb2license.Thepriceincludestheuseoftheproductaswellasserviceandsupport.Ifthelicenseisnotrenewed,thedatabasewillnothaveavalidlicense.TherearetwoDb2DirectEditions,dependingonwhatthedatabaserequirementsare:

• Db2DirectStandardEdition11.1-HasallthedatabasefeaturesofDb2WorkgroupServerEdition

• Db2DirectAdvancedEdition11.1-HasallthedatabasefeaturesofDb2AdvancedEnterpriseServerEdition


BothlicensesaresoldasamonthlylicensechargeusingtheVirtualProcessorCore(VPC)soldasamonthlylicensecharge.Customerscanchoosetomaintainthesedatabaseswitheither:

• LongTermSupportRelease(LTSR)support• ContinuousDelivery(CD)support

LTSRsupportwillgivethecustomerfixpacksonlywithnoadditionalfunctionalityaddedtotheproduct.CDsupportwillcontainthefixesfoundintheLTSRstreamandmayalsocontainnewfunctionalityandenhancements.

�Db2Developer-C(Community)EditionDb211.1.2.2introducesanewversionoftheDb2DeveloperEdition.ThisnewDb2Developer-CEditionhassimilarcharacteristicstotheexistingDeveloperEdition,exceptthatitisfreetodownloadandusebyallusers.Theproductisnotsupported(similartoExpress-CandTrialeditions)andcannotbeusedforproduction.Ithasnolimitsondatabaseenginefunctionalitybutitdoeshavelimitsontheenvironment(similartoExpress-C).Whyisthisneweditionimportantandwhywasitintroduced?Beingafreedownloadandwithoutfunctionallimitations,developerscandownloadandusethisfordevelopmentpurposes.Companiescanusethisforastart-upprojectwithoutcost.Itdoesnotexpire,andthuscanbeusedforalongperiodoftime(non-productiononly).Beingfullfunction,developersandteamscantestoutthemanyadvancedfeaturestheymaynotbeentitledtointheirotherforfeesystems,suchascompression,MPP,BLUAcceleration,pureScale,etc.Savingscanbehadinnon-productionenvironmentswheresupportisnotrequired(whensupportisrequired–aforfeeeditionmustbeused).ThereareafewlimitationswhenusingDeveloper-C.Themaximummachinessizewhenusingthisproductis:

• 4cores,16GBofmemory• 100GBofdatainusertablespaces

Someofthesupportingprogramsarenotincluded:Cognos,IDA,WebSphereAS/MQ,andDSMEnterpriseEdition.


�Db2DeveloperCommunityEditionAnewDownloadandGoenvironmenthasbeendevelopedthatincludesafullyconfiguredandfunctionalversionofDb2Developer-CinaDockerenvironment.ThisnewDownloadandGoapproachwillmakegettingacopyofDb2upandrunningamucheasierexperience.From10+screensandmultipleinputs,thenewapproachrequiresonlythreeclicksandauseridandpasswordtogetstarted.ThenecessaryDockercomponentsaredownloadedandconfiguredforyouoranexistingDockerinstallationcanbeused.Thedownloadisrelativelysmall(about15minutes)andexecutablefilesareavailableforMac,Windows,andLinuxenvironments.Thedownloadpackageincludes:

• Db2Developer-C11.1.2.2• DataServerManager2.1.4• DataStudio(optional)

Thispackagehasthesamelimitations(non-warranted)asDeveloper-CDevelopmentsuchasnon-productionuseonlyandcore,memory,andstoragelimitations.

OperatingSystemSupportAcustomerneedstobeawarethattherearenewoperatingsystems(O/S)requirementsforDb211.1.WhiletheirexistingDb2installationmaybesupportedatanolderO/Sreleaselevel,theywillnotbeabletoupgradetheirdatabaseunlesstheO/Shasbeenupdatedaswell.Db211.1introducessupportforPowerLinuxLE(LittleEndian)whichalsoincludessupportforpureScale.Db2supportsPowerLinuxLEwiththefollowingthreeoperatingsystems:

• RedHatEnterpriseLinux(RHEL)7.1+• SUSELinuxEnterpriseServer(SLES)12• Ubuntu14.04LTS

Db211.1hasalsoupdatedtheoperatingsystemsthatitsupports.Notethenewrequirementsinthefollowinglist.Supportfor32-bitserveroperatingsystemshasbeeneliminatedfornewinstancesinthisrelease,soWindows32-bitandLinux32-bitarenolongersupportedforthe


servereditionsbutcontinuetobeavailableaspartofthedeveloperedition.32-bitclientscontinuetobesupportedhowever.

• Intel64-bito Windows7,8.1,10,WindowsServer2012R2o RedHatEnterpriseLinux(RHEL)6.7+,7.1+o SUSELinuxEnterpriseServer(SLES)11SP4+,12o Ubuntu14.04LTS

• AIXVersion7.1TL3SP5+• zLinux

o RedHatEnterpriseLinux(RHEL)7.1+o SUSELinuxEnterpriseServer(SLES)12

StartinginDb211.1,someoperatingsystems,Linuxdistributionsandarchitecturesarenolongersupported.CustomersneedtolookatalternativeplatformstoupgradetoDb211.1.ThefollowingarenolongerbeingsupportedinDb211.1.

• HP-UX,Solaris• PowerLinuxBE,InspurK-UX

However,customerswhoarecurrentlyonDb210.5willcontinuetoreceiveservicefortheseplatformsuntiltheEndofServicedateforDb210.5.

VirtualizationSupportDb211.1supportsanumberofvirtualizationenvironments.Virtualizationcanimprovetheutilizationofprocessor/machineresourcesandalsogivethecustomeranenvironmenttotestsystemswithouthavingtoinvestinentiresystems.ThefollowingisalistofvirtualizedenvironmentsthatDb2supports:

• IBMSystemzo IBMProcessorResource/SystemManagero z/VMandz/KVMonIBMSystemz

• IBMPowero IBMPowerVMandPowerKVMandIBMWorkload

PartitionsonIBMPowerSystems• LinuxX86-64Platforms

o RedHatKVM


o SUSEKVM• VMWareESXi• Dockercontainersupport–Linuxonly• Microsoft

o Hyper-Vo MicrosoftWindowsAzureonx86-64Windows

Platformsonly• pureScalesupportonPowerVM/KVM,VMWare,andKVM

YouwillnotethatpureScaleissupportedinavirtualizedenvironmentandthatROCEAdapterscannowbevirtualizedinaVMWareenvironment(initially)sothatmultiplememberscansharethesamehardwareadapter.

UpgradeSupportUpgradingtoDb211.1issupportedfromDb210.5,Db210.1,orDb29.7.IfyouhaveanearlierversionofDb2,youmustfirstupgradetoDb210.5,Db210.1,orDb29.7beforeupgradingtoDb211.1.

• UpgradingtoaDb211.1non-rootinstallationissupportedfromaDb210.5,Db210.1,orDb29.7non-rootinstallation.UpgradingtoaDb211.1non-rootinstallationfromapre-Db211.1rootinstallationisnotsupported.

• InstancebitsizeisdeterminedbytheoperatingsystemwhereDb211.1isinstalled,andsupportfor32-bitkernelsand64-bitkernelshaschanged.

• UpgradingfromasystemwithmultiplecopiesofDb210.5,Db210.1,orDb29.7issupported.OnWindowsoperatingsystems,youmustbeawareoftherestrictionsoncoexistenceofpreviousversionsoftheDb2databaseproducts.

• Upgradingfromapartitioneddatabaseenvironmentwithmultipledatabasepartitionsissupported.

Restoringfulldatabaseofflinebackupsfrompre-Db211.1copiesissupported.However,rollingforwardoflogsfromapreviouslevelisnotpossible.ReviewBackupandrestoreoperationsbetweendifferentoperatingsystemsandhardwareplatformsforcompletedetailsaboutupgradesupportusingtheRESTOREDATABASEcommand.


AdministrationImprovementsDb211.1includesanumberofadministrativeenhancementsthatcustomerswillfinduseful.Fiveofthemthatarehighlightedhereinclude:

• RangePartitionTableReorganization• ADMIN_MOVE_TABLEimprovements• RemoteStorageOption• Db2HistoryFileBackup• BackupCompressionHardwareAcceleration

RangePartitionReorganization

ThemanageabilityoflargerangepartitionedtableshasbeenimprovedinDb211.1.AsinglepartitionofarangepartitionedtablecannowbereorganizedwiththeINPLACEoptionif:

• thetablehasnoglobalindex(i.e.non-partitionedindexes)• ONDATAPARTITIONisspecified

Thereorganizationcanonlyoccurononedatapartitionatatime,andthetablemustbeatleastthreepagesinsize.Thebiggestrestrictionofthisfeatureisthatthetablecannothaveanyglobalindexes.Ifaglobalindexexists,thereorgcommandwillnotrun.

ADMIN_MOVE_TABLE

TherearetwonewoptionsintheADMIN_MOVE_TABLEcommand:

• REPORT• TERM

TheREPORToptioncanbeusedtomonitortheprogressoftablemoves.Thecommandcalculatesasetofvaluestomonitortheprogressofsingleormultipletablemoves.ThefocusisontheCOPYandREPLAYphaseofarunningtablemove.TheREPORToptionrequiresatableschemaandnametogetinformationonaspecifictablemove.IfthesevaluesareleftblankorNULL,informationonalloutstandingtablemovesisretrieved.TheTERMoptioncanbeusedtoterminateatablemoveinprogress.TERMwillforceofftheapplicationrunningthetablemove,rollbackall


opentransactionsandsetthetablemovetoawell-definedoperationalstatus.Fromhere,thetablemovecanbecancelledorcontinued.

RemoteStorage

Db211.1deliversmoreflexibilityandoptionsforacquiring,sharingandstoringdatafilesandbackupimages,byallowscustomerstouseremotestorageforseveralDb2utilities:

• INGEST,LOAD,BACKUP,andRESTORE

Db2supportsremotestorageusingstoragealiasesfor:

• IBM®SoftLayer®ObjectStorage• AmazonSimpleStorageService(S3)

Db2HistoryFileBackup

TheDb2historyfilecontainsinformationaboutlogfilearchivelocation,logfilechainetc.Ifyouusesnapshotbackupsyouwanttohavecurrentinformationaboutlogfilelocationtoperformpointintimerecovery(RECOVERcommand).RECOVERneedsacurrentversionofthehistoryfiletobeavailable.TheNOTABLESPACEbackupoptionallowsyoutocreateacurrentandconsistentbackupofthehistoryfileinaconvenientway.ANOTABLESPACEbackupdoesnotcontaintablespaces,onlythehistoryfile.Anotablespacebackupisusedtoquicklybackupthehistoryfileandcanbedoneatdifferentfrequencythanadatabasebackup.RestorethehistoryfilebyusingtheexistingHISTORYFILEoptionwiththeRESTOREDATABASEcommand.

HardwareBackupCompression

Db2backupandlogarchivecompressionnowsupporttheNX842hardwareacceleratoronPOWER7+andPOWER8processors.Db2BACKUPsrequiretheuseofaspecificNX842librarytotakeadvantageofthehardwareacceleration.backup database <dbname> compress comprlib libdb2nx842.a

TheregistryvariableDB2_BCKP_COMPRESSIONcanbesettoNX842forthedefaultBACKUPcommandtousethehardwareacceleration.


Allowingforexistingcustomersthatareusingthebuilt-inDb2backupcompression(i.e.softwarecompression)tobenefitwithoutanychangestoscripts/jobs.Oncetheregistryvariableisset,usingthefollowingbackupcommandformatwillbenefitfromthehardwarecompressionaccelerator:backup database <dbname> compress

LogarchivecompressionisalsosupportedandcanbeconfiguredbyupdatingthedatabaseconfigurationparameterLOGARCHCOMPR1orLOGARCHCOMPR2toNX842:update database configuration for <dbname> using LOGARCHCOMPR1 NX842

2

Db2 BLU

Enhancements

EXTENDINGTHEPOWEROFIN-MEMORYCOMPUTING

Chapter2:Db2BLUEnhancements 18

Db2BLUEnhancements

TherehasbeensignificantenhancementsandcapabilitiesaddedtoDb2BLUtablesfromarchitecturetoperformanceimprovementsinthecolumnarengine.Db211.1addssupportforcolumn-organizedtablesinadatabasepartitionedenvironmentprovidedbyDb2DatabasePartitionFeature(DPF)allowingforpetabytescalingofBLUtablesinamassivelyparallelprocessing(MPP)architecture.InadditiontoMPPenablementofcolumnartables,Db211.1deliversadvancesinthecoreBLUAccelerationtechnologyforcolumn-organizedtables.TheseadvancesincludeNestedLoopJoin(NLJN)support,queryrewrite,fasterSQLMERGE,enhancementstomemorymanagement,furtherSIMDadvances,industryleadingparallelsort,improvedsortheaputilization,increasedSQLparallelismandwidevarietyofpushdownfunctionalityintotheBLUcolumnarengine.OtherenhancementsincludeBLUAccelerationsupportforIDENTITYandEXPRESSIONgeneratedcolumns,EuropeanLanguagesupportandNOTLOGGEDINITIALLYsupportforcolumn-organizedtables.Furtherinformationonsomeoftheseenhancementsarecoveredbelow.Theimprovementsnotonlyimproveperformanceofcolumnartables,butalsoincreaseconcurrentintra-parallelismdeliveringincreasedqueriesperhour,asseeninFigure4,DBInsightqueryperformance.

Figure4:DBInsightQueryPerformance

DB2 V10.5 FP5 DB2 V11.1QpH 703.85 955.82

0

200

400

600

800

1000

1200

Que

ries

Per H

our

Query Throughput BD Insights (800GB)

1.36x


MassivelyParallelProcessingArchitectureWithDb211.1,youcancreatecolumn-organizedtablesinapartitioneddatabaseenvironmentwiththeDatabasePartitioningFeatureofDb2.ThisallowsyoutoleverageBLUenhancementsinaMPPenvironment.InaDPFshared-nothingdatabase,dataispartitionedacrossmultipleserversorpartitionswitheachpartitionhavingitsowndedicatedresources–memory,processorsandstorage–toprocessdatalocallyandparallel.Db2allowsmultiplelogicalpartitionstobedeployedonthesamephysicalserver.Eachpartitionisresponsibleforasingleportion(i.e.partition)ofthedatainatableandthequeriesarebrokenupacrossallpartitions,withpartitionsworkinginparallel,leveragingthecomputepowerofallserversforfastqueryprocessing.Thepartitioningofthetableisdoneusingadistributionkeytodistributetabledataacrossasetofdatabasepartitions.Adistributionkeyconsistsofoneormorecolumnsofthetable.Tocreateapartitionedcolumn-organizedtableisthesameasforrow-organizedtables,viatheDISTRIBUTEBYclauseoftheCREATETABLEstatement,asbelow. CREATE TABLE sales(…) ORGANIZE BY COLUMN DISTRIBUTE BY (C1,C2)

IfDISTRIBUTEBYHASHisspecified,thedistributionkeysarethekeysexplicitlyincludedinthecolumnlistfollowingtheHASHkeyword(C1andC2intheexampleabove).Thecolumnsspecifiedmustbeasubsetofanyuniqueorprimarykeyonthetable.ImplicitselectionofdistributionkeyoccursiftheDISTRIBUTEBYHASHclauseisomittedandthetableisdefinedinadatabasepartitioninggroupwithmultiplepartitionsorifDISTRIBUTEBYRANDOMclauseisused.TherearetwomethodsthatDb2usestodistributedatawhenDISTRIBUTEBYRANDOMisspecified:

• Randombyunique:Ifthetableincludesauniqueorprimarykey,thecolumnsoftheuniqueorprimarykeyareusedasthedistributionkey.

• Randombygeneration:Db2willincludeanIMPLICITLYHIDDENcolumninthetabletogenerateandstoreageneratedvaluetouseinthehashingfunction.Thevalueofthecolumnwillbe


automaticallygeneratedasnewrowsareaddedtothetable.Bydefault,thecolumnnameisRANDOM_DISTRIBUTION_KEY.Ifitcollideswiththeexistingcolumn,anon-conflictingnamewillbegeneratedbyDb2.

FastCommunicationManagerTheFastCommunicationManager(FCM)communicationinfrastructureofDb2DPFhasbeenoptimizedforcolumnardataexchange.Thus,dataexchangeduringdistributedjoinsandaggregationprocessingoccursentirelywithintheBLUruntimeinnativecolumnarformat.Thefollowingspecificenhancementspertaintocolumn-organizedtablesinapartitionedenvironment:

• MPPawarequeryplanningandoptimizationforthecolumnvectorprocessingengine

• Anoptimizedvectorformatforcolumnardataexchangebetweenpartitions

• Acommontabledictionaryallowingdatatoremaincompressedacrossthenetwork

• Anoptimizedcommunicationsinfrastructuredesignedformulti-coreparallelism

BuiltonDb2'sDPFcapabilitiesmeansBLUtablesareabletoscale-outwellbeyondthatofasingleserverforimprovedresponsetime,handlinglargerdatasetsinthemulti-petabyterangeandmeansthatexistingDb2MPPdatawarehousescaneasilyleveragethein-memoryoptimizedcolumnartechnology.


FasterSQLMERGEprocessingTheMERGEstatementupdatesaTableorViewandusesdatafromasource(resultofatablereference).Rowsinthetargettablethatmatchthesourcecanbedeletedorupdatedasspecifiedandrowsthatdonotexistinthetargetcanbeinserted.Whenthetargetisaview,updates,deletesorinsertsoccurtotherowinthetable(s)onwhichtheviewisdefined.ThefollowingSQLisanexampleofaMERGEstatement.MERGE INTO archive ar USING (SELECT activity, description FROM activities) ac ON (ar.activity = ac.activity) WHEN MATCHED THEN UPDATE SET description = ac.description WHEN NOT MATCHED THEN INSERT (activity, description) VALUES (ac.activity, ac.description);

MERGEhasalreadybeensupportedagainstcolumnar-originatedtables,howeverDb211.1deliversperformanceimprovementsbypushingtheMERGEoperationdownintothecolumnarenginewhichmeansitoperatesinnativecolumnarformat.Thisalongwithafastersortengineimprovesperformancealso.

NestedLoopJoinSupportDb211.1deliversNestedLoopJoinsupportforcolumnar-orientedtables.ANestedLoopJoinisoneofthreetechniquesthattheDb2Optimizerusestoperformajoinbetweentwotables.HASHjoinistheotherjointechniquethatalreadysupportscolumnar-orientedtables.Anested-loopjoinisperformedbyscanningtheinnertableforeachaccessedrowoftheoutertableorbyperforminganindexlookupontheinnertableforeachaccessedrowoftheoutertable.NestedLoopJoinsupportoncolumnar-orientedtablesmeansthatevaluationsandactivitiescanbedonenativelyinBLU,therebyimprovingperformancedramatically.Thereisnoexternalenablementorsettingtobenefitfromthisnewcapability.ThisperformancebenefitisrealizedautomaticallywhentheDb2OptimizerselectsaNestedLoopJoinasthejointechnique.


SortProcessingEnhancementsDb211.1includessortinnovations,PARADIS,fromtheIBMTJWatsonResearchdivision.PARADISisanefficientparallelalgorithmforin-placeRadixsort.Inadditiontodeliveryindustryleadingparallelsort,Db211.1isalsoabletosortcompressedandencodeddata.ThisallowsmoreefficientandimprovedperformanceasprocessingisperformedwithintheBLUengine.Db211.1BLUsortenhancementscanincreaseBLUperformancebyasmuchas13.9x.PerformanceisbasedonmeasurementsandprojectionsusingstandardIBMbenchmarksinacontrolledenvironment.Theactualthroughputorperformancethatanyuserwillexperiencewillvarydependinguponmanyfactors,includingconsiderationssuchastheamountofmultiprogrammingintheuser'sjobstream,theI/Oconfiguration,thestorageconfiguration,andtheworkloadprocessed.Therefore,noassurancecanbegiventhatanindividualuserwillachieveresultslikethosestatedhere.MoredetailsonPARADIScanbefoundintheIBMResearchPaper:http://www.vldb.org/pvldb/vol8/p1518-cho.pdfTheDb2Explainfacilitycanbeutilizedtoexaminetheassociatedqueryaccessplans.TheaccessplanswillshowwhatpartoftheprocessingisdoneintherowenginevstheBLUengine.Allpartsbelowthe"CTQ"evaluatoraredoneintheBLUengine.InFigure5,wecanseethatbypushingdowntheSORTevaluatorwekeepthedataprocessinginBLUonlywhichcontributestothe4xspeedupobservedforthisquery.


Figure5:ColumnarEngineNativeSORTExample

QueryRewriteEnhancementsTherehavebeenenhancementstoQueryRewritewhendealingwithCorrelatedSubqueries.Theenhancementincludesthedecorrelationofcorrelatedsubqueries.Figure6isanexampleofacorrelatedsubquery.TheprocessingissuchthatthequerymatchestherowsintheLookuptableforaparticularKeyintheFacttable.Onceamatchisfound,weselectallrowsfromLookuptableforthatparticularKey.ThisprocessrepeatsforeachrowintheFacttable.

Figure6:CorrelatedSubqueryExample


Decorrelationallowsforbulkjoinprocessing,viaHASHjoinexploitation,insteadofarowatatime.Thereareseveralscenarioswhicharedecorrelatedsuchas:

• UPDATEstatementswithcorrelatedsubqueriesintheSETstatement

• CASEexpressionscontainingcorrelatedsubqueries• ORpredicatescontainingcorrelatedsubqueries• SubqueriescontainingGROUPBY

ThecorrespondingDb2QueryRewritegenerateddecorrelatedqueryforthecorrelatedsubqueryexampleinFigure6isasfollows:SELECT F.key FROM fact F LEFT OUTER JOIN lookup L ON F.key = L.key WHERE F.flag = 1 OR F.key IS NOT NULL;

Anested-loopjoinisthejoinmethodthatbestdealswithtruecorrelatedsubqueries,whichcannowalsobechosenasofDb211.1forquerieswheredecorrelationisnotpossible.

Push-downofOLAPfunctionsSQLOLAPfunctionsarepushed-downintotheBLUengineforprocessingandaredeliveredfordeeperin-databaseanalyticswithcolumn-organizedtables.Theseimprovementsincludefunctionssuchas:

• RANK,ROW_NUMBER,DENSE_RANK• FIRST_VALUE• AVG,COUNT,COUNT_BIG• MAX,MIN,SUM• FIRST_VALUE,RATIO_TO_REPORT• ROWS/RANGEBETWEENUNBOUNDEDPRECEDINGAND

UNBOUNDEDFOLLOWING• ROWSBETWEENUNBOUNDEDPRECEDINGANDCURENTROW

SQLFunctionsOptimizedforBLUSomeSQLsupportforBLUColumnartablesthatweremissinginearlierversionhasnowbeenincludedinDb211.1.WhichmeansincreasedperformanceasprocessingoccursnativelyintheBLUengineforcolumn-


organizedtables.Supportforcolumn-organizedtablesinclude,widerowsupport,allowingyoutocreateatablewhereitsrowlengthcanexceedthemaximumrecordlengthforthepagesizeofthetablespace.Othersupportincludes,logicalcharactersupport(CODEUNITS32),EuropeanLanguagesupport(Codepage819),IDENTITYandEXPRESSIONgeneratedcolumns,NOTLOGGEDINITIALLY,RowandColumnAccessControl(RCAC)andDeclaredGlobalTemporaryTable(DGTT)support,includingparallelinsertintonot-loggedDGTTfromBLUsource.

Inaddition,thereareaseriesofString,Numeric,Date&Timefunctionsoptimizedforcolumn-organizedtablesasnotedbelow:

• StringFunctionso LPAD/RPADo TO_CHARo INITCAP

• NumericFunctionso POWER,EXP,LOG10,LNo TO_NUMBERo MODo SIN,COS,TAN,COT,ASIN,ACOS,ATANo TRUNCATE

• DateandTimeFunctionso TO_DATEo MONTHNAME,DAYNAME

AutomaticDictionaryCreationDb211.1improvestheautomaticdictionarycreationofcolumn-organizedtables.ThecompressiondictionaryoccursduringINSERT,INGEST,IMPORTandisnowavailableearlierinprocessing.Theautomaticdictionarycreationimprovementsprovidebettercompressionrates,improvedqueryperformance,increasedthroughputforhighconcurrencyworkloadsandhigherPCTENCODED(percentencoded)valuesinSYSCAT.COLUMNS.

3

pureScale Continuous

Availability

EXTREMESCALABILITYANDAVAILABILITY

Chapter3:pureScaleContinuousAvailability 27

ContinuousAvailability

Db211.1introducessomeadditionalcapabilitiestoDb2pureScalethatwillprovidesimplificationwithinstallation&configuration,virtualization,workloadmanagement,manageabilityandadditionalDisasterRecovery(DR)configurations.Db2pureScalehasbeenprovidingscalabilityandcontinuousavailabilitythroughplannedandunplannedeventssinceversion9.8.TransparentscalabilitymeansthatasMembersareaddedtotheDb2pureScaleInstance,applicationscanscalewithoutanychangesorawarenessattheapplicationlevel.Bothscalabilityandavailabilityhavebeendesignedintothearchitectureandpossibleusingthehighlyreliableclustercachingfacility(CF)-capabilityportedfromthegoldstandardofavailability,Db2zOSSysplex.BeforewegetintothenewcapabilitiesdeliveredinDb211.1,let'stakeaquickreviewofthepureScalearchitecture.

pureScaleArchitectureThearchitectureofaDb2pureScalecluster(alsoreferredtoasapureScaleinstance),asshowninFigure7,isanactive/activedata-sharingenvironmentinwhichmultiplenodes–calledDb2members–handlethetransactionalworkloadofthesystemandtheyalsohaveequalandsharedaccesstoasinglecopyofthedatabaseondisk.AmemberisalogicalcomponentandessentiallyasingleinstanceofDb2Serverwithitsownbufferpool,memoryregion,logfiles,etc.ThesinglecopyofthedatabaseaswellasthelogfilesforeachmemberarestoredwithinthesamesharedstorageusingthehighlyscalableIBMSpectrumScale(GPFS)filesystem.Clientscanconnectintoanyofthemembersandneedonlyknowofoneastherearedefaultworkloadbalancingcapabilitiesthatwillautomaticallydistributetheworkloadacrossalltheactivemembers.Ifamemberfails,clientconnectionsareautomaticallyreroutedtohealthymembers.Acriticalelementofthearchitectureisthatthereisnomembertomembercommunications.ThememberscommunicatewiththeclusterCachingFacility(CF)forcentralizedlockingandcachemanagement.Foravailability,thereis


supportfortwoCFs,aprimaryandstandby.Thiscommunicationtakesplaceoverahighspeed,lowlatencyinterconnectviatheRDMA(RemoteDirectMemoryAccess)protocol.Alternatively,theuseofTCP/IPsocketsfortheinterconnectinsteadofanRDMA-basedinterconnectisalsosupported.TheDb2ClusterServices(CS)providesintegratedfailuredetection,recoveryautomationandtheclusteredfilesystem.AllthecomponentsarefullyintegratedintoDb2pureScale,inthattheyareallinstalledtogetheraspartofasingleinstallationprocess,theyareconfiguredaspartofinstallation,andtheyaremanagedandmaintainedtogetherallwithinpureScale.

Figure7:pureScaleArchitecture

DeploymentsDb2pureScalearchitectureallowsforhighlyflexibledeploymenttopologiesduetothelogicalaspectsofthemembersandCFs.ThisallowsforthemembersandCFstobedeployedinanynumberofcombinations,beitvirtual(LPAR,VM,KVM)orphysicalservers.Followingaresome

Clients

Shared Storage

Database

Logs Logs LogsLogs

Cluster Interconnect

MemberCS

MemberCS

MemberCS

MemberCS

Primary CF

CFCSSecondary CF

CFCS

DB2 pureScale Cluster (Instance)


exampledeploymentoptionsformembersandCF.Fortheexamples,wereferto'server'whichrepresentseithervirtualorphysicalimplementation.TheminimalrequirementforpureScaleisonememberandoneCF.However,therecommendeddeploymentconfiguration,asshowninFigure8,istwomembersandtwoCFsrunningacrosstwoservers.AsthisprovidesthecontinuouslyavailableconfigurationforwhichpureScaleisarchitected.

Figure8:TwoServers-TwoMembersTwoCFs

WecantakethesametwomembersandtwoCFconfigurationanddeployitoverthreeservers,asinFigure9,orfourserversasinFigure10.

Figure9:ThreeServers-TwoMembersTwoCFs

Figure10:FourServers-TwoMembersTwoCFs

Withthreeservers,youcanalsodeploythreemembersandtwoCFs,asshowninFigure11andwithfourservers,youcanhavefourmembersandtwoCFs(Figure12):

Member

CFp

Member

CFs


Figure11:ThreeServers-ThreeMembersTwoCFs

Figure12:FourServers-FourMembersTwoCFs

Thedecisiononwhichdeploymenttoutilizecomesdowntoresourcesontheserverandimpactonserverfailure.TakingFigure8andFigure9asanexample,aserverfailureinFigure8wouldresultinonememberandoneCFgoingofflineandtheworkloadcontinuesontheexistingmemberandCF.However,inFigure9withthreeservers,afailurewouldresultineitheraMemberorCFgoingofflineorbothamemberandCF.Ifit'sonlytheprimaryCF,thenbothmembercontinuetoprocesstransactionswiththesameprocessingcapacity.IfthememberorthememberandCFserverfail,thentheworkloadcontinuesontheexistingmemberandCFthesameastheFigure8failureexample.AsyouincreasethenumberofmembersyouincreasethescalabilityprovidedbyDb2pureScaleandminimizeimpactduringafailurescenario.Inadditiontothefullyactive/activeDb2pureScaledeployments,youcanalsodeploypureScaleusingan'active/passivelicensingmodel',asshownearlierinFigure3inChapter1.The'active/passive'isfromtheapplicationworkloadperspectiveandnotthecontinuouslyavailableaspect,asthisremains.Allapplicationworkloadsaredirectedtotheprimaryactivemember(s),sometimesreferredtoasthe"primary"memberwhileutilitiesandadministrativetasksareallowedonthesecondarymember.Thisisconfigurationisidealforoff-loadingbackupsandotheradministrativetasksfromprimarymembersaswellasensuringcontinuousavailableintheeventofamemberfailure.


VirtualizationDb2pureScalehassupportedavarietyofvirtualizationtechnologiessuchasIBMPowerVM,PowerKVM,RedHatKVMandVMWareESXi.Db211.1enhancesthevirtualizationsupportforVMWarebynowdeliveringRDMAoverConvergedEthernet(RoCE)supportinVMWarethroughRoCESingle-RootI/OVirtualization(SR-IOV)forRHEL7.2.SR-IOVstandardenablesonePCIExpress(PCIe)adaptertobepresentedasmultipleseparatelogicaldevices(VirtualFunctions)tovirtualmachines.ThisallowsthevirtualmachinestorunnativeRoCEandachievenearwirespeedperformance.ThiscapabilitycanbeenablewithDb2pureScalewhenusingMellanoxConnectX-3/ConnectX-3Pro/ConnectX-3VPIadaptersforEthernet.

InstallationTheDb2pureScaleinstallationprocessinDb211.1hasbeensimplifiedwithsmarterdefaults,intuitiveoptions,parallelandquickpre-deploymentvalidationacrosshosts.Allowinguserstobeupandrunningwithinhoursofstartingtheinstallationprocess.Therehasbeenre-engineeringeffortsininstallwhichreducesthecomplexitybyatleast40%forsocketsandtakesa30-stepnativeGPFSsetupdowntoasimple4-stepDb2installprocess.Db211.1providessimplificationinthesetupandconfigurationofGPFSreplication.GPFSreplicationisusedforcustomerswhowanttoprovideprotectionagainstacompleteSANstoragesubsystemfailure.TheGPFSreplication,Figure13,iscompletedwithafewdb2clustercommandsandisdefinedacrosstwoseparatestoragesubsystems.


Figure13:FourServers-FourMembersTwoCFsandGPFSReplication

WorkloadBalancingTherehasbeenadditionalcapabilityincludedinDb2pureScalewithworkloadbalancingwhenusingmembersubsets.MembersubsetsallowclientstoconnectandbebalancedacrossasubsetofthememberscomprisingtheDb2pureScaleinstance.Db211.1allowsyoutoexplicitlydefinethememberstobeusedasalternatesintheeventofprimarymemberfailures.AnalternatemembercanbeanyothermemberofthepureScaleInstance.Analternatememberofasubsetbehavesasanyothermemberandprocessingtransactionsbutisconsidereddormantforthatsubsetuntilafailureoccurstoaprimarymemberofthesubset.Uponafailure,thealternatememberwillbeassignedworkloadsfortherespectivesubsetinadditiontootherworkloadsthathasbeenassigned/runningonthatmember.Db2willattempttomaintainthesamenumberofprimarymembersintherespectivesubset,ifthereareenoughalternatesavailable.


AlternatemembersareassignedtoasubsetusingthenewlyaddedFAILOVER_PRIORITYoptionoftheWLM_ALTER_MEMBER_SUBSETDb2procedure.Aspartofthisenhancement,anewcolumn,FAILOVER_PRIORITY,wasaddedtotheSYSCAT.MEMBERSUBSETMEMBERSsystemcatalogview,asseeninTable1,toallowyoutoquicklyseewhichmembersareprimary,howmanyandorderingofalternatemembersforeachsubsetdefined.Table1:SYSCAT.MEMBERSUBSETMEMBERSCatalogView

Failoverpriorityhasvaluesfrom0-254.Memberswithfailoverpriorityof0,thedefaultifnotspecified,areconsideredprimarymembersofthesubset.Memberswithfailoverpriorityof1-254areconsideredalternatemembersofthesubset.Ifaprimarymemberfails,analternatememberisautomaticallyselectedtoservicethemembersubsetworkload.Thedeterminationofwhichalternatemembertoassigntothesubsetismadebychoosingthealternatememberwiththelowestfailoverpriority.Iftwoalternatemembershavethesameprioritydefined,thenthealternatememberwiththelowestmembernumberischosen.Ifaprimarymemberfails,analternatememberisautomaticallyselectedtoservicethemembersubsetworkload,fromalowerfailoverpriority.CreatingandalteringmembersubsetsisaDb2serversidedynamicallymanagedonlineoperation.

Assumingwealreadyhaveamembersubsetovermembers0and1calledBATCH,thefollowingexampleassignsmember2and4asanalternatememberwithfailoverpriorityof1and2,respectively,tothesubsetwithname'BATCH':CALL SYSPROC.WLM_ALTER_MEMBER_SUBSET ('BATCH', NULL, '(ADD 2 FAILOVER_PRIORITY 1, ADD 4 FAILOVER PRIORITY 2)');


Thefollowingquerycanbeusedtoviewthealternatememberinformationforsubsets: SELECT CAST(SUBSETNAME AS CHAR(10)) AS SUBSETNAME, MEMBER, FAILOVER_PRIORITY FROM SYSCAT.MEMBERSUBSETS ms, SYSCAT.MEMBERSUBSETMEMBERS msm WHERE ms.SUBSETID = msm.SUBSETID; ResultoftheabovequeryfortheBATCHexamplegiven:SUBSETNAME MEMBER FAILOVER_PRIORITY ---------- ------ ----------------- BATCH 0 0 BATCH 1 0 BATCH 2 1 BATCH 3 2 Inadditiontothecatalogview,thedb2pdcommandcanbeusedtoseethealternatememberinformation:db2pd –db <dbname> –membersubsetstatus TopreservethepriorDb211.1behaviorofmembersubsets,duringtheupgrade,thevalueofFAILOVER_PRIORITYattributeissetto0forallmembers.PriormembersubsetbehaviorcanalsobemaintainedfornewlycreatedsubsetsinDb211.1pureScale,bysettingaFAILOVER_PRIORTYof0forallthemembersintherespectivesubset.

TheDb211.1enhancementtomembersubsetsallowscustomerscurrentlyusingclientaffinitytomovetheirconfigurationtousingmembersubsetsandfailoverprioritysothattheycanexploitthenewbenefitssuchasdynamicserversidereconfiguration.SimplificationtosettingupclientaffinitywithhavingcontrolattheServervsclient–noneedtodb2dsdriver.cfg.

FailoverbehaviorDuringaplannedorunplannedeventtoamemberofasubset,theapplicationwillautomaticallyberoutedtothealternatememberwiththelowestfailoverpriorityandlowestmembernumber,whenmultiplealternatesaredefinedwiththesamefailoverpriority.Shouldasubsequenteventoccur,applicationswillberoutedtothenextalternatemember.


Ifthemembersubsetisdefinedasaninclusivesubsetandtherearenotenoughalternatememberstofailover,thentheapplicationusestheotheravailablemembersintheclusterthatarenotpartofthesubset

FailbackbehaviorOnceaprimarymemberoramemberwithlowerfailoverprioritybecomesavailable,applicationsfromthememberwiththehighestfailoverpriorityfailback.Thismeansthatallnewapplicationsconnecttothememberwithlowerprioritythatbecameavailableandexistingapplicationsconnectedtothealternatemember,withhigherfailoverpriority,cancompleteprocessingbeforefailingbacktothelowerprioritymember.

Manageability

Pre-Installation

Aspartoftheimprovementstoinstallation,Db211.1addsanewoption,adapterlist,tothedb2prereqcheckcommand.ThisoptionisusedtoverifythenetworkconnectivitythatallthehostsintheDb2pureScaleconfigurationarepingableusingRDMA.Theoptionrequiresaninputfiletobespecifiedwhichcontainsthelistofhostname,netnameandadapternamesforeachhosttobeverified.db2prereqcheck –adapter_list <adapter_list_filename>

Where<adapter_list_filename>specifiesthefilenamethatcontainsthelistofhostname,netname,andadapternamesforeachofthehosttobeverified.Theinputfilemusthavethefollowingformat:Table2:Adapterlistfilenameexample

Linesthatareprecededby#areconsideredcommentsandskipped.

#Hostname Netname Interface-Adapterhostname1 netname1 devicename-1hostname2 netname2 devicename-2hostname3 netname3 devicename-3


Post-Installation

Inadditiontotheenhancementstopre-installationabove,Db211.1deliversapost-installationunifiedhealthcheckoptiontothedb2clustercommand.Thedb2cluster–verifycommandperformsacomprehensivelistofcheckstovalidatethehealthoftheDb2pureScalecluster.Accordingly,analertisraisedforeachfailedcriterionandisdisplayedintheinstancemonitoringcommanddb2instance-list.Thevalidationsperformedinclude,butarenotlimitedto,thefollowing:

• ConfigurationsettingsinpeerdomainandGPFScluster• CommunicationsbetweenmembersandCFs(includingRDMA)• Replicationsettingforeachfilesystem• Statusofeachdiskinthefilesystem

DisasterRecoveryTherehavebeenenhancementstoDb2pureScaleinHADR(HighAvailabilityDisasterRecovery)andGDPC(GeographicallyDispersedDb2pureScaleCluster).BothprovidezerodatalossDisasterRecoverysolutionsintegratedanddeliveredaspartofDb2pureScale.

HADR

Db211.1enablesHADRsynchronous(SYNC)andnear-synchronous(NEARSYNC)supporttopureScaledeployments.ThisenhancementcombinesthecontinuousavailabilityoftheDb2pureScalefeaturewiththerobustdisasterrecoverycapabilitiesofHADRprovidinganintegratedzerodataloss(i.e.RPO=0)disasterrecoverysolution.

Figure14:HADRandpureScale


ThecompleteHADRfeaturessupportedwithpureScaleinDb211.1include:

• SYNC,NEARSYNC,ASYNCandSUPERASYNCmodes• Timedelayedapply• Logspooling• Bothnon-forced(roleswitch)andforced(failover)takeovers

GeographicallyDispersedDb2pureScaleCluster(GDPC)

Db211.1addsimprovedhighavailabilityforGeographicallydispersedDb2pureScaleclusters(GDPC)forbothRoCE&TCP/IP.pureScaleallowsformultipleadapterportspermemberandCFtosupporthigherbandwidthandimprovedredundancyattheadapterlevel.

Figure15:GeographicallyDispersedDb2pureScaleCluster

Inaddition,dualswitchescanbeconfiguredateachsiteinaGDPCconfigurationtoprovidegreateravailabilityintheeventofaswitchfailure.


InDb211.1apureScaleGDPCconfigurationexhibitstheequivalentfailurebehaviortoanon-GDPCpureScalecluster.Inaddition,thefour-switchconfigurationcansustainadditionalconcurrentfailurescenarios,suchasanytwoor3switchfailures,withoutanoutage.

4

Compatibility

Features

MAKINGITEASIERTOMOVEFROMONEDATABASESYSTEMTOANOTHER

Chapter4:CompatibilityFeatures 40

CompatibilityFeatures Movingfromonedatabasevendortoanothercansometimesbedifficultduetosyntaxdifferencesbetweendatatypes,functions,andlanguageelements.Db2alreadyhasahighdegreeofcompatibilitywithOraclePLSQLalongwithsomeoftheOracledatatypes.Db211.1introducessomeadditionaldatatypeandfunctioncompatibilitythatwillreducethemigrationeffortrequiredwhenportingfromothersystems.TherearesomespecificfeatureswithinDb2thataretargetedatNetezzaSQLandthatisdiscussedinaseparatesection.

OuterJoinOperatorDb2allowstheuseoftheOracleouter-joinoperatorwhenOraclecompatibilityisturnedonwithinadatabase.InDb211.1,theouterjoinoperatorisavailablebydefaultanddoesnotrequiretheDBAtoturnonOraclecompatibility.Db2supportsstandardjoinsyntaxforLEFTandRIGHTOUTERJOINS.However,thereisproprietarysyntaxusedbyOracleemployingakeyword:"(+)"tomarkthe"null-producing"columnreferencethatprecedesitinanimplicitjoinnotation.The(+)appearsintheWHEREclauseandreferstoacolumnoftheinnertableinaleftouterjoin.

Forinstance:SELECT * FROM T1, T2 WHERE T1.C1 = T2.C2 (+);

Isthesameas:SELECT * FROM T1 LEFT OUTER JOIN T2 ON T1.C1 = T2.C2;

Inthefollowingexample,wegetalistofdepartmentsandtheiremployees,aswellasthenamesofdepartmentswhohavenoemployees.

ThisexampleusesthestandardDb2outerjoinsyntax.SELECT DEPTNAME, LASTNAME FROM DEPARTMENT D LEFT OUTER JOIN EMPLOYEE E ON D.DEPTNO = E.WORKDEPT;


Theresultsfromthequeryare:DEPTNAME LASTNAME -------------------- --------------- SPIFFY COMPUTER SERV HAAS PLANNING THOMPSON INFORMATION CENTER KWAN SUPPORT SERVICES GEYER MANUFACTURING SYSTEM STERN ADMINISTRATION SYSTE PULASKI OPERATIONS HENDERSON SOFTWARE SUPPORT SPENSER SPIFFY COMPUTER SERV LUCCHESSI SPIFFY COMPUTER SERV O'CONNELL

Thisexampleworksinthesamemannerasthepreviousone,butusesthe"(+)"signsyntax.TheformatisalotsimplertorememberthanOUTERJOINsyntax,butitisnotpartoftheSQLstandard.SELECT DEPTNAME, LASTNAME FROM DEPARTMENT D, EMPLOYEE E WHERE D.DEPTNO = E.WORKDEPT (+);

CHARdatatypesizeincreaseTheCHARdatatypewaslimitedto254charactersinpriorreleasesofDb2.InDb211.1,thelimithasbeenincreasedto255characterstobringitinlinewithotherSQLimplementations. CREATE TABLE LONGER_CHAR ( NAME CHAR(255) );

BinaryDataTypesDb211.1introducestwonewbinarydatatypes:BINARYandVARBINARY.Thesetwodatatypescancontainanycombinationofcharactersorbinaryvaluesandarenotaffectedbythecodepageoftheserverthatthevaluesarestoredon.ABINARYdatatypeisfixedandcanhaveamaximumlengthof255bytes,whileaVARBINARYcolumncancontainupto32,672bytes.EachofthesedatatypesiscompatiblewithcolumnscreatedwiththeFORBITDATAkeyword.


TheBINARYdatatypewillreducetheamountofconversionrequiredfromotherdatabases.AlthoughbinarydatawassupportedwiththeFORBITDATAclauseonacharactercolumn,itrequiredmanualDDLchangeswhenmigratingatabledefinition.Thisexampleshowsthecreationofthethreetypesofbinarydatatypes.CREATE TABLE HEXEY ( AUDIO_SHORT BINARY(255), AUDIO_LONG VARBINARY(1024), AUDIO_CHAR VARCHAR(255) FOR BIT DATA );

InsertingdataintoabinarycolumncanbedoneusingBINARYfunctions,ortheuseofX'xxxx'modifierswhenusingtheVALUEclause.Forfixedstrings,youusetheX'00'formattospecifyabinaryvalueandBX'00'forvariablelengthbinarystrings.Forinstance,thefollowingSQLwillinsertdataintotheprevioustablethatwascreated.INSERT INTO HEXEY VALUES (BINARY('Hello there'), BX'2433A5D5C1', VARCHAR_BIT_FORMAT(HEX('Hello there')));

HandlingbinarydatawithaFORBITDATAcolumnwassometimestedious,sotheBINARYcolumnswillmakecodingalittlesimpler.Youcancompareandassignvaluesbetweenanyofthesetypesofcolumns.ThenextSQLstatementwillupdatetheAUDIO_CHARcolumnwiththecontentsoftheAUDIO_SHORTcolumn.ThentheSQLwilltesttomakesuretheyarethesamevalue.UPDATE HEXEY SET AUDIO_CHAR = AUDIO_SHORT; SELECT COUNT(*) FROM HEXEY WHERE AUDIO_SHORT = AUDIO_CHAR; 1 ----------- 1

�BooleanDataTypesTheBooleandatatype(true/false)hasbeenavailableinSQLPLandPL/SQLscriptsforsometime.However,theBooleandatatypecouldnot


beusedinatabledefinition.Db211.1.1.1nowallowsyoutousethisdatatypeinatabledefinitionanduseTRUE/FALSEclausestocomparevalues.ThissimpletablewillbeusedtodemonstratehowBOOLEANtypescanbeused. CREATE TABLE TRUEFALSE ( EXAMPLE INT, STATE BOOLEAN );

ThekeywordsforatruevalueareTRUE,'true','t','yes','y','on',and'1'.ForfalsethevaluesareFALSE,'false','f','no','n',and'0'.INSERT INTO TRUEFALSE VALUES (1, TRUE), (2, FALSE), (3, 0), (4, 't'), (5, 'no');

Nowwecanchecktoseewhathasbeeninsertedintothetable.EXAMPLE STATE ----------- ------ 1 1 2 0 3 0 4 1 5 0

RetrievingthedatainaSELECTstatementwillreturnanintegervaluefordisplaypurposes.1istrueand0isfalse(binary1and0).ComparisonoperatorswithBOOLEANdatatypeswilluseTRUE,FALSE,1or0oranyofthesupportedbinaryvalues.Youhavethechoiceofusingtheequal(=)operatorortheISorISNOTsyntaxasshowninthefollowingSQL.SELECT * FROM TRUEFALSE WHERE STATE = TRUE OR STATE = 1 OR STATE = 'on' OR STATE IS TRUE;

SynonymDatatypesDb2hasthestandarddatatypesthatmostdevelopersarefamiliarwith,likeCHAR,INTEGER,andDECIMAL.ThereareotherSQLimplementations


thatusedifferentnamesforthesedatatypes,soDb211.1nowallowsthesedatatypesassynonymsforthebasetypes.Thesenewdatatypesare:Table3:DataTypeSynonyms

Type Db2EquivalentINT2 SMALLINTINT4 INTEGERINT8 BIGINTFLOAT4 REALFLOAT8 FLOATThefollowingSQLwillcreateatablewiththesedatatypes.CREATE TABLE SYNONYM_EMPLOYEE ( NAME VARCHAR(20), SALARY INT4, BONUS INT2, COMMISSION INT8, COMMISSION_RATE FLOAT4, BONUS_RATE FLOAT8 );

Whenyoucreateatablewiththeseotherdatatypes,Db2doesnotusethese"types"inthesystemcatalog.WhatDb2willdoisusetheDb2typeinsteadofthesesynonymtypes.Ifyoudescribethecontentsofatable,youwillseetheDb2typesdisplayed,notthesesynonymtypes.DESCRIBE TABLE SYNONYM_EMPLOYEE; COLNAME TYPESCHEMA TYPENAME LENGTH SCALE NULLABLE --------------- ---------- -------- ------ ----- -------- NAME SYSIBM VARCHAR 20 0 Y SALARY SYSIBM INTEGER 4 0 Y BONUS SYSIBM SMALLINT 2 0 Y COMMISSION SYSIBM BIGINT 8 0 Y COMMISSION_RATE SYSIBM REAL 4 0 Y BONUS_RATE SYSIBM DOUBLE 8 0 Y

FunctionNameCompatibilityDb2hasawealthofbuilt-infunctionsthatareequivalenttothoseusedbyotherdatabasesystems,butwithadifferentname.InDb211.1,thesealternatefunctionnamesaremappedtoanequivalentDb2functionsothatnore-writeofthefunctionnamerequired.


ThefollowingtableliststhefunctionnamesandtheequivalentDb2functionthattheyaremappedto.Table4:FunctionSynonyms

Function Db2EquivalentCOVAR_POP COVARIANCESTDDEV_POP STDDEVVAR_POP VARIANCEVAR_SAMP VARIANCE_SAMPISNULL ISNULLNOTNULL ISNOTNULLLOG LNRANDOM RANDSTRPOS POSSTRSTRLEFT LEFTSTRRIGHT RIGHTBPCHAR VARCHAR(Castingfunction)DISTRIBUTEON DISTRIBUTEBY

5

Netezza

Compatibility

SUPPORTFORNETEZZASQLSYNTAXANDSTOREDPROCEDURES

Chapter5:NetezzaCompatibility 47

NetezzaCompatibility

Db2providesfeaturesthatenableapplicationsthatwerewrittenforaNetezzaPerformanceServer(NPS)databasetobeusedagainstaDb2database.TheSQL_COMPATglobalvariableisusedtoactivatethefollowingoptionalNPScompatibilityfeatures:

• Double-dotnotation-WhenoperatinginNPScompatibilitymode,youcanusedouble-dotnotationtospecifyadatabaseobject.

• TRANSLATEparametersyntax-ThesyntaxoftheTRANSLATEparameterdependsonwhetherNPScompatibilitymodeisbeingused.

• Operators-WhichsymbolsareusedtorepresentoperatorsinexpressionsdependsonwhetherNPScompatibilitymodeisbeingused.

• GroupingbySELECTclausecolumns-WhenoperatinginNPScompatibilitymode,youcanspecifytheordinalpositionorexposednameofaSELECTclausecolumnwhengroupingtheresultsofaquery.

• RoutineswritteninNZPLSQL-WhenoperatinginNPScompatibilitymode,theNZPLSQLlanguagecanbeusedinadditiontotheSQLPLlanguage.

SpecialOperatorCharactersDb2andNetezzabehavedifferentlywhenusingspecialoperatorcharacters.Thetablebelowliststhespecialcharactersandhoweachproductinterpretsthem.Table5:Db2SpecialCharacters

Character Db2Equivalent NetezzaI/&/~ LogicalOR,AND,Complement LogicalOR,AND,Complement^ LogicalXOR Power(also**)# None LogicalXOR


ThefollowingSQLwilldisplaytheDb2interpretationofthespecialcharacters.SET SQL_COMPAT = 'DB2'; WITH SPECIAL(OP, DESCRIPTION, EXAMPLE, RESULT) AS ( VALUES (' ^ ','XOR ', '2 ^ 3 ', 2 ^ 3), (' # ','NONE ', ' ',0) ) SELECT * FROM SPECIAL; OP DESCRIPTION EXAMPLE RESULT ---- ----------- --------- ----------- ^ XOR 2 ^ 3 1 # NONE 0

IfweturnonNPScompatibility,the^operatorbecomesa"power"operator,andthe#becomesanXORoperator.SET SQL_COMPAT = 'NPS'; WITH SPECIAL(OP, DESCRIPTION, EXAMPLE, RESULT) AS ( VALUES (' ^ ','POWER ', '2 ^ 3 ', 2 ^ 3), (' # ','XOR ', '2 # 3 ', 2 # 3) ) SELECT * FROM SPECIAL; OP DESCRIPTION EXAMPLE RESULT ---- ----------- --------- ----------- ^ POWER 2 ^ 3 8 # XOR 2 # 3 1

GROUPBYOrdinalLocationTheGROUPBYcommandbehavioralsochangesinNPSmode.ThefollowingSQLstatementgroupsresultsusingthedefaultDb2syntax:SET SQL_COMPAT='DB2'; SELECT WORKDEPT,INT(AVG(SALARY)) FROM EMPLOYEE GROUP BY WORKDEPT; WORKDEPT 2 -------- ----------- A00 70850 B01 94250 C01 77222 D11 58784 D21 51240 E01 80175 E11 45306 E21 47087


Ifyoutryusingtheordinallocation(similartoanORDERBYclause),youwillgetanerrormessage.SELECT WORKDEPT, INT(AVG(SALARY)) FROM EMPLOYEE GROUP BY 1; SQL0119N An expression starting with "WORKDEPT" specified in a SELECT clause, HAVING clause, or ORDER BY clause is not specified in the GROUP BY clause or it is in a SELECT clause, HAVING clause, or ORDER BY clause with a column function and no GROUP BY clause is specified. SQLSTATE=42803

IfNPScompatibilityisturnedon,thenthenyoucanusetheGROUPBYclausewithanordinallocation.SET SQL_COMPAT='NPS'; SELECT WORKDEPT, INT(AVG(SALARY)) FROM EMPLOYEE GROUP BY 1;

TRANSLATEFunctionThetranslatefunctionsyntaxinDb2is: TRANSLATE(expression, to_string, from_string, padding)

TheTRANSLATEfunctionreturnsavalueinwhichoneormorecharactersinastringexpressionmighthavebeenconvertedtoothercharacters.Thefunctionconvertsallthecharactersinchar-string-expinfrom-string-exptothecorrespondingcharactersinto-string-expor,ifnocorrespondingcharactersexist,tothepadcharacterspecifiedbypadding.Ifnoparametersaregiventothefunction,theoriginalstringisconvertedtouppercase.InNPSmode,thetranslatesyntaxis:TRANSLATE(expression, from_string, to_string)

Ifacharacterisfoundinthefromstring,andthereisnocorrespondingcharacterinthetostring,itisremoved.IfitwasusingDb2syntax,thepaddingcharacterwouldbeusedinstead.Note:IfORACLEcompatibilityisONthenthebehaviorofTRANSLATEisidenticaltoNPSmode.


Thisfirstexamplewilluppercasethestring.SET SQL_COMPAT = 'NPS'; VALUES TRANSLATE('Hello'); 1 ----- HELLO

Inthisexample,theletter'o'willbereplacedwithan'1'.VALUES TRANSLATE('Hello','o','1'); 1 ----- Hell1

Notethatyoucouldreplacemorethanonecharacterbyexpandingboththe"to"and"from"strings.Thisexamplewillreplacetheletter"e"witha"2"aswellas"o"with"1".VALUES TRANSLATE('Hello','oe','12'); 1 ----- H2ll1

Translatewillalsoremoveacharacterifitisnotinthe"to"list.VALUES TRANSLATE('Hello','oel','12'); 1 ----- H21

6

SQL Extensions

EXTENDINGSQLSYNTAXWITHMOREPOWERFULFEATURES

Chapter6:SQLExtensions 52

SQLExtensions

Db2hastheabilitytolimittheamountofdataretrievedonaSELECTstatementthroughtheuseoftheFETCHFIRSTnROWSONLYclause.InDb211.1,theabilitytooffsettherowsbeforefetchingwasaddedtotheFETCHFIRSTclause.

SimpleSQLwithFetchFirstClauseTheFETCHfirstclausecanbeusedinavarietyoflocationsinaSELECTclause.Thisfirstexamplefetchesonly5rowsfromtheEMPLOYEEtable.SELECT LASTNAME FROM EMPLOYEE FETCH FIRST 5 ROWS ONLY; LASTNAME --------------- HAAS THOMPSON KWAN GEYER STERN

YoucanalsoaddORDERBYandGROUPBYclausesintheSELECTstatement.NotethatDb2stillneedstoprocessalltherecordsanddotheORDER/GROUPBYworkbeforelimitingtheanswerset.So,youarenotgettingthefirst5rows"sorted".Youaregettingtheentireanswersetsortedbeforeretrievingjust5rows.SELECT LASTNAME FROM EMPLOYEE ORDER BY LASTNAME FETCH FIRST 5 ROWS ONLY; LASTNAME --------------- ADAMSON ALONZO BROWN GEYER GOUNOT

HereisanexamplewiththeGROUPBYstatement.ThisfirstSQLstatementgivesusthetotalanswerset-thecountofemployeesbyWORKDEPT.


SELECT WORKDEPT, COUNT(*) FROM EMPLOYEE GROUP BY WORKDEPT ORDER BY WORKDEPT; WORKDEPT 2 -------- ----------- A00 5 B01 1 C01 4 D11 11 D21 7 E01 1 E11 7 E21 6

AddingtheFETCHFIRSTclauseonlyreducestherowsreturned,nottherowsthatareusedtocomputetheGROUPingresult.SELECT WORKDEPT, COUNT(*) FROM EMPLOYEE GROUP BY WORKDEPT ORDER BY WORKDEPT FETCH FIRST 5 ROWS ONLY; WORKDEPT 2 -------- ----------- A00 5 B01 1 C01 4 D11 11 D21 7

OFFSETExtensionTheFETCHFIRSTnROWSONLYclausecanalsoincludeanOFFSETkeyword.TheOFFSETkeywordallowsyoutoretrievetheanswersetafterskipping"n"numberofrows.ThesyntaxoftheOFFSETkeywordis:OFFSET n ROWS FETCH FIRST x ROWS ONLY

TheOFFSETnROWSmustprecedetheFETCHFIRSTxROWSONLYclause.TheOFFSETclausecanbeusedtoscrolldownananswersetwithouthavingtoholdacursor.Forinstance,youcouldhavethefirstSELECTcallrequest10rowsbyjustusingtheFETCHFIRSTclause.Afterthatyoucouldrequestthefirst10rowsbeskippedbeforeretrievingthenext10rows.Theonethingyoumustbeawareofisthatthatanswersetcouldchangebetweencallsifyouusethistechniqueofa"moving"window.Ifrowsareupdatedoraddedafteryourinitialqueryyoumaygetdifferentresults.


ThisisduetothewaythatDb2addsrowstoatable.IfthereisaDELETEandthenanINSERT,theINSERTedrowmayendupintheemptyslot.Thereisnoguaranteeoftheorderofretrieval.Forthisreason,youarebetteroffusinganORDERBYtoforcetheorderingalthoughthistoowon'talwayspreventrowschangingpositions.Herearethefirst10rowsoftheemployeetable(notordered).SELECT LASTNAME FROM EMPLOYEE FETCH FIRST 10 ROWS ONLY; LASTNAME --------------- HAAS THOMPSON KWAN GEYER STERN PULASKI HENDERSON SPENSER LUCCHESSI O'CONNELL

Youcanspecifyazerooffsettobeginfromthebeginning.SELECT LASTNAME FROM EMPLOYEE OFFSET 0 ROWS FETCH FIRST 10 ROWS ONLY;

Nowwecanmovetheanswersetaheadby5rowsandgettheremaining5rowsintheanswerset.SELECT LASTNAME FROM EMPLOYEE OFFSET 5 ROWS FETCH FIRST 5 ROWS ONLY; LASTNAME --------------- PULASKI HENDERSON SPENSER LUCCHESSI O'CONNELL

FETCHFIRSTandOFFSETinSUBSELECTsTheFETCHFIRST/OFFSETclauseisnotlimitedtoregularSELECTstatements.Youcanalsolimitthenumberofrowsthatareusedina


subselect.Inthiscase,youarelimitingtheamountofdatathatDb2willscanwhendeterminingtheanswerset.Forinstance,sayyouwantedtofindthenamesoftheemployeeswhomakemorethantheaveragesalaryofthe3rdhighestpaiddepartment.(Bytheway,therearemultiplewaystodothis,butthisisoneapproach).Thefirststepistodeterminewhattheaveragesalaryisofalldepartments.SELECT WORKDEPT, AVG(SALARY) FROM EMPLOYEE GROUP BY WORKDEPT ORDER BY AVG(SALARY) DESC; WORKDEPT 2 -------- ------------------------------------------ B01 94250 E01 80175 C01 77222.5 A00 70850 D11 58783.63636363636363636363636363636 D21 51240 E21 47086.66666666666666666666666666667 E11 45305.71428571428571428571428571429

Weonlywantonerecordfromthislist(thethirdone),sowecanusetheFETCHFIRSTclausewithanOFFSETtogetthevaluewewant(Note:weneedtoskip2rowstogettothe3rdone). SELECT WORKDEPT, AVG(SALARY) FROM EMPLOYEE GROUP BY WORKDEPT ORDER BY AVG(SALARY) DESC OFFSET 2 ROWS FETCH FIRST 1 ROWS ONLY; WORKDEPT 2 -------- ------------------------------------------ C01 77222.5

Andhereisthelistofemployeesthatmakemorethantheaveragesalaryofthe3rdhighestdepartmentinthecompany. SELECT LASTNAME, SALARY FROM EMPLOYEE WHERE SALARY > ( SELECT AVG(SALARY) FROM EMPLOYEE GROUP BY WORKDEPT ORDER BY AVG(SALARY) DESC OFFSET 2 ROWS FETCH FIRST 1 ROW ONLY ) ORDER BY SALARY;


LASTNAME SALARY --------------- ----------- GEYER 80175.00 SPENSER 86150.00 HENDERSON 89750.00 THOMPSON 94250.00 PULASKI 96170.00 KWAN 98250.00 HAAS 152750.00

AlternateSyntaxforFETCHFIRSTTheFETCHFIRSTnROWSONLYandOFFSETclausecanalsobespecifiedusingasimplerLIMIT/OFFSETsyntax.TheLIMITclauseandtheequivalentFETCHFIRSTsyntaxareshownbelow.Table6:DataTypeSynonyms

Syntax EquivalentLIMITx FETCHFIRSTxROWSONLYLIMITxOFFSETy OFFSETyROWSFETCHFIRSTxROWSONLYLIMITy,x OFFSETyROWSFETCHFIRSTxROWSONLYWecanusetheLIMITclausewithanOFFSETtogetthevaluewewantfromthetable.SELECT WORKDEPT, AVG(SALARY) FROM EMPLOYEE GROUP BY WORKDEPT ORDER BY AVG(SALARY) DESC LIMIT 1 OFFSET 2; WORKDEPT 2 -------- ------------------------------------------ C01 77222.5

Andhereisthelistofemployeesthatmakemorethantheaveragesalaryofthe3rdhighestdepartmentinthecompany.NotethattheLIMITclausespecifiesonlytheoffset(LIMITx)ortheoffsetandlimit(LIMITy,x)whenyoudonotusetheLIMITkeyword.OnewouldthinkthatLIMITxOFFSETywouldtranslateintoLIMITx,ybutthatisnotthecase.Don'ttrytofigureouttheSQLstandardsreasoningbehindthesyntax!


SELECT LASTNAME, SALARY FROM EMPLOYEE WHERE SALARY > ( SELECT AVG(SALARY) FROM EMPLOYEE GROUP BY WORKDEPT ORDER BY AVG(SALARY) DESC LIMIT 2,1 ) ORDER BY SALARY; LASTNAME SALARY --------------- ----------- GEYER 80175.00 SPENSER 86150.00 HENDERSON 89750.00 THOMPSON 94250.00 PULASKI 96170.00 KWAN 98250.00 HAAS 152750.00

TableCreationwithSELECTTheCREATETABLEstatementcannowuseaSELECTclausetogeneratethedefinitionandLOADthedataatthesametime.ThesyntaxoftheCREATEtablestatementhasbeenextendedwiththeAS(SELECT...)WITHDATAclause:CREATE TABLE <name> AS (SELECT ...) [ WITH DATA | DEFINITION ONLY]

ThetabledefinitionwillbegeneratedbasedontheSQLstatementthatyouspecify.ThecolumnnamesarederivedfromthecolumnsthatareintheSELECTlistandcanbechangedbyspecifyingthecolumnsnamesaspartofthetablename:EMP(X,Y,Z,...)AS(...).Forexample,thefollowingSQLwillfailbecauseacolumnlistwasnotprovided:CREATE TABLE AS_EMP AS (SELECT EMPNO, SALARY+BONUS FROM EMPLOYEE) DEFINITION ONLY; SQL0153N The statement does not include a required column list. SQLSTATE=42908

YoucannameacolumnintheSELECTlistorplaceitinthetabledefinition.CREATE TABLE AS_EMP AS (SELECT EMPNO, SALARY+BONUS AS PAY FROM EMPLOYEE) DEFINITION ONLY;


YoucanDESCRIBEthetabletoseethedefinition.DESCRIBE TABLE AS_EMP; COLNAME TYPESCHEMA TYPENAME LENGTH SCALE NULLABLE ------- ---------- --------- ------ ----- -------- EMPNO SYSIBM CHARACTER 6 0 N PAY SYSIBM DECFLOAT 16 0 Y

TheDEFINITIONONLYclausewillcreatethetablebutnotloadanydataintoit.AddingtheWITHDATAclausewilldoanINSERTofrowsintothenewlycreatedtable.Ifyouhavealargeamountofdatatoloadintothetable,youmaybebetteroffcreatingthetablewithDEFINITIONONLYandthenusingLOADorothermethodstoloadthedataintothetable.CREATE TABLE AS_EMP AS (SELECT EMPNO, SALARY+BONUS AS PAY FROM EMPLOYEE) WITH DATA;

TheSELECTstatementintheCREATETABLEbodycanbeverysophisticated.Itcandoanytypeofcalculationorlimitthedatatoasubsetofinformation. CREATE TABLE AS_EMP(LAST,PAY) AS ( SELECT LASTNAME, SALARY FROM EMPLOYEE WHERE WORKDEPT='D11' FETCH FIRST 3 ROWS ONLY ) WITH DATA;

YoucanalsousetheOFFSETclauseaspartoftheFETCHFIRSTONLYtogetchunksofdatafromtheoriginaltable.CREATE TABLE AS_EMP(DEPARTMENT, LASTNAME) AS (SELECT WORKDEPT, LASTNAME FROM EMPLOYEE OFFSET 5 ROWS FETCH FIRST 10 ROWS ONLY ) WITH DATA;

CREATEFUNCTIONforAggregateUDFsTheupdatedCREATEFUNCTIONstatementallowsyoutocreateyourownaggregationfunctions.Anaggregatefunctionreturnsasinglevaluethatistheresultofanevaluationofasetoflikevalues,suchasthoseinacolumnwithinasetofrows.FunctionslikeSUM,AVG,andCOUNTareexamplesofbuilt-incolumnfunctions.


Aggregationfunctionscanbecreatedwithavarietyofprogramminglanguagesandmustcoverfourstagesoftheaggregationprocess.

• INITIATE–CalledwhenbeforetheSQLreturnsindividualrows.Theinitializationstepisusedtozerooutcountersandothervaluesthatareusedintheaggregationprocess.

• ACCUMULATE–Aseachrowisretrieved,theaccumulatestepiscalledtoincrementorprocessthevalueinsomeway.

• MERGE–InaDPF(partitioned)environment,thedatafromeachpartitionneedstobeconsolidatedtocomeupwithafinalresult.

• FINALIZE–ThefinalizestepwillcomputethefinalvaluethatthefunctionshouldreturntotheSQLstatement.

7

Date Functions

MOREWAYSTOEXTRACTANDMANIPULATEDATEFIELDS

Chapter7:DateFunctions 61

DateFunctions

ThereareplentyofnewdateandtimefunctionsfoundinDb211.1.Thesefunctionsallowyoutoextractportionsfromadateandformatthedateinavarietyofdifferentways.WhileDb2alreadyhasanumberofdateandtimefunctions,thesenewfunctionsallowforgreatercompatibilitywithotherdatabaseimplementations,makingiteasiertoporttoDb2.

ExtractFunctionTheEXTRACTfunctionextractsanelementfromadate/timevalue.ThesyntaxoftheEXTRACTcommandis: EXTRACT( element FROM expression )

ThisisaslightlydifferentformatfrommostfunctionsthatyouseeinDb2.Elementmustbeoneofthefollowingvalues:Table7:DateElements

ElementName DescriptionEPOCH Numberofsecondssince1970-01-0100:00:00.00.Thevaluecanbe

positiveornegative.MILLENNIUM(S) Themillenniumistobereturned.CENTURY(CENTURIES) Thenumberoffull100-yearperiodsrepresentedbytheyear.DECADE(S) Thenumberoffull10-yearperiodsrepresentedbytheyear.YEAR(S) Theyearportionistobereturned.QUARTER Thequarteroftheyear(1-4)istobereturned.MONTH Themonthportionistobereturned.WEEK Thenumberoftheweekoftheyear(1-53)thatthespecifieddayis

tobereturned.DAY(S) Thedayportionistobereturned.DOW Thedayoftheweekthatistobereturned.Notethat"1"represents

Sunday.DOY Theday(1-366)oftheyearthatistobereturned.HOUR(S) Thehourportionistobereturned.MINUTE(S) Theminuteportionistobereturned.SECOND(S) Thesecondportionistobereturned.MILLISECOND(S) Thesecondoftheminute,includingfractionalpartstoone

thousandthofasecondMICROSECOND(S) Thesecondoftheminute,includingfractionalpartstoonemillionth

ofasecond


ThisSQLwillreturneverypossibleextractvaluefromthecurrentdate.TheNOWkeywordisasynonymforCURRENTTIMESTAMP.WITH DATES(FUNCTION, RESULT) AS ( VALUES ('EPOCH', EXTRACT( EPOCH FROM NOW )), ('MILLENNIUM(S)', EXTRACT( MILLENNIUM FROM NOW )), ('CENTURY(CENTURIES)', EXTRACT( CENTURY FROM NOW )), ('DECADE(S)', EXTRACT( DECADE FROM NOW )), ('YEAR(S)', EXTRACT( YEAR FROM NOW )), ('QUARTER', EXTRACT( QUARTER FROM NOW )), ('MONTH', EXTRACT( MONTH FROM NOW )), ('WEEK', EXTRACT( WEEK FROM NOW )), ('DAY(S)', EXTRACT( DAY FROM NOW )), ('DOW', EXTRACT( DOW FROM NOW )), ('DOY', EXTRACT( DOY FROM NOW )), ('HOUR(S)', EXTRACT( HOURS FROM NOW )), ('MINUTE(S)', EXTRACT( MINUTES FROM NOW )), ('SECOND(S)', EXTRACT( SECONDS FROM NOW )), ('MILLISECOND(S)', EXTRACT( MILLISECONDS FROM NOW )), ('MICROSECOND(S)', EXTRACT( MICROSECONDS FROM NOW )) ) SELECT * FROM DATES; FUNCTION RESULT ------------------ --------------------------- EPOCH 1474090894.000000 MILLENNIUM(S) 2.000000 CENTURY(CENTURIES) 20.000000 DECADE(S) 201.000000 YEAR(S) 2016.000000 QUARTER 3.000000 MONTH 9.000000 WEEK 38.000000 DAY(S) 17.000000 DOW 7.000000 DOY 261.000000 HOUR(S) 5.000000 MINUTE(S) 41.000000 SECOND(S) 34.578000 MILLISECOND(S) 34578.000000 MICROSECOND(S) 34578000.000000

DATE_PARTFunctionDATE_PARTisliketheEXTRACTfunctionbutitusesthemorefamiliarfunctionsyntax: DATE_PART(element, expression)

InthecaseoftheDATE_PARTfunction,theelementmustbeplacedinquotes,ratherthanasakeywordintheEXTRACTfunction.inaddition,theDATE_PARTalwaysreturnsaBIGINT,whiletheEXTRACTfunctionwill


returnadifferentdatatypedependingontheelementbeingreturned.Forinstance,theSECONDsoptionforEXTRACTreturnsaDECIMALresultwhiletheDATE_PARTreturnsatruncatedBIGINT. WITH DATES(FUNCTION, RESULT) AS ( VALUES ('EPOCH', DATE_PART('EPOCH' ,NOW )), ('MILLENNIUM(S)', DATE_PART('MILLENNIUM' ,NOW )), ('CENTURY(CENTURIES)', DATE_PART('CENTURY' ,NOW )), ('DECADE(S)', DATE_PART('DECADE' ,NOW )), ('YEAR(S)', DATE_PART('YEAR' ,NOW )), ('QUARTER', DATE_PART('QUARTER' ,NOW )), ('MONTH', DATE_PART('MONTH' ,NOW )), ('WEEK', DATE_PART('WEEK' ,NOW )), ('DAY(S)', DATE_PART('DAY' ,NOW )), ('DOW', DATE_PART('DOW' ,NOW )), ('DOY', DATE_PART('DOY' ,NOW )), ('HOUR(S)', DATE_PART('HOURS' ,NOW )), ('MINUTE(S)', DATE_PART('MINUTES' ,NOW )), ('SECOND(S)', DATE_PART('SECONDS' ,NOW )), ('MILLISECOND(S)', DATE_PART('MILLISECONDS' ,NOW )), ('MICROSECOND(S)', DATE_PART('MICROSECONDS' ,NOW )) ) SELECT * FROM DATES; FUNCTION RESULT ------------------ -------------------- EPOCH 1474090894 MILLENNIUM(S) 2 CENTURY(CENTURIES) 20 DECADE(S) 201 YEAR(S) 2016 QUARTER 3 MONTH 9 WEEK 38 DAY(S) 17 DOW 7 DOY 261 HOUR(S) 5 MINUTE(S) 41 SECOND(S) 34 MILLISECOND(S) 34809 MICROSECOND(S) 34809000

DATE_TRUNCFunctionDATE_TRUNCcomputesthesameresultsastheDATE_PARTfunctionbutthentruncatesthevaluedown.Notethatnotallvaluescanbetruncated.Thefunctionsyntaxis: DATE_TRUNC(element, expression)

Theelementmustbeplacedinquotes,ratherthanasakeywordintheEXTRACTfunction.NotethatDATE_TRUNCalwaysreturnsaBIGINT.


Theelementsthatcanbetruncatedare:Table8:DATE_TRUNCElements

ElementName DescriptionMILLENNIUM(S) Themillenniumistobereturned.CENTURY(CENTURIES) Thenumberoffull100-yearperiodsrepresentedbytheyear.DECADE(S) Thenumberoffull10-yearperiodsrepresentedbytheyear.YEAR(S) Theyearportionistobereturned.QUARTER Thequarteroftheyear(1-4)istobereturned.MONTH Themonthportionistobereturned.WEEK Thenumberoftheweekoftheyear(1-53)thatthespecifieddayis

tobereturned.DAY(S) Thedayportionistobereturned.HOUR(S) Thehourportionistobereturned.MINUTE(S) Theminuteportionistobereturned.SECOND(S) Thesecondportionistobereturned.MILLISECOND(S) Thesecondoftheminute,includingfractionalpartstoone

thousandthofasecondMICROSECOND(S) Thesecondoftheminute,includingfractionalpartstoonemillionth

ofasecond

WITH DATES(FUNCTION, RESULT) AS ( VALUES ('MILLENNIUM(S)', DATE_TRUNC('MILLENNIUM',NOW )), ('CENTURY(CENTURIES)', DATE_TRUNC('CENTURY' ,NOW )), ('DECADE(S)', DATE_TRUNC('DECADE' ,NOW )), ('YEAR(S)', DATE_TRUNC('YEAR' ,NOW )), ('QUARTER', DATE_TRUNC('QUARTER' ,NOW )), ('MONTH', DATE_TRUNC('MONTH' ,NOW )), ('WEEK', DATE_TRUNC('WEEK' ,NOW )), ('DAY(S)', DATE_TRUNC('DAY' ,NOW )), ('HOUR(S)', DATE_TRUNC('HOURS' ,NOW )), ('MINUTE(S)', DATE_TRUNC('MINUTES' ,NOW )), ('SECOND(S)', DATE_TRUNC('SECONDS' ,NOW )), ('MILLISEC(S)', DATE_TRUNC('MILLISECONDS ',NOW )), ('MICROSEC(S)', DATE_TRUNC('MICROSECONDS ',NOW )) ) SELECT * FROM DATES; FUNCTION RESULT ------------------ -------------------------- MILLENNIUM(S) 2000-01-01 00:00:00.000000 CENTURY(CENTURIES) 2000-01-01 00:00:00.000000 DECADE(S) 2010-01-01 00:00:00.000000 YEAR(S) 2016-01-01 00:00:00.000000 QUARTER 2016-07-01 00:00:00.000000 MONTH 2016-09-01 00:00:00.000000 WEEK 2016-09-12 00:00:00.000000 DAY(S) 2016-09-17 00:00:00.000000 HOUR(S) 2016-09-17 05:00:00.000000 MINUTE(S) 2016-09-17 05:41:00.000000 SECOND(S) 2016-09-17 05:41:35.000000 MILLISEC(S) 2016-09-17 05:41:35.049000 MICROSEC(S) 2016-09-17 05:41:35.049000


ExtractingSpecificDaysfromaMonthTherearethreefunctionsthatretrievedayinformationfromadate.

• DAYOFMONTH–returnsanintegerbetween1and31thatrepresentsthedayoftheargument

• FIRST_DAY–returnsadateortimestampthatrepresentsthefirstdayofthemonthoftheargument

• DAYS_TO_END_OF_MONTH–returnsthenumberofdaystotheendofthemonth

ThefollowingexamplesassumethatthecurrentdateisSeptember17th,2016.VALUES NOW; 1 -------------------------- 2016-09-17 05:41:35.229000

DAYOFMONTHreturnsthedayofthemonth.VALUES DAYOFMONTH(NOW); 1 ----------- 17

FIRST_DAYwillreturnthefirstdayofthemonth.YoucouldprobablycomputethiswithstandardSQLdatefunctions,butitisaloteasierjusttousethisbuilt-infunction.VALUES FIRST_DAY(NOW); 1 -------------------------- 2016-09-01 05:41:35.399000

Finally,DAYS_TO_END_OF_MOTNHwillreturnthenumberofdaystotheendofthemonth.AZerowouldbereturnedifyouareonthelastdayofthemonth.VALUES DAYS_TO_END_OF_MONTH(NOW); 1 ----------- 13


DateAdditionFunctionsThedateadditionfunctionswilladdorsubtractdaysfromacurrenttimestamp.Thefunctionsthatareavailableare:

• ADD_YEARS–Addyearstoadate• ADD_MONTHS–Addmonthstoadate• ADD_DAYS–Adddaystoadate• ADD_HOURS–Addhourstoadate• ADD_MINUTES–Addminutestoadate• ADD_SECONDS–Addsecondstoadate

Theformatofthefunctionis:ADD_DAYS ( expression, numeric expression )

ThefollowingSQLwilladdone"unit"tothecurrentdate.WITH DATES(FUNCTION, RESULT) AS ( VALUES ('CURRENT DATE ',NOW), ('ADD_YEARS ',ADD_YEARS(NOW,1)), ('ADD_MONTHS ',ADD_MONTHS(NOW,1)), ('ADD_DAYS ',ADD_DAYS(NOW,1)), ('ADD_HOURS ',ADD_HOURS(NOW,1)), ('ADD_MINUTES ',ADD_MINUTES(NOW,1)), ('ADD_SECONDS ',ADD_SECONDS(NOW,1)) ) SELECT * FROM DATES; FUNCTION RESULT --------------- -------------------------- CURRENT DATE 2016-09-17 05:41:35.669000 ADD_YEARS 2017-09-17 05:41:35.669000 ADD_MONTHS 2016-10-17 05:41:35.669000 ADD_DAYS 2016-09-18 05:41:35.669000 ADD_HOURS 2016-09-17 06:41:35.669000 ADD_MINUTES 2016-09-17 05:42:35.669000 ADD_SECONDS 2016-09-17 05:41:36.669000


Anegativenumbercanbeusedtosubtractvaluesfromthecurrentdate.WITH DATES(FUNCTION, RESULT) AS ( VALUES ('CURRENT DATE ',NOW), ('ADD_YEARS ',ADD_YEARS(NOW,-1)), ('ADD_MONTHS ',ADD_MONTHS(NOW,-1)), ('ADD_DAYS ',ADD_DAYS(NOW,-1)), ('ADD_HOURS ',ADD_HOURS(NOW,-1)), ('ADD_MINUTES ',ADD_MINUTES(NOW,-1)), ('ADD_SECONDS ',ADD_SECONDS(NOW,-1)) ) SELECT * FROM DATES; FUNCTION RESULT --------------- -------------------------- CURRENT DATE 2016-09-17 05:41:35.749000 ADD_YEARS 2015-09-17 05:41:35.749000 ADD_MONTHS 2016-08-17 05:41:35.749000 ADD_DAYS 2016-09-16 05:41:35.749000 ADD_HOURS 2016-09-17 04:41:35.749000 ADD_MINUTES 2016-09-17 05:40:35.749000 ADD_SECONDS 2016-09-17 05:41:34.749000

ExtractingWeeks,Months,Quarters,andYearsfromaDateTherearefourfunctionsthatextractdifferentvaluesfromadate.Thesefunctionsinclude:

• THIS_QUARTER–returnsthefirstdayofthequarter• THIS_WEEK–returnsthefirstdayoftheweek(Sundayis

consideredthefirstdayofthatweek)• THIS_MONTH–returnsthefirstdayofthemonth• THIS_YEAR–returnsthefirstdayoftheyear

ThefollowingSQLwillreturnallthecurrent(THIS)valuesofthecurrentdate.WITH DATES(FUNCTION, RESULT) AS ( VALUES ('CURRENT DATE ',NOW), ('THIS_WEEK ',THIS_WEEK(NOW)), ('THIS_MONTH ',THIS_MONTH(NOW)), ('THIS_QUARTER ',THIS_QUARTER(NOW)), ('THIS_YEAR ',THIS_YEAR(NOW)) ) SELECT * FROM DATES;


FUNCTION RESULT --------------- -------------------------- CURRENT DATE 2016-09-17 05:41:35.879000 THIS_WEEK 2016-09-11 00:00:00.000000 THIS_MONTH 2016-09-01 00:00:00.000000 THIS_QUARTER 2016-07-01 00:00:00.000000 THIS_YEAR 2016-01-01 00:00:00.000000

ThereisalsoaNEXTfunctionforeachofthese.TheNEXTfunctionwillreturnthenextweek,month,quarter,oryeargivenacurrentdate. WITH DATES(FUNCTION, RESULT) AS ( VALUES ('CURRENT DATE ',NOW), ('NEXT_WEEK ',NEXT_WEEK(NOW)), ('NEXT_MONTH ',NEXT_MONTH(NOW)), ('NEXT_QUARTER ',NEXT_QUARTER(NOW)), ('NEXT_YEAR ',NEXT_YEAR(NOW)) ) SELECT * FROM DATES; FUNCTION RESULT --------------- -------------------------- CURRENT DATE 2016-09-17 05:41:35.979000 NEXT_WEEK 2016-09-18 00:00:00.000000 NEXT_MONTH 2016-10-01 00:00:00.000000 NEXT_QUARTER 2016-10-01 00:00:00.000000 NEXT_YEAR 2017-01-01 00:00:00.000000

NextDayFunctionTheprevioussetoffunctionsreturnedadatevalueforthecurrentweek,month,quarter,oryear(orthenextoneifyouusedtheNEXTfunction).TheNEXT_DAYfunctionreturnsthenextday(afterthedateyousupply)basedonthestringrepresentationoftheday.Thedatestringwillbedependentonthecodepagethatyouareusingforthedatabase.Thedate(fromanEnglishperspective)canbe:Table9:DaysandShortForm

Day ShortformMonday MONTuesday TUEWednesday WEDThursday THUFriday FRISaturday SATSunday SUN


ThefollowingSQLwillshowyouthe"day"afterthecurrentdatethatisMondaythroughSunday.WITH DATES(FUNCTION, RESULT) AS ( VALUES ('CURRENT DATE ',NOW), ('Monday ',NEXT_DAY(NOW,'Monday')), ('Tuesday ',NEXT_DAY(NOW,'TUE')), ('Wednesday ',NEXT_DAY(NOW,'Wednesday')), ('Thursday ',NEXT_DAY(NOW,'Thursday')), ('Friday ',NEXT_DAY(NOW,'FRI')), ('Saturday ',NEXT_DAY(NOW,'Saturday')), ('Sunday ',NEXT_DAY(NOW,'Sunday')) ) SELECT * FROM DATES; FUNCTION RESULT --------------- -------------------------- CURRENT DATE 2016-09-17 05:41:36.139000 Monday 2016-09-19 05:41:36.139000 Tuesday 2016-09-20 05:41:36.139000 Wednesday 2016-09-21 05:41:36.139000 Thursday 2016-09-22 05:41:36.139000 Friday 2016-09-23 05:41:36.139000 Saturday 2016-09-24 05:41:36.139000 Sunday 2016-09-18 05:41:36.139000

BetweenDate/TimeFunctionsThesedatefunctionscomputethenumberoffullseconds,minutes,hours,days,weeks,andyearsbetweentwodates.Ifthereisn'tafullvaluebetweenthetwoobjects(likelessthanaday),azerowillbereturned.Thesenewfunctionsare:

• HOURS_BETWEEN–returnsthenumberoffullhoursbetweentwoarguments

• MINUTES_BETWEEN–returnsthenumberoffullminutesbetweentwoarguments

• SECONDS_BETWEEN–returnsthenumberoffullsecondsbetweentwoarguments

• DAYS_BETWEEN–returnsthenumberoffulldaysbetweentwoarguments

• WEEKS_BETWEEN–returnsthenumberoffullweeksbetweentwoarguments

• YEARS_BETWEEN–returnsthenumberoffullyearsbetweentwoarguments


Theformatofallofthesefunctionsis:DAYS_BETWEEN( expression1, expression2 )

ThefollowingSQLwilluseadatethatisinthefuturewithexactlyoneextrasecond,minute,hour,day,weekandyearaddedtoit.CREATE VARIABLE FUTURE_DATE TIMESTAMP DEFAULT(NOW + 1 SECOND + 1 MINUTE + 1 HOUR + 8 DAYS + 1 YEAR); WITH DATES(FUNCTION, RESULT) AS ( VALUES ('SECONDS_BETWEEN',SECONDS_BETWEEN(FUTURE_DATE,NOW)), ('MINUTES_BETWEEN',MINUTES_BETWEEN(FUTURE_DATE,NOW)), ('HOURS_BETWEEN ',HOURS_BETWEEN(FUTURE_DATE,NOW)), ('DAYS BETWEEN ',DAYS_BETWEEN(FUTURE_DATE,NOW)), ('WEEKS_BETWEEN ',WEEKS_BETWEEN(FUTURE_DATE,NOW)), ('YEARS_BETWEEN ',YEARS_BETWEEN(FUTURE_DATE,NOW)) ) SELECT * FROM DATES; FUNCTION RESULT --------------- -------------------- SECONDS_BETWEEN 32230861 MINUTES_BETWEEN 537181 HOURS_BETWEEN 8953 DAYS BETWEEN 373 WEEKS_BETWEEN 53 YEARS_BETWEEN 1

MONTHS_BETWEENFunctionYoumayhavenoticedthattheMONTHS_BETWEENfunctionwasnotinthepreviouslistoffunctions.ThereasonforthisisthatthevaluereturnedforMONTHS_BETWEENisdifferentfromtheotherfunctions.TheMONTHS_BETWEENfunctionreturnsaDECIMALvalueratherthananintegervalue.Thereasonforthisisthatthedurationofamonthisnotaspreciseasaday,weekoryear.Thefollowingexamplewillshowhowthedurationisadecimalvalueratherthananinteger.Youcouldalwaystruncatethevalueifyouwantaninteger.WITH DATES(FUNCTION, RESULT) AS ( VALUES ('0 MONTH ',MONTHS_BETWEEN(NOW, NOW)), ('1 MONTH ',MONTHS_BETWEEN(NOW + 1 MONTH, NOW)), ('1 MONTH + 1 DAY',MONTHS_BETWEEN(NOW + 1 MONTH + 1 DAY, NOW)), ('LEAP YEAR ',MONTHS_BETWEEN('2016-02-01','2016-03-01')), ('NON-LEAP YEAR ',MONTHS_BETWEEN('2015-02-01','2015-03-01')) ) SELECT * FROM DATES;


FUNCTION RESULT --------------- --------------------------------- 0 MONTH 0.000000000000000 1 MONTH 1.000000000000000 1 MONTH + 1 DAY 1.032258064516129 LEAP YEAR -1.000000000000000 NON-LEAP YEAR -1.000000000000000

DateDurationFunctionsAnalternatewayofrepresentingdatedurationsisthroughtheuseofanintegerwiththeformatYYYYMMDDwheretheYYYYrepresentstheyear,MMforthemonthandDDfortheday.Datedurationsareeasiertomanipulatethantimestampvaluesandtakeupsubstantiallylessstorage.Therearetwonewfunctions.

• YMD_BETWEENreturnsanumericvaluethatspecifiesthenumberoffullyears,fullmonths,andfulldaysbetweentwodatetimevalues

• AGEreturnsanumericvaluethatrepresentsthenumberoffullyears,fullmonths,andfulldaysbetweenthecurrenttimestampandtheargument

ThisSQLstatementwillreturnvariousAGEcalculationsbasedonthecurrenttimestamp.WITH DATES(FUNCTION, RESULT) AS ( VALUES ('AGE+ 1 DAY ',AGE(NOW - 1 DAY)), ('AGE+ 1 MONTH ',AGE(NOW - 1 MONTH)), ('AGE+ 1 YEAR ',AGE(NOW - 1 YEAR)), ('AGE+ 1 DAY + 1 MONTH ',AGE(NOW - 1 DAY - 1 MONTH)), ('AGE+ 1 DAY + 1 YEAR ',AGE(NOW - 1 DAY - 1 YEAR)), ('AGE+ 1 DAY + 1 MONTH + 1 YEAR',AGE(NOW - 1 DAY - 1 MONTH - 1 YEAR)) ) SELECT * FROM DATES; FUNCTION RESULT ------------------------------ ----------- AGE + 1 DAY 1 AGE + 1 MONTH 100 AGE + 1 YEAR 10000 AGE + 1 DAY + 1 MONTH 101 AGE + 1 DAY + 1 YEAR 10001 AGE + 1 DAY + 1 MONTH + 1 YEAR 10101

TheYMD_BETWEENfunctionisliketheAGEfunctionexceptthatittakestwodatearguments.WecansimulatetheAGEfunctionbysupplyingtheNOWfunctiontotheYMD_BETWEENfunction.

Chapter7:DateFunctions 72WITH DATES(FUNCTION, RESULT) AS ( VALUES ('1 DAY ',YMD_BETWEEN(NOW,NOW - 1 DAY)), ('1 MON ',YMD_BETWEEN(NOW,NOW - 1 MONTH)), ('1 YR ',YMD_BETWEEN(NOW,NOW - 1 YEAR)), ('1 DAY + 1 MON ',YMD_BETWEEN(NOW,NOW - 1 DAY - 1 MONTH)), ('1 DAY +1 YR ',YMD_BETWEEN(NOW,NOW - 1 DAY - 1 YEAR)), ('1 DAY+1 MON +1 YR',YMD_BETWEEN(NOW,NOW - 1 DAY - 1 MONTH - 1 YEAR)) ) SELECT * FROM DATES; FUNCTION RESULT ------------------------------ ----------- 1 DAY 1 1 MONTH 100 1 YEAR 10000 1 DAY + 1 MONTH 101 1 DAY + 1 YEAR 10001 1 DAY + 1 MONTH + 1 YEAR 10101

OVERLAPSPredicateTheOVERLAPSpredicateisusedtodeterminewhethertwochronologicalperiodsoverlap.ThisisnotafunctionwithinDb2,butratheraspecialSQLsyntaxextension.Achronologicalperiodisspecifiedbyapairofdate-timeexpressions.Thefirstexpressionspecifiesthestartofaperiod;thesecondspecifiesitsend.(start1, end1) OVERLAPS (start2, end2)

Thebeginningandendvaluesarenotincludedintheperiods.Thefollowingsummarizestheoverlaplogic.Forexample,theperiods2016-10-19to2016-10-20and2016-10-20to2016-10-21donotoverlap.Thefollowingintervaldoesnotoverlap.VALUES CASE WHEN (NOW, NOW + 1 DAY) OVERLAPS (NOW + 1 DAY, NOW + 2 DAYS) THEN 'Overlaps' ELSE 'No Overlap' END; 1 ---------- No Overlap


Ifthefirstdaterangeisextendedbyoneday,thentherangewilloverlap.VALUES CASE WHEN (NOW, NOW + 2 DAYS) OVERLAPS (NOW + 1 DAY, NOW + 2 DAYS) THEN 'Overlaps' ELSE 'No Overlap' END; 1 ---------- Overlaps

Identicaldaterangeswilloverlap.VALUES CASE WHEN (NOW, NOW + 1 DAY) OVERLAPS (NOW, NOW + 1 DAY) THEN 'Overlaps' ELSE 'No Overlap' END; 1 ---------- Overlaps

UTCTimeConversionsDb2hastwofunctionsthatallowyoutotranslatetimestampstoandfromUTC(CoordinatedUniversalTime).

• TheFROM_UTC_TIMESTAMPscalarfunctionreturnsaTIMESTAMPthatisconvertedfromCoordinatedUniversalTimetothetimezonespecifiedbythetimezonestring.

• TheTO_UTC_TIMESTAMPscalarfunctionreturnsaTIMESTAMPthatisconvertedtoCoordinatedUniversalTimefromthetimezonethatisspecifiedbythetimezonestring.

Theformatofthetwofunctionsis:FROM_UTC_TIMESTAMP( expression, timezone)

TO_UTC_TIMESTAMP( expression, timezone)

Thereturnvaluefromeachofthesefunctionsisatimestamp.The"expression"isatimestampthatyouwanttoconverttothelocaltimezone(orconverttoUTC).Thetimezoneisanexpressionthatspecifiesthetimezonethattheexpressionistobeadjustedto.Thevalueofthetimezone-expressionmustbeatimezonenamefromtheInternetAssignedNumbersAuthority(IANA)timezonedatabase.Thestandard


formatforatimezonenameintheIANAdatabaseisArea/Location,where:

• AreaistheEnglishnameofacontinent,ocean,orthespecialarea'Etc'

• LocationistheEnglishnameofalocationwithinthearea;usuallyacity,orsmallisland

Examples:

• "America/Toronto"• "Asia/Sakhalin"• "Etc/UTC"(whichrepresentsCoordinatedUniversalTime)

Forcompletedetailsonthevalidsetoftimezonenamesandtherulesthatareassociatedwiththosetimezones,refertotheIANAtimezonedatabase.Db211.1usesversion2010coftheIANAtimezonedatabase.Theresultisatimestamp,adjustedfrom/totheCoordinatedUniversalTimezonetothetimezonespecifiedbythetimezone-expression.Ifthetimezone-expressionreturnsavaluethatisnotatimezoneintheIANAtimezonedatabase,thenthevalueofexpressionisreturnedwithoutbeingadjusted.

UsingUTCFunctionsOneoftheapplicationsforusingtheUTCistotakeatransactiontimestampandnormalizeitacrossallsystemsthataccessthedata.YoucanconvertthetimestamptoUTConinsertandthenwhenitisretrieved,itcanbeconvertedbacktothelocaltimezone.Thisexamplewilluseseveraltechniquestohidethecomplexityofchangingtimestampstolocaltimezones.ThefollowingSQLwillcreateourbasetransactiontable(TXS_BASE)thatwillbeusedthroughouttheexample.CREATE TABLE TXS_BASE ( ID INTEGER NOT NULL, CUSTID INTEGER NOT NULL, TXTIME_UTC TIMESTAMP NOT NULL );


TheUTCfunctionswillbewrittentotakeadvantageofalocaltimezonevariablecalledTIME_ZONE.Thisvariablewillcontainthetimezoneoftheserver(oruser)thatisrunningthetransaction.InthiscaseweareusingthetimezoneinToronto,Canada.CREATE OR REPLACE VARIABLE TIME_ZONE VARCHAR(255) DEFAULT('America/Toronto');

TheSETCommandcanbeusedtoupdatetheTIME_ZONEtothecurrentlocationwearein.SET TIME_ZONE = 'America/Toronto';

Toretrievethevalueofthecurrenttimezone,wetakeadvantageofasimpleuser-definedfunctioncalledGET_TIMEZONE.ItjustretrievesthecontentsofthecurrentTIME_ZONEvariablethatwesetup.CREATE OR REPLACE FUNCTION GET_TIMEZONE() RETURNS VARCHAR(255) LANGUAGE SQL CONTAINS SQL RETURN (TIME_ZONE);

TheTXSviewisusedbyallSQLstatementsratherthantheTXS_BASEtable.ThereasonforthisistotakeadvantageofINSTEADOFtriggersthatcanmanipulatetheUTCwithoutmodifyingtheoriginalSQL.NotethatwhenthedataisreturnedfromtheviewthattheTXTIMEfieldisconvertedfromUTCtothecurrentTIMEZONEthatwearein.CREATE OR REPLACE VIEW TXS AS ( SELECT ID, CUSTID, FROM_UTC_TIMESTAMP(TXTIME_UTC,GET_TIMEZONE()) AS TXTIME FROM TXS_BASE );

AnINSTEADOFtrigger(INSERT,UPDATE,andDELETE)iscreatedagainsttheTXSviewsothatanyinsertorupdateonaTXTIMEcolumnwillbeconvertedbacktotheUTCvalue.Fromanapplicationperspective,weareusingthelocaltime,nottheUTCtime.CREATE OR REPLACE TRIGGER I_TXS INSTEAD OF INSERT ON TXS REFERENCING NEW AS NEW_TXS FOR EACH ROW MODE DB2SQL BEGIN ATOMIC INSERT INTO TXS_BASE VALUES (

Chapter7:DateFunctions 76 NEW_TXS.ID, NEW_TXS.CUSTID, TO_UTC_TIMESTAMP(NEW_TXS.TXTIME,GET_TIMEZONE()) ); END / CREATE OR REPLACE TRIGGER U_TXS INSTEAD OF UPDATE ON TXS REFERENCING NEW AS NEW_TXS OLD AS OLD_TXS FOR EACH ROW MODE DB2SQL BEGIN ATOMIC UPDATE TXS_BASE SET (ID, CUSTID, TXTIME_UTC) = (NEW_TXS.ID, NEW_TXS.CUSTID, TO_UTC_TIMESTAMP(NEW_TXS.TXTIME,TIME_ZONE) ) WHERE TXS_BASE.ID = OLD_TXS.ID ; END / CREATE OR REPLACE TRIGGER D_TXS INSTEAD OF DELETE ON TXS REFERENCING OLD AS OLD_TXS FOR EACH ROW MODE DB2SQL BEGIN ATOMIC DELETE FROM TXS_BASE WHERE TXS_BASE.ID = OLD_TXS.ID ; END /

Atthispointintime(!)wecanstartinsertingrecordsintoourtable.WehavealreadysetthetimezonetobeToronto,sothenextinsertstatementwilltakethecurrenttime(NOW)andinsertitintothetable.Forreference,hereisthecurrenttimewhentheexamplewasrun.VALUES NOW; 1 -------------------------- 2016-09-17 05:53:18.956000

Wewillinsertonerecordintothetableandimmediatelyretrievetheresult.INSERT INTO TXS VALUES(1,1,NOW); SELECT * FROM TXS; ID CUSTID TXTIME ----------- ----------- -------------------------- 1 1 2016-09-17 05:53:19.056000


Notethatthetimestampappearstobethesameaswhatweinsert(plusorminusafewseconds).WhatactuallysitsinthebasetableistheUTCtime.SELECT * FROM TXS_BASE; ID CUSTID TXTIME_UTC ----------- ----------- -------------------------- 1 1 2016-09-17 09:53:19.056000

Wecanmodifythetimethatisreturnedtousbychangingourlocaltimezone.ThestatementwillmakethesystemthinkweareinVancouver.SET TIME_ZONE = 'America/Vancouver';

Retrievingtheresultswillshowthatthetimestamphasshiftedby3hours(Vancouveris3hoursbehindToronto).SELECT * FROM TXS; ID CUSTID TXTIME ----------- ----------- -------------------------- 1 1 2016-09-17 02:53:19.056000

So,whathappensifweinsertarecordintothetablenowthatweareinVancouver?INSERT INTO TXS VALUES(2,2,NOW); SELECT * FROM TXS; ID CUSTID TXTIME ----------- ----------- -------------------------- 1 1 2016-09-17 02:53:19.056000 2 2 2016-09-17 05:53:19.436000

ThedataretrievedreflectsthefactthatwearenowinVancouverfromanapplicationperspective.LookingatthebasetableandyouwillseethateverythinghasbeenconvertedtoUTCtime.SELECT * FROM TXS_BASE; ID CUSTID TXTIME_UTC ----------- ----------- -------------------------- 1 1 2016-09-17 09:53:19.056000 2 2 2016-09-17 12:53:19.436000

Finally,wecanswitchbacktoTorontotimeandseewhenthetransactionsweredone.YouwillseethatfromaTorontoperspective


thatthetransactionsweredonethreehourslaterbecauseofthetimezonedifferences.SET TIME_ZONE = 'America/Toronto'; SELECT * FROM TXS; ID CUSTID TXTIME ----------- ----------- -------------------------- 1 1 2016-09-17 05:53:19.056000 2 2 2016-09-17 08:53:19.436000

8

Binary Manipulation

NEWBINARYANDHEXADECIMALOPERATORS

Chapter8:BinaryManipulation 80

HexFunctions

SeveralnewHEXmanipulationfunctionshavebeenaddedtoDb211.1.Thereareaclassoffunctionsthatmanipulatedifferentsizeintegers(SMALL,INTEGER,BIGINT)usingNOT,OR,AND,andXOR.Inadditiontothesefunctions,thereareseveralfunctionsthatdisplayandconvertvaluesintohexadecimalvalues.

INTNFunctionsTheINTNfunctionsarebitwisefunctionsthatoperateonthe"two'scomplement"representationoftheintegervalueoftheinputargumentsandreturntheresultasacorrespondingbase10integervalue.Thefunctionnamesallincludethesizeoftheintegersthatarebeingmanipulated:Table10:ArgumentSizes

Identifier(N) IntegerSize2 SMALLINT–2bytes4 INTEGER–4bytes8 BIGINT–8bytesTherearefourfunctions:

• INTNAND-PerformsabitwiseANDoperation• INTNOR-PerformsabitwiseORoperation• INTNXORPerformsabitwiseexclusiveORoperation• INTNNOT-PerformsabitwiseNOToperation

ThisexamplewillshowthefourfunctionsusedagainstINT2datatypes. WITH LOGIC(EXAMPLE, X, Y, RESULT) AS ( VALUES ('INT2AND(X,Y)',1,3,INT2AND(1,3)), ('INT2OR(X,Y) ',1,3,INT2OR(1,3)), ('INT2XOR(X,Y)',1,3,INT2XOR(1,3)), ('INT2NOT(X) ',1,3,INT2NOT(1)) ) SELECT * FROM LOGIC; EXAMPLE X Y RESULT ------------ ------ ------ ------ INT2AND(X,Y) 1 3 1 INT2OR(X,Y) 1 3 3 INT2XOR(X,Y) 1 3 2 INT2NOT(X) 1 3 -2

Chapter8:BinaryManipulation 81

YoucanmixandmatchtheINT2,INT4,andINT8valuesinthesefunctionsbutyoumaygettruncationifthevalueistoobig.

TO_HEXFunctionTheTO_HEXfunctionconvertsanumericexpressionintoacharacterhexadecimalrepresentation.Forexample,thenumericvalue255representsx'FF'.ThevaluereturnedfromthisfunctionisaVARCHARvalueanditslengthdependsonthesizeofthenumberyousupply.VALUES TO_HEX(255); 1 -------- ff

RAWTOHEXFunctionTheRAWTOHEXfunctionreturnsahexadecimalrepresentationofavalueasacharacterstring.Theresultisacharacterstringitself.VALUES RAWTOHEX('00'); 1 ---- 3030

Thestring"00"convertstoahexrepresentationofx'3030'whichis12336inDecimal.So,theTO_HEXfunctionwouldconvertthisbacktotheHEXrepresentation. VALUES TO_HEX(12336); 1 -------- 3030

9

Regular Expressions

EXTENSIVEREGULAREXPRESSIONSUPPORTFORSEARCHINGANDDATAVALIDATION

Chapter9:RegularExpressions 83

RegularExpressions

Db211.1introducedsupportforregularexpressions.Regularexpressionsallowyoutodoverycomplexpatternmatchingincharacterstrings.NormalSQLLIKEsearchesarelimitedtoveryspecificpatterns,butregularexpressionshavearichsyntaxthatgivesyoumuchmoreflexibilityinsearching.

RegularExpressionExamplesTheexamplesinthissectionareusingalistofsomeoftheundergroundstationsofLondonUndergroundCentralline(existingstationsonly,nothistoricalones!).Thistablewillbeusedforalloftheexampleswithinthissection.CREATE TABLE CENTRAL_LINE ( STATION_NO INTEGER GENERATED ALWAYS AS IDENTITY, STATION VARCHAR(31), UPPER_STATION VARCHAR(31) GENERATED ALWAYS AS (UCASE(STATION)) );

Thelistofstationsis:Table1:LondonCentralLine(Subset)

WestRuislip RuislipGardens SouthRuislip NortholtGreenford Perivale HangerLane EalingBroadwayWestActon NorthActon EastActon WhiteCityShepherd'sBush HollandPark NottingHillGate QueenswayLancasterGate MarbleArch BondStreet OxfordCircusTottenhamCourtRoad

Holborn ChanceryLane St.Paul's

Bank LiverpoolStreet BethnalGreen MileEndStratford Leyton Leytonstone WansteadRedbridge GantsHill NewburyPark BarkingsideFairlop Hainault GrangeHill ChigwellRodingValley Snaresbrook SouthWoodford WoodfordBuckhurstHill Loughton Debden TheydonBoisEpping


RegularExpressionCommandsTherearesixregularexpressionfunctionswithinDb2including:

• REGEXP_COUNT–Returnsacountofthenumberoftimesthataregularexpressionpatternismatchedinastring.

• REGEXP_EXTRACT–Returnsoneoccurrenceofasubstringofastringthatmatchestheregularexpressionpattern.

• REGEXP_INSTR–Returnsthestartingorendingpositionofthematchedsubstring,dependingonthevalueofthereturnoptionargument.

• REGEXP_LIKE–ReturnsaBooleanvalueindicatingiftheregularexpressionpatternisfoundinastring.Thefunctioncanbeusedonlywhereapredicateissupported.

• REGEXP_MATCH_COUNT–Returnsacountofthenumberoftimesthataregularexpressionpatternismatchedinastring.

• REGEXP_REPLACE–Returnsamodifiedversionofthesourcestringwhereoccurrencesoftheregularexpressionpatternfoundinthesourcestringarereplacedwiththespecifiedreplacementstring.

• REGEXP_SUBSTR–Returnsoneoccurrenceofasubstringofastringthatmatchestheregularexpressionpattern.

Eachoneofthesefunctionsfollowsasimilarcallingsequence: REGEXP_FUNCTION(source, pattern, flags, start_pos, codeunits)

Theargumentstothefunctionare:

• Source–stringtobesearched• Pattern–theregularexpressionthatcontainswhatweare

searchingfor• Flag–settingsthatcontrolhowmatchingisdone• Start_pos–wheretostartinthestring• Codeunits–whichtypeunitofmeasurementstart_posrefersto

(forUnicode)

ThesourcecanbeanyvalidDb2stringincludingCHAR,VARCHAR,CLOB,etc.Start_posisthelocationinthesourcestringthatyouwanttostartsearchingfrom,andcodeunitstellsDb2whethertotreatthestart_posasanabsolutelocation(thinkbytelocation)oracharacterlocationwhichtakesintoaccounttheUnicodesizeofthecharacterstring.


CodeunitscanbespecifiedasCODEUNITS16,CODEUNITS32,orOCTETS.CODEUNITS16specifiesthatstartisexpressedin16-bitUTF-16codeunits.CODEUNITS32specifiesthatstartisexpressedin32-bitUTF-32codeunits.OCTETSspecifythatstartisexpressedinbytes.Patternandflagvaluesarecomplexandsoarediscussedinthefollowingsections.

RegularExpressionFlagValuesRegularexpressionfunctionshaveaflagspecificationthatcanbeusedtochangethebehaviorofthesearch.TherearesixpossibleflagsthatcanbespecifiedaspartoftheREGEXPcommand:Table11:RegularExpressionFlags

Flag Meaningc Specifiesthatmatchingiscase-sensitive(thedefaultvalue)I Specifiesthatmatchingiscaseinsensitivem Specifiesthattheinputdatacancontainmorethanoneline.Bydefault,the

'^'inapatternmatchesonlythestartoftheinputstring;the'$'inapatternmatchesonlytheendoftheinputstring.Ifthisflagisset,"^"and"$"alsomatchesatthestartandendofeachlinewithintheinputstring.

n Specifiesthatthe'.'characterinapatternmatchesalineterminatorintheinputstring.Bydefault,the'.'characterinapatterndoesnotmatchalineterminator.Acarriage-returnandline-feedpairintheinputstringbehavesasasingle-lineterminator,andmatchesasingle"."inapattern.

s Specifiesthatthe'.'characterinapatternmatchesalineterminatorintheinputstring.Thisvalueisasynonymforthe'n'value.

x Specifiesthatwhitespacecharactersinapatternareignored,unlessescaped.

RegularExpressionSearchPatternsRegularexpressionsusecertaincharacterstorepresentwhatismatchedinastring.Thesimplestpatternisastringbyitself.


SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'Ruislip'); STATION -------------------- West Ruislip Ruislip Gardens South Ruislip

Thepattern'Ruislip'willlookforamatchofRuislipwithintheSTATIONcolumn.Notethatthispatternwillalsomatch'WestRuislip'or'RuislipGardens'sinceweplacednorestrictiononwherethepatterncanbefoundinthestring.Thematchwillalsobeexact(casematters).ThistypeofsearchwouldbeequivalenttousingtheSQLLIKEstatement:SELECT STATION FROM CENTRAL_LINE WHERE STATION LIKE '%Ruislip%'; STATION -------------------- West Ruislip Ruislip Gardens South Ruislip

Ifyoudidn'tplacethe%atthebeginningoftheLIKEstring,onlythestationsthatstartwithRuislipwouldbefound.SELECT STATION FROM CENTRAL_LINE WHERE STATION LIKE 'Ruislip%'; STATION -------------------- Ruislip Gardens

IfyouwanttomatchRuislipwithupperorlowercasebeingignored,youwouldaddthe'i'flagaspartoftheREGEXP_LIKE(oranyREGEXPfunction).SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'RUISLIP','i'); STATION -------------------- West Ruislip Ruislip Gardens South Ruislip

AnchoringPatternsinaSearchBydefault,apatternwillbematchedanywhereinastring.OurpreviousexampleshowedhowRuislipcouldbefoundanywhereinastring.To


forceamatchtostartatthebeginningofastring,thecaratsymbol^canbeusedtoforcethematchtooccuratthebeginningofastring. SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'^Ruislip'); STATION -------------------- Ruislip Gardens

Tomatchapatternattheendofthestring,thedollarsign$canbeused.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'Ruislip$'); STATION -------------------- West Ruislip South Ruislip

Toforceanexactmatchwithastringyouwoulduseboththebeginningandendanchors.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'^Leyton$'); STATION -------------------- Leyton

Notethatifwedidn'tusetheendanchor,wearegoingtogetmorethanoneresult.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'^Leyton'); STATION -------------------- Leyton Leytonstone

MatchingpatternsacrossmultiplelinesSofartheexampleshavedealtwithstringsthatdonotcontainnewlinecharacters(orcarriagefeeds).Insomeapplications,datafromaninputpanelmayincludemultiplelineswhichmaycontainhardlinefeeds.Whatthismeansisthattherearemultiplelinesinthedata,butfromadatabaseperspective,thereisonlyonelineintheVARCHARfield.YoucanmodifythebehavioroftheRegularExpressionsearchbyinstructingittohonortheCRLFcharactersaslinedelimiters.


ThefollowingSQLwillinsertasinglelinewithmultipleCRLFcharactersinittosimulateamulti-linetextstring.CREATE TABLE LONGLINE (NAME VARCHAR(255)); INSERT INTO LONGLINE VALUES 'George' || CHR(10) || 'Katrina';

SearchingforKatrinaatthebeginningandendofstringdoesn'twork.SELECT COUNT(*) FROM LONGLINE WHERE REGEXP_LIKE(NAME,'^Katrina$'); 1 ----------- 0

WecanoverridetheregularexpressionsearchbytellingittotreateachNL/CRLFastheendofastringwithinastring.SELECT COUNT(*) FROM LONGLINE WHERE REGEXP_LIKE(NAME,'^Katrina$','m'); 1 ----------- 1

LogicalOROperatorRegularexpressionscanmatchmorethanonepattern.TheORoperator(|)isusedtodefinealternativepatternsthatcanmatchinastring.Thefollowingexamplesearchesforstationsthathave"ing"intheirnameaswellas"hill".SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'way|ing'); STATION -------------------- Ealing Broadway Notting Hill Gate Queensway Barkingside Roding Valley Epping

Somethingstobeawareofwhencreatingthesearchpattern.Spacesinthepatternsthemselvesaresignificant.Iftheprevioussearchpatternhadaspaceinoneofthewords,itwouldnotfindit(unlessofcoursetherewasaspaceinthestationname).


SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'way| ing'); STATION -------------------- Ealing Broadway Queensway

Usingthe"x"flagwillignoreblanksinyourpattern,sothiswouldfixissuesthatwehaveinthepreviousexample.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'way| ing','x'); STATION -------------------- Ealing Broadway Notting Hill Gate Queensway Barkingside Roding Valley Epping

Bracketscanbeusedtomakeitclearwhatthepatternisthatyouaresearchingforandavoidtheproblemofhavingblanksintheexpression.Bracketsdohaveaspecificusageinregularexpressions,buthereweareusingitonlytoseparatethetwosearchstrings.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(way)|(ing)'); STATION -------------------- Ealing Broadway Notting Hill Gate Queensway Barkingside Roding Valley Epping

CombiningPatternsAswefoundoutintheprevioussection,thereisanORoperatorthatyoucanusetoselectbetweentwopatterns.Howdoyourequestthatmultiplepatternsbepresent?First,wemustunderstandhowmatchingoccurswhenwehavemultiplestringsthatneedtobematchedthathaveanunknownnumberofcharactersbetweenthem.


Forinstance,howdowecreateapatternthatlooksfor"ing"followedby"way"somewhereinthestring?Regularexpressionsrecognizethe"."(period)characterasmatchinganything.Followingthepattern,youcanaddamodifierthatspecifieshowmanytimesyouwantthepatternmatched:Table12:LondonCentralLine(Subset)

Char Meaning* Matchzeroormoretimes? Matchzerooronetimes+ Matchoneormoretimes{m} Matchexactlymtimes{m,} Matchasleastaminimumofmtimes{m,n} Matchatleastaminimumofmtimesandnomorethanntimes

Thefollowingregularexpressionsearchesforapatternwith"ing"followedbyanycharactersandthen"way".SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(ing)*.(way)'); STATION -------------------- Ealing Broadway Queensway

Thepreviousanswergaveyoutworesults(EalingBroadwayandQueensway).Whytwo?Thereasonisthatweusedthe*inthewrongplace(asinglecharacterinawrongplacecanresultinverydifferentresults!).Whatwereallyneededtodowasplacea.*afterthe(ing)tomatch"ing"andthenanycharacters,beforematching"way".Whatourquerydidabovewasmatch0ormoreoccurrencesof"ing",followedbyanycharacter,andthenmatch"way".Hereisthecorrectquery.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(ing).*(way)') STATION

--------------------Ealing Broadway

Findingatleastoneoccurrenceofapatternrequirestheuseofthe+operator,orthebracketoperators.Thisexamplelocatesatleastoneoccurrenceofthe"an"stringinstationnames.


SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(an)+'); STATION -------------------- Hanger Lane Holland Park Lancaster Gate Chancery Lane Bank Wanstead Gants Hill Grange Hill

Ifwewanttofindanexactnumberofoccurrences,weneedtousethe{}notationtotelltheregularexpressionmatcherhowmanywewanttofind.Thesyntaxofthe{}matchis:

• {m}–Matchexactlymtimes• {m,}–Matchasleastaminimumofmtimes• {m,n}–Matchatleastaminimumofmtimesandnomorethann

times

Sothe"+"symbolisequivalenttothefollowingregularexpressionusingthe{}syntax.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(an){1,}'); STATION -------------------- Hanger Lane Holland Park Lancaster Gate Chancery Lane Bank Wanstead Gants Hill Grange Hill

Ifwewanttomatchexactly2'an'patternsinastring,wewouldthinkthatchangingtheexpressionto{2}wouldwork.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(an){2}'); No records found.

Sadly,wegetnoresults!Thiswouldappeartobethewrongresult,butit'sbecausewegotluckywithourfirstsearch!Thebestwaytofigureout


whatwematchedintheoriginalqueryistousetheREGEXP_EXTRACTandREGEXP_INSTRfunctions.

• REGEXP_EXTRACT–Returnsoneoccurrenceofasubstringofastringthatmatchestheregularexpressionpattern.

• REGEXP_INSTR–Returnsthestartingorendingpositionofthematchedsubstring,dependingonthevalueofthereturnoptionargument.

ThefollowingSQLgivesusacluetowhatwasfoundwiththe(an)pattern.SELECT STATION, REGEXP_INSTR(STATION,'(an)') AS LOCATION, REGEXP_EXTRACT(STATION,'(an)') AS EXTRACT FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(an)'); STATION LOCATION EXTRACT -------------------- ----------- -------------------- Hanger Lane 2 an Holland Park 5 an Lancaster Gate 2 an Chancery Lane 3 an Bank 2 an Wanstead 2 an Gants Hill 2 an Grange Hill 3 an

Whatyoushouldseeinthepreviousresultisthelocationwherethe"an"patternwasfoundinthestring.Notethatallwematchedwasthe"an"pattern,nothingelse.Sowhycan'tIfindtwo"an"patternsinthestring?Thereasonisthat(an){2}means"an"followedbyanother"an"!Wedidn'ttellthepatterntomatchanythingelse!Whatweneedtodoismodifythepatterntosaythatitcanmatch"an"followedbyanythingelse.Thepatternneedstobemodifiedto(an).*wherethe".*"meansanycharacterfollowingthe"an".Inordertotelltheregularexpressionfunctiontousethisentirepattern(an).*twice,weneedtoplacebracketsarounditaswell.Thefinalpatternis((an).*){2}.


SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'((an).*){2}'); STATION -------------------- Hanger Lane Chancery Lane

Youshouldfindthattwostationsmatchthepattern.ThefollowingSQLshowswhichpatternismatchedfirstintheSTATIONSnames.SELECT STATION, REGEXP_INSTR(STATION,'((an).*){2}') AS LOCATION, REGEXP_EXTRACT(STATION,'((an).*){2}') AS EXTRACT FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'((an).*){2}'); STATION LOCATION EXTRACT -------------------- ----------- -------------------- Hanger Lane 2 anger Lane Chancery Lane 3 ancery Lane

MatchingcharactertypesAsidefrommatchingentirestrings,youcanalsouseregularexpressiontolookforpatternsofcharacters.Thesimplestmatchingpatternistheperiod(.)whichmatchesanycharacter.Matchingastringofarbitrarylengthisthepattern".*".The"+"and"?"characterscanalsobeusedtomodifyhowmanycharactersyouwantmatched.Whataboutsituationswhereyouwanttocheckforcertainpatternsorcharactersinastring?Agoodexamplewouldbeasocialsecuritynumber,orcreditcardnumber.Therearecertainpatternsthatyouwouldfindfortheseobjects.Assumewehaveasocialsecuritynumberintheformatxxx-xx-xxxx.ItispossibletocreatearegularexpressionthatwouldreturntrueiftheSSNmatchedthepatternabove(itdoesn'ttellyouiftheSSNitselfisvalid,onlythatithastheproperformat).Regularexpressionsallowyoutocreatealistofcharactersthatneedtobematchedinsomethingcalledabracketexpression.Abracketexpressionhastheformat:[a-z] [A-Z] [0-9] [a-zA-z]


Theexamplesaboverepresentthefollowingsearchpatterns:

• [a-z]-matchanyseriesoflowercasecharactersbetweenaandz• [A-Z]-matchanyseriesofuppercasecharactersbetweenAandZ• [0-9]-matchanyvaliddigits• [a-zA-Z]-matchanylower-oruppercaseletters

Youcanalsoenumerateallofthecharactersyouwanttomatchbylistingthembetweenthebracketslike[abcdefghikjlmnopqrstuvwxyz].Theshortforma-ziseasiertoreadandlesspronetotypingerrors!ThefollowingexamplechecksforstationnamesthatbeginwiththeletterP-R.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'^[P-R]'); STATION -------------------- Ruislip Gardens Perivale Queensway Redbridge Roding Valley

IfyouwantedtoincludeallstationsthathavetheletterP-Rorp-e,youcouldaddtheconditionwithinthebrackets.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'[p-rP-R]'); STATION -------------------- West Ruislip Ruislip Gardens South Ruislip Northolt Greenford Perivale Hanger Lane Ealing Broadway North Acton Shepherd's Bush

BacktoourSSNquestion.Canaregularexpressionpatternbeusedtodeterminewhetherthestringisinthecorrectformat?ThedatawillbeintheformatXXX-XX-XXXXsotheregularexpressionneedstofindthethreenumericvaluesseparatedbydashes.


Thenumberpatterncanberepresentedwiththebracketexpression[0-9].Tospecifythenumberofcharactersthatneedtobefound,weusethebraces{}tospecifytheexactnumberrequired.Forthethreenumbersinthepatternwecanuse[0-9]{3},[0-9]{2},and[0-9]{4}.Addinginthedashesgivesusthefinalpattern.TheSQLbelowcheckstoseeifaSSNiscorrect.VALUES CASE WHEN REGEXP_LIKE('123-34-1422','[0-9]{3}-[0-9]{2}-[0-9]{4}') THEN 'Valid' ELSE 'Invalid' END; 1 ------- Valid

TheSSNisvalidintheexampleabove.Herearesomeotherexamplestoshowwhetherornottheregularexpressionpicksupalloftheerrors.WITH SSNS(SSN) AS ( VALUES '123-34-1322', 'ABC-34-9999', 'X123-44-0001', '123X-Y44-Z0001', '111-222-111' ) SELECT SSN, CASE WHEN REGEXP_LIKE(SSN,'[0-9]{3}-[0-9]{2}-[0-9]{4}') THEN 'Valid' ELSE 'Invalid' END FROM SSNS; SSN 2 -------------- ------- 123-34-1322 Valid ABC-34-9999 Invalid X123-44-0001 Valid 123X-Y44-Z0001 Invalid 111-222-111 Invalid

Ifyoucheckclosely,oneofthestringswasmarkedasvalid,althoughitisnotcorrect(X123-44-0001).Thereasonthisoccurredisthatthepatternwasfoundafterthe"X"anditwascorrect.Topreventthisfromhappening,weneedtoanchorthepatternatthebeginningtoavoidthissituation.Abetterpatternwouldbetoanchorbothendsofthepattern


sothereisnopossibilityofothercharactersbeingatthebeginningorendofthepattern.WITH SSNS(SSN) AS ( VALUES '123-34-1322', 'ABC-34-9999', 'X123-44-0001', '123X-Y44-Z0001', '111-222-111' ) SELECT SSN, CASE WHEN REGEXP_LIKE(SSN,'^[0-9]{3}-[0-9]{2}-[0-9]{4}$') THEN 'Valid' ELSE 'Invalid' END FROM SSNS; SSN 2 -------------- ------- 123-34-1322 Valid ABC-34-9999 Invalid X123-44-0001 Invalid 123X-Y44-Z0001 Invalid 111-222-111 Invalid

SpecialPatternsThepreviousexampleusedthe[0-9]syntaxtorequestthatonlynumbersbefoundinthepattern.Therearesomepredefinedpatternsthatdefinethesecommonpatterns.ThefirstargumentisPosixformat(ifitexists),thesecondistheescapecharacterequivalent,andthefinaloneistherawpatternitrepresents.Table13:RegularExpressionSpecialPatterns

Posix Escape Pattern Meaning[:alnum:] [A-Za-z0-9] Alphanumericcharacters \w [A-Za-z0-9_] Alphanumericcharactersplus"_" \W [^A-Za-z0-9_] Non-wordcharacters[:alpha:] \a [A-Za-z] Alphabeticcharacters[:blank:] \s,\t Spaceandtab \b Wordboundaries[:cntrl:] [\x00-\x1F\x7F] Controlcharacters[:digit:] \d [0-9] Digits \D [^0-9] Non-digits[:graph:] [\x21-\x7E] Visiblecharacters[:lower:] \l [a-z] Lowercaseletters[:print:] \p [\x20-\x7E] Visiblecharactersandthespacecharacter


[:punct:] [][!"#$%&'()*+,./:;<=>?@\^_`{|}~-]

Punctuationcharacters

[:space:] \s [\t\r\n\v\f] Whitespacecharacters \S [^\t\r\n\v\f] Non-whitespacecharacters[:upper:] \u [A-Z] Uppercaseletters[:xdigit:] \x [A-Fa-f0-9] HexadecimaldigitsForinstance,thefollowingthreestatementswillproducethesameresult.WITH SSNS(SSN) AS ( VALUES '123-34-1322', 'ABC-34-9999', 'X123-44-0001', '123X-Y44-Z0001', '111-222-111' ) SELECT 'Original', SSN, CASE WHEN REGEXP_LIKE(SSN,'^[0-9]{3}-[0-9]{2}-[0-9]{4}$') THEN 'Valid' ELSE 'Invalid' END FROM SSNS UNION ALL SELECT 'Posix', SSN, CASE WHEN REGEXP_LIKE(SSN,'^[:digit:]{3}-[:digit:]{2}-[:digit:]{4}$') THEN 'Valid' ELSE 'Invalid' END FROM SSNS UNION ALL SELECT 'Escape', SSN, CASE WHEN REGEXP_LIKE(SSN,'^\d{3}-\d{2}-\d{4}$') THEN 'Valid' ELSE 'Invalid' END FROM SSNS; 1 SSN 3 -------- -------------- ------- Original 123-34-1322 Valid Original ABC-34-9999 Invalid Original X123-44-0001 Invalid Original 123X-Y44-Z0001 Invalid Original 111-222-111 Invalid Posix 123-34-1322 Valid Posix ABC-34-9999 Invalid Posix X123-44-0001 Invalid Posix 123X-Y44-Z0001 Invalid Posix 111-222-111 Invalid

NegatingPatternsUptothispointintime,thepatternsthathavebeenusedarelookingforapositivematch.Insomecases,youmaywanttofindvaluesthatdonot


match.TheeasiestwayistonegatetheactualREGEXP_LIKEexpression.Thefollowingexpressionfindsallofthestationsthatstartandendwith"West".SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'^West$'); No records found.

AddingtheNOTmodifierinfrontoftheREGEXPfunctiongivesusthestationsthatdonotbeginwithWest.SELECT STATION FROM CENTRAL_LINE WHERE NOT REGEXP_LIKE(STATION,'^West$'); STATION -------------------- West Ruislip Ruislip Gardens South Ruislip Northolt Greenford Perivale Hanger Lane Ealing Broadway West Acton North Acton

Youcanalsonegatesomeofthesearchesinapatternbyusingthe[^...]syntaxwherethe^tellstheregularexpressionnottomatchthefollowingcharacters.Theexpression[^0-9]wouldmeanmatchanycharacterswhicharenotnumeric.However,regularexpressionshavesomethingcallednegativelookaroundswhichbasicallymeanfindthepatternwhichdoesnotmatch.Youcreatethispatternbyaddingthe(?!..)atthebeginningofthestring.Thesamequery(findingstationsthatdon'tstartwithWest)wouldbewrittenwiththislookaroundlogicfoundintheSQLbelow.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'^(?!West)'); STATION -------------------- Ruislip Gardens South Ruislip Northolt Greenford Perivale Hanger Lane Ealing Broadway


North Acton East Acton White City

CapturingParenthesisThepreviousexampleusedsomethingcalledanegativelookaroundwithcapturingparenthesis.Whenyouplaceapatternwithinasetofbrackets(...)thestringthatmatchesthispatternis"remembered".Thestringsthatarematchedcanbeusedinsubsequentpartsofyourregularexpression.Thisallowsaformofprogrammingwithinyourregularexpression!Eachsetofparenthesesthatarematchedareassociatedwithanumber,startingatoneandincrementingforeachsubsequentpatternmatch.Forinstance,thefollowingpatternwillhavethreematches:^([0-9]{3})-([0-9]{3})-([0-9]{3})$

ThisisliketheSSNexampleusedearlieroninthissection.Thedifferenceinthisexampleisthateachblockofnumbersisthesame(3digits).Thispatternwillmatchanysequenceofnumbersintheformat123-456-789.WITH SSNS(SSN) AS ( VALUES '123-456-789', '123-555-123', '890-533-098', '123-456-456' ) SELECT SSN, CASE WHEN REGEXP_LIKE(SSN,'^([0-9]{3})-([0-9]{3})-([0-9]{3})$') THEN 'Valid' ELSE 'Invalid' END FROM SSNS; SSN 2 ----------- ------- 123-456-789 Valid 123-555-123 Valid 890-533-098 Valid 123-456-456 Valid

Thesenumbersfitthepatternandshouldbevalid.Whenoneofthecapturingparenthesismatches,itwillrememberthestringthatitmatched.Forinstance,inthefirstexample(123-456-789),thefirstmatchwillfindthestring'123'.Thesecondmatchwillfind'456'andsoon.Wecanrefertothesematchedstringswiththespecialcontrolcharacters\n


wherenrepresentsthecapturingparenthesislocation.So\1wouldrefertothe'123'thatwasfound.\2wouldbefor'456'.Theregularexpressionwillbeupdatedsothatthelastpartofthepatternneedstobenumeric(0-9)butcan'tbesameasthefirstmatch.WITH SSNS(SSN) AS ( VALUES '123-456-789', '123-555-123', '890-533-098', '123-456-456' ) SELECT SSN, CASE WHEN REGEXP_LIKE(SSN,'^([0-9]{3})-([0-9]{3})-(?!\1)([0-9]{3})$') THEN 'Valid' ELSE 'Invalid' END FROM SSNS; SSN 2 ----------- ------- 123-456-789 Valid 123-555-123 Invalid 890-533-098 Valid 123-456-456 Valid

Inmanycases,itmaybeeasiertofindthepatternsthatmatchandthennegatetheREGEXPstatement!The(?...)syntaxisusedforavarietyofpurposesinregularexpressions:Table14:CapturingParentheses

Pattern Description Details(?:...) Non-capturing

parenthesesGroupstheincludedpattern,butdoesnotprovidecapturingofmatchingtext.Moreefficientthancapturingparentheses.

(?>...) Atomic-matchparentheses

Firstmatchoftheparenthesizedsubexpressionistheonlyonetried.Ifitdoesnotleadtoanoverallpatternmatch,backupthesearchforamatchtoapositionbeforethe"(?>"

(?#...) Free-formatcomment

(?#comment)

(?=...) Look-aheadassertion

Trueiftheparenthesizedpatternmatchesatthecurrentinputposition,butdoesnotadvancetheinputposition.

(?!...) Negativelook-aheadassertion

Trueiftheparenthesizedpatterndoesnotmatchatthecurrentinputposition.Doesnotadvancetheinputposition.


(?<=...) Look-behindassertion

Trueiftheparenthesizedpatternmatchestextthatprecedesthecurrentinputposition.Thelastcharacterofthematchistheinputcharacterjustbeforethecurrentposition.Doesnotaltertheinputposition.Thelengthofpossiblestringsthatismatchedbythelook-behindpatternmustnotbeunbounded(no*or+operators.)

(?<!...) NegativeLook-behindassertion

Trueiftheparenthesizedpatterndoesnotmatchtextthatprecedesprecedingthecurrentinputposition.Thelastcharacterofthematchistheinputcharacterjustbeforethecurrentposition.Doesnotaltertheinputposition.Thelengthofpossiblestringsthatismatchedbythelook-behindpatternmustnotbeunbounded(no*or+operators.)

Forefficiencyinmatching,thebestapproachistoplacestringsthatyouaresearchingforinnon-capturingparentheses(?:...)ratherthanthegeneric()parenthesis.Thefollowingexamplefindsallstationswith"West"inthename.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(West)'); STATION -------------------- West Ruislip West Acton

ThefollowingSQLisequivalent,exceptthatthematchedpatternisnotkeptforfutureuseinmatching.SELECT STATION FROM CENTRAL_LINE WHERE REGEXP_LIKE(STATION,'(?:West)'); STATION -------------------- West Ruislip West Acton

10

Statistical Functions

NEWANDUPDATEDSTATISTICALFUNCTIONSFORCOMPLEXDATAANALYSIS

Chapter10:StatisticalFunctions 103

StatisticalFunctions

Db2alreadyhasavarietyofStatisticalfunctionsbuiltin.InDb211.1,anumberofnewfunctionshavebeenaddedincluding:

• COVARIANCE_SAMP–Samplecovarianceofasetofnumberpairs• STDDEV_SAMP–Samplestandarddeviation• VARIANCE_SAMP–Samplevarianceofasetofnumbers• CUME_DIST-CumulativeDistribution• PERCENT_RANK-Percentilerank• PERCENTILE_DISCandPERCENTILE_CONT–Percentiles• MEDIAN• WIDTH_BUCKET

SamplingFunctionsThetraditionalVARIANCE,COVARIANCE,andSTDDEVfunctionshavebeenavailableinDb2foralongtime.Whencomputingthesevalues,theformulaeassumethattheentirepopulationhasbeencounted(N).Thetraditionalformulaforstandarddeviationis:

𝜎 = 1𝑁

𝑥' − 𝜇 *+

',-

Nreferstothesizeofthepopulationandinmanycases,weonlyhaveasample,nottheentirepopulationofvalues.Inthiscase,theformulaneedstobeadjustedtoaccountforthesampling.

𝑠 = 1

𝑁 − 1𝑥' − 𝑥 *

+

',-


ThethreefunctionsthatuseasamplingalgorithmareCOVARIANCE_SAMP,STDDEV_SAMP,andVARIANCE_SAMP.COVARIANCE_SAMPreturnsthesamplecovarianceofasetofnumberpairs. SELECT COVARIANCE_SAMP(SALARY, BONUS) FROM EMPLOYEE WHERE WORKDEPT = 'A00'; 1 ------------------------------------------ 5428750

STDDEV_SAMPreturnsthesamplestandarddeviation(divisionby[n-1])ofasetofnumbers.SELECT STDDEV_SAMP(SALARY) FROM EMPLOYEE WHERE WORKDEPT = 'A00'; 1 ------------------------------------------ 46863.03180546474203969595277486485

VARIANCE_SAMPreturnsthesamplevariance(divisionby[n-1])ofasetofnumbers.SELECT VARIANCE_SAMP(SALARY) FROM EMPLOYEE WHERE WORKDEPT = 'A00';

1 ------------------------------------------ 2196143750

AdditionalAnalyticalFunctionsDb211.1alsoincludesseveralanalyticfunctionsthatDataScientistswillfinduseful.ThesefunctionsincludeMEDIAN,CUME_DIST,PERCENT_RANK,PERCENTILE_DISC,PERCENTILE_CONT,andWIDTH_BUCKET.


TheMEDIANcolumnfunctionreturnsthemedianvalueinasetofvalues.SELECT MEDIAN(SALARY) AS MEDIAN, AVG(SALARY) AS AVERAGE FROM EMPLOYEE WHERE WORKDEPT = 'E21'; MEDIAN AVERAGE ----------------------------- ------------------------------------ 41895 47086.66666666666666666666666666667

CUME_DISTreturnsthecumulativedistributionofarowthatishypotheticallyinsertedintoagroupofrows.SELECT CUME_DIST(47000) WITHIN GROUP (ORDER BY SALARY) FROM EMPLOYEE WHERE WORKDEPT = 'A00'; 1 ------------------------------------------ 0.5

ThePERCENT_RANKcolumnfunctionreturnstherelativepercentilerankofarowthatishypotheticallyinsertedintoagroupofrows.SELECT PERCENT_RANK(47000) WITHIN GROUP (ORDER BY SALARY) FROM EMPLOYEE WHERE WORKDEPT = 'A00'; 1 ------------------------------------------ 0.4

ThePERCENTILE_DISCreturnsthevaluethatcorrespondstothespecifiedpercentilegivenasortspecificationbyusingdiscrete(DISC)orcontinuous(CONT)distribution.SELECT PERCENTILE_DISC(0.75) WITHIN GROUP (ORDER BY SALARY) FROM EMPLOYEE WHERE WORKDEPT = 'E21'; 1 ----------- 45370.00 SELECT PERCENTILE_CONT(0.75) WITHIN GROUP (ORDER BY SALARY) FROM EMPLOYEE WHERE WORKDEPT = 'E21'; 1 ---------------------- 4.49875000000000E+004


WIDTHBUCKETandHistogramExampleTheWIDTH_BUCKETfunctionisusedtocreateequal-widthhistograms.UsingtheEMPLOYEEtable,thisSQLwillassignabuckettoeachemployee'ssalaryusingarangeof35000to100000dividedinto13buckets.SELECT EMPNO, SALARY, WIDTH_BUCKET(SALARY, 35000, 100000, 13) FROM EMPLOYEE ORDER BY EMPNO; EMPNO SALARY 3 ------ ----------- ----------- 000010 152750.00 14 000020 94250.00 12 000030 98250.00 13 000050 80175.00 10 000060 72250.00 8 000070 96170.00 13 000090 89750.00 11 000100 86150.00 11 000110 66500.00 7 000120 49250.00 3

Wecanplotthisinformationbyaddingsomemoredetailstothebucketoutput.WITH BUCKETS(EMPNO, SALARY, BNO) AS (SELECT EMPNO, SALARY, WIDTH_BUCKET(SALARY, 35000, 100000, 13) aS BUCKET FROM EMPLOYEE ORDER BY EMPNO) SELECT BNO, COUNT(*) FROM BUCKETS GROUP BY BNO ORDER BY BNO ASC; BNO 2 ----------- ----------- 0 1 1 10 2 3 3 8 4 2 5 2 6 2 7 5 8 2 10 1


Plottingthedataresultsinthefollowingbarchart.

Figure16:HistogramofEmployeeSalaries

11

Hashing Functions

HASHINGFUNCTIONSFORGENERATINGUNIQUEVALUESFORCHARACTERSTRINGS

Chapter11:HashingFunctions 109

HashingFunctions

Hashingfunctionsaretypicallyusedtotakelargertextstringsandmapintoasmaller"hashed"value.Thesehashvaluescanbe4byte(HASH4),8byte(HASH8)orupto64byteslong(HASH).ThereasonforusingvariousHASHalgorithmsdependsonthesizeofthedataandhowmanycollisionsyouexpecttoget.AHASHfunctiondoesnotnecessarilyresultinauniquevalueforeverystringthatishashed.Toavoidcollisions,alargeHASHvalueisoftenused.ThetradeoffistheamountofcomputepowerrequiredtogeneratetheHASHvalueaswellasthesizeoftheHASHkey.HASHvaluescanbeusedforavarietyofpurposes.HASHvaluescanbeusedtocreateartificialpartitionkeysforatable,createachecksumfortransportingthedatatoanapplicationoranotherdatabase,orforindexingphrases.Theapplicationmustbeawarethatduplicatescouldoccursoadditionalcodingmayberequired.

HASH4-Four-byteHashEncodingHASH4isthesimplestofthehashingfunctions,butalsohasthepotentialforthemostcollisionswhencomputinghashvalues.TheHASH4takestwoarguments:thestringtobehashed,andthealgorithmthatistobeusedforthehash. HASH4(string, algorithm)

TheHASH4functionreturnsa4-byteintegervalue.Thealgorithmvaluecanbeeither0or1.Zero(0)isthedefaultvalueandusestheAdleralgorithm.One(1)usestheCRC32algorithm.TheAdleralgorithmprovidesafasterchecksumhash;however,ithaspoorcoveragewhenthestringsarelessthanafewhundredbytesinsize.Poorcoveragemeansthattwodifferentstringshashtothesamevalue,referredtoasa"collision".Inthiscase,usetheCRC32algorithm,orswitchtoHASH8instead.


ThefollowingSQLwillHASHaphraseusingthedefault(Adler)algorithm.VALUES HASH4('Hello there'); 1 ----------- 409338925

ChangingthephraseslightlyresultsinadifferentHASHvalue.VALUES HASH4('Hello therf'); 1 ----------- 409404462

TheSQLismodifiedtousetheCRC32algorithm.VALUES HASH4('Hello there',1); 1 ----------- -342989177

HASH8-Eight-byteHashEncodingTheHASH8functionisidenticaltotheHASH4functionexceptthatitreturnsan8-byte(BIGINT)value.TherearetwoargumentstotheHASH8function.Thefirstisthestringtobehashedandthesecondisthealgorithmtouse.WiththisversionofDb2,thereisonlyonevalueallowed(0)whichtellsthefunctiontousetheJenkinsalgorithm. HASH8(string, algorithm)

TheHASH8functionwillhaveahigherprobabilityofreturninguniquevaluesforallstrings,butthedownsideisthatitwillrequiremorecyclestocomputethehashvalue.VALUES HASH8('Hello there'); 1 -------------------- -844407019926684877


Supplyinganythingotherthanazeroforthesecondargumentwillresultinanerrorbeingreturned.VALUE HASH8('Hello there',1); SQL0104N An unexpected token "VALUE HASH8" was found following "BEGIN-OF-STATEMENT". Expected tokens may include: "<create_index>". SQLSTATE=42601

HASH-VariableHashEncodingTheHASHfunctionisthemostflexibleofallthehashingfunctions.IthasaformatidenticaltotheothertwoHASHfunctions: HASH(string, algorithm)

Therearefourchoicesforalgorithm:Table15:HASHAlgorithmOptions

# Algorithm Length Bytes ReturnValues0 MD5 128 16 2**1281 SHA1 160 20 2**1602 SHA2_256 256 32 2**2563 SHA2_512 512 64 2**512TheresultofthefunctionisaVARBINARYfield.NotethattheMD5andSHA1algorithmshavehadsecurityflawsidentifiedwiththemsoyoumaywanttoconsiderusingahigherstrengthalgorithmifyouareusingthisforsecuritypurposes.VALUES HASH('Hello there',0), HASH('Hello there',1), HASH('Hello there',2), HASH('Hello there',3); 1 ------------------------------------------------------------------ E8EA7A8D1E93E8764A84A0F3DF4644DE 726C76553E1A3FDEA29134F36E6AF2EA05EC5CCE 4E47826698BB4630FB4451010062FADBF85D61427CBDFAED7AD0F23F239BED89 567683DDBA1F5A576B68EC26F41FFBCC7E718D646839AC6C2EF746FE952CEF4CBE6DEA...

12

JSON Functions

EXTENDINGDB2TOMANIPULATEJSONDOCUMENTSWITHINRELATIONALTABLES

Chapter12:ExperimentalJSONFunctions 113

InternalDb2JSONRoutines

ThecurrentDb211.1releaseincludesadriver-basedJSONsolutionthatembracestheflexibilityoftheJSONdatarepresentationwithinarelationaldatabase.UserscanuseaJSONprogrammingparadigmthatismodeledaftertheMongoDBdatamodelandquerylanguageandstorethedatanativelyinDb2.TheJSONdataisstoredinabinary-encodedformatcalledBSON(BinaryJSON).BSONisdesignedtobelightweight,easilytraversedandveryefficientlyencodedanddecoded.UserscaninteractwithJSONdatainmanyways.

• TheycanadministerandinteractivelyqueryJSONdatausingacommandlineshell.

• TheycanprogrammaticallystoreandquerydatafromJavaprogramsusinganIBMprovidedJavadriverforJSONthatallowsuserstoconnecttotheirJSONdatathroughthesameJDBCdriverusedforSQLaccess.

• TheycanuseanydriverthatimplementsportionsoftheMongoDBprotocol.ThisallowsthemtoaccesstheirDb2JSONstorefromavarietyofmodernlanguages,includingnode.js,PHP,Python,andRuby,aswellasmoretraditionallanguagessuchasC,C++,andPerl.

TheseapproachestomanipulatingJSONwithinDb2useavarietyofuser-definedfunctions(UDFs).TheseUDFsarecataloguedwheneveraJSONdatastoreisdefinedwithinDb2.DevelopersdonothaveaccesstotheseSQL-basedroutinessincetheywereoriginallydesignedtobeusedonlybythesespecificJSONinterfaces.However,manycustomerswouldliketostoreandqueryaJSONcolumnthatisdefinedaspartofaDb2table.Whiletheseroutineshavenotbeenofficiallypublished,theyareavailableforcustomeruse.TheseroutinesarenotexternalizedintheDb2documentationbecausetheywereonlyusedbytheinternalAPI'sforqueryingandmanagingJSON.Forthisreason,theseroutineswerenotoriginallyconsidered"supported".Thischaptergivesyoudetailsofthesefunctionsandtheyarenowavailabletouseinyourapplications.NotethattheinternaluseoftheseroutinesbyDb2hasveryspecificusagepatterns,whichmeans


thatitispossiblethatyoumaygenerateasetofSQLthatmaynotbehandledproperly.Note:ThesefunctionsmaychangeinthefuturetoconformtotheSQLstandard.

Db2JSONFunctionsThereisonebuilt-inDb2JSONfunctionandseveralotherfunctionsthatmustberegisteredwithinDb2beforetheycanbeused.Thenamesofthefunctionsandtheirpurposearedescribedbelow.

• JSON_VAL–ExtractsdatafromaJSONdocumentintoSQLdatatypes

• JSON_TABLE–Returnsatableofvaluesforadocumentthathasarraytypesinit

• JSON_TYPE–ReturnsthedatatypeofaspecificfieldwithinaJSONdocument

• JSON_LEN–Returnsthecountofelementsinanarraytypeinsideadocument

• BSON2JSON–ConvertBSONformatteddocumentintoJSONstrings

• JSON2BSON–ConvertJSONstringsintoaBSONdocumentformat• JSON_UPDATE–Updateafieldordocumentusingsetsyntax• JSON_GET_POS_ARR_INDEX–Findavaluewithinanarray• BSON_VALIDATE–CheckstomakesurethataBSONfieldina

BLOBobjectisinacorrectformat

AsidefromtheJSON_VALfunction,allotherfunctionsinthislistmustbecataloguedbeforefirstbeingused.Ascriptthatcanbeusedtocatalogthesefunctions(DB2-V11-JSON-Examples.sql)isavailableinthedownloadfolderforthiseBook.Inaddition,alltheexamplesinthischapterarealsoincludedinthisscript.Pleaserefertotheappendixfordetailsonaccessingthisfileandotherusefulinformation.

�PathStatementRequirementsThreeoftheDb2JSONfunctionsarecataloguedintheSYSIBMschema(JSON_VAL,JSON2BSON,BSON2JSON)whiletheremainderare


cataloguedundertheSYSTOOLSschema.PriortoFP2,alloftheJSONfunctionsneededtobecataloguedmanually.AsofFP2,thecataloguingisdoneautomaticallyforyouwhenyouupgradetheinstanceortheinitialinstallation.IfyouareatapriorreleaseofDb2(11.1.1.1and10.5)alltheDb2JSONfunctionshavebeenplacedintotheSYSTOOLSschema.Thismeansthattoexecuteanyofthesecommands,youmustprefixthecommandwithSYSTOOLS,asinSYSTOOLS.JSON2BSON.Toremovethisrequirement,youmustupdatetheCURRENTPATHvaluetoincludeSYSTOOLSaspartofit.TheSQLbelowwilltellyouwhatthecurrentPATHis. VALUES CURRENT PATH; 1 ------------------------------------------------- "SYSIBM","SYSFUN","SYSPROC","SYSIBMADM","BAKLARZ"

UsethefollowingSQLtoaddSYSTOOLStothecurrentpath. SET CURRENT PATH = CURRENT PATH, SYSTOOLS;

Fromthispointonyouwon'tneedtoaddtheSYSTOOLSschemaonthefrontofanyofyourSQLstatementsthatrefertotheseDb2JSONfunctions.

CreatingTablesthatSupportJSONDocumentsTocreateatablethatwillstoreJSONdata,youneedtodefineacolumnthatcancontainbinarydata.TheJSONcolumnmustbecreatedasaBLOB(binaryobject)datatypebecausethenewVARBINARYdatatypewillnotworkwithanyoftheJSONfunctions.JSONDocumentsarealwaysstoredinBLOBSwhichcanbeaslargeas16M.Toensuregoodperformance,youshouldhaveaBLOBspecifiedasINLINEifpossible.Ifalargeobjectisnotinlined,orgreaterthan32Kinsize,theresultingobjectwillbeplacedintoalargetablespace.TheresultisthatBLOBobjectswillnotbekeptinbufferpools(whichmeansadirectreadisrequiredfromdiskforaccesstoanyBLOBobject)andthattwoI/Osarerequiredtogetanydocument–onefortherowandasecondforthedocument.ByusingtheINLINEoptionandkeepingtheBLOBsizebelow


thepagesize,youcanimprovetheretrievalperformanceofJSONcolumns.ThisSQLwillcreateacolumnthatissuitableforstoringJSONdata: CREATE TABLE TESTJSON ( JSON_FIELD BLOB(4000) INLINE LENGTH 4000 );

NotethatweareassumingthatthesizeoftheJSONobjectwillnotexceed4000charactersinsize.

JSONDocumentRepresentationDb2storesJSONobjectsinamodifiedbinary-encodedformatcalledBSON(BinaryJSON).Asmentionedatthebeginningofthischapter,BSONisdesignedtobelightweight,easilytraversedandveryefficientlyencodedanddecoded.AlltheJSONfunctionsexecuteagainstBSONobjects,nottheoriginalJSONtextstringsthatmayhavebeengeneratedbyanapplication.Tostoreandretrieveanentiredocumentfromacolumninatable,youmustuse:

• BSON2JSON–ConvertaBSONformatteddocumentintoaJSONstring

• JSON2BSON–ConvertJSONstringsintoaBSONdocumentformat

YoucanalsoverifythecontentsofadocumentthatisstoredinacolumnbyusingtheBSON_VALIDATEfunction:

• BSON_VALIDATE–CheckstomakesurethataBSONfieldinaBLOBobjectisinthecorrectformat

YoucannotuseJSONtoBSONconversionfunctionsthatareavailableinsomeprogramminglanguages.TheBSONformatusedbyDb2hassomeextensionstoitthatwillnotbehandledbytheseroutines,soyoumustusetheDb2BSON2JSONandJSON2BSONfunctionsexclusively.

JSON2BSON:InsertingaJSONDocumentInsertingintoaDb2columnrequirestheuseoftheJSON2BSONfunction.TheJSON2BSONfunction(andBSON2JSON)areusedtotransferdatain


andoutofatraditionalDb2BLOBcolumn.InputtotheJSON2BSONfunctionmustbeaproperlyformattedJSONdocument.IfthedocumentdoesnotfollowproperJSONrules,youwillgetanerrorcodefromthefunction. INSERT INTO TESTJSON VALUES ( JSON2BSON('{Name:"George"}'));

ApoorlyformattedJSONdocumentwillreturnanerror. INSERT INTO TESTJSON VALUES ( JSON2BSON('{Name:, Age: 32}')); SQL0443N Routine "JSON2BSON" (specific name "") has returned an error SQLSTATE with diagnostic text "JSON parsing error for: {Name:, Age: 32}, error code: 4 ". SQLSTATE=22546

BSON2JSON:RetrievingaJSONDocumentSincethedatathatisstoredinaJSONcolumnisinaspecialbinaryformatcalledBSON,selectingfromthefieldwillonlyresultinrandomcharactersbeingdisplayed. SELECT * FROM TESTJSON; JSON_FIELD -------------------- 0316000000024E616D65

IfyouwanttoextractthecontentsofaJSONfield,youmustusetheBSON2JSONfunction. SELECT BSON2JSON(JSON_FIELD) FROM TESTJSON; 1 -------------------- {"Name":"George"}

OnethingthatyouwillnoticeisthattheretrievedJSONhasbeenmodifiedslightlysothatallthevalueshavequotesaroundthemtoavoidanyambiguity.ThisisduetotheconversiontoBSONformatandbacktoJSON.Notethatwedidn'tnecessarilyrequirethequoteswhenweinsertedthedata.Forinstance,ouroriginalJSONdocumentcontainedthefollowingfield: {Name:"George"}


Whatgetsreturnedisslightlydifferent,butstillconsideredtobethesameJSONdocument. {"Name":"George"}

Youmustensurethatthenamingofanyfieldsisconsistentbetweendocuments."Name","name",and"Name"areallconsidereddifferentfields.Oneoptionistouselowercasefieldnames,ortousecamel-case(firstletteriscapitalized)inallyourfielddefinitions.Theimportantthingistokeepthenamingconsistentsoyoucanfindthefieldsinthedocument.

DeterminingDocumentSizeThesizeofBLOBfieldwillbedependentonthesizeofdocumentsthatyouexpecttostoreinthem.ConvertingaJSONdocumentintoaBSONformatusuallyresultsinasmallerdocumentsize.ThesampleJSON_EMPtable,with42documentscontains11,764bytesofdata,andwhenconvertedtoBSONcontains11,362bytes.TherewillbeoccasionswheretheBSONformatmaybelargerthantheoriginalJSONdocument.Thisexpansioninsizeoccurswhendocumentshavemanyintegerornumericvalues.Anintegertypetakesup4bytesinaBSONdocument.Convertingacharactervaluetoanativeintegerformatwilltakeupmorespacebutresultinfasterretrievalandcomparisonoperations.

BSON_VALIDATE:CheckingtheFormatofaDocumentDb2hasnonativeJSONdatatype,sotherenovalidationdoneagainstthecontentsofaBLOBcolumnwhichcontainsJSONdata.IftheJSONobjectisunderprogramcontrolandyouareusingtheJSON2BSONandBSON2JSONfunctions,youareprobablynotgoingtorunacrossproblemswiththedata.Ifyoubelievethatadocumentiscorruptedforsomereason,youcanusetheBSON_VALIDATEtomakesureitisokay(ornot!).Thefunctionwillreturnavalueof1iftherecordisokay,orazerootherwise.TheonerowthatwehaveinsertedintotheTESTJSONtableshouldbeokay. SELECT BSON_VALIDATE(JSON_FIELD) FROM TESTJSON;


1 ----------- 1

ThefollowingSQLwillinjectabadvalueintothebeginningoftheJSONfieldtotesttheresultsfromtheBSON_VALIDATEfunction. UPDATE TESTJSON SET JSON_FIELD = BLOB('!') || JSON_FIELD; SELECT BSON_VALIDATE(JSON_FIELD) FROM TESTJSON; 1 ----------- 0

RetrievingJSONDocumentsThelastsectiondescribedhowwecaninsertandretrieveentireJSONdocumentsfromacolumninatable.ThissectionwillexploreseveralfunctionsthatallowaccesstoindividualfieldswithintheJSONdocument.Thesefunctionsare:

• JSON_VAL–ExtractsdatafromaJSONdocumentintoSQLdatatypes

• JSON_TABLE–Returnsatableofvaluesforadocumentthathasarraytypesinit

• JSON_TYPE–ReturnsthedatatypeofaspecificfieldwithinaJSONdocument

• JSON_LEN–Returnsthecountofelementsinanarraytypeinsideadocument

• JSON_GET_POS_ARR_INDEX–Retrievetheindexofavaluewithinanarraytypeinadocument

SampleJSONTableCreation

ThefollowingSQLwillloadtheJSON_EMPtablewithseveralJSONobjects.TheserecordsaremodelledaroundtheSAMPLEdatabaseEMPLOYEEtable. CREATE TABLE JSON_EMP ( SEQ INT NOT NULL GENERATED ALWAYS AS IDENTITY, EMP_DATA BLOB(4000) INLINE LENGTH 4000 );


ThefollowingcommandwillloadtherecordsintotheJSON_EMPtable.OnlythefirstINSERTisdisplayed,butthereisatotalof42recordsincluded.ADb2scriptfile(DB2-V11-JSON-Examples.sql)containingalltheJSONexamplescanbefoundinthesamedirectoryasthiseBook.SeetheAppendixformoredetailsonhowtogetacopyofthisfile. INSERT INTO JSON_EMP(EMP_DATA) VALUES JSON2BSON( '{ "empno":"000070", "firstnme":"EVA", "midinit":"D", "lastname":"PULASKI", "workdept":"D21", "phoneno":[7831,1422,4567], "hiredate":"09/30/2005", "job":"MANAGER", "edlevel":16, "sex":"F", "birthdate":"05/26/2003", "pay": { "salary":96170.00, "bonus":700.00, "comm":2893.00 } }');

AdditionalJSON_DEPTTable

InadditiontotheJSON_EMPtable,thefollowingSQLwillgenerateatablecalledJSON_DEPTthatcanbeusedtodeterminethenameofthedepartmentanindividualworksin. CREATE TABLE JSON_DEPT ( SEQ INT NOT NULL GENERATED ALWAYS AS IDENTITY, DEPT_DATA BLOB(4000) INLINE LENGTH 4000 ); INSERT INTO JSON_DEPT(DEPT_DATA) VALUES JSON2BSON('{"deptno":"A00", "mgrno":"000010", "admrdept":"A00", "deptname":"SPIFFY COMPUTER SERVICE DIV."}'), JSON2BSON('{"deptno":"B01", "mgrno":"000020", "admrdept":"A00", "deptname":"PLANNING" }'), JSON2BSON('{"deptno":"C01", "mgrno":"000030", "admrdept":"A00", "deptname":"INFORMATION CENTER" }'), JSON2BSON('{"deptno":"D01", "admrdept":"A00", "deptname":"DEVELOPMENT CENTER" }'), ... JSON2BSON('{"deptno":"J22", "admrdept":"E01", "deptname":"BRANCH OFFICE J2" }');


JSON_VAL:RetrievingDatafromaBSONDocumentNowthatwehaveinsertedsomeJSONdataintoatable,thissectionwillexploretheuseoftheJSON_VALfunctiontoretrieveindividualfieldsfromthedocuments.Thisbuilt-infunctionwillreturnavaluefromadocumentinaformatthatyouspecify.Theabilitytodynamicallychangethereturneddatatypeisextremelyimportantwhenweexamineindexcreationinanothersection.TheJSON_VALfunctionhastheformat: JSON_VAL(document, field, type);

JSON_VALtakes3arguments:

• document–BSONdocument• field–Thefieldwearelookingfor(searchpath)• type–Thereturntypeofdatabeingreturned

Thesearchpathandtypemustbeconstants–theycannotbevariablessotheiruseinuser-definedfunctionsislimitedtoconstants.AtypicalJSONrecordwillcontainavarietyofdatatypesandstructuresasillustratedbythefollowingrecordfromtheJSON_EMPtable. { "empno":"200170", "firstnme":"KIYOSHI", "midinit":"", "lastname":"YAMAMOTO", "workdept":"D11", "phoneno":[2890], "hiredate":"09/15/2005", "job":"DESIGNER", "edlevel":16, "sex":"M", "birthdate":"01/05/1981", "pay": { "salary":64680.00, "bonus":500.00, "comm":1974.00 } }

Therearenumberoffieldswithdifferentformats,includingstrings(firstnme),integers(edlevel),decimal(salary),date(hiredate),anumberarray(phoneno),andastructure(pay).JSONdatacanconsistofnestedobjects,arraysandverycomplexstructures.TheformatofaJSONobject


ischeckedwhenusingtheJSON2BSONfunctionandanerrormessagewillbeissuedifitdoesnotconformtotheJSONspecification.TheJSON_VALfunctionneedstoknowhowtoreturnthedatatypebackfromtheJSONrecord,soyouneedtospecifywhattheformatshouldbe.Thepossibleformatsare:

Code Formatn DECFLOATi INTEGERI BIGINT(noticethisisalowercaseL)f DOUBLEd DATEts TIMESTAMP(6)t TIMEs:n AVARCHARwithasizeofnbeingthemaximumb:n ABINARYvaluewithnbeingthemaximumu Anintegerwithavalueof0or1.

RetrievingAtomicValues

Thisfirstexamplewillretrievethenameandsalaryoftheemployeewhoseemployeenumberis"200170" SELECT JSON_VAL(EMP_DATA,'lastname','s:20'), JSON_VAL(EMP_DATA,'pay.salary','f') FROM JSON_EMP WHERE JSON_VAL(EMP_DATA,'empno','s:6') = '200170'; 1 2 -------------------- ------------------------------------------ YAMAMOTO 64680

Ifthesizeofthefieldbeingreturnedislargerthanthefieldspecification,youwillgetaNULLvaluereturned,notatruncatedvalue. SELECT JSON_VAL(EMP_DATA,'lastname','s:7') FROM JSON_EMP WHERE JSON_VAL(EMP_DATA,'empno','s:6') = '200170'; 1 -------------------- -


Inthecaseofcharacterfields,youmayneedtospecifyalargerreturnsizeandthentruncateittogetasubsetofthedata. SELECT LEFT(JSON_VAL(EMP_DATA,'lastname','s:20'),7) FROM JSON_EMP WHERE JSON_VAL(EMP_DATA,'empno','s:6') = '200170'; 1 ------- YAMAMOT

RetrievingArrayValues

Selectingdatafromanarraytypewillalwaysgiveyouthefirstvalue(elementzero).Theemployeesallhaveextensionnumbersbutsomeofthemhavemorethanone.Someoftheextensionsstartwithazerososincethecolumnisbeingtreatedasanintegeryouwillgetonly3digits.It'sprobablybettertodefineitasacharacterstringratherthananumber! SELECT JSON_VAL(EMP_DATA, 'phoneno', 'i') FROM JSON_EMP; 1 ----------- 3978

Ifyouspecify":na"afterthetypespecifier,youwillgetanerrorifthefieldisanarraytype.HopefullyyoualreadyknowtheformatofyourJSONdataandcanavoidhavingtochecktoseeifarraysexist.Whatthisstatementwilltellyouisthatoneoftherecordsyouwereattemptingtoretrievewasanarraytype.Infact,allthephoneextensionsarebeingtreatedasarraytypeseventhoughtheyhaveonlyonevalueinmanycases. SELECT JSON_VAL(EMP_DATA, 'phoneno', 'i:na') FROM JSON_EMP; SQL20556N The operation failed because multiple result values cannot be returned from a scalar function "SYSIBM.JSON_VAL". SQLSTATE=22547

Ifyouneedtoaccessaspecificarrayelementinafield,youcanusethe"dot"notationafterthefieldname.Thefirstelementstartsatzero.Ifweselectthe2ndelement(.1)alltheemployeesthathaveasecondextensionwillhaveavalueretrievedwhiletheoneswhodon'twillhaveanullvalue.


RetrievingStructuredFields

Structuredfieldsareretrievedusingthesamedotnotationasarrays.Thefieldisspecifiedbyusingthe"field.subfield"formatandthesefieldscanbeanarbitrarynumberoflevelsdeep.Thepayfieldintheemployeerecordismadeupofthreeadditionalfields. "pay": { "salary":64680.00, "bonus":500.00, "comm":1974.00 }

Toretrievethesethreefields,youneedtoexplicitlynamethemsinceretrievingpayalonewillnotwork. SELECT JSON_VAL(EMP_DATA,'pay.salary','i'), JSON_VAL(EMP_DATA,'pay.bonus','i'), JSON_VAL(EMP_DATA,'pay.comm','i') FROM JSON_EMP WHERE JSON_VAL(EMP_DATA,'empno','s:6') = '200170'; 1 2 3 ----------- ----------- ----------- 64680 500 1974

Ifyouattempttoretrievethepayfield,youwillendupwithaNULLvalue,notanerrorcode.ThereasonforthisisthattheJSON_VALfunctioncannotformatthefieldintoanatomicvaluesoitreturnstheNULLvalueinstead.

DetectingNULLValuesinaField

Todeterminewhetherafieldexists,orhasanullvalue,youneedusethe"u"flag.Ifyouusethe"u"flag,thevaluereturnedwillbeeither:

• 1–Thefieldexists,andithasavalue(notnulloremptystring)• 0–Thefieldexists,butthevalueisnullorempty• null–Thefielddoesnotexist

IntheJSON_EMPtable,thereareafewemployeeswhodonothavemiddlenames.Thefollowingquerywillreturnavalueor1,0,orNULLdependingonwhetherthemiddlenameexistsforarecord.


SELECT JSON_VAL(EMP_DATA,'lastname','s:30'), JSON_VAL(EMP_DATA,'midinit','u') FROM JSON_EMP;

Theresultscontain40employeeswhohaveamiddleinitial,andtwothatdonot.Theresultscanbemisleadingbecauseanemployeecanhavethemidinitfielddefined,butnovalueassignedtoit: { "empno":"000120", "firstnme":"SEAN", "midinit":"", "lastname":"O''CONNELL",... }

Inthiscase,theemployeedoesnothaveamiddlename,butthefieldispresent.Todeterminewhetheranemployeedoesnothaveamiddlename,youwillneedtocheckforaNULLvalue(thefielddoesnotexist,orthefieldisempty)whenretrievingthemiddleinitial(9rows): SELECT COUNT(*) FROM JSON_EMP WHERE JSON_VAL(EMP_DATA,'midinit','s:40') IS NULL;

Ifyouonlywanttoknowhowmanyemployeeshavethemiddleinitialfield(midinit)thatisempty,youneedtoexcludetherecordsthatdonotcontainthefield(7rows): SELECT COUNT(*) FROM JSON_EMP WHERE JSON_VAL(EMP_DATA,'midinit','s:40') IS NULL AND JSON_VAL(EMP_DATA,'midinit','u') IS NOT NULL;

JoiningJSONTables

YoucanjointableswithJSONcolumnsbyusingtheJSON_VALfunctiontocomparetwovalues: SELECT JSON_VAL(EMP_DATA,'empno','s:6') AS EMPNO, JSON_VAL(EMP_DATA,'lastname','s:20') AS LASTNAME, JSON_VAL(DEPT_DATA,'deptname','s:30') AS DEPTNAME FROM JSON_EMP, JSON_DEPT WHERE JSON_VAL(DEPT_DATA,'deptno','s:3') = JSON_VAL(EMP_DATA,'workdept','s:3') FETCH FIRST 5 ROWS ONLY; EMPNO LASTNAME DEPTNAME ------ -------------------- ------------------------------ 000010 HAAS SPIFFY COMPUTER SERVICE DIV. 000020 THOMPSON PLANNING 000030 KWAN INFORMATION CENTER


000050 GEYER SUPPORT SERVICES 000060 STERN MANUFACTURING SYSTEMS

YouneedtoensurethatthedatatypesfrombothJSONfunctionsarecompatibleforthejointoworkproperly.Inthiscase,thedepartmentnumberandtheworkdepartmentarebothreturnedas3-bytecharacterstrings.Ifyoudecidedtouseintegersinsteadorasmallerstringsize,thejoinwillnotworkasexpectedbecausetheconversionwillresultintruncatedorNULLvalues.IfyouplanondoingjoinsbetweenJSONobjects,youmaywanttoconsidercreatedindexesonthedocumentstospeedupthejoinprocess.Moreinformationontheuseofindexesisfoundattheendofthischapter.

JSONDataTypesIfyouareunsureofwhatdatatypeafieldcontains,youcanusetheJSON_TYPEfunctiontodeterminethetypebeforeretrievingthefield.TheJSON_TYPEfunctionhastheformat: ID = JSON_TYPE(document, field, 2048);

JSON_TYPEtakes3arguments:

• document–BSONdocument• field–Thefieldwearelookingfor(searchpath)• searchpathsize–2048istherequiredvalue

The2048specifiesthemaximumlengthofthefieldparameterandshouldbeleftatthisvalue.WhenqueryingthedatatypeswithinaJSONdocument,thefollowingvaluesarereturned.ID TYPE ID TYPE1 Double 10 Null2 String 11 RegularExpression3 Object 12 Futureuse4 Array 13 JavaScript5 Binarydata14 Symbol6 Undefined15 Javascript(withscope)


ID TYPE ID TYPE7 Objectid 16 32-bitinteger8 Boolean 17 Timestamp9 Date 18 64-bitintegerThenextSQLstatementwillcreateatablewithstandardtypeswithinit. CREATE TABLE TYPES (DATA BLOB(4000) INLINE LENGTH 4000); INSERT INTO TYPES VALUES JSON2BSON( '{ "string" : "string", "integer" : 1, "number" : 1.1, "date" : {"$date": "2016-06-20T13:00:00"}, "boolean" : true, "array" : [1,2,3], "object" : {type: "main", phone: [1,2,3]} }');

ThefollowingSQLwillgeneratealistofdatatypesandfieldnamesfoundwithinthisdocument. SELECT 'STRING',JSON_TYPE(DATA, 'string', 2048) FROM TYPES UNION ALL SELECT 'INTEGER',JSON_TYPE(DATA, 'integer', 2048) FROM TYPES UNION ALL SELECT 'NUMBER',JSON_TYPE(DATA, 'number', 2048) FROM TYPES UNION ALL SELECT 'DATE',JSON_TYPE(DATA, 'date', 2048) FROM TYPES UNION ALL SELECT 'BOOLEAN', JSON_TYPE(DATA, 'boolean', 2048) FROM TYPES UNION ALL SELECT 'ARRAY', JSON_TYPE(DATA, 'array', 2048) FROM TYPES UNION ALL SELECT 'OBJECT', JSON_TYPE(DATA, 'object', 2048) FROM TYPES; 1 2 --------- ----------- ARRAY 4 BOOLEAN 8 DATE 9 NUMBER 1 INTEGER 16 STRING 2 OBJECT 3


ExtractingFieldsUsingDifferentDataTypesThefollowingsectionswillshowhowwecangetatomic(non-array)typesoutoftheJSONdocuments.Wearenotgoingtobespecificwhichdocumentswewantasidefromthefieldwewanttoretrieve.AtemporarytablecalledSANDBOXisusedthroughouttheseexamples: CREATE TABLE SANDBOX (DATA BLOB(4000) INLINE LENGTH 4000);

JSONINTEGERSandBIGINT

IntegerswithinJSONdocumentsareeasilyidentifiedasnumbersthatdon'thaveadecimalplaceinthem.TherearetwodifferenttypesofintegerssupportedwithinDb2andareidentifiedbythesize(numberofdigits)inthenumberitself.

• Integer–asetofdigitsthatdonotincludeadecimalplace.Thenumbercannotexceed−2,147,483,648to2,147,483,647

• Bigint–asetofdigitsthatdonotincludeadecimalplacebutexceedthatofaninteger.Thenumbercannotexceed–9,223,372,036,854,775,808to9,223,372,036,854,775,807

Youdon'texplicitlystatethetypeofintegerthatyouareusing.Thesystemwilldetectthetypebasedonitssize.TheJSON_TYPEfunctionwillreturnavalueof16forintegersand18foralargeinteger(BIGINT).Toretrieveavaluefromanintegerfieldyouneedtousethe"i"flagand"l"(lowercaseL)forbigintegers.ThisfirstSQLstatementwillcreatearegularintegerfield. INSERT INTO SANDBOX VALUES JSON2BSON('{"count":9782333}');

TheJSON_TYPEfunctionwillverifythatthisisanintegerfield(Type=16). SELECT JSON_TYPE(DATA,'count',2048) AS TYPE FROM SANDBOX; TYPE ----------- 16


Youcanretrieveanintegervaluewitheitherthe'i'flagorthe'l'flag.ThisfirstSQLstatementretrievesthevalueasaninteger. SELECT JSON_VAL(DATA,'count','i') FROM SANDBOX; 1 ----------- 9782333

WecanaskthatthevaluebeinterpretedasaBIGINTbyusingthe'l'flag,soJSON_VALwillexpandthesizeofthereturnvalue. SELECT JSON_VAL(DATA,'count','l') FROM SANDBOX; 1 -------------------- 9782333

ThenextSQLstatementwillcreateafieldwithaBIGINTsize.Notethatwedon'tneedtospecifyanythingotherthanhaveaverybignumber! DELETE FROM SANDBOX; INSERT INTO SANDBOX VALUES JSON2BSON('{"count":94123512223422}');

TheJSON_TYPEfunctionwillverifythatthisisabigintegerfield(Type=18). SELECT JSON_TYPE(DATA,'count',2048) AS TYPE FROM SANDBOX; TYPE ----------- 18

WecanchecktoseethatthedataisstoredinthedocumentasaBIGINTbyusingtheJSON_TYPEfunction. SELECT JSON_TYPE(DATA,'count',2048) FROM SANDBOX; 1 ----------- 18

Returningthedataasanintegertype'i'willfailsincethenumberistoobigtofitintoanintegerformat.Notethatyoudonotgetanerrormessage-aNULLvaluegetsreturned.


SELECT JSON_VAL(DATA,'count','i') FROM SANDBOX; 1 ----------- -

Specifyingthe'I'flagwillmakethedatabereturnedproperly. SELECT JSON_VAL(DATA,'count','l') FROM SANDBOX; 1 -------------------- 94123512223422

SincewehaveanintegerintheJSONfield,wealsohavetheoptionofreturningthevalueasafloating-pointnumber(f)orasadecimalnumber(n).Eitheroftheseoptionswillworkwithintegervalues. SELECT JSON_VAL(DATA,'count','n') AS DECIMAL, JSON_VAL(DATA,'count','f') AS FLOAT FROM SANDBOX; DECIMAL FLOAT ---------------------------------- ---------------------- 94123512223422 9.41235122234220E+013

JSONNUMBERSandFLOATINGPOINT

JSONnumbersarerecognizedbyDb2whenthereisadecimalpointinthevalue.FloatingpointvaluesarerecognizedusingtheExxspecifierafterthenumberwhichrepresentsthepowerof10thatneedstobeappliedtothebasevalue.Forinstance,1.0E01isthevalue10.TheJSONtypefornumbersis1,whetheritisinfloatingpointformatordecimalformat.TheSQLstatementbelowinsertsasalaryintothetable(usingthestandarddecimalplacenotation). INSERT INTO SANDBOX VALUES JSON2BSON('{"salary":92342.20}');

TheJSON_TYPEfunctionwillverifythatthisisanumericfield(Type=1). SELECT JSON_TYPE(DATA,'salary',2048) AS TYPE FROM SANDBOX; TYPE ----------- 1


Numericdatacanberetrievedineithernumber(n)formant,integer(i-notethatyouwillgettruncation),orfloatingpoint(f). SELECT JSON_VAL(DATA,'salary','n') AS DECIMAL, JSON_VAL(DATA,'salary','i') AS INTEGER, JSON_VAL(DATA,'salary','f') AS FLOAT FROM SANDBOX; DECIMAL INTEGER FLOAT ------------------------------ ----------- ---------------------- 92342.199999999997 92342 9.23422000000000E+004

Youmaywonderwhynumberformat(n)resultsinananswerthathasafractionalcomponentthatisn'texactly92342.20.ThereasonisthatDb2isconvertingthevaluetoDECFLOAT(34)whichsupportsahigherprecisionnumber,butcanresultinfractionsthatcan'tbeaccuratelyrepresentedwithinthebinaryformat.CastingthevaluetoDEC(9,2)willproperlyformatthenumber. SELECT DEC(JSON_VAL(DATA,'salary','n'),9,2) AS DECIMAL FROM SANDBOX; DECIMAL ----------- 92342.20

Afloating-pointnumberisrecognizedbytheExxspecifierinthenumber.TheBSONfunctionwilltagthisvalueasanumbereventhoughyouspecifieditinfloatingpointformat.ThefollowingSQLinsertsthefloatingvalueintothetable. INSERT INTO SANDBOX VALUES JSON2BSON('{"salary":9.2523E01}');

TheJSON_TYPEfunctionwillverifythatthisisafloating-pointfield(Type=1). SELECT JSON_TYPE(DATA,'salary',2048) AS TYPE FROM SANDBOX; TYPE ----------- 1

Thefloating-pointvaluecanberetrievedasanumber,integer,orfloating-pointvalue.


SELECT JSON_VAL(DATA,'salary','n') AS DECIMAL, JSON_VAL(DATA,'salary','i') AS INTEGER, JSON_VAL(DATA,'salary','f') AS FLOAT FROM SANDBOX; DECIMAL INTEGER FLOAT ------------------------- ----------- ---------------------- 92.522999999999996 92 9.25230000000000E+001

JSONBOOLEANVALUES

JSONhasadatatypewhichcanbetrueorfalse(Boolean).Db2doesn'thaveanequivalentdatatypeforBoolean,soweneedtoretrieveitasanintegerorcharacterstring(true/false).TheJSONtypeforBooleanvaluesis8.TheSQLstatementbelowinsertsatrueandfalsevalueintothetable. INSERT INTO SANDBOX VALUES JSON2BSON('{"valid":true, "invalid":false}');

Wewilldouble-checkwhattypethefieldisintheJSONrecord. SELECT JSON_TYPE(DATA,'valid',2048) AS TYPE FROM SANDBOX; TYPE ----------- 8

Toretrievethevalue,wecanaskthatitbeformattedasanintegerornumber. SELECT JSON_VAL(DATA,'valid','n') AS TRUE_DECIMAL, JSON_VAL(DATA,'valid','i') AS TRUE_INTEGER, JSON_VAL(DATA,'invalid','n') AS FALSE_DECIMAL, JSON_VAL(DATA,'invalid','i') AS FALSE_INTEGER FROM SANDBOX; TRUE_DECIMAL TRUE_INTEGER FALSE_DECIMAL FALSE_INTEGER ------------ ------------ ------------- ------------- 1 1 0 0

YoucanalsoretrieveaBooleanfieldasacharacterorbinaryfield,buttheresultsarenotwhatyouwouldexpectwithbinary.


SELECT JSON_VAL(DATA,'valid','s:5') AS TRUE_STRING, JSON_VAL(DATA,'valid','b:2') AS TRUE_BINARY, JSON_VAL(DATA,'invalid','s:5') AS FALSE_STRING, JSON_VAL(DATA,'invalid','b:2') AS FALSE_BINARY FROM SANDBOX; TRUE_STRING TRUE_BINARY FALSE_STRING FALSE_BINARY ----------- ----------- ------------ ------------ true 0801 false 0800

JSONDATE,TIME,andTIMESTAMPS

ThisfirstSQLstatementwillinsertaJSONfieldthatusesthe$datemodifier. INSERT INTO SANDBOX VALUES JSON2BSON('{"today":{"$date":"2016-07-01T12:00:00"}}');

QueryingthedatatypeofthisfieldusingJSON_VALwillreturnavalueof9(datetype). SELECT JSON_TYPE(DATA,'today',2048) FROM SANDBOX; 1 ----------- 9

Ifyoudecidetouseacharacterstringtorepresentadate,youcanuseeitherthe"s:x"specificationtoreturnthedateasastring,oruse"d"tohaveitdisplayedasadate.ThisfirstSQLstatementreturnsthedateasastring. INSERT INTO SANDBOX VALUES JSON2BSON('{"today":"2016-07-01"}'); SELECT JSON_VAL(DATA,'today','s:10') FROM SANDBOX; 1 ---------- 2016-07-01

Usingthe'd'specificationwillreturnthevalueasadate. SELECT JSON_VAL(DATA,'today','d') FROM SANDBOX; 1 ---------- 2016-07-01


Whatabouttimestamps?Ifyoudecidetostoreatimestampintoafield,youcanretrieveitinavarietyofways.ThisfirstsetofSQLstatementswillretrieveitasastring. INSERT INTO SANDBOX VALUES JSON2BSON('{"today":"' || VARCHAR(NOW()) || '"}'); SELECT JSON_VAL(DATA,'today','s:30') FROM SANDBOX; 1 ------------------------------ 2016-09-17-06.27.00.945000

RetrievingitasaDatewillalsowork,butthetimeportionwillberemoved. SELECT JSON_VAL(DATA,'today','d') FROM SANDBOX; 1 ---------- 2016-09-17

Youcanalsoaskforthetimestampvaluebyusingthe'ts'specification.Notethatyoucan'tgetjustthetimeportionunlessyouuseaSQLfunctiontocastit. SELECT JSON_VAL(DATA,'today','ts') FROM SANDBOX; 1 -------------------------- 2016-09-17 06:27:00.945000

Toforcethevaluetoreturnjustthetimeportion,eitherstorethedataasatimevalue(HH:MM:SS)stringorstoreatimestampandusetheTIMEfunctiontoextractjustthatportionofthetimestamp. SELECT TIME(JSON_VAL(DATA,'today','ts')) FROM SANDBOX; 1 -------- 06:27:00

JSONStrings

Forcharacterstrings,youmustspecifywhatthemaximumlengthis.Thisexamplewillreturnthesizeofthelastnamefieldas10characterslong.


SELECT JSON_VAL(DATA, 'lastname', 's:10') FROM JSON_EMP; 1 ---------- HAAS

Youmustspecifyalengthforthe's'parameterotherwiseyouwillgetanerrorfromthefunction.Ifthesizeofthecharacterstringistoolargetoreturn,thenthefunctionwillreturnanullvalueforthatfield. SELECT JSON_VAL(DATA, 'lastname', 's:8') FROM JSON_EMP; 1 -------- HAAS

JSON_TABLEFunctionThefollowingqueryworksbecausewedonottreatthefieldphonenoasanarray: SELECT JSON_VAL(DATA, 'phoneno', 'i') FROM JSON_EMP; 1 ----------- 3978

Bydefault,onlythefirstnumberofanarrayisreturnedwhenyouuseJSON_VAL.However,therewillbesituationswhereyoudowanttoreturnallthevaluesinanarray.ThisiswheretheJSON_TABLEfunctionmustbeused.TheformatoftheJSON_TABLEfunctionis: JSON_TABLE(document, field, type)

Theargumentsare:

• document–BSONdocument• field–Thefieldwearelookingfor• type–Thereturntypeofdatabeingreturned

JSON_TABLEreturnstwocolumns:TypeandValue.Thetypeisoneofapossible18valuesfoundinthetablebelow.TheValueistheactualcontentsofthefield.


ID TYPE ID TYPE1 Double 10 Null2 String 11 RegularExpression3 Object 12 Futureuse4 Array 13 JavaScript5 Binarydata14 Symbol6 Undefined15 Javascript(withscope)7 Objectid 16 32-bitinteger8 Boolean 17 Timestamp9 Date 18 64-bitinteger

TheTYPEfieldisprobablysomethingyouwouldn'trequireaspartofyourqueriessinceyouarealreadyspecifyingthereturntypeinthefunction.TheformatoftheJSON_TABLEfunctionislikeJSON_VALexceptthatitreturnsatableofvalues.YoumustusethisfunctionaspartofFROMclauseandatablefunctionspecification.Forexample,toreturnthecontentsofthephoneextensionarrayforjustoneemployee(000230)wecanusethefollowingJSON_TABLEfunction. SELECT PHONES.* FROM JSON_EMP E, TABLE( JSON_TABLE(E.EMP_DATA,'phoneno','i') ) AS PHONES WHERE JSON_VAL(E.EMP_DATA,'empno','s:6') = '000230'; TYPE VALUE ----------- -------------------- 16 2094 16 8999 16 3756

TheTABLE(...)specificationintheFROMclauseisusedfortablefunctions.TheresultsthatarereturnedfromtheTABLEfunctionaretreatedthesameasatraditionaltable.Tocreateaquerythatgivesthenameofeveryemployeeandtheirextensionswouldrequirethefollowingquery. SELECT JSON_VAL(E.EMP_DATA, 'lastname', 's:10') AS LASTNAME, PHONES.VALUE AS PHONE FROM JSON_EMP E, TABLE( JSON_TABLE(E.EMP_DATA,'phoneno','i') ) AS PHONES; LASTNAME PHONE ---------- -------------------- HAAS 3978 THOMPSON 3476 THOMPSON 1422


KWAN 4738 GEYER 6789 STERN 6423 STERN 2433 PULASKI 7831 PULASKI 1422 PULASKI 4567

Onlyasubsetoftheresultsisshownabove,butyouwillseethattherearemultiplelinesforemployeeswhohavemorethanoneextension.TheresultsofaTABLEfunctionmustbenamed(AS...)ifyouneedtorefertotheresultsoftheTABLEfunctionintheSELECTlistorinotherpartsoftheSQL.YoucanuseotherSQLoperatorstosortororganizetheresults.Forinstance,wecanusetheORDERBYoperatortofindoutwhichemployeeshavethesameextension.NotehowtheTABLEfunctionisnamedPHONESandtheVALUEScolumnisrenamedtoPHONE. SELECT JSON_VAL(E.EMP_DATA, 'lastname', 's:10') AS LASTNAME, PHONES.VALUE AS PHONE FROM JSON_EMP E, TABLE( JSON_TABLE(E.EMP_DATA,'phoneno','i') ) AS PHONES ORDER BY PHONE; LASTNAME PHONE ---------- -------------------- THOMPSON 1422 PULASKI 1422 SCHNEIDER 1422 O'CONNELL 1533 MEHTA 1533 ALONZO 1533 SCOUTTEN 1682 ORLANDO 1690

Youcanevenfoundouthowmanypeoplearesharingextensions!TheHAVINGclausetellsDb2toonlyreturngroupingswherethereismorethanoneemployeewiththesameextension. SELECT PHONES.VALUE AS PHONE, COUNT(*) AS COUNT FROM JSON_EMP E, TABLE( JSON_TABLE(E.EMP_DATA,'phoneno','i') ) AS PHONES GROUP BY PHONES.VALUE HAVING COUNT(*) > 1 ORDER BY PHONES.VALUE PHONE COUNT -------------------- ----------- 1422 3 1533 3


1793 2 2103 2 2167 2 2890 2 3332 2 3780 2

�InsubsequentreleasesofDb2,theJSON_TABLEfunctionwillbereplacedwiththeSQLstandardsversion.Thestandards-basedJSON_TABLEfunctioniscompletelydifferentthanthecurrentDb2implementation,soyoumustmakesuretousethecorrectfunctionpath(SYSTOOLSorSYSIBM)whenyouusethisfunctioninthefuture.

JSON_LENFunctionThepreviousexampleshowedhowwecouldretrievethevaluesfromwithinanarrayofadocument.Sometimesanapplicationneedstodeterminehowmanyvaluesareinthearrayitself.TheJSON_LENfunctionisusedtofigureoutwhatthearraycountis.TheformatoftheJSON_LENfunctionis: count = JSON_LEN(document,field)

Theargumentsare:

• document–BSONdocument• field–Thefieldwearelookingfor• count–NumberofarrayentriesorNULLifthefieldisnotanarray

Ifthefieldisnotanarray,thisfunctionwillreturnanullvalue,otherwiseitwillgiveyouthenumberofvaluesinthearray.Inourpreviousexample,wecoulddeterminethenumberofextensionsperpersonbytakingadvantageoftheJSON_LENfunction. SELECT JSON_VAL(E.EMP_DATA, 'lastname', 's:10') AS LASTNAME, JSON_LEN(E.EMP_DATA, 'phoneno') AS PHONE_COUNT FROM JSON_EMP E; LASTNAME PHONE_COUNT ---------- ----------- HAAS 1 THOMPSON 2 KWAN 1 GEYER 1 STERN 2 PULASKI 3 HENDERSON 1


SPENSER 1

JSON_GET_POS_ARR_INDEXFunctionTheJSON_TABLEandJSON_LENfunctionscanbeusedtoretrieveallthevaluesfromanarray,butsearchingforaspecificarrayvalueisdifficulttodo.OnewaytosearcharrayvaluesistoextracteverythingusingtheJSON_TABLEfunction. SELECT JSON_VAL(E.EMP_DATA, 'lastname', 's:10') AS LASTNAME, PHONES.VALUE AS PHONE FROM JSON_EMP E, TABLE( JSON_TABLE(E.EMP_DATA,'phoneno','i') ) AS PHONES WHERE PHONES.VALUE = 1422;

AneasierwaytosearchanarrayisbyusingtheJSON_GET_POS_ARR_INDEXfunction.ThisfunctionwillsearcharrayvalueswithouthavingtoextractthearrayvalueswiththeJSON_TABLEfunction.TheformatoftheJSON_GET_POS_ARR_INDEXfunctionis: element = JSON_GET_POS_ARR_INDEX(document, field)

Theargumentsare:

• document–BSONdocument• field–Thefieldwearelookingforanditsvalue• element–Thefirstoccurrenceofthevalueinthearray

Theformatofthefieldargumentis"{field:value}"anditneedstobeinBSONformat.ThismeansyouneedstoaddtheJSON2BSONfunctionaroundthefieldspecification. JSON2BSON( '{"field":"value"}' )

Thisfunctiononlytestsforequivalenceandthedatatypeshouldmatchwhatisalreadyinthefield.Thereturnvalueisthepositionwithinthearraythatthevaluewasfound,wherethefirstelementstartsatzero.InourJSON_EMPtable,eachemployeehasoneormorephonenumbers.ThefollowingSQLwillretrieveallemployeeswhohavetheextension1422: SELECT JSON_VAL(EMP_DATA, 'lastname', 's:10') AS LASTNAME FROM JSON_EMP


WHERE JSON_GET_POS_ARR_INDEX(EMP_DATA, JSON2BSON('{"phoneno":1422}')) >= 0; LASTNAME ---------- THOMPSON PULASKI SCHNEIDER

Ifweusedquotesaroundthephonenumber,thefunctionwillnotmatchanyofthevaluesinthetable.

UpdatingJSONDocumentsThereareacoupleofapproachesavailabletoupdatingJSONdocuments.OneapproachistoextractthedocumentfromthetableinatextformusingBSON2JSONandthenusingstringfunctionsorregularexpressionstomodifythedata.TheotheroptionistousetheJSON_UPDATEstatement.ThesyntaxoftheJSON_UPDATEfunctionis: JSON_UPDATE(document, '{$set: {field:value}}')

Theargumentsare:

• document–BSONdocument• field–Thefieldwearelookingfor• value–Thevaluewewanttosetthefieldto

TherearethreepossibleoutcomesfromusingtheJSON_UPDATEstatement:

• Ifthefieldisfound,theexistingvalueisreplacedwiththenewone

• Ifthefieldisnotfound,thefield:valuepairisaddedtothedocument

• �Ifyouuse$unset,andthevalueissettothenullkeyword,thefieldisremovedfromthedocument

Thefieldcanspecifyaportionofastructure,oranelementofanarrayusingthedotnotation.ThefollowingSQLwillillustratehowvaluescanbeaddedandremovedfromadocument.Asinglerecordthatcontains3phonenumberextensionsareaddedtoatable:


INSERT INTO SANDBOX VALUES JSON2BSON('{"phone":[1111,2222,3333]}');

Toaddanewfieldtotherecord,theJSON_UPDATEfunctionneedstospecifythefieldandvaluepair.UPDATE SANDBOX SET DATA = JSON_UPDATE(DATA,'{ $set: {"lastname":"HAAS"}}');

Retrievingthedocumentshowsthatthelastnamefieldhasnowbeenaddedtotherecord. SELECT BSON2JSON(DATA) FROM SANDBOX; 1 ---------------------------------------------- {"phone":[1111,2222,3333],"lastname":"HAAS"}

Ifyouspecifyafieldthatisanarraytypeanddonotspecifyanelement,youwillendupreplacingtheentirefieldwiththevalue. UPDATE SANDBOX SET DATA = JSON_UPDATE(DATA,'{ $set: {"phone":9999}}'); SELECT BSON2JSON(DATA) FROM SANDBOX; 1 ---------------------------------- {"phone":9999,"lastname":"HAAS"}

RunningtheSQLagainsttheoriginalphonedatawillworkproperly. UPDATE SANDBOX SET DATA = JSON_UPDATE(DATA,'{ $set: {"phone.0":9999}}'); SELECT BSON2JSON(DATA) FROM SANDBOX; 1 ------------------------------------------------------------ {"phone":[9999,2222,3333]}

�Todeleteafield,youmustuse$unsetinsteadof$setandusenullasthevalueforthefield.Toremovethelastnamefieldfromtherecord: UPDATE SANDBOX SET DATA = JSON_UPDATE(DATA,'{ $unset: {"lastname":null}}');


IndexingJSONDocumentsDb2supportscomputedindexes,whichallowsfortheuseoffunctionslikeJSON_VALtobeusedaspartoftheindexdefinition.Forinstance,searchingforanemployeenumberwillresultinascanagainstthetableifnoindexesaredefined: SELECT JSON_VAL(EMP_DATA, 'lastname', 's:20') AS LASTNAME FROM JSON_EMP WHERE JSON_VAL(EMP_DATA, 'empno', 's:6') = '000010'; LASTNAME -------------------- HAAS

Cost = 13.628412 Rows Operator (ID) Cost 1.68 RETURN ( 1) 13.6284 | 1.68 TBSCAN ( 2) 13.6283 | 42 Table: BAKLARZ JSON_EMP

Tocreateanindexontheempnofield,weusetheJSON_VALfunctiontoextracttheempnofromtheJSONfield. CREATE INDEX IX_JSON ON JSON_EMP (JSON_VAL(EMP_DATA,'empno','s:6'));

RerunningtheSQLresultsinthefollowingexplainplan: Cost = 6.811412 Rows Operator (ID) Cost 1.68 RETURN ( 1)


6.81141 | 1.68 FETCH ( 2) 6.8113 / \ 1.68 42 IXSCAN Table: ( 3) BAKLARZ 0.00484089 JSON_EMP | 42 Index: BAKLARZ IX_JSON

Db2cannowusetheindextoretrievetherecord.

SimplifyingJSONSQLInsertsandRetrievalFromadevelopmentperspective,youalwaysneedtoconvertdocumentstoandfromJSONusingtheBSON2JSONandJSON2BSONfunctions.Therearewaystohidethesefunctionsfromanapplicationandsimplifysomeoftheprogramming.OneapproachtosimplifyingtheconversionofdocumentsbetweenformatsistouseINSTEADOFtriggers.Thesetriggerscanintercepttransactionsbeforetheyareappliedtothebasetables.Thisapproachrequiresthatwecreateaviewontopofanexistingtable.ThefirststepistocreatethebasetablewithtwocopiesoftheJSONcolumn.OnewillcontaintheoriginalJSONcharacterstringwhilethesecondwillcontaintheconvertedBSON.Forthisexample,theJSONcolumnwillbecalledINFO,andtheBSONcolumnwillbecalledBSONINFO.TheuseoftwocolumnscontainingJSONwouldappearstrangeatfirst.ThereasonforthetwocolumnsisthatDb2expectstheBLOBcolumntocontainbinarydata.Youcannotinsertacharacterstring(JSON)intotheBSONcolumnwithoutconvertingitfirst.Db2willraiseanerrorsotheJSONcolumnistheretoavoidanerrorwhiletheconversiontakesplace.


Fromadebuggingperspective,wecankeepboththeCLOBandBLOBvaluesinthistableifwewant.ThetriggerwillsettheJSONcolumntonullaftertheBSONcolumnhasbeenpopulated. CREATE TABLE BASE_EMP_TXS ( SEQNO INT NOT NULL GENERATED ALWAYS AS IDENTITY, INFO VARCHAR(4000), BSONINFO BLOB(4000) INLINE LENGTH 4000 );

TouseINSTEADOFtriggers,aviewneedstobecreatedontopofthebasetable.NotethatweexplicitlyusetheSYSTOOLSschematomakesurewearegettingthecorrectfunctionusedhere. CREATE OR REPLACE VIEW EMP_TXS AS (SELECT SEQNO, BSON2JSON(BSONINFO) AS INFO FROM BASE_EMP_TXS);

AtthispointwecancreatethreeINSTEADOFtriggerstohandleinsert,updatesanddeletesontheview.OnINSERTtheDEFAULTkeywordisusedtogeneratetheIDnumber,theJSONfieldissettoNULLandtheBSONcolumncontainstheconvertedvalueoftheJSONstring. CREATE OR REPLACE TRIGGER I_EMP_TXS INSTEAD OF INSERT ON EMP_TXS REFERENCING NEW AS NEW_TXS FOR EACH ROW MODE DB2SQL BEGIN ATOMIC INSERT INTO BASE_EMP_TXS VALUES ( DEFAULT, NULL, SYSTOOLS.JSON2BSON(NEW_TXS.INFO)

);END

OnUPDATES,thesequencenumberremainsthesame,andtheBSONfieldisupdatedwiththecontentsoftheJSONfield. CREATE OR REPLACE TRIGGER U_EMP_TXS INSTEAD OF UPDATE ON EMP_TXS REFERENCING NEW AS NEW_TXS OLD AS OLD_TXS FOR EACH ROW MODE DB2SQL BEGIN ATOMIC UPDATE BASE_EMP_TXS SET (INFO, BSONINFO) = (NULL, SYSTOOLS.JSON2BSON(NEW_TXS.INFO)) WHERE BASE_EMP_TXS.SEQNO = OLD_TXS.SEQNO; END


Finally,theDELETEtriggerwilljustremovetherow. CREATE OR REPLACE TRIGGER D_EMP_TXS INSTEAD OF DELETE ON EMP_TXS REFERENCING OLD AS OLD_TXS FOR EACH ROW MODE DB2SQL BEGIN ATOMIC DELETE FROM BASE_EMP_TXS WHERE BASE_EMP_TXS.SEQNO = OLD_TXS.SEQNO; END

ApplicationswillonlydealwiththeEMP_TXSview.AnyinsertswillusethetextversionoftheJSONandnothavetoworryaboutusingtheJSON2BSONfunctionsincetheunderlyingINSTEADOFtriggerwilltakecareoftheconversion.ThefollowinginsertstatementonlyincludestheJSONstringsincethesequencenumberwillbegeneratedautomaticallyaspartoftheinsert.INSERT INTO EMP_TXS(INFO) VALUES ( '{ "empno":"000010", "firstnme":"CHRISTINE", "midinit":"I", "lastname":"HAAS", "workdept":"A00", "phoneno":[3978], "hiredate":"01/01/1995", "job":"PRES", "edlevel":18, "sex":"F", "birthdate":"08/24/1963", "pay" : { "salary":152750.00, "bonus":1000.00, "comm":4220.00} }');

SelectingfromtheEMP_TXSviewwillreturntheJSONinareadableformat: SELECT SEQNO, CAST(LEFT(INFO,50) AS VARCHAR(50)) FROM EMP_TXS;

SEQNO 2 ----------- --------------------------------------------------- 1 {"empno":"000010","firstnme":"CHRISTINE","midinit"}


ThebasetableonlycontainstheBSONbuttheviewtranslatesthevaluebackintoareadableformat.Anupdatestatementthatreplacestheentirestringworksasexpected. UPDATE EMP_TXS SET INFO = '{"empno":"000010"}' WHERE SEQNO = 1; SELECT SEQNO, CAST(LEFT(INFO,50) AS VARCHAR(50)) FROM EMP_TXS; SEQNO 2 ----------- -------------------------------------------------- 1 {"empno":"000010"}

IfyouwanttomanipulatetheBSONdirectly(saychangetheemployeenumber),youneedtorefertotheBASEtableinstead.UPDATE BASE_EMP_TXS SET BSONINFO = JSON_UPDATE(BSONINFO, '{$set: {"empno":"111111"}}') WHERE SEQNO = 1;

Andwecancheckitusingouroriginalview. SELECT SEQNO, CAST(LEFT(INFO,50) AS VARCHAR(50)) FROM EMP_TXS;

SEQNO 2 ----------- -------------------------------------------------- 1 {"empno":"111111"}

Summary

ThecurrentDb211.1release(andDb210.5)includesseveraluser-definedfunctions(UDFs)thatwereoriginallydesignedtobeusedbyinternalJSONinterfaces.Basedonfeedbackfromseveralcustomers,we'vedocumentedwhatisusedinternallywithinDb2.Whiletheseroutineshavenotbeenofficiallypublished,theyareavailableforcustomeruse.Note:RememberthatthesefunctionsmaychangeinthefuturetoconformtotheSQLstandard.

A

Appendix

ADDITIONALRESOURCESFORDB2

AppendixA:AdditionalResourcesforDb2 148

ResourcestoBuildYourDb2Skills

RelyonthewiderangeofIBMexperts,programs,andservicesthatareavailabletohelpyoutakeyourInformationManagementskillstothenextlevel.ParticipateinouronlinecommunitythroughdeveloperWorks.Findtutorials,articles,whitepapers,videos,demos,bestpractices,Db2Expressdownloads,andmore.Visitibm.com/developerworks/data.

IBMCertificationExamsFindindustry-leadingprofessionalcertificationexams,includingnewcertificationsforDb210.5withBLUAccelerationandDb211.1:

• Db210.5DatabaseAdministrationUpgradeExam(Exam311)• Db211.1DBAforLUW(Exam600)• Db210.5FundamentalsforLUW(Exam615)

Visitibm.com/certifyformoreinformationandexamavailability.

IBMTrainingIBMiscommittedtohelpingourclientsachievetheskillsandexpertisetotaketheircareerstothenextlevel.Weofferacomprehensiveportfoliooftechnicaltrainingandeducationservicesdesignedforindividuals,companies,andpublicorganizationstoacquire,maintain,andoptimizetheirITskillsinIBMSoftwareandIBMSystems.Visitibm.com/software/data/educationfordetailsandcourseavailability.

BigDataUniversityLearnaboutDb2andvariousbigdatatechnologiesatyourpaceandatyourplace.BigDataUniversityoffershelpfulonlinecourseswithinstructionalvideosandexercisestohelpyoumasternewconcepts.Coursecompletionismarkedwithafinalexamandacertificate.Visitbigdatauniversity.com.


InformationManagementBookstoreFindthemostinformativeDb2booksonthemarket,alongwithvaluablelinksandofferstosaveyoumoneyandenhanceyourskills.Visithttp://bit.ly/DB2_Books.

IBMSupportforDb2AccesstheIBMSupportPortaltofindtechnicalsupportinformationforDb211.1,includingdownloads,notifications,technicaldocuments,flashes,andmore.Visitibm.com/support.Wantaglimpseintothefuture?Checkoutthenewsupportexperiencebeta-https://ibm.biz/support-pilot.LookforanswerstoyourDb2questions?PleasetrydWAnswersforum-http://ibm.biz/dwAnswersDB2.

IBMDataMagazineThemagazine'smissionistodeliversubstantive,high-qualitycontentonthelatestdatamanagementdevelopmentsandIBMadvances,aswellascreateastrongcommunityoftheworld'stopinformationmanagementprofessionals.IBMDatamagazinevividlydemonstrateshowthesmartuseofdataandinformationadvancesbroadbusinesssuccess,providingthecontextthatenablesdatamanagementprofessionalsatalllevelstomakemoreinformedchoicesandcreateinnovative,synchronized,agilesolutions.Seemoreat:ibmdatamag.com.

InternationalDb2UserGroup(IDUG)IDUGisallaboutcommunity.Througheducationevents,technicalresources,uniqueaccesstofellowusers,productdevelopersandsolutionproviders,theyofferanexpansive,dynamictechnicalsupportcommunity.IDUGdeliversqualityeducation,timelyinformationandpeer-drivenproducttrainingandutilizationthatenableDb2userstoachieveorganizationalbusinessobjectives,andtodrivepersonalcareeradvancement.Visitidug.org.


JointheConversationStaycurrentasDb211.1evolvesbyusingsocialmediasitestoconnecttoexpertsandtocontributeyourvoicetotheconversation.Visitoneormoreofthefollowing:

• https://twitter.com/IBM_DB2• https://facebook.com/DB2community• http://bit.ly/BLUVideos• http://linkd.in/DB2Professional• https://twitter.com/ibmbigdata• http://www.planetdb2.com/• http://developer.ibm.com/data/db2/

AdditionaleBookMaterialTheDB2DEMOprogramletsyoutryoutmanyofthefeaturesthatarefoundinthiseBook.TheprogramandthecorrespondinginstallationmanualarefoundinthesamedirectorythatthiseBookisfoundin:https://ibm.ent.box.com/v/DB2v11eBookTouseDB2DEMO,youmustberunningonaWindowsworkstationandbeabletoconnecttoaDb2server.IfyouwanttotryDb2onyourworkstation,youcandownloadthefreeDb2DeveloperCommunityeditionfrom:http://www.ibm.com/us-en/marketplace/ibm-db2-direct-and-developer-editionsExamplesthatuseJupyterNotebookscanbefoundonourGithubrepositoryat:github.com/DB2-SamplesAnyupdatestotheeBookanddemonstrationsoftwarewillbefoundinthisdirectory.Ifyouhaveanycommentsorsuggestions,pleasecontacttheauthorsat:GeorgeBaklarz:[email protected]:[email protected]

TheDb211.1releasedeliversseveralsignificantenhancementsincludingDatabasePartitioningFeature(DPF)forBLUcolumnartechnology,improvedpureScaleperformanceandHighAvailabilityDisasterRecovery(HADR)support,andnumerousSQLfeatures.Thisbookwaswrittentohighlightmanyofthenewfeaturesandfunctionsthatarenowavailableinthisrelease,withoutyouhavingtosearchthroughvariousforums,blogs,andonlinemanuals.WehopethatthisbookgivesyoumoreinsightintowhatyoucannowaccomplishwithDb211.1,andincludeitonyourshortlistofdatabasestodeploy,whetheritisonpremise,inthecloud,orinvirtualizedenvironments.CoverageIncludes:

• AnoverviewofthenewpackagingchangesinDb211.1,alongwithpre-requisitesforinstallingtheproduct

• EnhancementstoDb2BLU,includingsupportforBLUcolumnartablesinaDPF(DataPartitioningFeature)environment

• pureScalecapabilitiesthatprovidecontinuousavailabilityinaproductionenvironment,alongwithalltheenhancementsthathavebeenmadetosimplifyinstallationandmanagementofacluster

• SQLandcompatibilityenhancements

GeorgeBaklarz,B.Math,M.Sc.,Ph.D.Eng.,hasspent31yearsatIBMworkingonvariousaspectsofdatabasetechnology.Georgehaswritten10booksonDb2andotherdatabasetechnologies.GeorgeiscurrentlypartoftheWorldwideCoreDatabaseTechnicalSalesTeam.EnzoCialini,B.Sc.,isaSeniorTechnicalStaffMemberandMasterInventorintheWorldwideCoreDatabaseTechnicalSalesTeamandformerlytheChiefQualityAssuranceArchitectforDb2&PureDataintheIBMTorontoDb2DevelopmentTeam.HeisalsoapublishedbookauthorandwrittenvariouspapersonDb2.Enzohas25yearsofexperienceindatabasetechnology,softwaredevelopment,testing,support,competitiveanalysisandproductiondeployments.

Db2 for Linux, Unix, and Windows Version 11 Highlights George Baklarz and Enzo Cialini

Db2 for Linux, Unix, and Windows - IBM

Documents

Transcript of Db2 for Linux, Unix, and Windows - IBM