The most important IBM DB2 features for SAP - IDUG

7 October 2009 • 14:15 – 15:15Platform: DB2 for LUW

Karl FleckensteinIBM Boeblingen, Germany

Session C12: The most important

IBM DB2 features for SAP

Presentation Abstract--------------------------------The unique partnership between SAP and IBM gave IBM the possibility to optimize “IBM DB2 for Linux, Unix, and Windows” for SAP applications, like SAP ERP or SAP NetWeaver BW. In addition a much better integration of IBM DB2 features into the SAP applications was achieved. DBAs should know about those aspects of the IBM DB2 integration into SAP applications with regard to operating IBM DB2 databases more efficiently. After a short introduction into the „IBM DB2 Optimized for SAP“ initiative and SAP‘s software components, special attention will be given to the seamless integration of important IBM DB2 features such as IBM DB2 monitoring functionality, autonomic memory manangement, autonomic storage, data compression, LOB inlining, HADR, DPF, MDC, online table move and Workload Management. SAP customers can benefit from the innovations immediately, typically within 6-8 weeks after IBM eGA. SAP certifies new IBM DB2 releases for new and current SAP releases, e.g. IBM DB2 9.7 is certified for SAP 4.6D and newer SAP releases.

Simplified database administration, configuration and administration, TCO reduction and performance advantages are the most significant benefits running SAP applications on IBM DB2 for LUW. I will also provide SAP customer experiences and best practices how IBM DB2 can be used more efficiently in an SAP environment.

2

Important Disclaimer

THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY.

WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.

IN ADDITION, THIS INFORMATION IS BASED ON IBM’S AND SAP’SCURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.

IBM AND SAP SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.

NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:

• CREATING ANY WARRANTY OR REPRESENTATION FROM IBM (OR ITS AFFILIATES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS); OR

• ALTERING THE TERMS AND CONDITIONS OF THE APPLICABLE LICENSE AGREEMENT GOVERNING THE USE OF IBM SOFTWARE.

3

Trademarks

IBM, the IBM logo, ibm.com, AIX and DB2 are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Acurrent list of IBM trademarks is available on the Web at www.ibm.com/legal/copytrade.shtml.

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.

Windows is a trademark of Microsoft Corporation in the United States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

SAP, SAP NetWeaver, SAP Business Information Warehouse, SAP BW, SAP NetWeaver BW and other SAP products and services mentioned herein are trademarks or registered trademarks of SAP AG in Germany and in several other countries.

Other company, product and service names may be trademarks or service marks of others.

4

Agenda

• IBM DB2 Optimized for SAP Initiative• SAP Software Overview• IBM DB2 Monitoring and Administration

• SAP DBA Cockpit• SAP DB Performance Warehouse

• Autonomic IBM DB2 Features• Self Tuning Memory Management• Automatic IBM DB2 Parameters

• Efficient Storage• Automatic Storage• Space Reclamation• Compression• LOB Inlining

• Integrated HA and DR Solution• SAP BW Optimizations

• Database Partitioning Feature (DPF)• Multi-Dimensional Clustering (MDC)

• Other Important IBM DB2 Features for SAP• Online Table Move• Workload Management

Objectives – what will attendees learn from this presentation:(1) Fully understand the „IBM DB2 optimized for SAP“ initiative(2) Overview about SAP's software components.(3) Understand which are the most important IBM DB2 featues for SAP. (4) Learn how the IBM DB2 features are seamless integrated into SAP

applications, like SAP ERP and SAP NetWeaver BW.(5) Understand the value proposition of the IBM DB2 features for SAP

customers.

5

The four pillars of “IBM DB2 Optimized for SAP”

Partnership• Joint SAP and IBM teams working on all levels of the product:

Design, Development, Test, Support• Excellent collaboration between SAP and IBM

Product Integration• One product, one maintenance strategy, one-stop service • All SAP relevant DB features are part of SAP OEM license

Technology Innovation• Joint technology roadmap with agreed deliverables• Synchronized release cycles between IBM and SAP

SAP runs IBM DB2 • SAP IT is a very satisfied customer of the IBM DB2 product;

about 1100 SAP systems are running on IBM DB2• All large SAP business systems are running on IBM DB2 for LUW

IBM DB2 follows SAP‘s maintenance strategy:•Your IBM DB2 release remains in service for the complete 7+2 and 5+1+2 maintenance of your SAP application (see SAP note 1168456 and IBM TechNote 1051638) *•As a result you have the freedom of choice

You can stay on your current IBM DB2 version if you desire.Or you can upgrade to the next IBM DB2 version on your current SAP release to benefit from the technology innovations provided by IBM and SAP

•Unmatched business value: No forced or undesired database upgrades

6

IBM DB2 Optimized for SAP Initiative

2005

2Q/05 NW 2004Streamlined installStreamlined adminInitial BI MDC expl.Auto storage EE

Version 8.2.2Automatic storage adminDeployment optimized for SAPTailored concurrency & perf improvements

1Q/07 NW 7.0Embedded databaseUnlimited scalabilityMinimal adminTCO (reduced storagecosts, self tuning)

Version 9.1Storage limits removedNear-0 admin

MemoryStorage

Selected Autonomic / TCO featuresCompression

2Q/08 NW 7.0 SR33Q/08 NW 7.0 EhP 1

Turn-key HA solutionPerf & TCO differentiators (MDC)Turn-key compressionPerformance WarehouseIntegrated Workload ManagementIntegrated MDC advisorDeferred Table Creation

Version 9.5Integrated & automatic HADRIntegrated Flash CopyTCO improvements

DPFMemory ManagementStatistics ManagementCompression

Perf improvements (eg MDC)Threaded ArchitectureDPF Scaling Improvements

2006

2008

2010

2012

2H/2009 Version 9.7360° MonitoringEven Deeper Deep Compression

40+% reduction in Indexes40+% reduction in TempsCompressed LOBs

Near-0 Storage AdminSimple MDC space reclaimEasy table space reclamationEase the path to Automatic Storage

Extending Online OperationsChange schema definitions onlineReorganization improvements

Extending Integrated HA+DR OS coverageBI query performance boost

SAP

-IBM DB2 V8.2.2 was the first IBM DB2 optimized for SAP realease-Four IBM DB2 optimized for SAP releases was delivered within 5 years. -The joint collaboration between SAP and IBM continues beyond IBM DB2 9.7 delivery; the discussion about new IBM DB2 optimized for SAP features beyond IBM DB2 9.7 has already started. -Features of new IBM DB2 releases are not only available for new SAP releases. Most of the new IBM DB2 features are also supported by older SAP releases, e.g. for SAP 4.6D systems.

7

Technology Innovation Roadmap

• SAP and IBM jointly define the content of “IBM DB2 Optimized for SAP” releases

• Mandatory design review and approval by SAP for all SAP relevant IBM DB2 line items before implementation starts

• Testing and verification during the entire IBM DB2 development cycle, far before the product goes GA

• Near-concurrent GA: SAP customers can benefit from the innovations immediately, typically within 6-8 weeks after IBM eGA

8

Technology Roadmap – Detailed Planning

Version 8.2.2Version 8.2.2 Version 9.1Version 9.1 Version 9.5Version 9.5 Version 9.7Version 9.7

Deployment,Configuration,Administration

TCO reduction

BI & Query Performance

Supportability,Backup & Recovery, HA

SAP tuner (DB2_WORKLOAD = SAP)Automatic storage (single node)Auto extend DMS tablespacesUniform page-size

Silent Install (Windows)

MDC roll out – stage 1MDC roll inIndex heuristicsConcurrency enhancementsUniform page-size

Complete deadlock analysisOne-shot optimizer service toolOptimization guidelines

Silent install (Unix)Multiple IBM DB2 deployments on the same machineOne IBM DB2 image serves all: Install, upgrade, patchInstallation free clientLarger index key partsLarger index names

Automatic configuration: IO cleaner, serverSelf tuning database global memoryDeep compressionAutomatic storage for DPF

SAP BI Optimizer enhancementsSort capacity cap

Resumable recover Redirected restore script from backup imageSQL access to list of required files for flash copyPre-packaged HA solution (2 node fail over)

Easy fixpack deploymentSA MP bundled and installed with IBM DB2 on AIX and LINUX

Simplified configuration by multi threaded architectureAdvanced Memory Management : Automation for all memory areasAutomatic dictionary creation for deep compressionSimpler space reclaim for automatic storage and regular DMS

Real Time StatisticsImproved MDC roll out for fast deletionAdvanced DPF MonitoringParallel index creation

HA Cluster Manager IntegrationAutomatic Backup/Log Retention ManagementSingle system view backup for DPF databasesEnhanced Backup using Flash Copy

TEMP table compressionIndex compressionMigration to Automatic StorageAutomatic MDC space reclaimLOB inlining (with compression)Support for LONG->LOB transition

Online REORG improvementsOnline Table MoveTablespace capacity increaseCompression support for replicated tablesTablespace space reclaim below HWM

Enhanced MonitoringDiagnostics improvements

Starjoin query improvementsDPF costing improvementsScan Sharing

9

• Integrated installation of IBM DB2 software during SAP install *• Seamless IBM DB2 feature integration into SAP• Integrated HA setup during SAP install *• One-step SAP-tailored IBM DB2 configuration: DB2_WORKLOAD=SAP• Full IBM DB2 administration and monitoring through SAP DBA Cockpit

*as of SAP NetWeaver 7.0 SR3 and SAP Applications based on NW 7.0 SR3 and higher

SAP on IBM DB2 is a fully integrated product

10

Agenda• IBM DB2 Optimized for SAP Initiative• SAP Software Overview• IBM DB2 Monitoring and Administration







11

SAP’s Software Portfolio

BW EP PI MDM

Web Applications ServerABAP

SAP Solutions for Small Business andMidsize Companies

SAP Business Suite• SAP ERP• SAP CRM• SAP SCM• SAP PLM• SAP SRM

SAP NetWeaverStandalone Engines

SAP Industry Solutions• SAP for Automotive• SAP for Banking• SAP for Healthcare• SAP for Retail• . . .

• SAP Business Objects Portfolio- Crystal Reports- Xcelsius

• SAP Business All-in-One• SAP Business One

• SAP Business By Design

SAP NetWeaver• User Productivity• Information Management• Business Process Management• Development and Composition• Lifecycle Management• Security and Identity Management• SOA Middleware

BWAccelerator

liveCacheTREX(Search)

WebDispatcher

IBM DB2 V8.2, V9.1,V9.5, 9.7 for LUW

Enterprise Software

IBM DB2 optimized components

J2EE

MI

IBM DB2 for LUW

SAP MaxDB

SAP delivers two categories of business software:-Enterprise software solutions for large customers -Solutions for small business and midsize companies

Following business applications are part ot the SAP Business Suite:-SAP ERP (enterprise resource planning)-SAP Customer Relationship Management (CRM)-SAP Supply Chain Management (SCM)-SAP Product Lifecycle Management (PLM)-SAP Supplier Relationship Management (SRM)

SAP Industry Solutions:-SAP offers specialized industry solutions for a large range of industries, e.g. Automotive, Banking, Healthcare, Retail, ...

The SAP Business Suite and the SAP Industry Solutions are SAP applications, which call SAP NetWeaver modules and SAP NetWeaver Standalone Engines.

SAP NetWeaver is SAP‘s basis software product for the SAP Business Suite applications and for the SAP Industry Solutions.SAP NetWeaver is an application platform that serves as the foundation for service-oriented architecture (SOA). It allows the integration and alignment of people, information, and business processes across business and technology boundaries. SAP NetWeaver has following components:-Business Warehouse (BW)-Enterprise Portal (EP)-Process Integration (PI)-Mobile Integration (MI)-Master Data Management (MDM)-Web Application Server with the ABAP and J2EE software stack.Most of the IBM DB2 features are integrated into the SAP ABAP stack and in the SAP BW application. MI and J2EE has also (but less) IBM DB2 optimizations.

Each SAP NetWeaver system has a database. Four IBM DB2 for LUW databases (V8.2.2, V9.1, V9.5 and 9.7) are currently supported by SAP NetWeaver. In addition SAP NetWeaver has following four standalone engines, which does not have a database.-Business Warehouse (BW) Accelerator to improve SAP BW query performanceW b Di t h

12

Typical 3-tiered SAP Landscape

Frontend• SAP GUI• Browser

ApplicationServers

DatabaseServers

Network

IBM DB2 DPF

SAP Supports the 2-tier or 3-tier architecture. Larger systems are typically installed as a 3-tier system, i.e. the database server and the application servers are running on different hardware. A SAP system running on IBM DB2 DPF can have more than one database server. IBM DB2 DPF product is used for SAP‘s OLAP based applicatoins, like SAP BW.SAP‘s OLTP based applications, like SAP ERP, are running with a single ESE database installation. They are not supported on IBM DB2 DPF.

13








14

SAP DBA Cockpit

DBA Cockpit - Backend

Collection of DB Performance and Configuration Data

DBA Cockpit

Browser based or SAP GUI

DB DB DB DB

DBA Cockpit - ST04• IBM DB2 monitoring and administration is

integrated in SAP applications• Single point of entry for DB administrators• Access to all DBs in a SAP system landscape

New DBA Cockpit • Rollout with NW 7.00 EhP1 (Q1 2009)• Browser based solution on Web-Dynpro ABAP• More functionality and exploitation of new IBM

DB2 features• Easy navigation and guided procedures• Better preparation and monitoring of data • Allows monitoring of remote database systems

The SAP DBA Cockpit, which can be called with SAP transaction ST04, is a complete database administration and monitoring tool fully integrated into the SAP NetWeaver application.

15

SAP DB Performance Warehouse (1)

SAP BW

SMD InfoCubes,DPW InfoCubes

DB DB DBDB DB DBDB DB

DBA Cockpit - Backend

Extensive Collectionof Performance Data

SMD Backend

Extraction of SAPPerformance Data

Sol

utio

nLa

ndsc

ape

DBA Cockpit

Extensible Analysis-Application for DBAs

SMD Navigation

SMD Analysis on BI

• Central monitoring of all SAP systems in the system landscape• Central monitoring of all SAP and non- SAP databases in the system landscape• Monitoring of historical performance data.• Combined analysis of application and database related performance data

The SAP Solution Manager (SM) is the central software component of the SAP DB Performance Warehouse solution. The SAP Solution Manager includes the complete SAP BW application. Both performance metrics, from SAP applications and from databases are stored as multi-dimensional InfoCubes within SM‘s SAP BW database. This is the basis for the advanced monitoring capability of historical performance data.

16

With DB PW System Administrators can• see history and long term trends • easily modify or create reports and alerts• slice and dice into monitoring data• correlate data from different areas

With DB PW SAP can• easily incorporate customer reports into SAP

standard delivery• correlate data from SAP application with

DB or other third party products for improving end-to-end support

Roadmap• Rollout with NW 7.00 EhP1 (Q1 2009)• More BW content and incorporation of

feedback with future NW 7.00 EhPs

Drill Down

SAP DB Performance Warehouse (2)

17

• New „Time Spent“ monitor New metrics for wait times, component times and counters implemented as „light weight“ table functions on• System level

• Database• Database partition• WLM service class

• Activity level • based on SQL cache

• New event monitors• Lock event monitor: Deadlocks,

lock-timeouts, lock waits > x seconds• Package cache event monitor• Event monitor for UOW (not used within SAP)

• New object metrics for tables, indexes (e.g. index access statistics), tablespaces, database containers (e.g. I/O read/write times), bufferpools

Lock Wait20%

Sort Processing12%

BP Read Wait7%

BP Write Wait5%

Log Write Wait9%

Commit Processing

7%

Compile Processing

9%

FCM Send Wait3%

FCM Receive Wait7%

Others Wait7%

Others Processing14%

z

More efficient and more powerful performance analysis and tuning• Largest time consumers and wait times are easily visible• History for all important metrics • Support of problem-oriented performance tuning

Enhanced IBM DB2 9.7 Monitoring –Planned exploitation by SAP

SAP plans to integrate the new DB2 9.7 monitoring functionality into the SAP NetWeaver application. 360 degree database monitoring will be possible with the new DB2 9.7 monitoring features.

18

New monitoring features in SAP DBA Cockpit for IBM DB2 9.7• Planned delivery with next SAP Solution Manager 4.0 EhP2• New IBM DB2 UOW overview (former application monitoring)

• Details about coordinator agent and assigned subagents• “Time Spent” metrics for active UOWs• “Time Spent” metrics for active SQL statement currently running inside UOW• “Activity Trace” records selected metrics for an application over time

• Full integration of new object metrics including integration into SAP DB Performance Warehouse

• Integration of new lock event monitor including graphical display of lock wait events• Deadlocks, lock-timeouts, lock waits > x seconds• Basic information is always collected: Application context, WLM attributes,

Client PiD, Connect User, …• Extended information selectively switched on/off: Statement history inclusive bind values

Enhanced IBM DB2 9.7 Monitoring –Planned SAP DBA Cockpit Integration (1)

19

New monitoring features in SAP DBA Cockpit for IBM DB2 9.7• Integration of new Package Cache History

• Ability to see former execution plans for comparison

• History for almost all IBM DB2 monitoring data• Local system for short-term history, e.g. 2 weeks (configurable)• SAP DB Performance Warehouse for long-term history

• “Time Spent” metrics with sophisticate drill-down capabilities from top to detailed performance data through interactive charts

Enhanced IBM DB2 9.7 Monitoring –Planned SAP DBA Cockpit Integration (2)

drill-down into

package cache by selected metric

drill-down by WLM Service

Class

Global Time-Slice drill-down by DB

PartitionLock Wait20%

Sort Processing12%

BP Read Wait7%

BP Write Wait5%

Log Write Wait9%

Commit Processing

7%

Compile Processing

9%

FCM Send Wait3%

FCM Receive Wait7%

Others Wait7%

Others Processing14%

z

Drill-down examples of planned „Time Spent“ monitoring feature:-Drill down by package cache: Show 10 statements with largest lock wait time-Drill down by partition: Show lock wait time on all partitions of the last week-Drill down by WLM Service Class (e.g. srvice classes might be designed for SAP dialog, batch, update processes): You can drill-down by WLM servide class and see the lock wait times per WLM service class.

20








21

Autonomic IBM DB2 database

Environment under which a database is running changes permanently• Volatile or changing workload• New service packs or upgrades for DB or application• Activation of additional application components

Changing environment is a cost factor• Ongoing efforts for DB optimization and tuning• Risk for productive environment• Large SAP customers have many SAP systems installed

IBM DB2 optimized for SAP addresses the problem • IBM DB2 has many autonomic features which are fully supported by SAP • Many DB parameters are tuned automatically in a SAP environment• IBM DB2 @ SAP is an integrated product

22

Autonomic IBM DB2 database – Main FeaturesDB2_WORKLOAD = SAP (since IBM DB2 V8.2.2)• More than 20 DB parameters are affected• Automatically adopted for FixPacks or

new DB Versions

Auto Runstats (IBM DB2 V8.2.2) and Real Time Statistics (IBM DB2 V9.5)• Table and index statistics are created

automatically when needed

Automatic Storage (IBM DB2 V9.1, IBM DB2 9.7) details later

Automatic Backup and Log File Retention (IBM DB2 V9.5)

• Automate delete of old backup images andlog files

Easy FixPack Installation (IBM DB2 V9.5)• Automate binding of packages against

the database

Self Tuning Memory Manager (IBM DB2 V9.1, IBM DB2 V9.5)• Autonomic feature for IBM DB2 memory management• All relevant memory segments can be tuned automatically

•>db2set –all | grep DB2_WORKLOAD•[i] DB2_WORKLOAD=SAP•[i] DB2_TRUST_MDC_BLOCK_FULL_HINT=YES [DB2_WORKLOAD]•[i] DB2_ATS_ENABLE=YES [DB2_WORKLOAD]•[i] DB2_RESTRICT_DDF=YES [DB2_WORKLOAD]•[i] DB2_SET_MAX_CONTAINER_SIZE=20000000000[DB2_WORKLOAD]•[i] DB2_CREATE_DB_ON_PATH=YES [DB2_WORKLOAD]•[i] DB2_OPT_MAX_TEMP_SIZE=10240 [DB2_WORKLOAD]•[i] DB2_TRUNCATE_REUSESTORAGE=IMPORT [DB2_WORKLOAD]•[i] DB2_MDC_ROLLOUT=DEFER [DB2_WORKLOAD]•[i] DB2_SKIPINSERTED=YES [DB2_WORKLOAD]•[i] DB2_VIEW_REOPT_VALUES=YES [DB2_WORKLOAD]•[i] DB2_OBJECT_TABLE_ENTRIES=65532 [DB2_WORKLOAD]•[i] DB2_OPTPROFILE=YES [DB2_WORKLOAD]•[i] DB2_IMPLICIT_UNICODE=YES [DB2_WORKLOAD]•[i] DB2_MEM_TUNING_RANGE=10,20 [DB2_WORKLOAD]•[i] DB2_INLIST_TO_NLJN=YES [DB2_WORKLOAD]•[i] DB2_MINIMIZE_LISTPREFETCH=YES [DB2_WORKLOAD]•[i] DB2_REDUCED_OPTIMIZATION=4,INDEX,JOIN,NO_TQ_FACT,

NO_HSJN_BUILD_FACT, STARJN_CARD_SKEW,NO_SORT_MGJOIN, CART OFF,CAP OFF [DB2_WORKLOAD]

•[i] DB2NOTIFYVERBOSE=YES [DB2_WORKLOAD]•[i] DB2_INTERESTING_KEYS=YES [DB2_WORKLOAD]•[i] DB2_EVALUNCOMMITTED=YES [DB2_WORKLOAD]•[i] DB2_ANTIJOIN=EXTEND [DB2_WORKLOAD]•[i] DB2COMPOPT=327685,192 [DB2_WORKLOAD]•[i] DB2_RR_TO_RS=YES [DB2_WORKLOAD]•[i] DB2_DROP_NO_WAIT=YES [DB2_WORKLOAD]•[i] DB2_GRP_LOOKUP=,TOKEN [DB2_WORKLOAD]

•>db2set –all | grep DB2_WORKLOAD•[i] DB2_WORKLOAD=SAP•[i] DB2_TRUST_MDC_BLOCK_FULL_HINT=YES [DB2_WORKLOAD]•[i] DB2_ATS_ENABLE=YES [DB2_WORKLOAD]•[i] DB2_RESTRICT_DDF=YES [DB2_WORKLOAD]•[i] DB2_SET_MAX_CONTAINER_SIZE=20000000000[DB2_WORKLOAD]•[i] DB2_CREATE_DB_ON_PATH=YES [DB2_WORKLOAD]•[i] DB2_OPT_MAX_TEMP_SIZE=10240 [DB2_WORKLOAD]•[i] DB2_TRUNCATE_REUSESTORAGE=IMPORT [DB2_WORKLOAD]•[i] DB2_MDC_ROLLOUT=DEFER [DB2_WORKLOAD]•[i] DB2_SKIPINSERTED=YES [DB2_WORKLOAD]•[i] DB2_VIEW_REOPT_VALUES=YES [DB2_WORKLOAD]•[i] DB2_OBJECT_TABLE_ENTRIES=65532 [DB2_WORKLOAD]•[i] DB2_OPTPROFILE=YES [DB2_WORKLOAD]•[i] DB2_IMPLICIT_UNICODE=YES [DB2_WORKLOAD]•[i] DB2_MEM_TUNING_RANGE=10,20 [DB2_WORKLOAD]•[i] DB2_INLIST_TO_NLJN=YES [DB2_WORKLOAD]•[i] DB2_MINIMIZE_LISTPREFETCH=YES [DB2_WORKLOAD]•[i] DB2_REDUCED_OPTIMIZATION=4,INDEX,JOIN,NO_TQ_FACT,

NO_HSJN_BUILD_FACT, STARJN_CARD_SKEW,NO_SORT_MGJOIN, CART OFF,CAP OFF [DB2_WORKLOAD]

•[i] DB2NOTIFYVERBOSE=YES [DB2_WORKLOAD]•[i] DB2_INTERESTING_KEYS=YES [DB2_WORKLOAD]•[i] DB2_EVALUNCOMMITTED=YES [DB2_WORKLOAD]•[i] DB2_ANTIJOIN=EXTEND [DB2_WORKLOAD]•[i] DB2COMPOPT=327685,192 [DB2_WORKLOAD]•[i] DB2_RR_TO_RS=YES [DB2_WORKLOAD]•[i] DB2_DROP_NO_WAIT=YES [DB2_WORKLOAD]•[i] DB2_GRP_LOOKUP=,TOKEN [DB2_WORKLOAD]

23

Autonomic IBM DB2 – DB2 V9.5 Memory Model

INSTANCE_MEMORY - upper limit for a database partition

DATABASE_MEMORY - database memory

Lock Buffer ( LOCKLIST and MAXLOCKS)

Shared Sort Heap (SORTHEAP )

Package Cache (PCKCACHESZ)

Utility Heap (UTIL_HEAP_SZ)

Catalog Cache (CATALOGCACHE_SZ)

Database Heap (DBHEAP)

Bufferpools

4K, 8K , 16K, 32K

APPL_MEMORY - application memory

Application Heap (APPLHEAPSZ)

Private Sort Heap (SORTHEAP )

Query Heap (QUERY_HEAP_SZ)

Statement Heap (STMTHEAP)

Statistics Heap (STAT_HEAP_SZ)

Agent Stack (AGENT_STACK_SZ )

Java Interpreter Heap (JAVA_HEAP_SZ)

System Monitor Heap ( MON_HEAP_SZ )

Audit Monitor Heap ( AUDIT_BUF_SZ )

Instance memory has a new semantic and comprises database and application memory

Simplified Memory Management with IBM DB2 V9.5

24

Autonomic IBM DB2 –SAP Recommendations for STMM

STMM recommendation for IBM DB2 V9.5• SELF_TUNING_MEM = ON • INSTANCE_MEMORY = <value>

• DATABASE_MEMORY = AUTOMATIC

• LOCKLIST = AUTOMATIC• MAXLOCKS = AUTOMATIC• PCKCACHESZ = AUTOMATIC• SHEAPTHRES_SHR = AUTOMATIC• SORTHEAP = AUTOMATIC

• ALTER BUFFERPOOL <name> SIZE AUTOMATIC

Only increased when needed• APPL_MEMORY = AUTOMATIC • STMTHEAP = AUTOMATIC• APPLHEAPSZ = AUTOMATIC• STAT_HEAP_SZ = AUTOMATIC• DBHEAP = AUTOMATIC

STMM recommendation for IBM DB2 V9.1• SELF_TUNING_MEM = ON •

• DATABASE_MEMORY = <value>

• LOCKLIST = AUTOMATIC

• MAXLOCKS = AUTOMATIC• PCKCACHESZ = AUTOMATIC• SHEAPTHRES_SHR = AUTOMATIC• SORTHEAP = AUTOMATIC

• ALTER BUFFERPOOL <name> SIZE AUTOMATIC

25


• SAP’s DBA Cockpit• SAP’s Performance Warehouse






26

IBM DB2 9.7 – Enhanced Automatic Storage (1)IBM DB2 Automatic Storage (AS)• Fully integrated into SAP installation and

DBA Cockpit • Greatly reduce the effort of database administration• More flexibility with IBM DB2 9.7

IBM DB2 9.7 New Features• Add new AS paths to non AS Databases

ALTER DATABASE <dbname> ADD STORAGE ON <path1>, <path2> …

• Easy drop of existing AS paths, which are not used any more for future database growingALTER DATABASE <dbname> DROP STORAGE

ON <path1>, <path2> …

- State of storage path changed: “In Use” “Drop Pending”- Future growth on dropped paths will not occur- Each affected tablespace must be rebalanced before dropped path can be removed.

• Start data rebalancing (online) to move existing data to the new AS pathsALTER TABLESPACE <tsname> REBALANCE

P R E V I E W

27

IBM DB2 9.7 – Improved Space Reclamation (1)

BBB

AAB

AAA

RIDsKey Value

Index Page

0001

0006

0003

0002

0004

0005

0009

0001

0002

0003

0004

0005

0006

Tablespace

BBB

AAB

AAA

OR RIDsKey Value

Index Page

0001

0002

0003

0004

0005

0006

0007

0001

00020003

0004

0005

0006

TablespaceRIDsOR RIDs

EMP Page

0001

0006

0003

0002

0004

0005

0009

0001

0002

0003

0004

0005

0006

0007

Normal DMS and AS Tablespaces•RIDs are pointing directly to data pages •Moving of data pages online within tablespace is impossible

Reclaimable DMS and AS Tablespaces•Object relative RIDs are pointing to EMPpages. Only EMP pages contain the physical address of data pages.

•Moving of data pages online within a tablespace is now possible.

•Fully supported by SAP’s DBA cockpit•Reclaimable tablespaces are default*

*existing tablespaces cannot be altered to be reclaimable

28

IBM DB2 9.7 – Improved Space Reclamation (2)

CREATE DATABASE testdbCREATE TABLESPACE ts MANAGED BY AUTOMATIC STORAGECREATE TABLE t1(key int) IN tsCREATE TABLE t2 LIKE t1 IN tsCREATE TABLE t3 LIKE t1 IN ts

// Populate tables with records to achieve extent usage

DROP TABLE t1 // extents in FREE stateDROP TABLE t3 // extents in FREE state

ALTER TABLESPACE ts REDUCE MAX

Automatic Storage Tablespaces• ALTER TABLESPACE <tsname> REDUCE [ <size>| MAX | STOP ]

DMS Tablespaces• ALTER TABLESPACE <tsname> LOWER HIGH WATER MARK [ STOP ]• ALTER TABLESPACE <tsname> REDUCE …

29

Row Compression (1)

Dictionary Compression

• Dictionary contains repeated information from the rows in the table

• Common sequences of consecutive bytes in row replaced with 12 bit symbol

• Compression candidates can be across page and column boundaries or within column

• Can be enabled at CREATE table time

• Existing tables can be ALTERed to enable compression. Table reorganization necessary

• Automatic Dictionary Creation possible with IBM DB2 Version 9.5

Compression Dictionary

Uncompressed Records

CompressedRecords

x’01C‘

……..

Canada Ontario Toronto

….. …

WalldorfToronto2 Ontario MillerCanada

WalldorfToronto1 Ontario SmithCanada

1 x’01C‘ Smith

2 x’01C‘ Miller

30

Row Compression (2)

Significant Lower TCO with IBM DB2 Row Compression in SAP space

Key Figures• Better bufferpool utilization and less I/O• Saving of up to 90% storage for single

tables; up to 50% for whole SAP application

• Dialog response time could be improved by up to 20%

• Backup and Restore time could be reduced

• CPU user time went up by about 5%

No Compression (vmstat output)

0%

20%

40%

60%

80%

100%

0 10 20 30 40 50 60 70 80 90

seconds

cpu

%

idle

iowait

sys

usr

Row Compression (vmstat output)

0%

20%

40%

60%

80%

100%

0 10 20 30 40 50 60 70 80

seconds

cpu

%

idle

iowait

sys

usr

31

Row Compression (3)

Monitoring and Administration• IBM DB2 row compression is integrated

part of SAP DBA Cockpit• Compression adviser exists to identify

table candidates for row compression (/ISIS/ZCOMP - SAP note 980067)

• Checks for IBM DB2 row compression are implemented in SAP BI

Installation and Migration• Automatic Dictionary Creation possible

for installation and migration• Data sampling implemented in R3load

for creating optimal compression dictionaries

32

IBM DB2 9.7 - Index Compression (1)

Index RID-List Compression

Index Prefix CompressionBBB

AAB

AAA

RIDsKeyValue

CREATE INDEX I ON T (

F VARCHAR(3) )

Index Page

0001

0002

0003

0005

0007

0008

0009

BBB

AAB

AAA

RIDsKeyValue

0001

1

2

0005

2

0008

1

Index Page with RID List CompressionIBM DB2 Index Compression

• Is a transparent feature of the IBM DB2 engine*

• Reduces index size of a SAP system by 40% and more

• Reduces I/O data transfer and increase buffer pool hit ratio of a SAP system

• Works for unique indexes and secondary indexes as well

• Is fully supported for all SAP releases in standard and extended maintenance (4.6D – 7.20)

• Is default for tables with that are IBM DB2 row compressed*

• Is covered by the SAP OEM license for IBM DB2 product at no extra costs

* an index reorganization is required to compress existing indexes 0008

0007

0006

0005

0003

0002

0001

RIDsKey Value

CREATE UNIQUE INDEX I ON T (

F VARCHAR(3) )

Index Page

AAA

AAB

AAC

AAD

BBA

BBB

BBC

Index Page with Prefix Compression

0008

0007

0006

0005

0003

0002

0001

RIDsKey Value

AAA

B

C

D

BBA

B

C

33

IBM DB2 9.7 - Index Compression (2)

With IBM DB2 Compression you can reduce the size of your SAP system by 50% and more

-233 GB / -33 %297 GB / -142 %705 GBΣ Absolute

-408 GB / -58 %

135 GB / 100 %

162 GB / 38 %

Size After Row Compression

-64 GB / -22 %0 GBΔ Relative

71 GB / 47 %135 GBINDEXES

162 GB / 38 %570 GBDATA/LOB/LONG

Size After Index Compression

Size Before Compression

IBM DB2 Index and Row Compression savings with real customer data• SAP R/3 4.6D System

This slide shows the compression ratio of a real SAP R/3 4.6D customer database. After applying row and index compression, the database size was only a third of the original database size.

34

IBM DB2 9.7 – Temp Table Compression

IBM DB2 Temp Table Compression• Is a transparent feature of the IBM DB2 engine• Reduce the amount of required temporary disk

space• Works like IBM DB2 Row Compression for large

temporary tables• Improves query performance with large sort

operations:• Reduces I/O data transfer for large sorts• More sorts can be processed in bufferpool

• Is fully supported for all SAP releases in standard and extended maintenance (4.6D – 7.20)

• Is default for SAP systemsCompression Dictionary

Uncompressed Records

CompressedRecords

x’01C‘

……..

Canada Ontario Toronto

….. …

WalldorfToronto2 Ontario MillerCanada

WalldorfToronto1 Ontario SmithCanada

1 x’01C‘ Smith

2 x’01C‘ Miller

Temp Table compression is an important feature for large sort operations, which usually spills to temporary tablespaces. This feature reduces the size of required temporary table space. In general the sort operations run faster due to reduced I/O operations.

35

IBM DB2 9.7 – LOB Inlining (1)

2K Byte

4321

C1I252 BCREATE TABLE T1

( I INTEGER, C1 CLOB(1G) )

1K Byte

LOB Data PageTable Data PageDDL Statement

LOB Data PageTable Data PageDDL Statement

1K Byte

2K Byte

4321

C1I

252 B

CREATE TABLE T1

( I INTEGER, C1 CLOB(1G) )

IBM DB2 LOB Inlining• Is a transparent feature of the IBM DB2

engine*• Reduces LOB space in a SAP system• Allows buffering and compression of

LOB fields• Accelerates access of small LOB fields

and improves query performance• Is fully supported for all SAP releases

in standard and extended maintenance (4.6D – 7.20)

• Is default for SAP systems*

* a table reorganization/move is required to inline existing LOB fileds

36

IBM DB2 9.7 – LOB Inlining (2)

More LOBs will be used in future SAP releases• IBM DB2 Migration from LONG to LOB data type

• More LOBs are used by SAP applications

≤ 144CLOB(99K) LOGGED≤ 120CLOB(32K)

LOGGEDLCHR

≤ 120BLOB(32K) LOGGED≤ 120BLOB(32K)

LOGGEDLRAW

≤ 252

≤ 252

Bytes in data page

DBCLOB(512M) LOGGED≤ 252CLOB(1G)

LOGGEDSTRG

≤ 252

Bytes in data page

BLOB(1G) LOGGED

BLOB(1G) LOGGEDRSTR

UnicodeNon-UnicodeABAP DDIC Type

IBM DB2 LOB Data Types as used with SAP

The LOB inlining feature is important to reduce storage used for LOBs and to increase performance of LOB access. This will be more important for future SAP releases, because SAP applications store data more and more in LOB datatypes.

37








38

Integrated HA and DR Solution

* Currently limited to Linux, AIX and Solaris

• Built-in near-sync standby database with rollingupdate capability (IBM DB2 HADR)

• Integrated cluster manager for fail-over automation *• Integrated into SAP installation, which generates

the Tivoli SA MP configuration file.• IBM DB2 tool “db2haicu” is available for cluster

configuration• Fully supported for all SAP releases in service

Log replication with IBM DB2 HADR

db2agent

TablesIndexes

LogBuffer

LogWriter

Logs

Replay MasterShredder

Redo MasterRedo Workers

TablesIndexes

LogWriter

Logs

HADR

Virtual IP

ClientClientClient

HADR

Primary DB Server Secondary DB Server

Shared Disk

Primary DB Server

DB Instance

Standby DB Server

DB Instance

Virtual IP

ClientClientClient

Shared disk

SAP integration:-SAP includes the Tivoli SA MP setup into the SAPINST install dialog. -Following two High Availability solutions are supported by the SAP installation:

-Log replication with the IBM DB2 HADR feature. Depending on the customer input, SAPINST generates the Tivoli SA MP Input Configuration File.-Shared disk solution

Log Replication with HADR•Can be used for an onsite Hot Standby High Availability (HA) or for an offsite Disaster Recovery (DR) solution •Failover in seconds:

Cluster manager assigns the virtual IP address to standby DB server Rollback of uncommitted transactions on standby DB serverFast takeover, because the database on standby DB server is consistent. No DB shutdown/restart on standby DB server is needed.

Shared Disk:•High Availability (HA) solution •Failover can take a while:

Cluster manager assigns the virtual IP address to standby DB serverMounts the shared diskStarts the IBM DB2 instance on standby DB server IBM DB2 triggers a crash recovery

IBM DB2 automatically maintains cluster configuration changes during operations, like add node, add tablespace

NO scripting is required!

39








40

IBM DB2 Database Partitioning Feature (DPF) (1) • Shared-nothing scale out architecture with proven linear scale-out capability• Mature technology –almost all SAP on IBM DB2 customers run their Data Warehouse with DPF• Fully supported since SAP BW 2.0 • Fully integrated into SAP installation and DBA Cockpit• SAP BW support on IBM InfoSphere Balanced Warehouse:

http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101467

DB Part. 3 DB Part. 4DB Partition 0 DB Part. 1

SAP BasisData

DB Server 1 DB Server 2

Fast Communication

Appl. Server 1 Appl. Server N

Dimension

Aggregates

. . .

DB Part. 2

Fact

Operational Data Store (DSO)

Aggregates

Bufferpool IBMDEFAULTBP

Data Transfer during Query Processing

Psersistent Storage Area (PSA)

This slide shows •the data flow within SAP BW •the SAP extended star schema on which a SAP BW query operates. •the recommended table space layout of a partitioned SAP BW/DB2 database. Very large tables can be distributed across multiple IBM DB2 database partitions.

To improve query performance typically the large SAP BW tables are distributed across multiple DB2 database partitions. The amount of database partitions defines how many DB2 agents are processing a query in parallel (1 DB2 agent is used per database partition). During query execution the dimension tables which are located in partition 0 must be send to all partitions which contain fact data.

Predefined Partitioning Keys are automatically generated when an SAP BW object (InfoCube, PSA, ODS) is activated.

41

IBM DB2 Database Partitioning Feature (DPF) (2)

Queries

Customer Scenario• SAP BW query performance test • Compare IBM DB2 with Non-IBM DB2 database. Same hardware was used • Eight logical IBM DB2 database partitions were configured on a 8-way database server

Single-user Test• Set of 27 most important SAP BI queries run in a

loop• At any time one SAP BI query was running on

the systemResult: On average IBM DB2 LUW is 44%

faster than the non-IBM DB2 database.

Multi-user Test• Several user processes run the set of 27 most

important SAP BI queries in a loop• At any time several SAP BI queries were running

in parallel on the system• CPU utilization on the DB server: 90%- 100%

Result: On average IBM DB2 LUW is 23% faster than the non-IBM DB2 database.

Sec

onds

DB2 LUWNon-DB2

DB2 LUWNon-DB2

Sec

onds

42

Multi-Demensional Clustering (MDC) (1)MDC is Optimized for Multiple Key Access• MDC organizes table data along one or more columns defined as MDC dimensions

• Records with the same values in one or more columns are stored physically together in blocks of pages. Blocks correspond to tablespace extents.

• MDC block indexes point to blocks instead of single rows (block indexes are small)• Improves query performance for more than one key column (MDC dimensions) • Risk of high disk space consumption, if wrong MDC dimensions are chosen

• Queries run faster by factors, if query restrictions match MDC definition

SELECT * FROM Sales WHERE Region = WEST- 2 big block I/Os to retrieve pages containing region WEST- All sequential I/O (efficient)

SELECT * FROM Sales WHERE Year = 2005- 2 big block I/Os to retrieve pages containing year 2005- All sequential I/O (efficient)

EAST,2005 WEST,2005 WEST,2006

Region

Year0

50

100

150

200

250

300

350

400

450

Seco

nds

1 2 3 4 5 6SAP BW Query

Index vs. Multi-Dimensional Clustering

Index Clustering MDC on Calmonth, Dim1

Query Restrictions match MDC Definition

43

Multi-Demensional Clustering (MDC) (2)Fast Delete with MDC Rollout• Delete predicate must match with MDC definition• Delete with deferred RID Index cleanup

• Background processing of RID indexes kicked in after Commit

• Avoids random I/O • RID index clean-up in parallel (one background

task per RID index)• Logging during delete is significantly reduced

• No key deletions logged during delete• Background tasks periodically commit, limiting

transaction log consumption• Delete is factor 20 or more faster depending on the

number of RID indexes

Region

Year… 2004, 2005, 2005, 2006, …

RegionEast South West

MDC

RID

MDC Space Reclaim (IBM DB2 9.7)• Reorg TABLE <mdc-table-name> RECLAIM EXTENTS ONLY • Easy space reclaim after data roll-out in MDC tables • Can be part of automated maintenance policy• Fully integrated into SAP DBA Cockpit

Processing steps of the statement “DELETE … WHERE Region = West”-Affected blocks marked free and removed from region block index-Rows deleted with per-page processing; one small log record per page-Maintenance of RID index „Year“ is deferred to background processing

44

Multi-Dimensional Clustering (MDC) (3)MDC is integrated into SAP BW and SAP DBA Cockpit• Supported with SAP BW 3.5 and SAP NetWeaver BI 7.x• SAP BW user interface to specify MDC for InfoCubes, Datastore Objects, and PSA Tables• MDC advisor and size estimation tool is available in SAP DBA Cockpit• Re-clustering tool is available

Create SAP BW objects with MDC MDC Advisor integration

45








46

Online Table Move (1)• Stored procedure for online table move

• Fully logged operations• Small window where a X-lock is required• Small active log space usage• Minimal impact on Update/Delete/Insert operations

• Deployment• IBM DB2 V9.1, V9.5: Stored procedure SAPTOOL.ONLINE_TABLE_MOVE is delivered as part

of the db2sap library (SAP Note 103544)• IBM DB2 9.7: Delivered as IBM DB2 stored procedure SYSTOOLS.ADMIN_TABLE_MOVE

• Integration into SAP’s table move tool “DB6CONV” (SAP Note 362325)• Usage scenarios:

• Online enablement of IBM DB2 features• Index & table compression• Large RID• Move tables to IBM DB2 9.7 reclaimable tablespaces• Enablement of LOB inlining

• Online structural table changes• Add/remove columns• Enable/disable MDC

• Change tablespace physical layout• Change page size or extent size• Move to tablespace with large object table (DB2_OBJECT_TABLE_ENTRIES=65535)

• Misc• Reduce tablespace High Watermark• Online table redistribute (DPF environment)

DBAs like the Onlie Table Move feature, because it is can be used for many important DBA operations.

47

Online Table Move (2)• On the command line (single step)

call saptools.online_table_move(‘SAPR3’,’SVERS’,’TGT_D’,’TGT_I’,’’,’’,’’,’MOVE’)• On the command line (multi step)

call saptools.online_table_move(‘SAPR3’,’SVERS’,’TGT_D’,’TGT_I’,’’,’’,’’,’INIT’)call saptools.online_table_move(‘SAPR3’,’SVERS’,’’,’’,’’,’’,’’,’COPY’)call saptools.online_table_move(‘SAPR3’,’SVERS’,’’,’’,’’,’’,’’,’REPLAY’)call saptools.online_table_move(‘SAPR3’,’SVERS’,’’,’’,’’,’’,’’,’SWAP’)

TARGET TABLESOURCE TABLE STAGING TABLE

… imi2i1

cm… … cnc2c1

A… … AAA

… cmc2c1

… CCC

… BBB

… YYY

… ZZZ

… UZZ

INSERT

DELETE

UPDATEBefore image

UPDATEAfter image

TRIGGER

… imi2i1

B… … BBB

U… … ZZZ

Y… … YYY

C… … CCC

Z… … ZZZ

cm… … cnc2c1

A… … AAA

… imi2i1

B… … BBB

Y… … YYY

C… … CCC

Z… … ZZZ

Y… … UYY Y… … UYY

U… … ZZZ

COPY

Capture changes

REPLAY

48

IBM DB2 Workload Management (WLM)• Prioritize and control work in an IBM DB2 database along various aspects, for

example• SAP process types (dialog, batch, spool,… )• SAP end user names• SAP application server groups

• Integrated into SAP Installation, SAP DBA Cockpit and SAP Solution Manager *• SAP delivers a WLM configuration, which can be optionally build from

SAP DBA Cockpit.

* As of NetWeaver 7.0 EhP 1 and Solution Manager 7.0 EhP 1

49

• News, Blogs, Backgrounds, Docs: https://www.sdn.sap.com/irj/sdn/db6• Discussion Forum: https://www.sdn.sap.com/irj/sdn/forum?forumID=187

• IBM DB2 optimized for your SAP software: Customer reference stories, white papers, …http://www.ibm.com/solutions/sap/us/detail/landing/J233701A22235G06.html

Additional Information in SDN

50

Karl FleckensteinIBM Boeblingen, Germany

[email protected]

Session C12 The most important IBM DB2 features for SAP

Thank you for attending

The most important IBM DB2 features for SAP - IDUG

Documents

Transcript of The most important IBM DB2 features for SAP - IDUG