Senior DB2 LUW DBA and Data Stage and Cognos Infrastructure lead forChurch Mutual Insurance Company based in Merrill Wisconsin. I preferred tobe called Kohli

2012 DB2's GOT TALENT winner on The DB2Night Show.

Reviewing Raul Chong and Clara Liu tentatively titled book UnderstandingDB2 : The Essentials

Attended both the IDUG NA Denver and EMEA Germany conferences.

1

Company URL : http://www.churchmutual.com/

2

-About Church Mutual

-Need for a new Policy Admin System ? To get more revenue

-Why and How data is the biggest natural resource and how it can helporganizations succeed.

3

-Technologies used at CMIC

- Technical Architecture

4

-Coming to why we all are here today, to learn and here is the TechnicalAgenda.

5

-What 2 new DBA’s support.

6

Database score is poor

Look at metrics SQL SYNC Read % -explain

IX Read Efficiency.

Async write %

SQL BP Hit Ratio

7

Av Logical Reads Av Execution time. 3.07 3.95 1.99, 132 ( big numbers ) Notethis is on our dev server with multiple isntance for multiple apps so CPU andmemory are shared and I will explain how we tackled those later.Av rows readIref , relative CPU cost Response time execution time- these are metrics

8

Total Cost -338,983

Tb Scan over predicate Street name

9

Tells us to create 2 indexes over street name and Data item id.

99% improvement if we do that

Creating the IX is very easy using DBI, you just need to click on the IX nameand Run. You will talk about Change Management.

DBI is very methodical it creates a full Report.

And stores every detail on changes in its Repository .

10

Cost drops from -338,983 to 161. how easy was that.

11

Av Logical Reads Av Execution time. 3.07 3.95 1.99, 132 ( big numbers )becomes 0.01,0.033 0.004, 13.26 (note the 4.5 second query is an IBM queryBig improvements everywhere. Advise and explains have not been this easy.Iref , relative CPU cost Response time execution time- these are metrics

12

Execution time of each query goes down. Good for Performance of the AppDB

13

Less Rows read . Mountains become Flat lands, seems we came from theHimalayas to the Antarctica ( both are cold )

14

Better Index performance. Looks like the trajectory of a plane landing.

15

Before AV TX Time is 3.6

Inside time 0.92 (25%)

Av outside time 2.7( 75%)

16

Before AV TX Time is 4.179

Inside time 0.125 (3%)

Av outside time 4.05( 97%)

Perplexed ? This is our dev server which is 1 lpar and 9 intances on both WASand for DB2 . Resource contention for WAS.

17

DB2 does not like using indexes over expressions.

Cardinality of an index is important

Index over var chars are not good

Where Views and MQT’s have helped us.

18

Story: We selected 1.8 M rows on 2 executions and were selecting everything.

Total size of this select statement is around( DBI provides the metrics in thecomments section when we extract the query)1KB

Total rows retrieved 900,000 .

So size of 1 execution is 900,000 x 1= 0.85 GB

WAS choked up after 2-3 execution threw heap dumps.

This is where we should use fetch first n rows and not everything.

19

Expression over a columns.

20

Create a Table with an auto generated trigger which converts text to uppercase.

Run the above query over the table with expression over column.

We are converting whatever we have in the column to lower.

21

As seen in the explain in spite of having an Index the cost of query is 194

22

We are querying the exact string.

23

Cost of the query is 15 if we do not use an expression over the column.

24

Storing the Data in either case : meaning end user can pass name in any case.But data in the table should be stored as upper.

25

Cost is 22 when we use inverse of an expression over a column and not 194when we have an expression over a column

26

Conclusions

27

First Problem can be solved by creating an Index

Second and Third we cant use and Index as you need to perform a full indexscan or Table scan which means reading all index/table leaf pages and theirentries from top to bottom

28

• Change the search Criteria from '%abc' to'cba%'

• Create a reverse function.

• Add generated columns with reversed value of the searched column, thenIndex it.

• Index range scan be used efficiently for case of searched value endingwith abc.

29

Create Table with an auto generated column with a reverse trigger. And thenload data

30

After Loading the data this is how your table looks like.

31

Querying over the real column the Total Cost comes out to be 1628 , Index notbeing used, clear Table scan.

32

Querying over Triggered column with a reverse name. The cost comes out tobe 1638 without an index.

33

Create an Index over Rev column and Full name ( to avoid a fetch ) The costdropped from 1638 to 15 ( pretty good )

There are different ways you can implement this on the UI.

34

Summary top search text.

35

Bit mask assigned for write and execute

36

Bit Mask assigned for Read and Execute

37

Frequency of alphabets used for Bit Assignment.

38

This is how you create a String Mask Function.

39

Create the Table .

Includes an auto generate column which uses this bit mask Function. So whenyou insert something bits will be assigned to each Alphabet.

So we Create Table Create Indexes now we Load a few rows of data andAlways Run Stats.

40

Query 1 : normal wildcard search scans the IX or the table

Query 2 : the query is important: in the first part of where we search thefunction to be only used from the alphabet passed for the coulmn.

In the second where we locate all the entire string being passed

41

Add more data to check whether this really works, normal Wildcard search

42

Add more data to check whether this really works, Special Wild card search.

43

Summary of how to counter wild card searches and its benefits.

44

How using testdata.db2 in helped us generate random and diverse test data.

http://pic.dhe.ibm.com/infocenter/dzichelp/v2r2/index.jsp?topic=%2Fcom.ibm.db2z9.doc.sqlref%2Fsrc%2Ftpc%2Fdb2z_bif_rand.htm

http://publib.boulder.ibm.com/infocenter/db2luw/v9/index.jsp?topic=%2Fcom.ibm.db2.udb.admin.doc%2Fdoc%2Fr0000862.htm

45

How changing integer length can spread more data across the column

46

The query above is used to create all the foreign key content from the systemtables

We can use the same to same approach to load and export data w/ db2move

47

The query above collects all the content to delete foreign keys from systemtables

We can use the same to same approach to load and export data w/ db2move

48

Horrible Statemtent performance

Multiple queries being fired all outer joins, no table scans to be seen

49

The queries are generated by Hibernate.

Even DB2’s explain graph is not compressed enough to give us a good viewover explain graph

Explain over liltte data.

50

The queries are generated by Hibernate.

Even DB2’s explain graph is not compressed enough to give us a good viewover explain graph

Explain over more data.

51

Our Database design .

To find 1 result the query joins all tables, reads data from all and gives resultfrom 2

1 being the parent and other the child.

52

Why was this design created, Normalization to the core.

53

Denormalized the Database table

54

Explain after the design change with very little data.

55

Explain after a lot of data no Index

56

Explain after a lot of data and an Index. And final summary.

57

• Restored Production database to 2 instance of DB2, one on 9.7 and theother on 10.1

• Took the top 10 executing queries on that DB and executed over both.

• Above is Statement performance is on 9.7

58

• Restored Production database to 2 instance of DB2, one on 9.7 and theother on 10.1

• Took the top 10 executing queries on that DB and executed over both.

• Above is Statement performance is on 10.1.2

59

DBI’s Explain is “one stop shopping” on a single pane of glass. The statementis shown top left, the Explain plan in center left, the tables involved, theirstatistics and attributes at top right, and, for any highlighted table on top right,indexes and their statistics are shown plus a describe of the table. No need torun queries against SYSCAT.INDEXES, SYSCAT.TABLES,SYSCAT.COLUMNS, or run multiple DESCRIBE commands. The Explainpresentation is concise (compressed) but the “graphical” view is availabletoo. The steps of the explain plan are easily traced, along with costs andstatistics for each step.

60

More Sort Steps in 10.1

DBI’s Explain is “one stop shopping” on a single pane of glass. The statementis shown top left, the Explain plan in center left, the tables involved, theirstatistics and attributes at top right, and, for any highlighted table on top right,indexes and their statistics are shown plus a describe of the table. No need torun queries against SYSCAT.INDEXES, SYSCAT.TABLES,SYSCAT.COLUMNS, or run multiple DESCRIBE commands. The Explainpresentation is concise (compressed) but the “graphical” view is availabletoo. The steps of the explain plan are easily traced, along with costs andstatistics for each step.

61

After upgrading 9.7 to 10.1.2 the Statement performance looks the same.

62

Workload compare. Yes DBI has this functionality where you can compareworkloads of different queries fired on the same DB over a period of time. Inthis case I leveraged this functionality to compare worklods b/w 9.7 and 10.1

- We can see that execution time have improved although index efficiency isthe same.

63

Clearly Execution times have dropped

64

Clearly Cpu is being consumed less in comparison with 9.7

65

However no change in Index ref that is the same number of rows are beingread and fetched.

66

Results and Conclusion.

67

Nmons for write and reading data, I/O is poor which makes a DBA look bad asqueries do not perform.

Peopl will argue its not a DBA role but a DBA should know what it is.

68

Queue DepthSpreading and Striping

Memory increase

69

Spreading mechanism.

70

Nmons for reads and write after Spreading and Queue depths were improved.

71

What is striping.

72

73

74