Post on 20-Mar-2023
Senior DB2 LUW DBA and Data Stage and Cognos Infrastructure lead forChurch Mutual Insurance Company based in Merrill Wisconsin. I preferred tobe called Kohli
2012 DB2's GOT TALENT winner on The DB2Night Show.
Reviewing Raul Chong and Clara Liu tentatively titled book UnderstandingDB2 : The Essentials
Attended both the IDUG NA Denver and EMEA Germany conferences.
1
-About Church Mutual
-Need for a new Policy Admin System ? To get more revenue
-Why and How data is the biggest natural resource and how it can helporganizations succeed.
3
Database score is poor
Look at metrics SQL SYNC Read % -explain
IX Read Efficiency.
Async write %
SQL BP Hit Ratio
7
Av Logical Reads Av Execution time. 3.07 3.95 1.99, 132 ( big numbers ) Notethis is on our dev server with multiple isntance for multiple apps so CPU andmemory are shared and I will explain how we tackled those later.Av rows readIref , relative CPU cost Response time execution time- these are metrics
8
Tells us to create 2 indexes over street name and Data item id.
99% improvement if we do that
Creating the IX is very easy using DBI, you just need to click on the IX nameand Run. You will talk about Change Management.
DBI is very methodical it creates a full Report.
And stores every detail on changes in its Repository .
10
Av Logical Reads Av Execution time. 3.07 3.95 1.99, 132 ( big numbers )becomes 0.01,0.033 0.004, 13.26 (note the 4.5 second query is an IBM queryBig improvements everywhere. Advise and explains have not been this easy.Iref , relative CPU cost Response time execution time- these are metrics
12
Less Rows read . Mountains become Flat lands, seems we came from theHimalayas to the Antarctica ( both are cold )
14
Before AV TX Time is 4.179
Inside time 0.125 (3%)
Av outside time 4.05( 97%)
Perplexed ? This is our dev server which is 1 lpar and 9 intances on both WASand for DB2 . Resource contention for WAS.
17
DB2 does not like using indexes over expressions.
Cardinality of an index is important
Index over var chars are not good
Where Views and MQT’s have helped us.
18
Story: We selected 1.8 M rows on 2 executions and were selecting everything.
Total size of this select statement is around( DBI provides the metrics in thecomments section when we extract the query)1KB
Total rows retrieved 900,000 .
So size of 1 execution is 900,000 x 1= 0.85 GB
WAS choked up after 2-3 execution threw heap dumps.
This is where we should use fetch first n rows and not everything.
19
Create a Table with an auto generated trigger which converts text to uppercase.
Run the above query over the table with expression over column.
We are converting whatever we have in the column to lower.
21
Storing the Data in either case : meaning end user can pass name in any case.But data in the table should be stored as upper.
25
Cost is 22 when we use inverse of an expression over a column and not 194when we have an expression over a column
26
First Problem can be solved by creating an Index
Second and Third we cant use and Index as you need to perform a full indexscan or Table scan which means reading all index/table leaf pages and theirentries from top to bottom
28
• Change the search Criteria from '%abc' to'cba%'
• Create a reverse function.
• Add generated columns with reversed value of the searched column, thenIndex it.
• Index range scan be used efficiently for case of searched value endingwith abc.
29
Querying over the real column the Total Cost comes out to be 1628 , Index notbeing used, clear Table scan.
32
Querying over Triggered column with a reverse name. The cost comes out tobe 1638 without an index.
33
Create an Index over Rev column and Full name ( to avoid a fetch ) The costdropped from 1638 to 15 ( pretty good )
There are different ways you can implement this on the UI.
34
Create the Table .
Includes an auto generate column which uses this bit mask Function. So whenyou insert something bits will be assigned to each Alphabet.
So we Create Table Create Indexes now we Load a few rows of data andAlways Run Stats.
40
Query 1 : normal wildcard search scans the IX or the table
Query 2 : the query is important: in the first part of where we search thefunction to be only used from the alphabet passed for the coulmn.
In the second where we locate all the entire string being passed
41
How using testdata.db2 in helped us generate random and diverse test data.
http://pic.dhe.ibm.com/infocenter/dzichelp/v2r2/index.jsp?topic=%2Fcom.ibm.db2z9.doc.sqlref%2Fsrc%2Ftpc%2Fdb2z_bif_rand.htm
http://publib.boulder.ibm.com/infocenter/db2luw/v9/index.jsp?topic=%2Fcom.ibm.db2.udb.admin.doc%2Fdoc%2Fr0000862.htm
45
The query above is used to create all the foreign key content from the systemtables
We can use the same to same approach to load and export data w/ db2move
47
The query above collects all the content to delete foreign keys from systemtables
We can use the same to same approach to load and export data w/ db2move
48
Horrible Statemtent performance
Multiple queries being fired all outer joins, no table scans to be seen
49
The queries are generated by Hibernate.
Even DB2’s explain graph is not compressed enough to give us a good viewover explain graph
Explain over liltte data.
50
The queries are generated by Hibernate.
Even DB2’s explain graph is not compressed enough to give us a good viewover explain graph
Explain over more data.
51
Our Database design .
To find 1 result the query joins all tables, reads data from all and gives resultfrom 2
1 being the parent and other the child.
52
• Restored Production database to 2 instance of DB2, one on 9.7 and theother on 10.1
• Took the top 10 executing queries on that DB and executed over both.
• Above is Statement performance is on 9.7
58
• Restored Production database to 2 instance of DB2, one on 9.7 and theother on 10.1
• Took the top 10 executing queries on that DB and executed over both.
• Above is Statement performance is on 10.1.2
59
DBI’s Explain is “one stop shopping” on a single pane of glass. The statementis shown top left, the Explain plan in center left, the tables involved, theirstatistics and attributes at top right, and, for any highlighted table on top right,indexes and their statistics are shown plus a describe of the table. No need torun queries against SYSCAT.INDEXES, SYSCAT.TABLES,SYSCAT.COLUMNS, or run multiple DESCRIBE commands. The Explainpresentation is concise (compressed) but the “graphical” view is availabletoo. The steps of the explain plan are easily traced, along with costs andstatistics for each step.
60
More Sort Steps in 10.1
DBI’s Explain is “one stop shopping” on a single pane of glass. The statementis shown top left, the Explain plan in center left, the tables involved, theirstatistics and attributes at top right, and, for any highlighted table on top right,indexes and their statistics are shown plus a describe of the table. No need torun queries against SYSCAT.INDEXES, SYSCAT.TABLES,SYSCAT.COLUMNS, or run multiple DESCRIBE commands. The Explainpresentation is concise (compressed) but the “graphical” view is availabletoo. The steps of the explain plan are easily traced, along with costs andstatistics for each step.
61
Workload compare. Yes DBI has this functionality where you can compareworkloads of different queries fired on the same DB over a period of time. Inthis case I leveraged this functionality to compare worklods b/w 9.7 and 10.1
- We can see that execution time have improved although index efficiency isthe same.
63
Nmons for write and reading data, I/O is poor which makes a DBA look bad asqueries do not perform.
Peopl will argue its not a DBA role but a DBA should know what it is.
68