DW - 2nd - Introduction To DW & BI

44
Introduction to Data Warehouse & Business Intelligence 1 Hadziq Fabroyir . Department of Informatics

Transcript of DW - 2nd - Introduction To DW & BI

Page 1: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

1

Introduction toData Warehouse & Business Intelligence

Page 2: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

2

Makin Banyak IstilahOLTP, OLAP, DSS, Data Warehouse, Data Mining.

Tapi intinya adalah:Teknologi untuk mengambil keputusan berdasarkan data

yang cukup

Page 3: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

3

Sejarah SingkatPemakaian Komputer

Page 4: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

4

Page 5: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

5

Cuma Beda: “Online”

1978

DSS

Page 6: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

6

Perkembangan Teknologi Database

1950•Komputer hanya di AS•terbatas di kalangan militer, pemerintah, dan akademis•Belum ada Model Basis Data

1960•komputer untuk bisnis •IMS dari IBM •model pohon dan model jaringan

Page 7: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

7

Model Kebanyakan Jaman DuluModel Pohon Model Jaringan

Page 8: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

8

Model Relational

Page 9: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

9

Perkembangan Teknologi Database

1970 •baru ada model relasional. Digagas oleh Edgar F. Codd.

1972 •prototipe relational database dibuat.

1975 •IBM bikin DBMS versi komersialnya.

1977 •Oracle menyusul.

1980 •dBASE banyak yang pake, karena running di atas DOS yang lagi naik daun.

Page 10: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

10

Relational ModelFounding Father

Page 11: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

11

Page 12: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

12

OLTP(OnLine Transaction Processing)

Tidak semua transaksi yang online bisa dibilang online.

Seperti contohnya adalah proses transaksi kliring

Dulu tahun 1980, transaksi online mungkin dapat disebut sebagai pemrosesan yang interaktif atau waktu-nyata (real-time).

Sehingga kliring seperti contoh di atas itu termasuk offline atau lebih dikenal dengan istilah batch.

Page 13: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

13

Decision Support System

Istilah Decision Support System diperkenalkan P.G.W Keen dan M.S.Scott Morton lewat bukunya : Decision Support System: An Organizational Perspective” tahun 1978.

Arsitekturnya terdiri atas: Database (knowledge base),

Model (the Decision context and user criteria), dan

Antar muka.

Page 14: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

14

Sejarah Pengambilan Keputusan

Sampai dekade 1970 masih banyak software yang belum memakai SQL dan relational database.

Pada masa itu, data komputer sudah dipakai untuk membantu pengambilan keputusan menggunakan: Riset Operasional, Teori Manajemen, dan Teori Perilaku.

DSS pada saat itu berkenaan dengan data agregat (data yang diperoleh dengan memakai operasi-operasi agregat seperti SUM, COUNT, AVG, MIN, MAX).

Jelasnya RDBMS dan SQL sangat memudahkan pembuatan data agregat, hal yang sulit dalam COBOL.

Page 15: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

15

Data WarehouseIstilah ini baru diperkenalkan pada tahun 1988 (10 tahun setelah diperkenalkannya istilah DSS)

Yang memperkenalkan adalah W. H. Inmon dalam bukunya “Data Architecture: The Information Paradigm”

Page 16: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

16

Data WarehouseMuncullah rumusan bahwa subjek datawarehouse berkenaan dengan:

Pengolahan data agregatTipe snapshot, bitmap index, function indexTipe Table Partition dan Index PartitionOperasi star join dan operasi-operasi yang aware terhadap partitionStar-schema (ad0hoc database design)

Page 17: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

17

OLAP(OnLine Analytical Processing)

Muncul tahun 1993 oleh Edgar F. Codd, S. B. Codd, dan C. T. Salley dalam dokumen untuk Arbor Corporation berjudul “Providing OLAP (OnLine Analytical Processing) to User-Analyst: An IT Mandate”.

Analytical Processing diterapkan ke datawarehousing bukan production database.

Lagi-lagi tidak ada dasar ilmiah. Kejadian yang mirip dengan DSS.

Page 18: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

18

OLAP(OnLine Analytical Processing)

Euforia banyak saintis dan akademis tentang OLAP telah mereda. Berikut adalah butir-butir berikut sebagai dasar ilmiah untuk OLAP:

Tipe: TABLE, REPORT, dan CUBEOperasi: GROUPING, ROLLUP, CUBENilai: pemakaian nilai NULL bagi nilai-nilai ‘sel’ di kubus (cube) dan report untuk data agregat.

Page 19: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

19

Page 20: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

20

Data MiningIstilah ini dikenalkan tahun 1996 oleh Peter Adriaans dan Dolf Zatinge dalam bukungan “Data Mining” oleh Addison-Wesley. Mereka menulis aspek-aspek dan teknik-teknik ilmiah yang bisa dipakai untuk Data Mining.

Secara umum Data Mining adalah mengolah data.

Page 21: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

21

Teknik & Aturan Data Mining

Secara khusus, data mining sebagai pengolahan data memakai teknik atau aturan yang di antaranya adalah sebagai berikut:

Association ruleClassification ruleClustering rulePrediction rule

Page 22: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

22

Analisis dan Pengambilan Keputusan

Page 23: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

23

It’s All About Technology …

Apabila Data Mining, OLAP, Datawarehouse, dan lain-lain dianggap sebagai teknologi, maka:

Semuanya mengarah ke satu hal: untuk mengambil keputusanSia-sia menyimpan data, membuat algoritme, dan lain-lain, jika tidak dipakai untuk mengambil keputusan.Dengan memanfaatkan semua teknologi tersebut (dengan bijaksana), keputusan yang diambil adalah well-informed decision.

Page 24: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

24

KesamaanSelain kesamaan tujuan inti, semua teknologi di atas juga memiliki kesamaan:

Untuk mengolah data-data agregatMemakai sumber data yang berukuran sangat besarMemakai analisis

Page 25: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

25

Microsoft’s MindMicrosoft Corp memakai frasa Analysis

Service untuk semua fasilitas-fasilitas di SQL Server 9 DBMS.

Page 26: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

26

Restropeksi Sejarah

Relational Model (1969)

DSS(1978)

OLAP(1993)

Page 27: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

27

Page 28: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

28

Business Intelligence

Analysis of corporate data that influences business decision making

Page 29: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

29

Page 30: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

30

The Tools Evolution

EM 2000

QA 2000

AM 2000

EM 2000

QA 2000

AM 2000

SQL 2000 SQL 2005

SQL Management

Studio

BI Development

Studio

Target Audience:

DBA

Developer

Page 31: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

31

SQL ’05 BI Components

•SQL Server Management StudioManage•Business Intelligence Development StudioDesign•Integration ServicesSynthesize

•Universal Dimensional Model (UDM)Store•Reporting ServicesDeliver

Page 32: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

32

SQL Server Management Studio

“Replacement” for SQL Server Enterprise Manager and Query Analyzer

Visual Studio 2005 IDE

Manage all DatabasesRelational Databases

Analysis Databases

CE Databases

Page 33: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

33

Page 34: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

34

Integration Services

Formerly Data Transformation Services (DTS)

Build and debug complex integration packages

Separation of Control Flow and Data Flow

Integrated Source Control

Page 35: DW - 2nd - Introduction To DW & BI

Hadziq Fabroyir . Department of Informatics

35

Integration Services

Page 36: DW - 2nd - Introduction To DW & BI

New England Code Camp IV: “Developer’s Gone Wild”

Universal Dimensional

ModelUnified logical model for both relational and OLAP analysis databases with high performance and scalability

Capture and model all of your data

Relational reporting and OLAP converge through a single relational model

Page 37: DW - 2nd - Introduction To DW & BI

New England Code Camp IV: “Developer’s Gone Wild”

SQL Server 2005 Enterprise BI A Unified Dimensional Model

DW

Datamart

DatamartBI Applications

MOLAP

MOLAP

OLAP Browser

Reporting Tool

UDM

Page 38: DW - 2nd - Introduction To DW & BI

New England Code Camp IV: “Developer’s Gone Wild”

Illustration

Page 39: DW - 2nd - Introduction To DW & BI

New England Code Camp IV: “Developer’s Gone Wild”

Which models are available with SQL ’05?

Analytical problem Examples AlgorithmsClassification: Assign cases to predefined classes

Credit risk analysisChurn analysisCustomer retention

Decision TreesNaive BayesNeural Nets

Segmentation: Taxonomy for grouping similar cases

Customer profile analysisMailing campaign

ClusteringSequence Clustering

Association: Advanced counting for correlations

Market basket analysisAdvanced data exploration

Decision TreesAssociation

Time Series Forecasting: Predict the future

Forecast salesPredict stock prices

Time Series

Prediction: Predict a value for a new case based on values for similar cases

Quote insurance ratesPredict customer income

All

Deviation analysis: Discover how a case or segment differs from others

Credit card fraud detectionNetwork infusion analysis

All

Page 40: DW - 2nd - Introduction To DW & BI

New England Code Camp IV: “Developer’s Gone Wild”

Reporting Services

Deliver traditional and interactive reports

Single platform and tools for all types of structured data (relational, hierarchical, multidimensional)

Single platform for authoring, management, and delivery of reports

Page 41: DW - 2nd - Introduction To DW & BI

New England Code Camp IV: “Developer’s Gone Wild”

Reporting Services

Page 42: DW - 2nd - Introduction To DW & BI

New England Code Camp IV: “Developer’s Gone Wild”

Reporting Services

Page 43: DW - 2nd - Introduction To DW & BI

Server Middle Tier Client

AnalysisServerAnalysisServer

IIS

IIS

XM

LA

IS

AP

IX

MLA

IS

AP

I

PTS

/OLED

BP

TS

/OLED

B

AnalysisServerAnalysisServer

TCP

HTTP IISIIS

AS2005

AS2000

XM

LA

XM

LA

Application

XMLA

XMLA

Application

HTTP

XML for Analysis AS 2000 vs. AS 2005

Page 44: DW - 2nd - Introduction To DW & BI

New England Code Camp IV: “Developer’s Gone Wild”

ResourcesSQL Server Developer Center

http://msdn.microsoft.com/sql/

OLAP Bloghttp://www.sqljunkies.com/weblog/mosha

SQL Server 2005 Data Mininghttp://www.sqlserverdatamining.com

SQL Server Integration Serviceshttp://www.sqlis.com

Introduction to SQL Server Report Builderhttp://www.developer.com/db/article.php/3520116