Testing dan Implementasi Testing dan Implementasi Sistem InformasiSistem Informasi

TujuanTujuan

Tercapainya tujuan implementasi Tercapainya tujuan implementasi sistem informasi yang mendukung sistem informasi yang mendukung keberlanjutan dalam pengelolaan keberlanjutan dalam pengelolaan bisnis untuk mencapai visi dan target bisnis untuk mencapai visi dan target utama bisnis. utama bisnis.

Efisiensi dan efektifitas bisnis proses; Efisiensi dan efektifitas bisnis proses;

Leader Commitment(e-Leadership)

S/W

H/W

HR

N/W

INF.

Services

Quality of Services

Continuity of Services

Correctness

System Improvement

Information Systems

General ControlGeneral Control ManagementManagement HardwareHardware SoftwareSoftware System SupportSystem Support DatabaseDatabase NetworkingNetworking

Logical SecurityLogical Security OperationsOperations ContinuityContinuity Physical SecurityPhysical Security Systems Systems

developmentdevelopment

General Controls - General Controls - ManagementManagement

OrganizationOrganization PlanningPlanning TrainingTraining SecuritySecurity Resource managementResource management FacilitiesFacilities OperationsOperations

General ControlsGeneral ControlsHardwareHardware

Hardware InteractionHardware Interaction ProcessorsProcessors

MainframesMainframes DesktopsDesktops LaptopsLaptops

Input devicesInput devices Tape, CD, scanner, smart card reader, RFID, Tape, CD, scanner, smart card reader, RFID,

dlldll Output devicesOutput devices

General ControlsGeneral ControlsSoftwareSoftware

Systems interactionsSystems interactions Operating systemsOperating systems ApplicationsApplications Integrated systemsIntegrated systems UtilitiesUtilities

Software FunctionsSoftware Functions

Application ProcessingApplication Processing

Data Data ProcessorProcessor OnlineOnline

Management Communications Management Communications TransactionTransaction

ControlControl

Access ControlAccess Control

Operating System FunctionOperating System Function

General Controls ProgramGeneral Controls ProgramIntegrityIntegrity

Program integrityProgram integrity TestingTesting AccessAccess MaintenanceMaintenance

General Controls General Controls System SupportSystem Support

Primary objectivePrimary objective Control environmentControl environment Central vs. DecentralizedCentral vs. Decentralized Control requirementsControl requirements

General ControlsGeneral ControlsDatabaseDatabase

DefinedDefined Logical viewLogical view Physical viewPhysical view

ProductsProducts How they workHow they work SecuritySecurity

Access ControlAccess Control AuthenticationAuthentication

BackupBackup Contingency PlanContingency Plan

Information Network

General Controls - General Controls - NetworkingNetworking

Basics of networkingBasics of networking Topologies: star, bus, hybrid,Topologies: star, bus, hybrid, Protocols: TCPIP, PPP, …Protocols: TCPIP, PPP, …

Components: router, switch, firewallComponents: router, switch, firewall Internal Connectivity: VPN, Internal Connectivity: VPN, Performance: speed, bandwidth, CIRPerformance: speed, bandwidth, CIR

General ControlsGeneral ControlsNetwork ConfigurationsNetwork Configurations

CentralizedCentralized DecentralizedDecentralized DistributedDistributed Client-ServerClient-Server

General ControlsGeneral ControlsNetworkingNetworking

Internet, intranet, extranet technologiesInternet, intranet, extranet technologies ServicesServices

External connectivityExternal connectivity ObjectivesObjectives Protection stepsProtection steps Firewall configurationFirewall configuration

Website issuesWebsite issues Distributed executablesDistributed executables ““Cookies”Cookies” Digital signaturesDigital signatures

General ControlsGeneral ControlsPhysical SecurityPhysical Security

Physical protectionsPhysical protections PracticesPractices FacilitiesFacilities

General Controls General Controls Logical SecurityLogical Security

PoliciesPolicies PasswordsPasswords PracticesPractices

General ControlsGeneral ControlsOperationsOperations

Data center operationsData center operations SchedulingScheduling Media managementMedia management Production environmentProduction environment

General ControlsGeneral ControlsProgrammingProgramming

System programmingSystem programming Critical functionsCritical functions Controlling functionsControlling functions Make the system programmer your Make the system programmer your

friendfriend Application programmingApplication programming

Program maintenanceProgram maintenance Roles of the programmerRoles of the programmer

General ControlsGeneral ControlsContinuity PlanningContinuity Planning

Proper planningProper planning ScenariosScenarios Sufficient resources and commitmentSufficient resources and commitment Human sideHuman side Media managementMedia management

Inventory ProcessesInventory Processes Onsite and offsite verificationOnsite and offsite verification

General Controls General Controls Continuity PlanningContinuity Planning

Analysis terhadap ancaman (threats)Analysis terhadap ancaman (threats) Analysis terhadap processesAnalysis terhadap processes Mencari/mempertimbangkan alternativesMencari/mempertimbangkan alternatives Penentuan dan pengembangan rencana Penentuan dan pengembangan rencana

(plan) contigency(plan) contigency Documentation pendukung rencana Documentation pendukung rencana

(plan)(plan) Pengujian rencana (Plan)Pengujian rencana (Plan)

IT Contingency Planning IT Contingency Planning ProcessProcess

To develop and maintain an effective IT To develop and maintain an effective IT contingency plan, organizations should use contingency plan, organizations should use the following approach:  the following approach:  Develop the contingency planning policy statementDevelop the contingency planning policy statement Conduct the business impact analysis (BIA)Conduct the business impact analysis (BIA) Identify preventive controlsIdentify preventive controls Develop recovery strategiesDevelop recovery strategies Develop an IT contingency planDevelop an IT contingency plan Plan testing, training, and exercisesPlan testing, training, and exercises Plan maintenance.Plan maintenance.

Testing Contingency PlanTesting Contingency Plan

Stage 1 - Senior Staff ReviewStage 1 - Senior Staff Review The senior staff selects an internally-publicized date and time to The senior staff selects an internally-publicized date and time to

review all contingency plans. Aside from ensuring overall business review all contingency plans. Aside from ensuring overall business soundness, this review also serves to recognize people who have soundness, this review also serves to recognize people who have thoughtfully completed their assignment. Knowledge of a firm thoughtfully completed their assignment. Knowledge of a firm date for a senior staff review will increase quality, accuracy and date for a senior staff review will increase quality, accuracy and timeliness.timeliness.

Stage 2 - Interdepartmental ReviewsStage 2 - Interdepartmental Reviews Each department should review another department’s plans. The Each department should review another department’s plans. The

goal of this stage is to find bottlenecks, identify conflicts and goal of this stage is to find bottlenecks, identify conflicts and allocate resources. If possible, departments that are allocate resources. If possible, departments that are "downstream" in the business process can review the plans of "downstream" in the business process can review the plans of "upstream" departments."upstream" departments.

Testing Contingency PlanTesting Contingency Plan

Stage 3 - Failures in Critical SystemsStage 3 - Failures in Critical Systems This testing can be localized within departments. It involves This testing can be localized within departments. It involves

simulating system or vendor failures. You don't actually have to simulating system or vendor failures. You don't actually have to shut down critical equipment or processes - you can role-play a shut down critical equipment or processes - you can role-play a "what if" scenario. You can either run a "surprise" drill or plan a "what if" scenario. You can either run a "surprise" drill or plan a role-playing event for a specific time.role-playing event for a specific time.

Stage 4 - The Real DealStage 4 - The Real Deal This testing involves short-term shutdowns in key areas. If This testing involves short-term shutdowns in key areas. If

possible, these tests should be conducted in a real-time possible, these tests should be conducted in a real-time environment. The goal, of course, is to fully test the contingency environment. The goal, of course, is to fully test the contingency plan. Concentrate this last phase of testing only on areas that plan. Concentrate this last phase of testing only on areas that have a high business priority and a high risk for failure.have a high business priority and a high risk for failure.

Software Testing Software Testing TechniquesTechniques

Strategy:Strategy: A strategy for software testing integrates software test A strategy for software testing integrates software test

case design techniques into a well – planned series of case design techniques into a well – planned series of steps that result in the successful construction of software steps that result in the successful construction of software

Common Characteristics of Software Testing Common Characteristics of Software Testing StrategiesStrategies Testing begins at module level and works outward Testing begins at module level and works outward

towards the integration of the entire system. towards the integration of the entire system. Different testing techniques are appropriate at different Different testing techniques are appropriate at different

points in time. points in time. Testing is conducted by the developer of the software and Testing is conducted by the developer of the software and

for large projects by an independent test group. for large projects by an independent test group. Testing and debugging are different activities, but Testing and debugging are different activities, but

debugging must be accommodated in any testing strategy debugging must be accommodated in any testing strategy

Validation and Verification Validation and Verification (V&V)(V&V)

Validation (Product Oriented)Validation (Product Oriented) Validation is concerned with whether the Validation is concerned with whether the right right

functions of the program have been properly functions of the program have been properly implementedimplemented, and that this function will properly , and that this function will properly produce the correct output given some input produce the correct output given some input value.value.

Verification (Process Oriented)Verification (Process Oriented) Verification involves checking to see whether the Verification involves checking to see whether the

program conforms to specificationprogram conforms to specification. I.e the . I.e the right tools and methods have been employed. right tools and methods have been employed. Thus it focuses on process correctness.Thus it focuses on process correctness.

Software Quality Assurance Software Quality Assurance InvolvementInvolvement

Seven Principles Software Seven Principles Software TestingTesting

To test a program is to try to make it failTo test a program is to try to make it fail Tests are no substitute for specifications.Tests are no substitute for specifications. Regression testing: Regression testing: Any failed Any failed

execution must yield a test case, to execution must yield a test case, to remain a permanent part of the project’s remain a permanent part of the project’s test suite.test suite.

Applying oracles: Applying oracles: Determining success Determining success or failure of tests must be an automatic or failure of tests must be an automatic process.process.

Seven Principles Software Seven Principles Software TestingTesting

Manual and automatic test cases: Manual and automatic test cases: An An effective testing process must include both effective testing process must include both manually and automatically produced test cases.manually and automatically produced test cases.

Empirical assessment of testing strategies: Empirical assessment of testing strategies: Evaluate any testing strategy, however attractive Evaluate any testing strategy, however attractive in principle, through objective assessment using in principle, through objective assessment using explicit criteria in a reproducible testing process.explicit criteria in a reproducible testing process.

Assessment criteria: Assessment criteria: A testing strategy’s most A testing strategy’s most important property is the number of faults it important property is the number of faults it uncovers as a function of time.uncovers as a function of time.

Testing from low-level to Testing from low-level to high level (Testing in high level (Testing in

Stages)Stages) Except for small programs, systems should Except for small programs, systems should notnot be tested as be tested as a single unit. a single unit.

Large systemsLarge systems are built out of are built out of sub-systemssub-systems, which are , which are built out of built out of modulesmodules that are that are composed of procedures composed of procedures and functionsand functions. The . The testing testing process should therefore process should therefore proceed in stagesproceed in stages where testing is carried out where testing is carried out incrementallyincrementally in conjunction with system implementation. in conjunction with system implementation.

The most widely used testing process consists of five stages. The most widely used testing process consists of five stages.

Component Component testingtesting

Unit TestingUnit Testing

VerificationVerification(Process (Process

Oriented)Oriented)

White Box Testing White Box Testing TechniquesTechniques

(Tests that are derived (Tests that are derived from knowledge of from knowledge of

the the program’s program’s structure and structure and

implementationimplementation))

Module TestingModule Testing

Integrated testingIntegrated testing

Sub-System Sub-System TestingTesting

System TestingSystem Testing

User testingUser testing Acceptance Acceptance TestingTesting

ValidationValidation(Product (Product

Oriented)Oriented)

Black Box Testing Black Box Testing TechniquesTechniques

(Tests are derived from (Tests are derived from the the program program

specificationspecification))

The stages in the testing The stages in the testing process are as follows:process are as follows:

Unit testing:Unit testing: (Code Oriented) (Code Oriented) Individual componentsIndividual components are tested to ensure are tested to ensure

that they operate correctly. Each component is that they operate correctly. Each component is tested tested independentlyindependently, without other system , without other system components.components.

Module testing:Module testing: A module is a A module is a collection of dependent collection of dependent

componentscomponents such as an such as an object classobject class, an , an abstract data typeabstract data type or some or some looser collection looser collection of procedures and functionsof procedures and functions. A module . A module encapsulates related componentsencapsulates related components so it can so it can be tested without other system modules.be tested without other system modules.

3232

Proses TestingProses Testing

UnitTesting

ModuleTesting

Sub-systemTesting

SystemTesting

AcceptanceTesting

Component Testing Integration TestingUserTesting

The stages in the testing The stages in the testing process (cont.)process (cont.)

Sub-system testing:Sub-system testing: (Integration Testing) (Integration Testing) (Design Oriented)(Design Oriented) This phase involves testing This phase involves testing collections of modulescollections of modules, which , which

have been have been integrated into sub-systemsintegrated into sub-systems. Sub-systems may . Sub-systems may be be independently designed and implementedindependently designed and implemented. The most . The most common problems, which arise in large software systems, are common problems, which arise in large software systems, are sub-systems sub-systems interface mismatchesinterface mismatches. The sub-system test . The sub-system test process should therefore concentrate on the process should therefore concentrate on the detection of detection of interface errorsinterface errors by rigorously exercising these interfaces. by rigorously exercising these interfaces.

System testing:System testing: The sub-systems are integrated to make up the entire The sub-systems are integrated to make up the entire

system. The testing process is concerned with finding errors system. The testing process is concerned with finding errors that result from unanticipated interactions between sub-that result from unanticipated interactions between sub-systems and system components. It is also concerned with systems and system components. It is also concerned with validating that the system meets its functional and non-validating that the system meets its functional and non-functional requirements.functional requirements.

The stages in the testing The stages in the testing process (cont.)process (cont.)

Acceptance testing:Acceptance testing: This is the final stage in the testing process This is the final stage in the testing process

before the system is before the system is accepted for operational accepted for operational useuse. The system is tested with . The system is tested with data supplieddata supplied by by the the system clientsystem client rather than simulated test rather than simulated test data. Acceptance testing may reveal errors and data. Acceptance testing may reveal errors and omissions in the omissions in the systems requirements systems requirements definition( user – oriented)definition( user – oriented) because real data because real data exercises the system in different ways from the exercises the system in different ways from the test data. Acceptance testing may also reveal test data. Acceptance testing may also reveal requirement problems where the system facilities requirement problems where the system facilities do not really meet the do not really meet the users needs users needs (functional)(functional) or the or the system performance (non-system performance (non-functional)functional) is unacceptable. is unacceptable.

The stages in the testing The stages in the testing process (cont.)process (cont.)

Acceptance testing is sometimes called Acceptance testing is sometimes called alpha testingalpha testing. . Bespoke systems are developed for a single client. The Bespoke systems are developed for a single client. The alpha testing process continues until the alpha testing process continues until the system system developer and the clientdeveloper and the client agrees that the delivered agrees that the delivered system is an acceptable implementation of the system system is an acceptable implementation of the system requirements.requirements.

When a system is to be marketed as a software product, When a system is to be marketed as a software product, a testing process called beta testing is often used. a testing process called beta testing is often used.

Beta testingBeta testing involves delivering a system to a involves delivering a system to a number number of potential customersof potential customers who agree to use that system. who agree to use that system. They report They report problems to the system developersproblems to the system developers. This . This exposes the product to real use and detects errors that exposes the product to real use and detects errors that may not have been anticipated by the system builders. may not have been anticipated by the system builders. After this After this feedback, the system is modifiedfeedback, the system is modified and either and either released fur further beta testing or for general sale.released fur further beta testing or for general sale.

Testing StrategiesTesting Strategies

Top-down testing Top-down testing Where testing starts with the most Where testing starts with the most abstract abstract

componentcomponent and works and works downwards.downwards. Bottom-up testing Bottom-up testing

Where testing starts with the Where testing starts with the fundamental fundamental componentscomponents and works and works upwards.upwards.

Thread testing Thread testing Which is used for Which is used for systems with multiple processessystems with multiple processes

where the processing of a transaction threads its way where the processing of a transaction threads its way through these processes.through these processes.

Stress testing Stress testing Which relies on stressing the system by Which relies on stressing the system by going going

beyond its specified limitsbeyond its specified limits and hence testing and hence testing how how wellwell the system the system can cope with over-load can cope with over-load situationssituations..

Testing StrategiesTesting Strategies

Back-to-back testing Back-to-back testing Which is used when Which is used when versions of a systemversions of a system are available. are available.

The systems are The systems are tested togethertested together and their and their outputs outputs are compared.are compared.

Performance testing. Performance testing. This is used to test the run-time performance of software.This is used to test the run-time performance of software.

Security testing.Security testing. This attempts to verify that protection mechanisms built This attempts to verify that protection mechanisms built

into system will protect it from improper penetration.into system will protect it from improper penetration. Recovery testing.Recovery testing.

This forces software to fail in a variety ways and verifies This forces software to fail in a variety ways and verifies that recovery is properly performed.that recovery is properly performed.

Testing StrategiesTesting Strategies

Large systems are usually tested using a mixture of these Large systems are usually tested using a mixture of these strategies rather than any single approach. Different strategies rather than any single approach. Different strategies may be needed for different parts of the system strategies may be needed for different parts of the system and at different stages in the testing process. and at different stages in the testing process.

Whatever testing strategy is adopted, it is always sensible Whatever testing strategy is adopted, it is always sensible to adopt an to adopt an incremental approach to sub-system and incremental approach to sub-system and system testingsystem testing. Rather than integrate all components into . Rather than integrate all components into a system and then start testing, the system should be a system and then start testing, the system should be tested incrementallytested incrementally. Each increment should be tested . Each increment should be tested before the next increment is added to the system. This before the next increment is added to the system. This process should continue until process should continue until all modulesall modules have been have been incorporated into the system. incorporated into the system.

When a module is introduced at some stage in this process, When a module is introduced at some stage in this process, tests, which were previously unsuccessful, may now, detect tests, which were previously unsuccessful, may now, detect defects. These defects are probably due to defects. These defects are probably due to interactions interactions with the new modulewith the new module. The source of the problem is . The source of the problem is localized to some extent, thus localized to some extent, thus simplifying defect location simplifying defect location and repair.and repair.

Unit Testing Coding Focuses on each module and whether it works properly. Makes heavy use of white box testing

Integration Testing

Design Centered on making sure that each module works with another module.

Comprised of two kinds:1. Top-down and2. Bottom-up integration.Or focuses on the design and construction of

the software architecture. Makes heavy use of Black Box testing.

(Either answer is acceptable)

Validation Testing

Analysis Ensuring conformity with requirements 

Systems Testing Systems Enginee

ring

Making sure that the software product works with the external environment,e.g.,computer system,other software products.

Driver and Stubs Driver and Stubs   Driver: dummy main program Driver: dummy main program Stub: dummy sub-program Stub: dummy sub-program This is due to the fact that the This is due to the fact that the

modules are not yet stand alone modules are not yet stand alone programs therefore drive and or stubs programs therefore drive and or stubs have to be developed to test each have to be developed to test each unit.unit.

The responsibility for testing between The responsibility for testing between the Project & Software Qualitiy the Project & Software Qualitiy

Assurance (S.Q.A.)Assurance (S.Q.A.) Unit Test is the responsibility of the Unit Test is the responsibility of the

Development TeamDevelopment Team System Testing is the responsibility of SQASystem Testing is the responsibility of SQA User Acceptance Testing is the User Acceptance Testing is the

Responsibility of the User Representatives Responsibility of the User Representatives TeamTeam

Technology Compliance Testing is the Technology Compliance Testing is the responsibility of the Systems Installation & responsibility of the Systems Installation & Support Group. Support Group.

Web Application TestingWeb Application Testing

Does the User Interface promote Does the User Interface promote Usability?Usability? NavigationNavigation InteractrionInteractrion Abstract Interface Design and Abstract Interface Design and

ImplementationImplementation Does aesthetics of Web App. Appropriate Does aesthetics of Web App. Appropriate

for the app domain and pleasing to the for the app domain and pleasing to the user?user?

Web Application TestingWeb Application Testing Is the content designed in a manner that Is the content designed in a manner that

imparts the most information with the imparts the most information with the least effort.least effort.

Is navigation efficient and straightforward?Is navigation efficient and straightforward? Is Web App. Architecture provided for Is Web App. Architecture provided for

user: structure content and function, flow user: structure content and function, flow of navigation for efficient use ?of navigation for efficient use ?

Are components designed in manner to Are components designed in manner to reduce complexity and enhanced reduce complexity and enhanced correctness, reliability and performance?correctness, reliability and performance?

Security Test: SQL InjectSecurity Test: SQL Inject

Technical Metrics for Web Technical Metrics for Web AppsApps

Representative Tools:Representative Tools: Netmechanic Tools: improve web-performance Netmechanic Tools: improve web-performance

((www.netmechnic.com)) NIST Web Matrics Testbed NIST Web Matrics Testbed

(zing.ncsl.nist.gov/WebTools/)(zing.ncsl.nist.gov/WebTools/) Web Static Analyzer Tool (WebSAT)Web Static Analyzer Tool (WebSAT) Web Category Analysis Tool (Web CAT)Web Category Analysis Tool (Web CAT) Web Variable Instrument Program (WebVIP): Web Variable Instrument Program (WebVIP):

capture log user interactioncapture log user interaction Framework for Logging Usability Data (FLUD)Framework for Logging Usability Data (FLUD) VisVIP: visualization of user navigation pathsVisVIP: visualization of user navigation paths TreeDec: adds navigation aids to the web siteTreeDec: adds navigation aids to the web site

Web SecurityWeb Security

FirewallFirewall AuthenticationAuthentication EncryptionEncryption AuthorizationAuthorization

Performace TestPerformace Test Response TimeResponse Time Unacceptable rateUnacceptable rate Component responsible of degradation Component responsible of degradation

performanceperformance Impact of degradation to securityImpact of degradation to security What happens when load over the What happens when load over the

maximum capacitymaximum capacity Load TestingLoad Testing Stress TestingStress Testing

Load TestingLoad Testing

P = N x T x DP = N x T x D N: concurrent userN: concurrent user T: On Line TransactionT: On Line Transaction D: Data load processed by the Server D: Data load processed by the Server

per transactionper transaction