The Core of International Software Testing Standard, ISO/IEC29119

As one of the Delegates of KATS (Korean Agency for Technology and Standards)MKE (Ministry Of Knowledge Economy)

As one of the Delegates of KATS (Korean Agency for Technology and Standards)MKE (Ministry Of Knowledge Economy)

STA Consulting Inc. Wonil Kwonwww.softwaretesting.co.krSTA Consulting Inc. Wonil Kwonwww.softwaretesting.co.kr

Who is presenting?• Presenter : Wonil KWON (wonil@sten.or.kr)

– Representative of STA Consulting Inc. and STEN (Software Test Engineers Network)

– President of KTB (Korea Testing Board of ISTQB)

– Korean Representative of ISO/IEC 29119, Software Engineering: Software Testing (President of Korean Local Working Group)

– Board member of SW quality certification by MKE (Ministry of Knowledge Economy)

– Experience at ETRI(Electronics and Telecommunications Research Institute), TTA

– Consulting and Training – Hyundai Mobis, Prudential Life Insurance, LG Electronics, LSIS, Samsung Electronics, SUN Microsystems, IBM, Samsung SDS, LG Hitachi, Hanwha S&C, Posdata, Humax, AhnLab, Kookmin Bank, Industrial Bank of Korea, etc.

– Books

• Practical Software Testing Foundation 2nd edition

• Software Testing Terminology

• Learn Software Testing Through Questions

• Testing Embedded Software (Translated)

www.softwaretesting.co.kr

Contents

3

How about this?

Do you see the value of the international software testing standard?

Do you see the value of the international software testing standard?

What contents do you expect of the international

SW testing standard?

What contents do you expect of the international

SW testing standard?

Do you think the int’l standard world got to recognize the importance of SW testing?

Do you think it is worthwhile SW testing getting attention

from the int’l standard?

Do you think the int’l standard world got to recognize the importance of SW testing?

Do you think it is worthwhile SW testing getting attention

from the int’l standard? In what conditions are you going to use

the standards?

In what conditions are you going to use

the standards?

Aren’t you confused by multiple testing concepts

and terminologies with the similar meanings, vice

versa?

Aren’t you confused by multiple testing concepts

and terminologies with the similar meanings, vice

versa?

Need Standard?

Standard

6

Standard

7

Quality and Standards

Types of Standard

OrganizationalStandards

OrganizationalStandards

MarketStandards

MarketStandards

ProfessionalStandards

ProfessionalStandards

IndustryStandardsIndustry

Standards

NationalStandardsNational

Standards

InternationalStandards

InternationalStandards

ISO/IEC JTC1 SC7

10

SC7 Working Groups

Some of the WG26 Members• Dr. Stuart Reid (UK) - convener• Ms. Ursula Parker, representing IEEE (USA)• Ms. Tafline Murnane (Australia)• Mr. Wonil KWON (Korea)• Dr. Alastair Walker (South Africa)• plus....

11

Process Context

ISO/IEC 29119 - Scope

13

ISO/IEC 29119 - OverviewISO/IEC 29119

Test management

process

Test management

process

Test

str

ateg

yTe

st s

trat

egy

Test

pro

cess

mon

itorin

gTe

st p

roce

ss m

onito

ring

Test

pro

ject

com

plet

ion

Test

pro

ject

com

plet

ion

Test processTest process

Test

pla

nnin

gTe

st p

lann

ing

Test

des

ign

Test

des

ign

Test

exe

cutio

nTe

st e

xecu

tion

Bug

repo

rtin

gBu

g re

port

ing

Test

com

plet

ion

Test

com

plet

ion

Status reporting

Status reporting

Proj

ect

stat

us r

epor

ting

Proj

ect

stat

us r

epor

ting

Test

sta

tus

repo

rtin

gTe

st s

tatu

s re

port

ing

Test environment

support

Test environment

support

Unit testingUnit testingUnit testingUnit testingExamples - Unit testingExamples - Unit testing

TM Process Documentation Test Documentation

Interim status report

Test environment

report

!!!!TM : Test Management

IEEE 1008, ISO/IEC 12207 aligned with software development lifecycle

IEEE 829

BS 7925-2

BS 7925-1 (IEEE 829)

Organizational Test Process

Organizational Test Process

Organizational Test Policy and Strategy

Documentation

Test

str

ateg

yTe

st s

trat

egy

Test

Pol

icy

Test

Pol

icy

Part 1: Concepts & Vocabulary

15

Part 2: Testing ProcessISO/IEC 29119 - 4 Layered Model ISO/IEC 29119 - 4 Layered Model

16

Organizational Test Processes

Organizational Test Strategy Process

Create/Update Test Strategy

Test Policy

IT ProjectManagement Policy

Test Strategy

Organizational Test Policy Process

Create/Update Test PolicyIT Policy

Management PolicyTest Policy

Test Policy Test Policy Updates

Project Test Management Process

Test StrategyInterim ProjectTest Status Reports

Project TestCompletion Report

17

Project Test Management Process

Comments onOrganizationalTest Strategy usage

ProjectTestCompletionReport

Create/Update Project Test

Plan(TM1)

Monitor and Control(TM2)

ReportTest Project Completion

(TM3)

ProjectTest Plan

ProjectTestStatusReports

Project Test Plan Updates

Project Test Management Process

TestStrategy

Organizational Test Strategy Process

Project Management Plan,Product Documentation,Software Development Plan, Regulatory Standards

Level Test Plan,Level Test Status Reports

Project Test Plan

Level Test CompletionReport

Control Directives

Test Level Process

18

Test Level Process

ProjectTestPlan

Test Level

Status Report

Level Test CompletionReports

Control Directives

Project Test Management Process

ReportAnomalies

(TL6)

ReportTest Level Completion

(TL8)

Level Test Plan

TestSpecification

TestResults

[Issuenoticed]

[No issues noticed]

Anomaly Report

Test Level Process

Set-up & Maintain TestEnvironment

(TL4)

Plan Testing (TL1)

RunTests(TL5)

Test Environment Readiness Report

Specific TestEnv’t Requirements

Level Test CompletionReport

Design & BuildTests(TL3)

Test Execution Cycle

Close Down Testing(TL7)

Control Directives

Monitor and Control(TL2)

Test Process Info

Asset Info

Measures

Level Test Plan Updates

Project Management Plan,Product Documentation,Software Development Plan, Regulatory Standards

19

Test Activity Descriptions Example – Design & build tests (TL3)

• Each testing activity is described using a standard template.(following ISO 24774), for example:

PurposeThe purpose of Design and Build Tests is to design and document test cases that will be executed during testing.

Entry Criteria•the Test Plan has been agreed•the Test Item Specification(s) is available

Inputs•Test Plan•Test Item Specification(s) •System Requirements

Roles•Test Designer

Tasks1.Analyse the test item specification(s) to identify specification items;2.Determine the test conditions by identifying which specification items to test using the selection criteria from the Test Plan;3.Derive test coverage items to be exercised by applying the test case design technique(s) to the test conditions;4.Derive test cases by selecting input values to exercise the test coverage items and corresponding expected results;5.Create a test set by rationalising the test cases (i.e. removing redundancy, duplication, etc.);6.Derive a test script by ordering the test cases within the test set using their pre- and post-conditions;7.Identify and document any specific test environment requirements that are not already included in the Test Plan.

Outputs •Test Specification•Specific Test Environment Requirements

Outcomes •the Test Specification has been documented.

Exit Criteria•the Test Specification has been agreed. 20

Test levels

21

Part 3: Documentation

22

Part 4: Techniques

23

Conclusions

24

Conclusions

25

Some examples of ISO/IEC 29119 usage and spread-ing in Korea

*Testing Treasure Map info : sites.google.com/site/swtestingmap

• Testing knowledge map

• SSTC conference• Local seminar &

presentation (on-site incl.)

• Industry project• Gov. supported project !

Testing process guideline

• Training course• Online training course

• I want you to make absolute full use of it through the contribution to this lovely international testing standard

• I want you to make absolute full use of it through the contribution to this lovely international testing standard

• We all want this standard to turn out to be reflecting the “Best Practices” (for us to enjoy the value of standard)

• We all want this standard to turn out to be reflecting the “Best Practices” (for us to enjoy the value of standard)

Conclusion

IEEE 1008BS 7925-2

26

IEEE 829BS 7925-2

Process Assessment

Part 5

ISO 15504TMMi, TPI

Reference:

A Vision for International Standardization in Software and Systems Engineering

François Coallierfrancois.coallier@etsmtl.ca

Sixth International Conference on the Quality of Information and

Communications TechnologyIEEE 2007

The New International Software Testing Standard

Stuart Reidsreid@testing-solutions.com

27

28

• Current status of standard. See website

• Current status of standard. See website

• Need more testing experts involvement

• Need more national bodies participation

• Need more testing experts involvement

• Need more national bodies participation

www.softwaretestingstandard.org