Project Metrics - PMI-NH

Copyright Copyright ©© ShareDynamicsShareDynamics, Inc., Inc. 11

Presented toPresented to

PMI NH ChapterPMI NH Chapter

December 5December 5thth, 2007, 2007

byby

Joseph Raynus Joseph Raynus

Project MetricsProject Metrics


The Role of Metrics in PMOThe Role of Metrics in PMO

There are 5 common PMO function groups:There are 5 common PMO function groups:

�� Group 1: Monitoring and controlling project Group 1: Monitoring and controlling project

performanceperformance

�� Group 2: Development of PM competencies Group 2: Development of PM competencies

and methodologiesand methodologies

�� Group 3: MultiGroup 3: Multi--Project ManagementProject Management

�� Group 4: Strategic ManagementGroup 4: Strategic Management

�� Group 5: Organizational LearningGroup 5: Organizational Learning

“The Multi-Project PMO” Dr. Brian Hobbs (A white paper for PMI) 2007


Group 1: Monitoring and Controlling Group 1: Monitoring and Controlling

Project PerformanceProject Performance

–– Reporting project status to upper Reporting project status to upper

managementmanagement

–– Monitor and control project performanceMonitor and control project performance

–– Implement and operate project information Implement and operate project information

systemsystem

–– Develop and maintain project scorecardDevelop and maintain project scorecard

“The Multi-Project PMO” Dr. Brian Hobbs (A white paper for PMI) 2007


The ParadigmThe Paradigm

Paradigm: A set of assumptions, concepts, values, and practices that constitutes a way of viewing reality for the community that shares them, especially in an intellectual discipline.

Our experience shapes our perception


Measure is Measure is ““To KnowTo Know”” guess is guess is ““to Missto Miss””

Measure

AnalyzePerformance

Manage and Control


Measurement PhilosophyMeasurement Philosophy

�� What is sufficient?What is sufficient?

�� Who owns the Who owns the

measure?measure?

�� What are the risks?What are the risks?

�� What is strategic What is strategic

value?value?

�� Which Executive?Which Executive?

�� Which Sponsor?Which Sponsor?


Personal ScorecardPersonal Scorecard

Values I’m admired, independent, and wealthy

Mission Enjoy my life as much as possible

Vision I’m healthy and good-looking by 1/2008

Strategies 1. Exercise 2. Diet

Target Weight lose of 20 lbs by 7/2007

Action Jog 5 miles/ wk No ice cream, Plan Gym 2/wk chocolate, or beer

Measuring Heart rate, body fat, Body mass index, recovery favorite jeans

Rewards New running gear New clothes and iPod


Balanced MeasurementsBalanced Measurements


How to MeasureHow to Measure


Customers

Internal

Learning

& Growth

Fewer Planes

More Passengers

Increase Profitability

Focus on Process Redesign

Fast Ground Turnaround

On-TimeFlights

Lowest Cost

Will our people do that?

–Educate and compensate personnel for cycle time

reduction

What must the internal focus be?

– Fast turnaround

How will we do that?

–Attract targeted customer segments who value price and

on time arrivals

What will drive operating

efficiency?

–More customers on fewer planes

Financial

AnyAirline –Operating Efficiency

© PerformanceMeasurement.ca©pm2.ca


Profitability

Closed contracts Product cost Revenue

Customer

Loyalty

On time

Delivery

Customer

satisfaction Market share

Resource

Management Project

management

Training

Process

implementation

Productivity

SDLC

Stra te gi c

Ope ra ti ona l

Strategic Operational AlignmentStrategic Operational Alignment


What do we Know?What do we Know?

To understand what we To understand what we ““knowknow””, we need , we need

to ask questions:to ask questions:

��Do we help our employees develop the Do we help our employees develop the

right skill?right skill?

��Do we hire the right people?Do we hire the right people?

��Do we really know what customer Do we really know what customer

wants?wants?


Transition Strategy to OperationsTransition Strategy to Operations

Vision

Mission

Strategy

Operations

MeasureDown

ReportUp


Why to MeasureWhy to Measure

�� To minimize project riskTo minimize project risk

�� To make decisionsTo make decisions

�� Establish performance goalsEstablish performance goals

�� Allocate and prioritize resourcesAllocate and prioritize resources

�� Inform management about needed change Inform management about needed change

�� Share results of performance in pursuing Share results of performance in pursuing those goalsthose goals


““How is your project going?How is your project going?””


Project Project ““HealthHealth”” QuestionsQuestions

�� Are we on schedule? Are we on schedule?

�� How much $$$ did we How much $$$ did we spend so far? spend so far?

�� Do we have needed Do we have needed skills? skills?

�� Any scope Changes? Any scope Changes?

�� Are we keeping track of Are we keeping track of quality problems? quality problems?

TIMETIME

COSTCOST

RESOURCESRESOURCES

SCOPESCOPE

QUALITYQUALITY


What is a Key Performance What is a Key Performance

Indicator (KPI)?Indicator (KPI)?

�� KPIKPI’’s are highs are high--level snapshots of a business or level snapshots of a business or

project project ““healthhealth”” based on specific predefined based on specific predefined

measures measures �� KPIKPI’’s help organizations define and measure s help organizations define and measure

progress towards original goals progress towards original goals

�� KPI represents the rate of change or deviation KPI represents the rate of change or deviation

from these original goalsfrom these original goals

�� KPI is an indicator of a trend measuring planned KPI is an indicator of a trend measuring planned

versus actual process performance over timeversus actual process performance over time


PMBOKPMBOK

Project Management

KPI

(Monitor & Control project work)KPI

(Resources Control)

KPI

(Scope Control)KPI

(Performance reporting)

KPI

(Schedule Control)KPI

(Risk Management)

KPI

(Cost Control)KPI

(Vendor management)

KPI

(Quality Management & Control)


Effective KPIEffective KPI’’ss

–– Goal orientedGoal oriented

–– Applied to projects, processes, and resourcesApplied to projects, processes, and resources

–– Interpreted based on understanding of the Interpreted based on understanding of the

organizational context, environment, and goalsorganizational context, environment, and goals

–– Monitor and control changesMonitor and control changes

Rule of Thumb: Develop your ownKPI to answer your own questions


Key Project Performance GoalsKey Project Performance Goals

�� Meet the scheduleMeet the schedule

�� Stay within budgetStay within budget

�� Improve productivityImprove productivity

�� Control scope Control scope

�� Reduce customer complaintsReduce customer complaints


Conflicts! Again..?Conflicts! Again..?

Copyright © Fraunhofer CESE & Fraunhofer IESE 2007


R&DPerspective Objective Measure

Customer Deliver Quality

product on time

Development

progress100%

Target

Perspective Objective Measure

Customer Increase

Customer

Satisfaction

Reduce

customer

complaints

Customer support

90%

Target

CompanyPerspective Objective Measure

Customer Develop new

markets

Growth rate in

product sale

30%

Target

Goals Often ConflictGoals Often Conflict


What do I What do I NeedNeed to Know?to Know?

The Question is not:

What metrics should I use?

Rather:

•What do I want to know?•Why are we collecting the data?• How do we use the data?


The Goal Question Metric Methodology The Goal Question Metric Methodology

(GQM)(GQM)

�� GQM is a goal oriented approach that helps GQM is a goal oriented approach that helps to define why and what to measureto define why and what to measure

�� GQM is a topGQM is a top--down approach for the down approach for the selection, development, and tailoring of selection, development, and tailoring of measuresmeasures

�� This paradigm ensures the purpose for This paradigm ensures the purpose for collecting each measure collecting each measure

�� Provides information for decision making Provides information for decision making and actionsand actions


GQM StructureGQM Structure

Copyright © Fraunhofer CESE & Fraunhofer IESE 2007


Hierarchy of Measurement Terms Hierarchy of Measurement Terms

(Customer Requests)(Customer Requests)


GoalGoal-- Metric ConnectionMetric Connection


GQM ExampleGQM Example

� Cost per trained employee

� Cost per untrained

� Process cost per trained

� Cost of materials used

What is the total process cost per trained versus untrained employee by category?

� Number of employees by

category

� Number of employees that

require training

� % of trained employees

What are employee process training needs by category?

Process activities by category

What activities compose the process?

Improve employee productivity

MetricQuestionGoal


Requirements Management ExampleRequirements Management Example

Number of requirementsrejectedNumber of changesapproved

Is the number of changes manageable?

Type of change Reason for changeProject phase in whichthe change was requested

Why are the requirements changed?

# of initial requirements# of final requirements# of changes perrequirement

What is the level of requirement volatility?

Keep product requirements under control

MetricQuestionGoal


What is the strategy and

critical success factors?

Organizational Learning

Strategic

Objective

Number of employees to train

How many % have a job 3 months after the training?

Measure

How do we track

strategy execution?

100

60%

Target

What is our Target?

Focus training content for manufacturing

Action Plans

Marketing campaign for trainings

What are the key

action plans to achieve the target?

Define MeasuresDefine Measures


Personnel

Financial Performance

Resources andCost

Staff Experience

Effort

Earned Value

Cost

Resources and Cost Indicators


Project Measures Used to:Project Measures Used to:

�� Monitor and control project performanceMonitor and control project performance

�� Achieve alignment of organizational goals Achieve alignment of organizational goals and objectives with project objectivesand objectives with project objectives

�� Drive process improvementsDrive process improvements

�� Maximize the effectiveness of project effortMaximize the effectiveness of project effort

�� Improve crossImprove cross--functional collaborationfunctional collaboration


Productivity

Completion

EVM

PeopleChange

Risk

Quality

Project Management BoardProject Management Board

Progress


Project ProgressProject Progress

�� Cost Cost

�� Earned ValueEarned Value

�� ProductivityProductivity

�� Task CompletionTask Completion


Cost MetricCost Metric

-20

0

20

40

60

80

100

J F M A

BCWS

ACWP

CV

�� Provides insight for project Provides insight for project managers into the actual managers into the actual costcost

�� Compares actual versus Compares actual versus plannedplanned

�� Metric Relation:Metric Relation:

–– Requirements StabilityRequirements Stability

–– Development ProgressDevelopment Progress

–– Fault ProfileFault Profile

–– Schedule PerformanceSchedule Performance

–– Cost PerformanceCost Performance


Earned ValueEarned Value

�� Cost Performance Index = BCWP/ACWPCost Performance Index = BCWP/ACWP

(Historical measure of average productivity over (Historical measure of average productivity over

the life of the project)the life of the project)

�� Schedule Performance Index = BCWP/BCWSSchedule Performance Index = BCWP/BCWS


Task CompletionTask Completion

Completion of tasks during the currentCompletion of tasks during the current

reporting period:reporting period:

–– Total DueTotal Due

–– Completed on TimeCompleted on Time

–– Completed LateCompleted Late

–– Total Overdue = Total Due Total Overdue = Total Due –– (Completed on (Completed on

Time + Completed late)Time + Completed late)


Scope ChangeScope Change

0

10

20

30

40

50

60

70

80

90

Jan March

Added

Mod.

Deleted

�� Requirements Volatility MetricRequirements Volatility Metricrepresents the level of change represents the level of change in the approved requirements in the approved requirements baselinebaseline

�� Requirements Stability MetricRequirements Stability Metricallows project manager to allows project manager to determine the cause and determine the cause and source of requirements source of requirements changeschanges

�� Metric RelationMetric Relation–– Requirements TraceabilityRequirements Traceability–– Cost/ScheduleCost/Schedule–– Design StabilityDesign Stability–– Breadth of TestingBreadth of Testing

Requirements Volatility Metric (RVM)


PersonnelPersonnel

�� Turnover per MonthTurnover per Month–– Measure for risk assessmentMeasure for risk assessment

�� Overtime per MonthOvertime per Month–– Overtime Hr../Base Hrs.Overtime Hr../Base Hrs.

–– Target ~ 10%Target ~ 10%

–– More than 20% More than 20% -- adds to riskadds to risk


Risk MeasuresRisk Measures

The Risk ExposureThe Risk Exposure�� Shows each risk plotted by its Shows each risk plotted by its

cost consequence and probabilitycost consequence and probability

Risk ReserveRisk Reserve�� Shows total risk exposure for cost and schedule Shows total risk exposure for cost and schedule

compared with the current cost and time risk compared with the current cost and time risk

reservesreserves


Quality MeasuresQuality Measures

0

20

40

60

80

100

Jan

March

Open

Closed

�� Fault Profile MetricFault Profile Metricprovides a summary of provides a summary of

problems & change data problems & change data

�� Fault Profile MetricFault Profile Metricprovides insight into the provides insight into the

number and type of number and type of

problems and the ability to problems and the ability to

fix known faultsfix known faults

�� Metric RelationMetric Relation–– Cost/ScheduleCost/Schedule

–– Requirements TraceabilityRequirements Traceability


–– Breadth of TestingBreadth of Testing

–– Depth of TestingDepth of Testing

Fault Profile Metric


Schedule Metric (Phase Duration)Schedule Metric (Phase Duration)

�� Reports actual project Reports actual project progress in relation to the progress in relation to the original scheduleoriginal schedule

�� Measures the degree of Measures the degree of completeness of completeness of development effort and development effort and readiness to proceed to readiness to proceed to the next stagethe next stage

�� Metric RelationMetric Relation


–– Development ProgressDevelopment Progress

–– Fault ProfileFault Profile

–– Cost PerformanceCost Performance

0

10

20

30

40

50

60

70

80

Phase1 Phase2

Planned

Actual


KPI as a Decision ToolKPI as a Decision Tool

0

10

20

30

40

50

60

70

80

Req Coding

Planned

Actual

-20

0

20

40

60

80

100

J F M A

BCWS

ACWP

CV

Issue

Decision Feedback

DecisionKPI

Volatility

Volatility

Volatility

Data

Data

Metrics

Measurements


Types of DashboardsTypes of Dashboards

StrategicStrategic�� Monitors the Monitors the

executionexecution

of strategic goalsof strategic goals�� Uses cascading scoreUses cascading score

cardscards�� EmphasizesEmphasizes

collaboration collaboration

TacticalTactical�� Tracks processes Tracks processes

and projects and projects

�� Emphasizes analysisEmphasizes analysis

OperationalOperational�� Tracks missionTracks mission

critical processescritical processes

�� EmphasizesEmphasizes

monitoringmonitoring


Why to use DashboardsWhy to use Dashboards

� For presentations or on-screen access by your team

� Keep your stakeholders informed without meetings

� Streamline your meetings by ONLY focusing on those areas

needing attention

� Communicate to a wide range of stakeholders with confidence

that everyone will understand

�� Align your organization with key performance indicators by usingAlign your organization with key performance indicators by using

a hierarchy of dashboards that measure an objective at all levela hierarchy of dashboards that measure an objective at all levels s

of the organization of the organization

�� Analyze the key success metrics through time phased analysis, Analyze the key success metrics through time phased analysis,

drill down components, and root cause graphics drill down components, and root cause graphics


SummarySummary

�� Establish clear strategy, goals and targetsEstablish clear strategy, goals and targets

�� Establish relationship between internal Establish relationship between internal goals and environmental inputs goals and environmental inputs

–– influence of outside environmentinfluence of outside environment

�� Set quantitative measurement goals to Set quantitative measurement goals to compare with actual resultscompare with actual results

�� Construct KPIConstruct KPI’’s to achieve and maintain s to achieve and maintain stability in dynamic internal and external stability in dynamic internal and external environmentenvironment


Questions???Questions???


About the SpeakerAbout the Speaker

Joseph Joseph RaynusRaynus, Founder and Principal Consultant at , Founder and Principal Consultant at ShareDynamicsShareDynamics, Inc. has over 25 years of providing , Inc. has over 25 years of providing vision and strategic direction in the areas of vision and strategic direction in the areas of Corporate Quality, Information Technology, and Corporate Quality, Information Technology, and Software Lifecycle Management to his clients that Software Lifecycle Management to his clients that include Sun Microsystems, Oracle, US Air Force, include Sun Microsystems, Oracle, US Air Force, and Us Marine Corp among others.and Us Marine Corp among others.

Joseph is a certified SEI CMM assessor. He also Joseph is a certified SEI CMM assessor. He also participated as a nonparticipated as a non--Government advisor to the Government advisor to the Source Selection Evaluation Board (SSEB) during a Source Selection Evaluation Board (SSEB) during a major US Air Force acquisition.major US Air Force acquisition.


Publications:Publications:

““Software Process Improvement with CMMSoftware Process Improvement with CMM””. .

ArtechArtech House, 1999. The book is sold by all House, 1999. The book is sold by all

major book sellers and translated into major book sellers and translated into

Japanese and Chinese languages Japanese and Chinese languages

Joseph is currently working on a new book Joseph is currently working on a new book

titled titled ““Quantitative Business Performance Quantitative Business Performance

Management: Challenge, Change, and DashboardManagement: Challenge, Change, and Dashboard””, ,

Taylor & Francis Publishers. Due out Sept. 1, Taylor & Francis Publishers. Due out Sept. 1, 2008.2008.


Contact InformationContact Information

ShareDynamics, Inc.ShareDynamics, Inc.

15 Pelham Rd.15 Pelham Rd.

Lexington, MA 02421Lexington, MA 02421

781781--863863--96339633

[email protected]@sharedynamics.com

Project Metrics - PMI-NH

Documents

Transcript of Project Metrics - PMI-NH