. 44
( 67 .)


Functional Alignment
To address the question of whether IT is doing things right, the Performance
Measurement framework uses a multidimensional approach4 that includes ele-
ments of cost performance, service level, and quality and process measurements.

These three dimensions are interconnected, and IT management needs to rec-
ognize and be aware of these performance relationships. There is an intuitive
understanding of the relationship between cost and quality or service level. Less
intuitive is the fact that the specific relationship between cost and quality is
determined by the quality of the processes involved (i.e., the marginal cost of
an improvement in project delivery is determined by the quality of our project
management processes). See Exhibit 10.4.

EXHIBIT 10.4 Performance Dimensions
Cost Performance

Process Development Quality and Service Level

Cost Performance
Cost performance and cost metrics will always be part of the IT Performance
Measurement suite of measures. Organizations are continually looking to save
IT-related costs wherever possible, but the methods for doing so are not always
obvious and cost analysis is not always straightforward. For example, although
the value of e-mail systems is indisputable, the cost of operating them can be
the subject of much controversy. Ascertaining costs is a crucial aspect of any
cost-saving exercise: A granular and accurate breakdown of system expenses is
mandatory for identifying potential savings. Organizations pursuing a managed
approach to investment in e-mail systems require an in-depth understanding
of mail system costs to formulate a strategy for reducing overall mail system
Within the context of IT as a cost center and with the primary goal of IT
being to manage and control costs, then cost measures are appropriate indica-
tors of success. When the context changes and IT is expected to add value, then
Frameworks and Process Overview

new measures are needed to support the new criteria for success. Indeed, within
the new added-value context, managing with cost performance as the guiding
star could land the organization on rocky shores. Additional performance meas-
urement dimensions are needed.

Quality and Service Levels
As the performance context for IT shifts toward added-value services, the meas-
urement framework must give more emphasis to customer-focused metrics such
as service level and quality. See Exhibit 10.5.

EXHIBIT 10.5 Elements of Service Level and Quality
Elements of Service Level Elements of Quality
Application • Availability of the development • Reliability of the development
Development resource process
Portfolio • Responsiveness of the • Reliability of the development
development resource result
Applications • Availability of the application • Functionality of the application
Portfolio • Responsiveness of the • Accuracy of the application and
application its data
Infrastructure • Availability of the infrastructure • Functionality of the infrastructure
Portfolio element element
• Responsiveness of the • Reliability of the infrastructure
infrastructure element element
Services • Availability of the service • Reliability of the service process
Portfolio resource • Reliability of the service result
• Responsiveness of the service

Process Measurement
Although cost and service level data are useful, they are lagging indicators that
communicate results but are not useful as levers to manage or implement change.
Process measurement (and management) is a way to affect the IT organization™s
performance drivers. Measuring and managing processes is about measuring and
managing the causes behind cost and service-level results.
The NIE Performance Measurement practice recommends a combination of
operational and management processes for assessment. Example processes are
shown in Exhibit 10.6. These processes will not be equally important, as their
impact will depend on the specific needs of the IT organization within its spe-
cific business context. The framework for assessing and managing processes,
including assessment categories, is illustrated in Exhibit 10.7.

EXHIBIT 10.6 Example of Available IT Process Models

Software Development (SEI) IT Management (COBIT)
Requirements Management Define a Strategic IT Plan
Software Project Planning Define the Information Architecture
Software Project Tracking and Oversight Define the IT Organization and Relationships
Software Quality Assurance Manage the IT Investment
Software Configuration Management Manage Human Resources
Organization Process Definition
System Engineering (SEI)
Integrated Software Management
Derive and Allocate Requirements
Software Product Engineering
Integrate System
Verify and Validate System
Ensure Quality
Monitor and Control Technical Effort

EXHIBIT 10.7 Process Measurement Example

0 1 2 4 5
Process Maturity

Critical Success Enabled by a documented methodology that is supported
Factors by validated data and a structured transparent decision-
making process

√ Provides for a prioritization scheme for the business
objectives and quantification of business requirements

Ο Achieves buy-in from IT senior management team
Key Goal Percent of strategic plans that are translated into
Indicators long- and short-range plans leading to individual
Key Performance .5 Yrs Age of Strategic Plan
.39 Plan Quality Index, including timelines of development
effort, adherence to structured approach, and
completeness of plan

Managing IT Strategy Implementation
It is critical that IT be able to implement its own strategic objectives. The fun-
damental framework for this is to track each strategic objective as a project,
which means naming an owner for that objective, identifying related initiatives,
assigning resources, and establishing milestones (see Exhibit 10.8). An additional
component of the Strategy Implementation framework is the identification of
linkages to IT portfolios from which specific measures can be obtained to assess
the impact of the strategic objective as it is implemented. The purpose of any

EXHIBIT 10.8 Tracking IT Strategy Implementation

Strategic Objective: Planning and Budgeting Process Enhancement
John Barnes, IT/CFO
Brad Snyder, IT Budget Director
Project Management Office (PMO) implementation
Related Initiatives Portfolio Management tool implementation
Common IT Metrics initiative
IT Budget staff
Resources Budget staff in each business unit
Step 1 Template development
Milestones and
Step 2 FY2003 Baseline
Step 3 FY2004 Budget
Management Portfolio”Connect budgets to business objectives
Linkages to Process and Project Portfolio”Address project resource allocation
Resource Portfolios Lights-On Portfolio”Address alignment issues and the retirement of
low-value systems

strategic objective is to improve the performance of IT services and capabilities
as reflected in the portfolios.

IT, like most organizations, needs to pay attention to short-term, tactical per-
formance as well as to long-term, strategic performance. Correspondingly, there
are measures needed for the identification and diagnosis of operational issues,
and measures necessary to maintain focus on strategic goals and assess progress
toward their achievement.
The measures described within the Performance Measurement practice pro-
vide a broad spectrum of metrics that can be used for diagnostic as well as strate-
gic purposes. The metrics included in this scorecard are identified through a
process that links strategic objectives to specific resource and process drivers and
to specific metrics associated with those drivers. Most, if not all, of an IT Per-
formance Scorecard™s measures will come from the Performance Measurement
framework as presented in this book.
Exhibit 10.9 shows the overall results of portfolio assessment in Performance
Measurement. Each application in the Application Portfolio can be assessed, first
for alignment, then service level/quality, then for current and future resources.
Taken together, this begins to comprise a complete picture of the Application
Portfolio. Management can assess the components of the Portfolio and make
reasonable decisions about retirement, future investment, or operational con-
cerns about services and quality. For example, in the exhibit, Financial Infor-
mation is the lowest from the Alignment perspective. Does that mean that the

EXHIBIT 10.9 Integrating Performance Measures with IT Portfolios

Service Level



Customer Care Systems 2.9 1.9 2.2 $7M -
Manufacturing 1.1 1.2 1.2 $30M -
Sales Force Automation 2.4 2.1 2.5 $2M ++
Manufacturing 2.5 1.1 2.1 $46M +
Product Planning 1.5 1.2 1.6 $3M +
E-Business 2.2 1.6 1.8 $9M -
Financial Information .8 2.2 2.3 $3M -
Data Warehouse 1.1 2.1 1.1 $1M +

Alignment Efficiency
Connection to Strategy Budget Conformance

Service Level Quality
Availability Functionality
Responsiveness Accuracy

financial system needs a refresh? Does it signal a need to consider alternatives?
Similarly, the Manufacturing application ranks lowest in service and quality. The
same questions apply, particularly since that application area is consuming the
largest amount of resources. Again, this means that management can make
informed decisions.

Performance Measurement most directly supports the actions tied to strategy
and resource management principles of Right Decisions/Right Results, but has
an indirect impact on the others as well.

Actionable Strategic Intentions. Performance Measurement starts with the
company™s strategic intentions and maps key IT activities (and related meas-
ures) to them. Through Performance Measurement, IT™s actions can be
focused on the connection to strategic intentions. This focus will improve
how IT contributes to the achievement of the strategic intentions.
Actions Tied to Strategy. Performance Measurement closes the loop from
strategy to action. When IT measures the right things and ties its actions to


. 44
( 67 .)