CIO Executive Board

IT Performance Diagnostic

Key Attributes of the World-Class Information Technology Organization

A Competency Diagnostic Tool

Detailed competency map of the Information Technology organization Targeted diagnostic exercise for evaluating current approaches against leading practices Structured competency-to-practice map enabling quick identication of most relevant best practices

Introduction to the Competency Diagnostic

Purpose
Key Attributes of the World-Class Information Technology Organization serves a threefold purpose by: Presenting a detailed competency map of the Information Technology organization; this diagnostic provides members with a clear understanding of the critical attributes of leading Information Technology teams Providing a guided diagnostic exercise that enables IT executives to evaluate their organizations current approaches against a standard and to prioritize areas for focused improvement based on potential impact on company performance Linking results to the CIO Executive Boards extensive database of IT strategy-related practices, research briefs, and benchmarking data, allowing members to quickly identify appropriate best practices and accelerate their improvement efforts

Key Attributes of the World-Class IT Organization: A Competency Diagnostic

Attributes of Interest to the CEO/CFO
1

Strategy and Planning We chart ITs strategic direction based on the businesss articulated strategies.

Enterprise Architecture We maintain an architecture blueprint that clearly links technology choices to business capabilities.
Performance (Circle below):

Business Case Discipline We employ a standard business case template for all IT investments that captures project life-cycle costs, benets, and risks.
Performance (Circle below):

Portfolio Management We help the business prioritize projects using a dened set of objective, weighted criteria.

Value Demonstration We track metrics that demonstrate ITs contribution to the achievement of desired business outcomes.
Performance (Circle below):

Performance Reporting We provide regular reports tailored by channel and content for specic business and IT audiences.
Performance (Circle below):

Risk Management We have created a principled framework for assessing relative risk and sequence risk mitigation investments accordingly.
Performance (Circle below):

Security Policies and Standards We have dened and communicated a set of enterprise information security policies and standards
Performance (Circle below):

Disaster Recovery and Business Continuity Planning We develop and regularly test enterprise plans to ensure continuous support of core business processes.
Performance (Circle below):

10

Availability Management

We ensure the availability of systems based on SLAs that consider business criticality.

Performance (Circle below):

Performance (Circle below):

Performance (Circle below):

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Capabilities Alignment Diagnostic

Embedded Architectural Discipline

Project ROI Contracts

Enterprise-Level Portfolio Prioritization

Executive Satisfaction Survey

Balanced Scorecard Life-Cycle Management

Enterprise Risk Assessment

Employee Awareness Campaign

Enterprise-Level Business Continuity Planning

User-Centric Operational Indicators

11

Life-Cycle Cost Efciency

25

Process Digitization

We employ a standard methodology to identify opportunities for business process automation and enhancement.

I. IT Governance

II. IT Performance Measurement and Value Demonstration

How to Use This Diagnostic

III. Security and Business Continuity Planning

We effectively manage assets across their life cycle to maximize efciencies.

Performance (Circle below):

1 1

2 2

3 3

4 4

5 5

Importance (Circle below):

Competency map of the Information Technology organization

Performance (Circle below):

1 1

2 2

3 3

4 4

5 5

Importance (Circle below):

Please select the value (1 to 5) that best describes your IT organization using the two grading scales provided. The top scale measures performance in each competency and the bottom scale measures the importance of each competency.
9

IV. Infrastructure Delivery and Management

End-to-End Asset Management

12

Cost Transparency

Top-Down Business Process Optimization

Competency

VIII. Business Enablement

24

Security Policies and Standards We have dened and communicated a set of enterprise information security policies and standards
Performance (Circle below):

We use unit-level visibility into infrastructure cost drivers to provide internal customers with the ability to effectively manage their demand.
Performance (Circle below):

Performance Importance

1 1

2 2

3 3

4 4

5 5

Importance (Circle below):

1 1

2 2

3 3

4 4

5 5

Technology Innovation

Grading Scale
Performance 5 = We are excellent at this 4 = We are good at this 3 = We are average at this 2 = We are poor at this 1 = We are terrible at this Importance 5 = Critical 4 = High Priority 3 = Moderate Priority 2 = Low Priority 1 = Not a Priority

Importance (Circle below):

We proactively scan for and exploit opportunities to deploy new technologies in support of the business.
Performance (Circle below):

User-Friendly Infrastructure Product Catalog

13

Requirements Denition

1 1

2 2

3 3

4 4
Prototype Labs

5 5

Importance (Circle below):

VII. Talent Management

VI. Vendor Management

V. Applications Delivery and Management

We create alternative scenarios of cost vs. functionality for proposed IT projects to help business sponsors make informed scoping choices.
Performance (Circle below):

Diagnostic exercise for evaluating IT practice

1 1

2 2

3 3

4 4

5 5

Importance (Circle below):

Alternative Cost and Functionality Scenarios

23

IT-Enabled Collaboration We have created a toolkit and a set of communities that allow distributed staff to effectively share information and collaborate.
Performance (Circle below):

22

Data and Knowledge Management We provide the business with accurate, timely, required information.
Performance (Circle below):

21

Performance Management We closely link compensation of senior IT staff to desired outcomes such as simplication or cost reduction.
Performance (Circle below):

20

Leadership Development We provide high potential IT staff with customized training and development plans, including formal training, rotations, and project-based work.
Performance (Circle below):

19

IT Staff Development We provide IT staff with transparency into role-specic competency requirements and opportunities for technical and business skills development.
Performance (Circle below):

18

Vendor Segmentation We segment vendors based on strategic value and allocate management resources accordingly.
Performance (Circle below):

17

Vendor Performance

16

Oversight We utilize a metrics-based scorecard to select and evaluate vendors based on standard performance criteria.
Performance (Circle below):

Maintenance Cost Containment

We segment our applications portfolio based on business value to facilitate legacy system retirement and prioritize maintenance work.
Performance (Circle below):

15

Project Management and Execution

14

Application Design

We manage projects using a standard methodology to meet budget, scope, and schedule.
Performance (Circle below):

We reuse application components and services to build new functionality and integrate existing systems.
Performance (Circle below):

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

1 1

2 2

3 3

4 4

5 5

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Community Seeding

One View of the Company

Architecture Gain-Sharing Incentive Plan

Personalized Leadership Development Planning

Development Opportunity Brokering

Value-Based Vendor Segmentation

Vendor Performance Scorecard

Value-Based Maintenance

Stage-Gated Project Execution

Best-in-Breed Component Architecture Updated September 2003

Recommended Strategic Research

CIO Execut ive Board CIO Execut ive Board

Recommended Short Answer Research

CIO Executive Board

Web-Based Resource Centers Structured gateway to the CIO Executive Board research resources

urity ive Board ormation Sec g CIO Execut Trends in InfContinuity Plannin gement ndor Mana and Business Strategic Ve cing ur Scorecards and Outso IT Balanced

Short-Answer Research Library

A Work in Progress
Our Competency Diagnostic is based on the leading practice research that the CIO Executive Board has conducted to date. As we increase both the depth and breadth of our coverage of IT strategy and planning practices, we will continue to update the diagnostic with future releases. As always, CIO Executive Board staff remain ready to provide customized research to supplement the existing research database available to you on our Web site at www.cioexecutiveboard.com. Should you be interested in commissioning any customized work, please contact your account director or visit our Web site. Customized research is included in your membership and, like all other elements of CIO Executive Board membership, carries no additional charge.

A Guide to Using the Diagnostic

Instructions
1. Evaluate IT Strategy and Management Activities Getting Started Assemble the IT strategy and management team. Participants will help assess the organizations level of performance and potential impact across 25 attributes of IT management excellence. If desired, individual members may complete the diagnostic survey online at the CIO Executive Boards Web site, www.cioexecutiveboard.com. By submitting this exercise online, members can receive a customized Web page and an automated e-mail response containing links to relevant research. Scoring the Organization For each of the 25 attributes featured on the diagnostic, rate your organization against the two criteria, using the following scoring system: Performance Against Objective Rate the manner in which the organization currently performs this activity, using the following scale: 5 = We are excellent at this 4 = We are good at this 3 = We are average at this 2 = We are poor at this 1 = We are terrible at this Importance of Objective Rate the potential impact on the value created for customers and the rm if this activity were performed to a world-class standard, using the following scale: 5 = Critical 4 = High Priority 3 = Moderate Priority 2 = Low Priority 1 = Not a Priority

2. Prioritize Areas for Improvement

1 2

Performance Against Objective

Upon entering all 25 attribute scores, you can focus the organizations attention on the areas of highest importance but lowest current performance.
1 2 3 4 5

Importance of Objective

3. Identify Leading Practices for Adoption For each attribute you have prioritized for focus, identify best practice examples from the CIO Executive Boards archives. The diagnostic itself points to a number of best practices; the CIO Executive Boards Web site provides many more and can be searched by each of the 25 attributes to assist you in identifying pertinent research to support your efforts.

Note to Members
CIO Executive Board staff are at your service to help translate these best practices for your adoptionincluding discussing practice details, providing supplemental research, or networking you with other CIO Executive Board members who have worked through similar challenges. There is no additional cost to you for any of these services.

Achieving Leading Practice

The Diagnostic Directs Members to the Most Relevant Strategy and Management Practices
Illustrative Application

For each competency

Portfolio Management We help the business prioritize projects using a dened set of objective, weighted criteria.
Performance (Circle below):

A Framework for Enterprise-Level Portfolio Management and Prioritization

To enable portfolio prioritization, Schlumberger establishes a Project Ofce with global reach and creates an IT project portfolio with principled project categories
Schlumberger Asset Classes and Target Allocations Confer competitive Confer competitive advantage, e.g., incubating advantage, e.g., incubating new platforms or responses new platforms, or responses to competitors offerings to competitors offerings
IT Planning and Standards Business Segment IT

1 1

2 2

3 3

4 4

5 5
IT Security

Importance (Circle below):

Schlumberger IT Organizational Structure Schematic

CIO

IT Controller and HR

Knowledge Management and Project Ofce

Realize measurable Realize measurable business benets business benets (revenue generation (revenue generation or cost savings) or cost savings)
Business Opportunity 25%

Project Ofce

the CIO Execuutive Board has identied and proled case studies

EnterpriseLevel Portfolio Prioritization

Structure Six FTEs with six to ten years of experience at Schlumberger Diverse functional and BU background

Charter Project management processes, tools, governance IT portfolio management

Optimize the use of Optimize the use of existing IT assets existing IT assets and and reducecosts reduce IT IT costs

Innovation 25%

Infrastructure 40%

Legal or Legal or regulatory regulatory compliance compliance

Mandatory 10%

Sugarland, Tex. Montrouge, France Austin, Tex.

Initial project inventory includes 250 projects, or 50 percent of ongoing IT work Based on interviews with 125 project managers and sponsors

Source: Schlumberger Limited; CIO Executive Board research.

Responsive IT Portfolio Prioritization/Schlumberger

25

Creating a Common Vocabulary for Life-Cycle IT Costs and Value

An enterprise-standard business case template completed by a project sponsor provides enterprise-wide business case comparability
Schlumberger Project Business Case Template Illustrative
Date: 6/4/02 Project: Strategic Customer Extranet Upgrade 3.5 100%

and project life-cycle cost visibility

Percentage of Life-Cycle Cost Captured in Project Business Case Illustrative

Communication

that provide detailed descriptions of leading management practices in this area of IT strategy

BENEFITS Revenue from New Customers Revenue from Existing Customers Revenue from Additional Asset Use Cost Reductions Additional Flexibility Total Bene ts COSTS Evaluation Hardware Software Development Deployment Management Support Communication Downtime Subtotal Costs Grand Total Costs Total Benets Total Costs = Value Added 1 32.9 93.2 = 39.7 Start-Up 2.0 15.0 12.0 30.0 10.0 N/A N/A N/A N/A 69.0 35.4 20.5 2.0 50.0 25.0 132.9 Operational N/A 5.0 6.0 8.0 N/A
Deployment Hardware Maintain Upgrade Software Maintain Upgrade Support

Development (Maintenance & Enhancement)

Project Life-Cycle Cost

Management

0.5 3.5 0.4 0.8 24.2 93.2 10%

Evaluation Hard/Software Purchase

Development

Start-Up Costs Captured in Historical Business Case Creation

Ongoing Costs Captured with New Standard Template

Source: Schlumberger Limited; CIO Executive Board research.

Responsive IT Portfolio Prioritization/Schlumberger

27

Supplementing Value with Risk

Project Ofce Augments Project Value Scoring with a More Detailed Project Risk Assessment
Schlumberger Project Risk-Assessment Scorecard Illustrative

Project Stafng Resource availability Skills availability Availability of experienced staff

Technological Risk Newness of technology to company and world Scalability Data security and privacy Disaster recovery capabilities Business Risk

Project Schedule Realism of schedule Criticality of on-time delivery

Business impact of system downtime Business impact of lost or improperly released data Organizational Risk Project process repeatability Organizational diversity of project team

Project Scope
7 4 1 0 8 5 2 . 9 6 3 x + =

Size and complexity of project Measurability of success Ability to accommodate scope reduction without loss of critical functionality Project Dependencies Dependence on other projects Dependence on externally developed technologies

Political Risk Senior management commitment Likelihood that funding is jeopardized

Source: Schlumberger Limited; CIO Executive Board research.

Responsive IT Portfolio Prioritization/Schlumberger

29

CIO Executive Board

Key Attributes of the World-Class IT Organization: A Competency Diagnostic

Attributes of Interest to the CEO/CFO
3 4 5 6 7 8 9 10

Availability Management

Strategy and Planning We chart ITs strategic direction based on the businesss articulated strategies.

Enterprise Architecture We maintain an architecture blueprint that clearly links technology choices to business capabilities.
Performance (Circle below): Performance (Circle below): Performance (Circle below): Performance (Circle below): Performance (Circle below): Performance (Circle below): Performance (Circle below):

Business Case Discipline We employ a standard business case template for all IT investments that captures project life-cycle costs, benets, and risks.
1
Importance (Circle below): Importance (Circle below): Importance (Circle below): Importance (Circle below): Importance (Circle below): Importance (Circle below):

Portfolio Management We help the business prioritize projects using a dened set of objective, weighted criteria.

Value Demonstration We track metrics that demonstrate ITs contribution to the achievement of desired business outcomes.

Performance Reporting We provide regular reports tailored by channel and content for specic business and IT audiences.

Risk Management We have created a principled framework for assessing relative risk and sequence risk mitigation investments accordingly.
1 1
Enterprise Risk Assessment Employee Awareness Campaign

Security Policies and Standards We have dened and communicated a set of enterprise information security policies and standards
1 1 2 3 4 5 1 2 1 3 4 5 2 2 3 3 4 4 5 5

Disaster Recovery and Business Continuity Planning We develop and regularly test enterprise plans to ensure continuous support of core business processes.

We ensure the availability of systems based on SLAs that consider business criticality.

Performance (Circle below):

Performance (Circle below):

Performance (Circle below):

1 1
Project ROI Contracts Enterprise-Level Portfolio Prioritization Executive Satisfaction Survey Balanced Scorecard Life-Cycle Management

2 1 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 2 3 4 5 1 1 2 3 4 5

1 1
Enterprise-Level Business Continuity Planning

2 2

3
Importance (Circle below):

4 3 4

5 5
User-Centric Operational Indicators

Importance (Circle below):

Importance (Circle below):

Importance (Circle below):

Capabilities Alignment Diagnostic

Embedded Architectural Discipline

11

Life-Cycle Cost Efciency

We effectively manage assets across their life cycle to maximize efciencies.

25

Process Digitization

I. IT Governance

II. IT Performance Measurement and Value Demonstration

How to Use This Diagnostic
Please select the value (1 to 5) that best describes your IT organization using the two grading scales provided. The top scale measures performance in each competency and the bottom scale measures the importance of each competency.
8

III. Security and Business Continuity Planning

Performance (Circle below):

1 1

2 2

3
Importance (Circle below):

4 3 4

5 5
End-to-End Asset Management

We employ a standard methodology to identify opportunities for business process automation and enhancement.

Performance (Circle below):

Importance (Circle below):

IV. Infrastructure Delivery and Management

12

Cost Transparency

Top-Down Business Process Optimization

Competency
Performance (Circle below):

Security Policies and Standards We have dened and communicated a set of enterprise information security policies and standards
1
Importance (Circle below):

We use unit-level visibility into infrastructure cost drivers to provide internal customers with the ability to effectively manage their demand.
Performance (Circle below):

VIII. Business Enablement

2 2 3 4 5 3 4 5

Performance Importance
1

1 1

2 2

3
Importance (Circle below):

4 3 4

5 5
User-Friendly Infrastructure Product Catalog

24

Technology Innovation

Scoring Scale

We proactively scan for and exploit opportunities to deploy new technologies in support of the business.

13

Requirements Denition

Performance (Circle below):

Performance 5 = We are excellent at this 4 = We are good at this 3 = We are average at this 2 = We are poor at this 1 = We are terrible at this

Importance 5 = Critical 4 = High Priority 3 = Moderate Priority 2 = Low Priority 1 = Not a Priority

Importance (Circle below):

We create alternative scenarios of cost vs. functionality for proposed IT projects to help business sponsors make informed scoping choices.
Performance (Circle below):

Prototype Labs

VII. Talent Management

VI. Vendor Management

V. Applications Delivery and Management

1 1

2 2

3
Importance (Circle below):

4 3 4

5 5
Alternative Cost and Functionality Scenarios

23

22

21

20

19

18

17

Vendor Performance

16

IT-Enabled Collaboration We have created a toolkit and a set of communities that allow distributed staff to effectively share information and collaborate.
Performance (Circle below): Performance (Circle below):

Data and Knowledge Management We provide the business with accurate, timely, required information.

Performance Management We closely link compensation of senior IT staff to desired outcomes such as systems simplication or cost reduction.

Maintenance Cost Containment Oversight We utilize a metrics-based scorecard to select and evaluate vendors based on standard performance criteria.
Performance (Circle below): Performance (Circle below): Performance (Circle below):

15

Project Management and Execution

14

Application Design

Leadership Development We provide high potential IT staff with customized training and development plans, including formal training, rotations, and project-based work.
1
Importance (Circle below):

IT Staff Development We provide IT staff with transparency into role-specic competency requirements and opportunities for technical and business skills development.
5 1 2 3
Importance (Circle below):

Vendor Segmentation We segment vendors based on contract value and strategic contribution and allocate management resources accordingly.
4 5 1 2 3
Importance (Circle below):

We segment our applications portfolio based on business value to facilitate legacy system retirement and prioritize maintenance work.
Performance (Circle below):

We manage projects using a standard methodology to meet budget, scope, and schedule goals.
Performance (Circle below):

We reuse application components and services to build new functionality and integrate existing systems.
Performance (Circle below):

Performance (Circle below):

Performance (Circle below):

1
Importance (Circle below):

2 1 1
Architecture Gain-Sharing Incentive Plan

3 2 3 4 5 1 2 3

4 4

4 5 1 2 3 4 5
Development Opportunity Brokering

5 1 2 3 4 5
Value-Based Vendor Segmentation

1 1

2 2

3
Importance (Circle below):

4 3 4

5 5
Vendor Performance Scorecard

1 1

2 2

3
Importance (Circle below):

4 3 4
Value-Based Maintenance

5 5

1 1

2 2

3
Importance (Circle below):

4 3 4

5 5
Stage-Gated Project Execution

1 1

2 2

3
Importance (Circle below):

4 3 4

5 5
Best-in-Breed Component Architecture Updated September 2003

Importance (Circle below):

Importance (Circle below):

Community Seeding

One View of the Company

Personalized Leadership Development Planning

Scoring Your Organization

Prioritizing Areas for Improvement
Using the performance and importance scores, plot each attribute on the prioritization grid found to the right For example, if attribute 9 scored as 5 for importance and 1 for performance, enter the number 9 in the upper rightmost box on the grid Upon entering all attribute scores, you can focus the organizations attention on the attributes in the upper-right section of the gridthe areas of highest importance but lowest current performance
9 8 10
6

13

Performance Against 3 Objective

12 19 20 3 5

11

5 1 2 3
Importance of Objective

Leveraged for a Variety of Purposes

Shape Strategic Thinking Drive Towards Effective Change Build a Compelling Business Case

Diagnostic used to prioritize short- and long-term performance improvement opportunities on a quarterly or semiannual basis. Exercise can be customized by members to reect industry- or businessspecic concerns.

Diagnostic permits IT leadership to pressure test strategic goals with day-to-day staff and identify discrepancies in perception of current performance. Integrating highpotential employees into the planning process effectively addresses potential roadblocks to project implementation. Target Audience Seniormost IT Executive IT Leadership Team High-Potential IT Staff

Diagnostic results used as a data point in support of a business case for improvement initiatives that require senior executive support. Diagnostic can also be completed by CIO and senior business leadership at off-site planning sessions to gain 360 insight into IT performance. Target Audience Seniormost IT Executive CFO and Executive Team IT Leadership Team

Target Audience Seniormost IT Executive

Accessing Leading Practices Research

Our short answer research library and Web site provide members with the ability to search for IT strategy and management best practices at their convenience
Short Answer Research Library
CIO Executive Board

CIO Executive Board Web Site

Short-Answer Research Library

www.cioexecutiveboard.com
Issue Brief: Case examples of leading organizations approach to common challenges Member Networking: CIO Executive Board brokered introductions to industry experts and peers Vendor Proles: Overview of leading vendors for a specic category of products or services, including key contacts and representative clients Presentation Builder: Tool for incorporating graphics and charts from CIO Executive Board research into members own presentations Peer Database: List of member experts available for networking conversations Discussion Boards: Threaded discussion on emerging issues

and Working Council staff are also available for in-person support and briengs
Member Meetings Annual Executive Retreats for CIOs: Interactive presentation of key ndings to intimate audience of IT principals IT Executive Briengs: Formal presentation of key ndings to broad audience of IT leaders Emerging Issue Teleconferences: Brief presentations, member Q&A, and discussion on most requested topics On-Site Briengs Formal Research Presentation: Formal presentation to convey research ndings to a large audience, followed by Q&A Roundtable Research Review: More intimate presentation with questions and discussion throughout session Interactive Working Session: Interactive discussion of member issues, which may include breakout sessions and group priority-setting exercises

Unlimited Service
The annual membership contribution covers the entire range of CIO Executive Board services.

CIO Executive Board

www.cioexecutiveboard.com
Corporate Executive Board 2000 Pennsylvania Avenue NW Washington, DC 20006 Telephone: +1-202-777-5000 Fax: +1-202-777-5100 The Corporate Executive Board Company (UK) Ltd. Victoria House 3763 Southampton Row London WC1B 4DR UK Telephone: +44-(0)20-7632-6000 Fax: +44-(0)20-7632-6001
CIO133UN11