Internal use only

GPL
Product Risk Analysis (Related to an information system)
Guidance Manual

Status: Owner: Date: Document id: Version:

Final GPL team 1 June 2005

Product Risk Analysis information system guidance.doc

2.0

GPL Product Risk Analysis (Related to an information system) Guidance Manual

Table of contents
STATUS:..................................................................................................................................................1 STATUS:..................................................................................................................................................1 FINAL......................................................................................................................................................1 FINAL......................................................................................................................................................1 OWNER:..................................................................................................................................................1 OWNER:..................................................................................................................................................1 GPL TEAM..............................................................................................................................................1 GPL TEAM..............................................................................................................................................1 ..................................................................................................................................................................1 ..................................................................................................................................................................1 DATE:......................................................................................................................................................1 DATE:......................................................................................................................................................1 1 JUNE 2005.............................................................................................................................................1 1 JUNE 2005.............................................................................................................................................1 DOCUMENT ID:.....................................................................................................................................1 DOCUMENT ID:.....................................................................................................................................1 PRODUCT RISK ANALYSIS INFORMATION SYSTEM GUIDANCE.DOC ...............................1 PRODUCT RISK ANALYSIS INFORMATION SYSTEM GUIDANCE.DOC ...............................1 VERSION:...............................................................................................................................................1 VERSION:...............................................................................................................................................1 2.0..............................................................................................................................................................1 2.0..............................................................................................................................................................1 TABLE OF CONTENTS........................................................................................................................2 1. INTRODUCTION ...............................................................................................................................4 1.1. Information system as a product ......................................................................................................4 PRODUCTS IN GENERAL ARE INSTRUMENTS AND SERVICES DELIVERED TO CLIENTS. IT IS IMPORTANT FOR THE PROJECT MANAGER TO NOTE THAT PRODUCT DEVELOPMENT PROCESSES GOVERNING NEW AND CHANGED PRODUCTS ARE SEPARATE FROM PROJECT MANAGEMENT PROCESSES. PRODUCTS ARE GOVERNED BY THE PRODUCT REVIEW PROCEDURES AND PROJECTS BY THE GPL IN COMBINATION WITH A PROJECT GOVERNANCE PROCESS...................................................4 IN THIS MANUAL THE PRODUCT IS A NEW OR ENHANCED INFORMATION SYSTEM. THEREFORE THE PRODUCT RISK ANALYSIS AS DESCRIBED IN THIS MANUAL IS ONLY CONCERNED WITH USING THE INFORMATION SYSTEM! .........................................4 FOR READABILITY OF THIS MANUAL THE TERMS PRODUCT, PRODUCT RISK AND PRODUCT RISK ANALYSIS ARE USED, WITHOUT THE ADDITION OF THE TERM INFORMATION SYSTEM..................................................................................................................4 1.2. What is Product Risk Analysis (for an information system).............................................................4 1.3. Why Product Risk Analysis..............................................................................................................4 1.4. Target Readership.............................................................................................................................5
GPL Team 78471528.doc Page 2 of 16 Internal use only 1 June 2005

GPL Product Risk Analysis (Related to an information system) Guidance Manual 1.5. How to use the manual......................................................................................................................5 2. ANALYSING PRODUCT RISKS ......................................................................................................7 2.1. Stakeholders selection......................................................................................................................7 'STAKEHOLDERS SELECT THEMSELVES (OR EXCLUDE THEMSELVES). THIS IS ACHIEVED BY AN ALL STAKEHOLDERS LIST AT THE PROJECT INITIATION PHASE AND BY SURVEY THE STAKEHOLDERS SELF SELECT OR SELF ELIMINATE THEMSELVES (AUDIT TRAIL TO BE RETAINED BY THE PROJECT MANAGER). THEREAFTER THE PROJECT MANAGER IS RESPONSIBLE TO ENSURE PARTICIPATION OF THE 'INCLUDED' STAKEHOLDERS, IN THE ANALYSIS PHASE, THE DEVELOPMENT PHASE (AS ADVISORS), THE TESTING PHASE AND THE IMPLEMENTATION PHASE.' ....7 2.2. Identify product risks........................................................................................................................7 2.2.1. Creation of specific checklists....................................................................................................8 2.2.2. Prepare Result table....................................................................................................................8 2.3. Determine priority using probability and impact..............................................................................9 2.4. Define measures to mitigate product risks......................................................................................10 2.5. Using the results.............................................................................................................................11 3. APPENDICES.....................................................................................................................................13 THE CHECKLISTS ARE A COLLECTION OF PRODUCT RISK QUESTIONS THAT COULD BE AN INDICATION OF A POSSIBLE PRODUCT RISK. THE PRODUCT RISK QUESTIONS ARE SUBDIVIDED IN 5 CHECKLISTS FOR EACH DIFFERENT, WELL-KNOWN GROUP OF STAKEHOLDERS OR DEPARTMENT:.............................................................................................13 USERS;...................................................................................................................................................13 MARKETING;......................................................................................................................................13 IT PRODUCTION & INFRASTRUCTURE;.....................................................................................13 IT DEVELOPMENT;...........................................................................................................................13 SECURITY DEPARTMENT...............................................................................................................13 M (MUST TEST);..................................................................................................................................13 S (SHOULD TEST);..............................................................................................................................13 C (COULD TEST);................................................................................................................................13 W (WONT TEST)................................................................................................................................13

GPL Team 78471528.doc

Page 3 of 16

Internal use only 1 June 2005

GPL Product Risk Analysis (Related to an information system) Guidance Manual

1. 1.1.

INTRODUCTION Information system as a product

Products in general are instruments and services delivered to clients. It is important for the Project Manager to note that product development processes governing new and changed products are separate from project management processes. Products are governed by the Product Review Procedures and Projects by the GPL in combination with a Project Governance Process. In this manual the product is a new or enhanced information system. Therefore the product risk analysis as described in this manual is only concerned with using the information system! For readability of this manual the terms product, product risk and product risk analysis are used, without the addition of the term information system.

1.2.

What is Product Risk Analysis (for an information system)

In testing an information system, two types of risk are involved: project delivery risks and product risks. Project delivery risks are related to problems that may endanger the testing in a project. The project delivery risks are essential to a project manager and/or test manager in governing the project. Project delivery risks need to be monitored to assure carrying out the project according to plan. An example of a project delivery risk is the risk of not having enough resources available to carry out the activities of the project in the planned time. The Project delivery risks are beyond the scope of this document. Product risks concern the operational risks that may arise when using an information system in day-to-day business (and are therefore also called business risks in PRINCE2). The product risks should be the basis for defining what should be tested, because the goal of testing is to get information on whether these risks will become effective when using the system or not. The decision whether or not to accept a system should be based on the knowledge of the business risks that are covered and the ones that are not covered. Product risks are a subset of the operational risks. Product risks are operational risks that focus on the use of the information system in the operational situation. Operational risks are defined on the Operational Risk Management intranet site (http://www.orm.intranet/) as: the risk of direct or indirect loss resulting from inadequate or failed internal processes, people and systems or from external events. An example of a product risk is that an incorrect interest calculation will cause erroneous accounts and thus may result in losing either money or customers.

1.3.

Why Product Risk Analysis

The product risk analysis is a necessary component of Risk and Requirements Based Testing (RRBT). The product risk analysis is used to prepare an efficient test and to prioritise the results of the test specification (for example the test clusters and test cases). The list of product risks and their priority can be used when doing the test specification and the planning and execution of the tests. The premise is that the highest risks are elaborated the most and will be tested first. This approach of RRBT will also have the result that at any point in time the most critical part has been tested. Additional information on RRBT is available on the GPL website.

GPL Team 78471528.doc

Page 4 of 16

Internal use only 1 June 2005

GPL Product Risk Analysis (Related to an information system) Guidance Manual The bottom line regarding test efficiency is that doing a Product Risk Analysis is a small effort that can save much time and budget in testing, because testing requires an enormous effort in IT Development projects (20% to 50% of the total cost after Initiation) The product risks identified will also be used to cross-check the requirements and/or specifications. For most product risks a requirement should exist (if for example the product risk is that a long responsetime reduces productivity, a requirement must exist stating the maximum responsetime), and the requirements should be reflected in the product risks that were identified. If during this check differences are found they can be corrected in an early stage of the project. From literature it is very well known that correcting a fault early in the lifecycle is far cheaper than correcting the defect during testing. The picture below shows the context of the Product Risk Analysis:

PID

Stakeholders

Operational Risk Management

Product Risk Analysis

Requirements & Functional Specifications

Test Strategy

Test Plan

Test Specification

1.4.

Target Readership
The product risk analysis is carried out by the test manager (or one or more members of the test team) in co-operation with the functional stakeholders who own the risks. All key stakeholders are invited to assess the product risks. The comprehensive set of product risks is relevant for the development activities, the test specification and the test execution. So mutual understanding of and agreement on the product risks is of great value to the development team, test team, and different other stakeholders. So the audience for the product risk analysis are all parties involved in testing.

1.5.

How to use the manual

This Product Risk Analysis Guidance describes in Chapter 2 the approach how to determine product risks, and how they are used during testing. The final result is a table containing prioritised product risks and their impact on testing. This result table will be used during creation of the test strategy and test plan and the priorities in the table are the basis for prioritising test cases during the test specification. Checklists are important instruments in the analysis, example checklists are provided in the Appendices. Those example checklists can be used as a starting point for creating a specific checklist for a specific test. Note that in smaller projects or situations with a very tight timetable

GPL Team 78471528.doc

Page 5 of 16

Internal use only 1 June 2005

GPL Product Risk Analysis (Related to an information system) Guidance Manual the testers may decide not to create a checklist but to fill in the result table right away, based upon previous experience, common sense or other sources that are quickly available. In that case the product risk analysis may not be as profound as usual but still there will be a product risk analysis in time to serve as the basis for the subsequent testing activities.

GPL Team 78471528.doc

Page 6 of 16

Internal use only 1 June 2005

GPL Product Risk Analysis (Related to an information system) Guidance Manual

2.

ANALYSING PRODUCT RISKS

The following steps are taken in the product risk analysis: 1. Stakeholders selection; 2. Identify product risks; 3. Determine priority using probability and impact; 4. Define measures to mitigate the product risks. The results of the product risk analysis are contained in a result table. This table is filled in during the steps 2, 3 and 4. In the last section of this chapter it is explained what is done with the results.

2.1.

Stakeholders selection
'Stakeholders select themselves (or exclude themselves). This is achieved by an all stakeholders list at the Project Initiation Phase and by survey the stakeholders self select or self eliminate themselves (audit trail to be retained by the Project Manager). Thereafter the Project Manager is responsible to ensure participation of the 'included' stakeholders, in the Analysis Phase, the Development Phase (as advisors), the Testing Phase and the Implementation Phase.'

Stakeholders of an information system are for example: the users (who are making use of the information system); the marketing department (that wants to have a marketable product in a short time-tomarket); the security department (that does not wish information to be accessible to outsiders); the IT production & infrastructure department (that appreciates if new or changed systems do not impact the working of the infrastructure); the IT development department (who of course are responsible for creating the system); the IT maintenance department (who appreciate maintainable software). In having an interest, the stakeholders also have a responsibility. It is for the stakeholders that the test is being carried out. Therefore the stakeholders are important parties in the product risk analysis. However responsible is the Executive/Client/User representatives. In order to analyse the risks associated to an information system, it is useful to analyse all different business processes using the descriptions that were made in the Analysis Phase of the GPL. Especially, when considering the impact after a risk occurs, it is important to follow the process flows. What happens with a business scenario when the information system works incorrectly? Analysing the business processes may also lead to identifying additional stakeholders. These should then also be included in the product risk analysis.

2.2.

Identify product risks

Identification of product risks is done in two steps. First checklists are made that will create a basis for structured product risk identification. After that the actual analysis is carried out and the resulting product risks are captured in the result table.

GPL Team 78471528.doc

Page 7 of 16

Internal use only 1 June 2005

GPL Product Risk Analysis (Related to an information system) Guidance Manual However if an Operational Risk Assessment has been done earlier in the project or in the business unit that will use the information system the results can be used. Then the product risks result table can be filled in directly with the purpose of identifying the impact on testing.

2.2.1.

Creation of specific checklists In identifying the product risks, the aim is to discover what the product risks are and where they are to be found. Checklists with questions are used to assist the various stakeholders in identifying the product risks. The goal of using checklists is to have a solid list of items that inspires all stakeholders in identifying the product risks they may run resulting from the new/changed system. A specific checklist per stakeholder needs to be prepared in advance by the person responsible for doing the Product Risk Analysis before contacting the stakeholders. This improves the outcome of the analysis. For creating the specific checklist example checklists can be used. Refer to the Appendices for the example checklists for the various stakeholders and how to prepare the specific checklist. The example checklists also contain a column for priority, here the stakeholder can indicate the importance of the topic covered by the question in the checklist. This is used in specifying the final priority (as described in section 2.3) of a distinct product risk in the result table. The example checklists in the appendices also contain the related quality attribute (according to ISO 9126) for the product risk questions. Quality attributes (refer to Test Strategy guidance manual) help in identifying risk areas. During creation of the checklists, and during the product risk analysis itself, ensure that all quality attributes have been covered. Bear in mind that a checklist is just a tool. If the time is too short to create checklists or it is a small project, then for example a workshop could be organised to identify the product risks. In that case the results of the product risk analysis are directly captured in the result table.

2.2.2.

Prepare Result table Input for product risk identification can be: prepared specific checklists past experience operational risk assessment PID (section on risks) Feasibility Study / Requirements specs. (business process descriptions and risk descriptions) The prepared checklist can be used as a starting point. When contacting the stakeholders for filling in the checklist they should consider and identify product risks based on their knowledge and experience. A workshop with all stakeholders is useful to evaluate the results and also to enrich the product risk analysis. During the workshop what-if questions are an effective means to stimulate thinking about the product risks. For example the question what if the calculation of interest is erroneous? will identify product risks for various stakeholders. The user will see the risk of losing money or customers, the marketing department will see the risk of not being able to reach the marketing goals. In case of doubts and/or discussions, experiences of the different stakeholders can bring forward important information. All experiences help in predicting and analysing the product risks.

GPL Team 78471528.doc

Page 8 of 16

Internal use only 1 June 2005

GPL Product Risk Analysis (Related to an information system) Guidance Manual To keep a manageable project the number of product risks to be handled should not be too high. As a rule of thumb for an average project about 20 to 30 product risks would be a reasonable number.

The result table will be structured as in the next example: Product Risk short name Quality Attribute Product Risk description Product Risk short name Product Risk description

Probabil .

Impact

Priority

Product Risk mitigation / Test Quality Attribute Probabil . Impact Priority

Product Risk mitigation / Test

2.3.

Determine priority using probability and impact

If checklists are used a preliminary priority by a stakeholder can be assigned to a product risk. In the workshop or in another way the final priority should be agreed. The following text will help in assessing and agreeing the final priority. The level of a risk is based on the probability and the impact (related costs or negative effects) of the risk when it occurs. A higher risk needs more attention and test effort. Determine probability In general the probability of a product risk in a business process is dependent on: frequency of use of the process likeliness of occurring the amount of users executing the process: The use of experiences and knowledge of stakeholders is very important. Note that this information is often a rough estimation and thus it works best to express the probability in High, Medium or Low. So if a process is used by all users, every day, the whole day long, even a failure that is not likely to happen often will have a high probability and may thus lead to a high product risk classification. Determine impact After risks and related probability are clear the impact is determined by looking at the probable costs. The costs are not only expenses, but also non-financial negative impacts. For example, a negative article in a newspaper. The impact may be different for different stakeholders. Some stakeholders will look at the financial costs, others will look at image and negative publicity. In case of conflicting interests the stakeholders should discuss and together decide upon the product risk for the specific situation for the organisation as a whole.

GPL Team 78471528.doc

Page 9 of 16

Internal use only 1 June 2005

GPL Product Risk Analysis (Related to an information system) Guidance Manual

Just as the probability the impact is often a rough estimation and thus it works best to express the impact in High, Medium or Low. Determine priority Priority is stated by using the MoSCoW categorisation. In product risk analysis, MoSCoW stands for: M: Must test; S: Should test; C: Could test; W: Wont test.

As a rule of the thumb the following formula for the probability/impact combinations can be used: High, High results in Must Test. High, Medium: Should Test Medium, Medium: Could test Low, Low: Wont test Refer to the example result table in the next section for an example of assigning the priority in this way. The next table gives an example of a more implicit use of the probability and impact of risks, and how to determine the priority level.
financial consequences for our customer

All customers One customer All customers One customer No workaround Workaround No workaround Workaround

Must test Should test Should test Could test Could test Wont test Could test Wont test

If this goes wrong then

no financial consequences for our customer financial consequences for our department no financial consequences for our department

2.4.

Define measures to mitigate product risks

The priority, probability and impact of a risk determine the next steps to be taken. There are four optional steps to follow: 1. Do nothing. The impact or probability of the risk is low; 2. Act afterwards. The impact is low. If the risk comes out, repair will be easy; 3. Take measures in the project to prevent the risk from happening. Options are: Design different procedures, maybe including checks on the risks; Provide work instructions and training;

GPL Team 78471528.doc

Page 10 of 16

Internal use only 1 June 2005

GPL Product Risk Analysis (Related to an information system) Guidance Manual Change requirements and specifications of the information system; Simulate the process and check on the risks; 4. Test the risk and repair if there are problems. For each measure that has been taken to mitigate a product risk it must be verified, for example by means of testing, whether the risk has been sufficiently mitigated. If a test is the measure to be taken this may impact the content of the various test phases (and if necessary test types). Some product risks can best be checked in a system test (like risks about technical situations) and other product risks can better be checked in an acceptance test (for example the risk that the system is not according to the operational procedures used in the organisation). Example of some items of a completed result table: Product Risk short name Interest calculation incorrect probability
Quality Attribute Functionality

Probabil .

Impact High

Priority Must test

High Product Risk description Product Risk mitigation / Test If the interest calculation is incorrect the business Extensive test of the interest calculation according will lose customers or money. to the functional requirements Product Risk short name Quality Probabil Impact Priority End-of-day processing within 8 hours Medium Should test Attribute . Efficiency High

Product Risk description

Product Risk mitigation / Test

If the end-of-day processing is not ready in 8 hours Performance test in a production-like the end-users can not use the system at start of day. environment Product Risk short name Quality Probabil Impact Priority Lay-out of printouts not user-friendly Attribute Medium Could test . Usability Medium

Product Risk description

Product Risk mitigation / Test

If the lay-out of the printouts is not user-friendly the Do a review (static test) on the lay-out design in users will not be able to work efficiently an early stage Product Risk short name Quality Probabil Impact Priority Wont test The colours of error messages are Low Attribute . wrong Functionality Low

Product Risk description

If the colour of error messages is black instead of red they will not attract extra attention

Product Risk mitigation / Test

If a tester comes across wrong colours during other tests he will report them Quality Probabil Impact Priority Should test High Attribute . Usability Medium

Product Risk short name Human error causes wrong processing

Product Risk description When an operator starts processes in a wrong order this may cause serious problems in the processing.

Product Risk mitigation / Test

The operators will be given proper training. An exam is used to verify their knowledge level.

2.5.

Using the results

As explained in the previous section the results of the product risk analysis will further be used in creating the test strategy, and subsequently the test plan. The result table can be copied directly into the section of the Test Strategy template regarding product risks. Another option is to keep the result table as a separate deliverable, this is advised if it is a large table to be used by many persons. The product risks are the basis of the acceptance criteria.

GPL Team 78471528.doc

Page 11 of 16

Internal use only 1 June 2005

GPL Product Risk Analysis (Related to an information system) Guidance Manual

The product risks, acceptance criteria together with the requirements or functional specifications are the input for the test specification. The priorities identified for the product risks are used to prioritise the test clusters, test conditions and test cases. Finally, during test execution, these priorities are the basis for assigning the priority of test findings (also called incidents) that result from testing. The product risks identified will also be used to cross-check the requirements and/or specifications. For each product risk a requirement should be specified. If for example a product risk is that the organisation can not work properly if the response time is more than 2 seconds there should be a requirement that states this response time. Also the tester might find requirements that obviously should have been mentioned in the product risk analysis, and thus the product risk analysis can be completed. Insight in the product risks involved will also help quality management during the development of an information system. The outcomes of reviewing and testing will be input for improving the deliverables of the development process.

GPL Team 78471528.doc

Page 12 of 16

Internal use only 1 June 2005

GPL Product Risk Analysis (Related to an information system) Guidance Manual

3.

APPENDICES
How to use the example checklists

Appendix A

The appendices show example checklists for product risk determination for various departments. The following checklists are an aid for determining the product risks related to an information system. The checklists are not extensive and must be supplemented with other product risks questions in each specific test that is set up. The checklists may be extended with extra information on operational risks for a specific business area, the checklist Library of Key Risk Indicators can be found on the intranet page http://www.orm.intranet/ under key risk indicators. The checklists are a collection of product risk questions that could be an indication of a possible product risk. The product risk questions are subdivided in 5 checklists for each different, well-known group of stakeholders or department: Users; Marketing; IT production & infrastructure; IT development; Security department. The last column is used to determine the priority and thereby test effort related to one product risk indication. In this column the stakeholder can indicate the importance of the area covered by the question in the checklist. This preliminary priority is used in specifying the final priority of a distinct product risk in the result table. A stakeholder can fill in the following values: M (Must Test); S (Should Test); C (Could Test); W (Wont Test). The tables below contain a column with quality attributes according to ISO9126. As defined in GPL the 6 quality characteristics of ISO9126 are used. For information also the sub characteristics are shown but they are not required.

Appendix B
ID 1.1 1.2 1.3 1.4 1.5 1.6

Example checklist product risks: Users

Quality Attribute Functionality Functionality Usability Usability Usability Usability
Subcharacteristic (ISO9126) Accuracy Suitability Operability Understandability Learnability Understandability

Product Risk Indication Must the information system check the entered data? Must the information system check the relationship between different entered fields? Are the end-users familiar with the user interface of the (new) information system? Are the system messages (e.g. error messages) clear and helpful? Are help files and/or online instructions available? Is the system menu and/or navigation structure clear and logical?
GPL Team 78471528.doc

Priority (MoSCoW)

Page 13 of 16

Internal use only 1 June 2005

GPL Product Risk Analysis (Related to an information system) Guidance Manual ID 1.7 1.8 1.9 1.10 1.11 1.12 1.13 1.14 1.15 1.16 1.17 1.18 1.19 1.20 1.21 1.22 1.23 1.24 1.25 1.26 1.27 1.28 1.29 1.30 1.31 1.32 1.33 1.34 Product Risk Indication Are all windows and fields clear, logical and self-evident? Are all views, summaries and reports clear, logical and self-evident? Do all views, summaries and reports contain the relevant and necessary information? Is the user manual clear, logical and complete? Are there interfaces with other information systems and databases? Are there alternatives for processing information when the information system is not available? Must the information system be available all the time? Must the information system be available in different languages? Is there a peak load for the information system during the day (a stress moment)? Are there maximum processing and response times required for the information system? Are there many concurrent users? Can end-users customise the information system to personal wishes or demands? Is the information system consistent with existing standards and procedures (organisation policies)? Is the information system used for processing financial information? Are the information and/or available data confidential? Is there a critical relationship between the information system and the physical environment? Is the information system critical or non-critical for the business processes? Are the necessary business procedures stable? Can the information system affect work of other departments? How many users will work simultaneously with the information system? How many different user roles are available for the information system? (Different authorisation) Is the information system an online or batch process (or a combination of both)? Is the information system necessary for the complete business process? Is management information relevant output of the information system? Are the end-users familiar with the use of information systems for their daily work? How much of the existing functionality has been changed compared to the previous version of the information system? Is the security or maintenance of historical information important? Are there complex calculations available in the information system?
GPL Team 78471528.doc

Quality Attribute Usability Usability Functionality Usability Functionality Reliability Reliability Usability Efficiency Efficiency Efficiency Usability Functionality Functionality Functionality Reliability Reliability Functionality Portability Efficiency Functionality Usability Reliability Functionality Usability Maintainability Functionality Functionality

Subcharacteristic (ISO9126) Attractiveness Understandability Suitability Learnability Interoperability Recoverability Maturity Operability Time behaviour Time behaviour Resource utilisation Usability compliance Functionality compliance Accuracy Security Maturity Maturity Suitability Adaptability Resource utilisation Suitability Operability Recoverability Accuracy Learnability

Priority (MoSCoW)

Changeability Functionality compliance Accuracy

Page 14 of 16

Internal use only 1 June 2005

GPL Product Risk Analysis (Related to an information system) Guidance Manual ID 1.35 1.36 1.37 1.38 1.39 Product Risk Indication Can other information systems have a negative influence on the operation of the information system? Is the processing time of entered data critical for the business process? Is the working status of the information system always clear? Is the new information system a replacement of an existing information system? Can the system functions be operated in different ways? Quality Attribute Reliability Efficiency Usability Functionality Usability
Subcharacteristic (ISO9126) Fault tolerance Time behaviour Understandability Suitability Operability

Priority (MoSCoW)

Appendix C
ID 2.1 2.2 2.3 2.4 2.5 2.6

Example checklist product risks: Marketing

Quality Attribute Usability Portability Functionality Portability Functionality Maintainability
Subcharacteristic (ISO9126) Operability Adaptability Functionality compliance Installability Interoperability Changeability

Product Risk Indication Is the information system used for sales purposes? Must the information system be compliant for different operating system and/or technical platforms? Is the information system compliant with all privacy legislation? Will the installation be performed by the end-users? Is the information system geographical distributed and used? Is time-to-market a important success factor for the information system?

Priority (MoSCoW)

Appendix D
ID 3.1 3.2 3.3 3.4

Example checklist product risks: IT Production & infrastructure

Quality Attribute Maintainability Usability Maintainability Maintainability
Subcharacteristic (ISO9126) Analysability Operability Analysability Analysability

Product Risk Indication Is the helpdesk able to report on the status of the information system? Is there a System Administration Guide to operate the information system? Is the information system an online or batch process (or a combination of both)? Is there an error report for the batch functions of the information system?

Priority (MoSCoW)

Appendix E
ID 4.1 4.2 4.3

Example checklist product risks: IT Development & Maintenance

Quality Attribute Usability Reliability Reliability
Subcharacteristic (ISO9126) Understandability Fault tolerance Recoverability

Product Risk Indication Is there a brief functional manual or functional description to explain the information system? Are backups made of the different releases and data of the information system? In case of a malfunction or breakdown how fast must the information system be corrected and restored?
GPL Team 78471528.doc

Priority (MoSCoW)

Page 15 of 16

Internal use only 1 June 2005

GPL Product Risk Analysis (Related to an information system) Guidance Manual ID 4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11 4.12 4.13 4.14 4.15 4.16 4.17 4.18 Product Risk Indication Is there an alternative in case the hardware of the information system breaks down? Is the information system (solution) new technology or proven technology? How handles or restores the information system a failed or interrupted transaction or procedure? Are contingency plans for system failures available? Do changes of the settings of client PC configuration have influences (affect) on the information system? Is data of other system stored (saved) in the information system? Must the information system operate on the existing technical platform and infrastructure? Will there be future technical modifications to the infrastructure? Are the Create, Read, Update and Delete (CRUD) functions implemented for all entities? Are dependencies and relationships checked when deleting an entity? Are components of other systems of third parties used by the information system? Is there a fall back scenario in case of problems during implementation? Is there a maximum time to run a batch? Do changes to the information system introduce many defects? Is there a maximum time to problem solving in case of a system error? Quality Attribute Reliability Reliability Reliability Reliability Usability Functionality Portability Portability Maintainability Functionality Functionality Reliability Efficiency Maintainability Reliability
Subcharacteristic (ISO9126) Fault tolerance Maturity Recoverability Recoverability Usability compliance Interoperability Co-existence Adaptability Testability Accuracy Suitability Fault tolerance Time behaviour Stability Recoverability

Priority (MoSCoW)

Appendix F
ID 5.1 5.2 5.3 5.4 5.5 5.6 5.7

Example checklist product risks: Security department

Quality Attribute Functionality Functionality Functionality Maintainability Functionality Functionality Maintainability
Subcharacteristic (ISO9126) Security Security Security Maintainability compliance Security Security Analysability

Product Risk Indication Are there procedures and business rules for authorisation and registration of different users? Is the information system secured with passwords? Is use by and registration of users logged? Is the information system in accordance with ING information risk standards and policy? Are there interfaces with systems outside the organisation? Are transactions and messages encrypted? Is an audit trail obligated for the information system?

Priority (MoSCoW)

GPL Team 78471528.doc

Page 16 of 16

Internal use only 1 June 2005