NON FUNCTIONAL REQUIREMENTS WHITE PAPER/GUIDEBOOK

The Guide to Analysis and Design of Non-functional requirements

This paper was submitted for eWORLDFORUM 2011 conference

Page 1

TABLE OF CONTENTS
Introduction ........................................................................................ 1 Effectiveness ....................................................................................... 2 Efficiency .............................................................................................. 3 Disaster Recovery and High Availability ................................. 4 Maintainability ................................................................................... 5 Reliability.............................................................................................. 6 Availability ........................................................................................... 7 Scalability ............................................................................................. 8 Performance and Time Response............................................... 9 Compatibility / Interoperability .............................................. 10 Usability ............................................................................................. 11 Documentation ................................................................................ 12 Robustness ........................................................................................ 13 Conclusion ......................................................................................... 14 Useful information ......................................................................... 14

This paper was submitted for eWORLDFORUM 2011 conference

Page 2

Non-functional Requirement white paper

INTRODUCTION
80% of the requirements for success of a project are Non-functional requirements; the remainder is the specification of functional requirements. Why is it that analysts spend the more than 90% of their time documenting functional requirements at the cost of the non-functional? It is ironic that for a system to be fully functional, it is dependent on non-functional requirements. There are two components to non-functional requirements; the hard and the soft capabilities. Given some of the focus of Business Analyst training, guidelines and methodology, non-functional requirements are seldom well documented. I have put together this short, concise guide booklet to serve as a entry-point for business analysis. Since system projects have a high dependence on technology systems, incorrect specification of non-functional requirements have a huge impact on solution success.

Purpose of Document
Guide books by nature are never fully comprehensive; it is not a definitive document. This booklet serves as a pragmatic direction tool, a compass. It helps analysts navigate. Since each project is a new distinct journey, the white paper serves as a checklist and a reference tool. The final tweaks and alignment in a project depends on the skill of the analyst. As the analyst continues to use this tool, individuals skills will be honed. More use means more competence. To ensure ongoing learning it is imperative that this booklet evolves as the team becomes more adept with nonfunctional requirements definition and testing. This document is not an original piece of work. Major portions are extracted from various sources. It was always the intention to produce a piece of work, without over-engineering the research process, to provide quick high-value benefit at the least cost and time. This end result serves that purpose well.

Document Audience
Business Analysts, Project Managers, Architects and Developers, Change Managers and Testing personnel will find this document useful during conceptual definition, project analysis and design, build and deployment stage. While this document was intended for the growth and development of the Business Analysts, it is appropriate for other roles as well.

1|Page

Non-functional Requirement white paper

EFFECTIVENESS
Definition
Efficiency is about process - the use of resources and energy such as people, technology, production, waste management and energy management, including output and productivity. Effectiveness is about outcome, impact and quality how well the solution addresses the problem, how well the product works, how well the person fulfils their role, including cost effectiveness and value for money. Effectiveness focuses primarily on doing only what will bring about a goal. If a plan includes an action that would entail a waste of resources in which the desired goal is not possible to be achieved, then the theory of effectiveness would eradicate

Example 2:
Rolls-Royce value effectiveness over efficiency: to them, quality is everything, and they sacrifice efficiency to achieve it.

EFFICIENCY means: saving TIME, MONEY or EFFORT EFFECTIVENESS means how well the job gets done. i.e. the quality of the output

Focus Areas
Production (maximising productivity) Process extent to which the process is effective in producing the desired outcomes. Operations in order to support the system, to what extent is the business and support operations capable and competent to deliver adequate capacity to run and maintain the processes and infrastructure.

Example 1:
A company could spend millions of dollars on commercials and billboards to get a product out there - being effective but not efficient. That same company could make a silly viral video and "leak" it for free on the internet or any number of guerilla marketing tactics. That would be efficient, but may not be totally effective.

2|Page

Non-functional Requirement white paper

EFFICIENCY
Definition
Efficiency is a measure of time, cost and effort. Measures of an efficient information system include its productivity, processing time, operational costs and level of automations.

Focus Areas
Measurement of efficiency
In order to determine the efficiency of a system, metrics should be in place. Metrics are detailed measures that feed Key Performance Indicators (KPI). Efficiency IT metrics measure the performance of an IT system. An effective IT metrics program should measure many aspects of performance including throughput, speed, and availability of the system. There should also be an organized way of documenting and reporting the findings of efficiency IT metrics. Although measuring efficiency of an IT system is important for evaluating and improving performance of an IT system, it is also important to make sure that an IT system is being utilized in a proper way to ensure effectiveness of business processes and is also in line with business objectives.

Ease to use
To help the organization to increase efficiency and reduce costs the system solution that is chosen should easily intergrate with business processes and provide an easy to use solution

Level of support
The level of support determines how efficient is the system. Support includes both people and system. Some systems can be fully automated but complicated in such a way that staff members needs added support . Such a system cannot be classified as efficient. Also if the system itself required a lot of support being it physical or monetary it is not efficient.

3|Page

Non-functional Requirement white paper

DISASTER RECOVERY AND HIGH AVAILABILITY

Definition
The DR Plan indicates the exact steps to be taken by business to recover from any type of disaster. The plan would encapsulate relocating employees to getting a copy of your data restored immediately on a fully functioning server located off site in order to continue normal business operations. A High Availability (HA) solution must provide uninterruptible service (24/7). Each HA solution may include identical pairs of servers, firewalls, switches, routers, etc. If one of these hardware devices fails, the backup unit must seamlessly kick-in and service must not be interrupted. This process is known as failover.

Responsibilities Organizational practices and standards Disaster recovery drills System backup and restore procedures Data and system level recovery Alternative practices and fail-over plans

Focus areas
SLA defining explicit uptime and performance levels and specific remittance for under compliance o Mean time to recovery i.e. basic and full recovery o Mean time between failures o Required availability percentage i.e. how much of outage time can be tolerated over a period of time o Planned restrictions schedule i.e. how often and for how long planned maintenance activity can interfere with full, unencumbered availability. o DR Mode performance tolerance i.e. how much performance degradation the business can deal with and for how long after failing over to a secondary DR site. o Max and min decision time i.e. how much time occurs before deciding on the recovery solution for the crisis situation Criticality for HA o Mission critical e.g. internet, POS, support services o Business critical e.g. payroll, procurement o Business critical Business intelligence e.g. supply chain intelligence

4|Page

Non-functional Requirement white paper

MAINTAINABILITY
Definition
A characteristic of design and installation, expressed as the probability that an item will be retained in or restored to a specified condition within a given period of time, when the maintenance is performed in accordance with prescribed procedures and resources. Maintainability is a design parameter and it is used to: correct defects meet new requirements make future maintenance easier, or cope with a changed environment;

Lack of experience (People who did the work have left SBSA due to being consultants etc). Expensive maintainability is not part of the budget, and therefore not carried out. Design flaw. The original project scope paid little or no attention to including maintainability as a factor. Meaning that project stakeholders overlook it.

Maintainability is about a product's ability to stay the same or to adapt to change and is a compared-towhat question.

Focus Areas
Maintainability is the ability of an item to be maintained, whereas maintenance constitutes a series of actions necessary to ensure an item or items is restored or retained in an effective operational state. Key aspects of maintainability include updating documentation pertaining to a product or service, through making necessary corrections and enhancements to documents as changes are made. It also includes continuous testing of item/s ensuring that maintainability occurs.

SBSA specific Issues

Maintainability issues within SBSA include (but are not limited to) the following: Work is never updated and a gap begins to emerge. Documents become less and less useful as it falls out of sync with the product. Within the SBSA environment this is most clearly seen with regards to FSSs. The product changes but the specification describing the product is still describing the very first version of the project.

5|Page

Non-functional Requirement white paper

RELIABILITY
Definition
Reliability pertains to how hardware, software and operators consistently perform or under perform under different scenarios and situations. Reliability is divided into 3 categories

Hardware reliability
Is the probability of a hardware component failing?

Testing metrics must take two approaches to comprehensively evaluate the reliability. The first approach is the evaluation of the test plan, ensuring that the system contains the functionality specified in the requirements. This activity should reduce the number of errors due to lack of expected functionality. The second approach, one commonly associated with reliability, is the evaluation of the number of errors in the code and rate of finding/fixing them.

Specific SBSA Issues

No or poorly defined non functional requirements pertaining to reliability Lack of adequate redundancy and recovery mechanisms result in failures with serious consequences Insufficient focus placed on adopting good design principles due to project delivery time pressures Lack of validation of reliability NFR, after project has been implemented to ensure NFR has been met.

Software reliability
Is the probability that the software system will function properly without failure over a certain time period.

Operator reliability
Is the probability that a system user (or operations support) will make an error

Focus Areas
Thorough testing Extensive training of users Investment in high quality products Follow good design principles

Hardware Reliability Metrics

Hardware components can wear out. This could result in failure. The intent is to measure the time that the component is operating (uptime) before it fails i.e. Mean Time Between Failures (MTBF). This will assist in proactive maintenance schedules, ensuring maximum uptime.

Software Reliability Metrics

Design and Code Reliability Metrics It is generally accepted that more complex modules are more difficult to understand and have a higher probability of defects than less complex modules. Design and code metrics are quite technical e.g. Weighted Methods per Class (WMC) is a predictor of how much time and effort is required to develop and maintain the class. The higher the WMC, the more testing is necessary.

Testing Reliability Metrics 6|Page

Non-functional Requirement white paper

AVAILABILITY
Definition
Availability is the proportion of time a system is in a functioning condition. The ratio of (a) the total time a functional unit is capable of being used during a given interval to (b) the length of the interval (eg. 99.5%) Availability indicates when a system is operational as well as how reliable it is during operational periods.

Budget available to operate the system

Budget approved by management to operate the system plays a major role to determine the parameters to be decided for Availability. 365x24x7 costs dearly!

SBSA Specific issues

1. 2. System unavailability- Some systems not available when there are people in place to operate them. Vendor support unavailability -Vendor may not provide support to keep the system available according to the parameters defined in other Non Functional Requirements such as Hours/days of operation, Reliability, speed etc. Systems operate on a 24/7 basis- Some internal facing systems may only be needed when there are people in place to operate them. Slow response times during operating timesSystems response times affect service to customers, staff morale and work throughput.

Focus area
Hours of operation
Length of the interval for which the system should run .What are the hours that a given system will be available?

3.

Days of operation

What days will the system be operational? Not all systems operate on a 24/7 basis. Some internal facing systems may only be needed when there are people in place to operate them.

4.

Reliability

Is the consistency of your measurement, or the degree to which a System measures the same way each time. Focus area for NFR is during a system's hours of operation, what reliability is needed? The higher the number, the greater is the cost.

Cost of running system

An amount paid for making the system available for the parameters defined in other Non Functional Requirements such as Hours/days of operation, Reliability, speed etc.

Internal Support required operating the system What support will be given to keep the

system available according to the Hours/days of operation and reliability parameters defined

Vendor Support required operating the system (If system is outsourced)

What support will be given by Vendor/Outsourcing Company to keep the system available?

Maximum acceptable time for starting the system

What will be the turnaround time for a system to start up?

7|Page

Non-functional Requirement white paper

SCALABILITY
Definition
Scalability is a desirable property of a system, a network, or a process, which indicates its ability to either handle growing amounts of work in a graceful manner or to be enlarged.

To scale vertically (or scale up) means to add resources to a single node in a system, typically involving the addition of CPUs or memory to a single computer.

Scale horizontally (scale out)

To scale horizontally (or scale out) means to add more nodes to a system, such as adding a new computer to a distributed software application. An example might be scaling out from one web server system to three.

Focus areas
The concept of scalability applies to technology as well as business settings. The base concept is consistent - The ability for a business or technology to accept increased volume without impacting the contribution margin (= revenue - variable costs). For example, a given piece of equipment may have capacity from 1-1000 users, and beyond 1000 users, additional equipment is needed or performance will decline (variable costs will increase and reduce contribution margin). Scalability can be measured in various dimensions, such as:

Load scalability

It is the ability for a distributed system to easily expand and contract its resource pool to accommodate heavier or lighter loads. Alternatively, the ease with which a system or component can be modified, added, or removed, to accommodate changing load.

Geographic scalability

It is the ability to maintain performance, usefulness, or usability regardless of expansion from concentration in a local area to a more distributed geographic pattern.

Administrative scalability

It is the ability for an increasing number of organizations to easily share a single distributed system.

Functional scalability:

It is the ability to enhance the system by adding new functionality at minimal effort.

Scale vertically (scale up)

8|Page

Non-functional Requirement white paper

PERFORMANCE AND TIME RESPONSE

Definition
Traditionally, response time is often defined as the interval from when a user initiates a request to the instant at which the first part of the response is received at by the application. However, such a definition is not usually appropriate within a performance related application requirement specification.

Performance tests, Network sensitivity tests, Volume tests (Sociability sensitivity Tests, Tuning Cycle Tests, Protocol Tests, Thick Client Application Tests, Thin Client Application Tests)

Application

Key Concepts
The definition of response time must incorporate the behaviour, design and architecture of the system under test. Time to display order details Average time to display order details less than 5.0 seconds. 90th percentile time to display order details less than 7.0 seconds. The above specification, or response time service level agreement, is a reasonably tight specification that is easy to validate against. The following form part and parcel of the performance and response time testing

Types of tests: Loads tests, Failover tests, Soak tests, Stress tests, Targeted infrastructure tests,

9|Page

Non-functional Requirement white paper

COMPATIBILITY / INTEROPERABILITY
Description:
If products designed for the new standard can receive, read, view or play older standards or formats, then the product is said to be backward-compatible. The reverse is forward compatibility, which implies that old devices allow (or are expected to allow) data formats generated by new (or future) devices, perhaps without supporting all new features. A standard supports forward compatibility if older product versions can receive, read, view or play the new standard.

Key Concepts:
These requirements address the need for the product to interface with other applications or systems without interfering with the operation of those other applications or systems. They may consider or specify: Named applications or systems with which to interface Messaging format, medium, and content Compatibility with products from different vendors Conversion requirements for hardware, applications, or data Middleware architecture Messaging and transmission protocols Frequency Multiple time zones Process timing and sequencing

10 | P a g e

Non-functional Requirement white paper

USABILITY
Definition
Usability is generally acknowledged as a factor of system quality representing the answer to many interactions with technology. It describes the quality of products and systems from the point of view of humans who use them. Usability is crucial to create websites that your customers will want to return to. If they cant use the site, they wont stay.

Help and documentation

Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and be concise.

Training
Users need to be trained before they can start using the system and be provided with training manuals which they can always refer to when using the system.

SBSA specific issues

1. Page Layout and Content-If customers don't find relevant content quickly; they will be more likely to leave. Keep your pages clean and simple. Try removing elements, and see if your page needs them, if the page functions without them - take them out. Download speed-After 10 seconds; your customer has lost interest in your page, no matter how interested they were in the topic. Advanced Technology Usage-The best solution is to avoid beta-level technology until it has been in use for at least one year.

Focus Area
Visibility of system status
The system should always keep users informed about what is going on, through appropriate feedback within reasonable time. Probably the two most important things that users need to know at your site are "Where am I?" and "Where can I go next?" Instructions for use of the system should be visible or easily retrievable.

2.

3.

Match between system and the real world

The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

User control and freedom

Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue.

Consistency and standards

Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.

Error prevention

Error message is a careful design which prevents a problem from occurring in the first place. Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

11 | P a g e

Non-functional Requirement white paper

DOCUMENTATION
Definition
Non Functional requirements documentation is the communicable material that details the system architecture. It explains how the system is supposed to be rather than do. It also references documentation that forms part of the development which contains the all the impacted areas of the system development and implementation. The questions that one should ask is what kind of documentation are required and what audience is to be addressed by each document.

Use Case Model

Use Case Documents have a specific section for nonfunctional requirements. These should normally apply specifically to the use case in question. It may be, however, that requirements that apply across more than one use case have been included. If so, add the requirements to the non-functional requirements document and reference them in the use case document.

SBSA Specific Issue

1. Unavailability of SBSA structural information, such as infrastructure architecture and organization structure makes it difficult to do impact analysis. 2. Standard methodology that should be enforced and implemented to ensure elimination of loop hole.

Focus Areas
Project Overview
Project Overview contains a summary of known major technical constraints. Add these to the non-functional requirements document and update the project overview with references to them.

Stakeholder Needs List

These are single line statements of needs gather early in the project from all stakeholders. Scan through the list and look for those needs that would not have been included in the use case documents. Add them to the appropriate section of the non-functional requirements documents as single sentence statements. Update the stakeholder needs list with the reference to the requirement in the non-functional requirements document.

Business Process Model

When the business process model is created notes may have been added to the processes as attachment to activities on activity diagrams regarding stakeholder requirements as they have been mentioned by the stakeholder representative or process owner. Hint: Search all relevant activities for notes relating to non-functional requirements and add them to the non-functional requirements document. Update the activity notes with the reference to the requirement in the non-functional requirements document.

12 | P a g e

Non-functional Requirement white paper

ROBUSTNESS
Definition
Robustness is the quality of being able to withstand stresses, pressures, or changes in procedure or circumstance. A system or design may be said to be "robust" if it is capable of coping well with variations (sometimes unpredictable variations) in its operating environment with minimal damage, alteration or loss of functionality. Robustness is defined as the degree to which a system operates correctly in the presence of exceptional inputs or stressful environmental conditions Robustness testing is a testing methodology to detect vulnerabilities of a component under unexpected inputs or in a stressful environment.

Focus Areas
"Happy-path testing" (that is, testing that is only meant to show that the system meets its functional requirements). While testing to ensure that requirements are met is necessary, often tests aimed at ensuring that the system handles errors and failures appropriately are neglected Robustness testing is known by many different names. Many of these are equivalent, and some are used to define a specific type of robustness testing. Many of these terms are defined below.

Adhoc testing: a testing phase where the tester tries to break the system by randomly trying the systems functionality. This can include negative testing. Boundary value analysis/testing: an analysis that focuses on corner cases or values that are usually out of range as defined by the specification. This means that if a function expects all values in the range of negative 100 to positive 1000, test inputs would include negative 101 and positive 1001. Compatibility testing: testing whether software is compatible with other elements of a system with which it should operate (e.g., browsers, operating systems, or hardware) Endurance testing: checks for memory leaks or other problems that may occur with prolonged execution End-to-end testing: testing a complete application environment in a situation that mimics real-world use (such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems, if appropriate) Exhaustive testing: testing that covers all combinations of input values and preconditions for an element of the software under test Exploratory testing: Exploratory testing has several uses: the one that is germane to robustness testing is to provide rapid feedback on a products quality on short notice, with little time, without advance thought, or early in development when the system may not be stable. Negative testing: testing aimed at showing that software does not work. This is also known as test to fail or dirty testing.

13 | P a g e

Non-functional Requirement white paper

CONCLUSION
Non-functional requirements definition and design done early in the SDLC pre-empts flaws and inherent constraints in the business and technical systems. With careful consideration, looking at the various aspects this whitepaper will ensure that a more comprehensive solution is created. This document will guide Business Analysts through the SDLC process indicating NF touch points, roles, responsibilities, artefacts and deliverables. As the team grows, and the skill of the team grows, this document will evolve and provide a useful tool for the work of Business Analysts.

USEFUL INFORMATION
The following list provides some details that supplement the categories listed in this whitepaper. Accessibility Auditability Back-up and store Compliance Data Purification Escrow Flexibility/customizability Legislation that requires conformance Modifiability Operability Portability Quality (e.g. faults discovered, faults delivered, fault removal efficacy) Resilience Safety Supplier contracts and SLAs Archiving and backup strategy Availability (see service level agreement) Capacity, current and forecast Configuration management Content and Data Retention/Purging Extensibility (adding features, and carry-forward of customizations at next major version upgrade) Global/Geographical reach Legal and licensing issues or patentinfringement-avoidability Network topology Physical environment Price Recovery / recoverability (e.g. mean time to recovery - MTTR) Resource constraints (processor speed, memory, disk space, network bandwidth etc) Security Technical or organizational standards compliance Audit and control Backup Certification Conversion Dependency on other parties Failure management Integrity Longevity Open source Platform compatibility Privacy Reliability (e.g. mean time between failures - MTBF) Resources and Management issues Software, tools, standards etc. Testability

14 | P a g e