Software Quality Assurance

Lecture 1

MS-SE

M Zeeshan Asaf

1
Quality expectations

• Our (as developers) objective is to deliver software

system that...

• does what it is supposed to do

• needs to be “validated“

• does the things correctly

• needs to be “verified“

• show/demonstrate/prove it
• modeling/analysis needed

2
Meeting Quality Expectations

• Difficulties in achieving good quality:

• size: MLOC products common
• complexity
• environmental stress/constraints
• flexibility/adaptability expected

• Other difficulties/factors:
• product type
• cost and market conditions
• Others .. (discussed in Part III of the book)

• No silver bullet", but...

• SQE (software quality engineering) helps

3
 Overview
• MEETING PEOPLE’S QUALITY EXPECTATIONS
• The software systems must do what they are
supposed to do. In other words, they must do the
right things.
• They must perform these specific tasks correctly or
satisfactorily. In other words, they must do the
things right.

4
Overview
• Main tasks for software quality engineering
• quality planning;
• execution of selected QA or software validation
and verification activities;
• measurement and analysis to provide
convincing evidence to demonstrate software
quality to all parties involved.

5
An Initial Definition of (Software) Quality
•Quality =
• ...meeting the customer’s requirements,
• ...at the agreed cost,
• ...within the agreed timescales.
•Quality = “Fitness for purpose”
•Quality = Customer satisfaction

6
Standard Definitions of (Software) Quality
• IEEE Glossary: Degree to which a system, component, or
process meets (1) specified requirements, and (2)
customer or user needs or expectations

• ISO 8402: The totality of features and

characteristics of a product or service that bear
on its ability to satisfy specified or implied needs

7
 Quality Problems in Large Software

• Pervasive use of software

• Growing reliance on software
• Large software systems

8
Software Failures: The Statistics
Cancelled
31%

Successful
16%

Challenged
53%

9
Software Failures: Cost Overruns

Overrun Amount % cases

Under 20% 15.5%

21 - 50% 31.5%
51 - 100% 29.6%
101 - 200% 10.2%
201 - 400% 8.8%
Over 400% 4.4%

Average cost for “failed” projects is 189% of the original

estimate, i.e. almost double.
10
 Software Failures: Time Overruns

Overrun Amount % cases

Under 20% 13.9%

21 - 50% 18.3%
51 - 100% 20.0%
101 - 200% 35.5%
201 - 400% 11.2%
Over 400% 1.1%

Average time for “failed” projects is 222% of original estimate, i.e.

more than double.
11
 Software Failures: Content Deficiencies

Under 25% 4.6%

25 – 49% 27.2%
50 – 74% 21.8%
75 – 99% 39.1%
100% 7.3%
On average “challenged” projects deliver 61% of specified functions
12
Turning Failure to Success: SQE

Failure Failure

Success

Content
Failure

13
SQE for Meeting Quality Expectations

•Expectations for quality and reliability

• Validation: software systems must do what
they are supposed to do
• Verification: must perform these tasks
correctly, without problems
• Measurement/analysis: supported by evidence
based on concrete measurements

14
The SQE Process
1. Quality planning: plans assure that
– Software development, evaluation, and acceptance standards are developed,
documented and followed
– The results of quality reviews are given to appropriate management
– That test results adhere to acceptance standards
2. Software testing as the primary means to ensure
quality
3. Other alternatives for quality assurance (QA) e.g.
defect prevention, process improvement, inspection,
formal verification, fault tolerance, safety assurance,
and damage control
4. Measurement and analysis to close the feedback loop
for assessment and quantifiable improvement

15
 The SQE Process
• This is the general subject of Software Quality
Engineering, which is organized into three Major topics:
• Software testing as a primary means to ensure software
quality;
• Other alternatives for quality assurance (QA), including
defect prevention, process improvement, inspection,
formal verification, fault tolerance, safety assurance, and
damage control;
• Measurement and analysis to close the feedback loop for
quality assessment and quantifiable improvement.

16
Scope of Major SQE Activities

Software Quality Engineering

Quality Assurance

Testing

17
 Software Failures: Some Famous Examples

• Taurus (London Stock Exchange)

• 11 years late & 13,200% over budget
• London Ambulance Service
• faults led to reversion to manual processes
• NHS Service
• 2008 was the start date, still not finished!

18