Test Destiny The Psyche of A Professional Tester

March 31 – April 2, 2009
Optimizing the Testing

Effort with Keyword-
Driven Frameworks
Brian Massey, IBM
Quick Bio
Product Manager
No worries – not here to sale you anything (but to ensure the
kids get fed I have to mention them)
As part of Rational Services organization helped develop and

deploy frameworks for our customers
Established Software Quality Centers of Excellence for Companies
Experience as Developer, Tester, Architect, Dev Manager, QA

Manager
Used to build Air Defense Systems for a living

Developed and tested embedded systems
Father of 6 so prepared for anything

STPCon Spring 2009
IBM Software Group Rational software
Market Impact on Business Innovation

Delicate balance for CIOs
Market Pressures
IBM CEO studies of 2006 & 2008
Tremendous drive for business transformation and change
Increasing gap between expectation of change & organization’s
history to manage it
Recent global “Secular Bear Market” may take years to
recover
Survival means drastic changes in expenditures
Advice
Manage what you can control – economy is not one of
them
Spending – where it counts for long term growth
Growth & Earnings Assumptions – don’t trust your models
Focus on quality
Reduce debt
Sources: Sequoia Capital Report Oct 2008 & IBM CEO Study May 2008
The Psyche of A Professional Tester ®

STPCon Spring 2009
Who are the testers in your organization?

f Does your test organization lack clout?
f Why are testers perceived negatively?
f Are testers junior level people -- hired as a tester as they
work their way up in the organization?
f Is the skill set for the art we love lower than it should be?
f When the quality people perform the quality job – is it only
temporary?
f Do testers get the respect and training needed?

STPCon Spring 2009
Who are the testers in your organization?
Why is it that a company will invest millions

in building an application but then does not
invest adequately in the last line of defense
to delivering a quality application?

STPCon Spring 2009
Application Failure - The Costs

f Typically 5 to 15 flaws in every 1,000 lines of code
f Dealing with defects: (DOD & SEI)
Finding each bug takes up about 75 minutes
Fixing bugs takes two to nine hours per bug
On the outside, that’s ~150 hours,or roughly $30,000, to cleanse
every 1,000 lines of code
f Software errors cost U.S. economy $60 billion annually (0.6% of
the gross domestic product) (NIST)
f $22.2 billion could be eliminated with more effective identification
(NIST)
f 40% of unplanned downtime is due to application failure (Gartner)

STPCon Spring 2009
The Quality Zone

There is a new dimension beyond that which is known to
software development. It is the middle ground between
deployment and development, between art and science, and
it lies between the pit of developments fears, and the summit
of their knowledge. This is the dimension of quality. It is an
area which we call ...

STPCon Spring 2009
Testing in the “Quality Zone”
f Identified “typical” profiles from data gathered.

f These profiles identify contrasting types of testers (the
good, the bad and the ugly).
f Just because there is a bit of truth to these doesn’t mean
we can’t have a little fun.
f The “Quality Zone” while stretching the imagination has a
firm hand in reality.

STPCon Spring 2009
Tester Personality Profiles -- Junior Tester

f Description
Entry level/Junior tester
Duh Oh,
f The Good Found a
Can follow well written manual scripts defect
Rarely deviates from scripts
Happy doing mundane tasks
May have lots of enthusiasm
Often believes they are making a difference
f The Bad and Ugly
May have trouble writing scripts
Rarely deviates from scripts
Has no power or authority
Not very technical
Very little expected of them

STPCon Spring 2009
Tester Profiles – Coop Tester
Big Mac Inc.

Will Test for Happy Meals

STPCon Spring 2009
Tester Profiles – Coop Tester

f Description
Temporary tester
f The Good
May be highly trained (college coop)
Often tries to make a difference
Tells it like it is (only temporary)
May want to move up in organization
f The Bad and Ugly
Usually only temporary
Not very well trained (unskilled)
Often filling a role because the thought is “Anyone can test”
Not well respected (contractor and a tester)

STPCon Spring 2009
Tester Profiles – Techno Geek/Caffeine Freak Tester

f Description
Developers or other high strung individuals
f The Good
Finds more bugs than anyone
Solves the difficult problems
Runs a lot of tests
Can write very “creative” scripts
Results are often solution oriented
f The Bad and Ugly
Probably won’t be able to replicate
Defect reports are very “creative”
Usually does not follow scripts (finds
more interesting things)
Test because they were told to

STPCon Spring 2009
Tester Personality Profiles – NASA Tester

Fail it – they
f Description misspelled their
comments
AKA -- Test Cop
f The Good
Loves to find and report defects
Very process driven
Empowered by management
Loves their job
Intelligent and systematic
f The Bad and Ugly
The person developers love to hate
Bad perception by others -- – any
process deviation could get you shot
Usually detail oriented and standards
driven

STPCon Spring 2009
Tester Personality Profiles – Apathetic Tester

f Description
Yep – It’s
Experienced tester who has fought many battles corn
(losing most)
f The Good
Very methodical
Writes great, highly reproducible bug
reports
Often has a wealth of untapped knowledge
Could get great benefit from implementing
their ideas
f The Bad and Ugly
Has their own ways of doing things
Nothing is really high priority
Will resist change (especially automation)

STPCon Spring 2009
The Facts (Our Data)
f Developed questionnaire to obtain data set

f Surveyed individuals from approximately 15 companies
f Surveyed Managers, Testers and Developers
f Followed up surveys with interviews

STPCon Spring 2009
Questions asked (from questionnaire)

f How the Software world perceives testing
f Why were “ineffective” testers ineffective
f Top attributes and attitudes that made testers effective
f How “effective” testers could improve
f How many testers they worked with in their career
f How many of those were “effective” testers
f Rank characteristics of testers in order of importance
f Training accomplished by the respondent

STPCon Spring 2009
How do you think the software engineering world

perceives testing?
f “Not altogether as important as it really is.”
f “People that are hired to point out flaws in product and
therefore diminish the perceived ability of the developer.”
f “I believe this is determined by the culture of each
company you ask. I’ve been at some places where
software engineering felt that the test organization was
completely useless and others where they thanked their
lucky stars that they had them.”

STPCon Spring 2009
How do you think the software engineering world

perceives testing?
f “Necessary evil or as a curse.”
f “Something that can be cut out when we run short on
time.”
f “Initially, they do not value testing until they have
something that blows up in production and then they
develop a great sense of the need for someone to check
behind them and give them that additional view point.”

STPCon Spring 2009
Results
f Those surveyed deemed that approximately 57% or less
of testers they worked with were effective
f Number ranged from 5% to 100% on individual responses
f Individuals with more experience reported 2/3 the
effectiveness of testers compared to those with less
experience (47% vs. 71%)
f Developers thought 63% testers effective
f QA thought 64% testers effective
f Managers thought 43% testers effective
f Perceived value of QA – QA 3.9, Dev 2.5, Management
4.3, Overall 3.9 (out of 10)

STPCon Spring 2009
Results
Henry, Joel and Bob Blasewitz. 1994. “Do we really need SQA to produce quality software? No! Well maybe.
It depends. Yes!” ACM Software Engineering Notes 19 (2): 63-64

STPCon Spring 2009
Why were testers ineffective?

f “Bored – didn’t like their job”
f “Couldn’t think independently”
f “Poor communication skills”
f “Submissive personality”
f “Tired of battling developers”
f “Unfamiliar with X (too many to list)”
f “Unable to use a computer”
f “Did not understand Y (too many to list)”
f “Didn’t use enough common sense”

STPCon Spring 2009
What makes a Tester effective (attitudes)?

f “Work on being part of ‘team’ ”
f “Remove personal preference from real problems”
f “Like to break things”
f “Strong desire (passion) for quality”
f “Ability to think about a problem/requirement from various
perspectives”
f “Enjoys a challenge”
f “Non-defeatist/problem solving attitude”
f “High degree of confidence”

STPCon Spring 2009
What makes a Tester effective (attributes)?

f “Understanding of software development life cycle”
f “Understand programming”
f “Good people skills”
f “Able to work with programmers to fix problems”
f “Analytical methodology (scientific)”
f “Problem solving skills”
f “Embrace new technology”
f “Becomes an expert on what is being tested”

STPCon Spring 2009
How can testers be more effective?

f “Keep up with latest technology”
f “Provide constructive criticism”
f “Don’t try to fix the problem yourself”
f “Don’t be afraid to go outside the lines (knowing when
they’re NOT done)”
f “Use IBM/Rational tools and process where appropriate”
f “Know when to be less skeptical”
f “Know when to be more skeptical”
f “Develop and fine tune intuition”

STPCon Spring 2009
Characteristics
f 23 Characteristics of Individuals
f Ranking was 1 to 23 (1 being the top characteristic)
f Lower the number the more important the characteristic
f The bottom indicates less important (not unimportant)
f Following charts show average of all characteristics with
breakdown of Development, Testing, and Management
perceptions

STPCon Spring 2009
Top Characteristics of Tester

Overall Developer QA Manager
Organized 4.4
1 Organized 7.7 Communication Skills 4.3
4.3 Problem Solver 8.1
8.1 Organized 4.4
Communication Understands Users Communication

2 Skills
8.1
Perspective
5.0 Independent 9.1
Skills
6.6
QA lacked conformity
3 Problem Solver 9.0 Organized 5.7 Out of the box thinker 9.4 Self Motivated 7.3
4 Development/Management
Self Motivated 9.3 Diplomatic had
6.3 clear winners
Communication Skills 9.7 Problem Solver 8.9
Understands Understands Users

5 Users Perspective
9.4 Self Motivated 7.0
Perspective
10.1 Intelligent 9.0
f Very diverse
Organized topon
attitude overall
what but ranks
not in top
at5the
for QA
top
f Diplomacy more important to developers

STPCon Spring 2009
Less desired characteristics (what was on the bottom)

Overall Developer QA Manager
19 Intuitive 12.9 Process Centric 15.3

15.3 Good People Person 13.4 Diplomatic 15.0
20 Diplomatic 15.2 Flexible 17.0 Diplomatic 17.3 Self Sufficient 16.4
Programming Effective
21 Experience
16.4 Effective Modeler 17.0 Detail Oriented 17.3
Modeler
18.1
Programming
22 Effective Modeler 18.3
Exp 19.7
19.7 Programming Exp 17.6
17.6 Independent 19.0
Detail
23 Detail Oriented 19.4
19.4 Detail Oriented 21.3 Effective Modeler 18.6
Oriented
22.5
Management ranked programming experience

f Veryhigher
Surprise than
-- Process
clear Testers or Developers
less(Effective
losers important toModeler & Detail Oriented)
developers
f QA/management want QA to tell it like it is (non-diplomatic)
Detail Oriented at the Bottom

STPCon Spring 2009
Want more data – Survey from Qaforums.com

f Suppose that you have a friend who has some experience
in software testing but is now unemployed. She has been
unemployed for long enough that she is willing to spend
some money taking courses and writing exams in order to
get a certificate.
Q1 -- Which type of certification do you think would MOST

likely help her land a testing job? (Up to 2 choices)
Q2 -- Which type of certification do you think would

LEAST likely help her land a testing job? (Up to 2 choices)

STPCon Spring 2009
Want more data – Results from Survey

f From general forum (Everyone)
Certifications Most Helpful Least Helpful
Programming 65% 13%
Networking 3% 58%
Testing 23% 39%
Test Tool 74% 6%
Project Management 16% 19%
f From very senior level testers

Rated network knowledge more valuable (very important for
load/stress testing and much other client/server testing)
Rated project mgmt knowledge more valuable
Rated the testing certificates as less valuable.

STPCon Spring 2009
So who is John Doe?

f Short lived Fox TV Show
f Had knowledge of literally everything in the world, yet no
memory of who -- or even what -- he is
f Mostly saw the world in black and white
f Had all the answers – except info on himself
f Helped cops solve crimes uses his special gift to help
them solve "impossible" crimes each week
f While watching the show I thought he would make a great
SW Tester

STPCon Spring 2009
Tester Personality Profiles -- John Doe
f Systematic and Ad Hoc

f Thorough
f Well trained
f Problem solver
f Loves his job
f Well respected by peers and organization
f Provides value to the project
f One of the top individuals on the project

STPCon Spring 2009
Tester Personality Profiles -- John Doe (as a Tester)

f John Doe loves to break things and is also capable
enough to determine why (knows everything)
f John Doe is a problem solver, methodical, and ad-hoc at
the same time (sees the world B/W and Color)
f John Doe leads the organization to obtain a higher
perception of quality
f John Doe takes responsibility for improving the testing
process
f John Doe uses special gifts to help developers find and
resolve defects

STPCon Spring 2009
Organizational Challenge
f What is your mission?

f Where does quality fit into your mission?
f What is quality?
f What is an error?
f What is the cost of an error?

STPCon Spring 2009
Organizational Challenge
f Why not do what other industries do

f Need to change SW industry mind set about quality
f Must have skilled professionals doing the quality job
f Have tools and roles that streamline all types of testing
f Do more than say “test it”
f Realize the value of quality software
f Build quality into the product
f Realize quality is function of people and process, not tools

STPCon Spring 2009
Professional Challenge
f We can fix the problem, we have the ability and
responsibility
f Need to justify our existence
f Realize the more we know the better we test
f Provide value to your project
f Get involved early – quality begins at requirements
f Help build quality into the product
f "Quality is not an act. It is a habit" -- Aristotle

STPCon Spring 2009
Professional Challenge
Stand up and say “Hello, I’m John Doe We can
and I am a Tester” change. Be
proud to
Will test for test.
food.
Before After

STPCon Spring 2009
The Quality Zone

There is a new dimension beyond that which is known to
software development. It is the middle ground between
deployment and development, between art and science, and
it lies between the pit of developments fears, and the summit
of their knowledge. This is the dimension of quality. It is an
area which we call ...

38
Thank you!
For more information:
Brian Massey
Senior Product Manager
IBM/Rational Software
678-248-4523
jbmassey@us.ibm.com
www.ibm.com/rational

Test Destiny The Psyche of A Professional Tester

Enviado por

Dados do documento

Descrição original:

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Test Destiny The Psyche of A Professional Tester

Enviado por

Direitos autorais:

Formatos disponíveis

March 31 – April 2, 2009

Optimizing the Testing

 As part of Rational Services organization helped develop and

 Established Software Quality Centers of Excellence for Companies

 Experience as Developer, Tester, Architect, Dev Manager, QA

 Used to build Air Defense Systems for a living

 Father of 6 so prepared for anything

Market Impact on Business Innovation

The Psyche of A Professional Tester ®

Who are the testers in your organization?

The Psyche of A Professional Tester ®

Who are the testers in your organization?

Why is it that a company will invest millions

The Psyche of A Professional Tester ®

Application Failure - The Costs

The Psyche of A Professional Tester ®

The Quality Zone

The Psyche of A Professional Tester ®

Testing in the “Quality Zone”

f Identified “typical” profiles from data gathered.

The Psyche of A Professional Tester ®

Tester Personality Profiles -- Junior Tester

The Psyche of A Professional Tester ®

Tester Profiles – Coop Tester

Big Mac Inc.

The Psyche of A Professional Tester ®

Tester Profiles – Coop Tester

The Psyche of A Professional Tester ®

Tester Profiles – Techno Geek/Caffeine Freak Tester

The Psyche of A Professional Tester ®

Tester Personality Profiles – NASA Tester

The Psyche of A Professional Tester ®

Tester Personality Profiles – Apathetic Tester

The Psyche of A Professional Tester ®

The Facts (Our Data)

f Developed questionnaire to obtain data set

The Psyche of A Professional Tester ®

Questions asked (from questionnaire)

The Psyche of A Professional Tester ®

How do you think the software engineering world

The Psyche of A Professional Tester ®

How do you think the software engineering world

The Psyche of A Professional Tester ®

The Psyche of A Professional Tester ®

The Psyche of A Professional Tester ®

Why were testers ineffective?

The Psyche of A Professional Tester ®

What makes a Tester effective (attitudes)?

The Psyche of A Professional Tester ®

What makes a Tester effective (attributes)?

The Psyche of A Professional Tester ®

How can testers be more effective?

The Psyche of A Professional Tester ®

The Psyche of A Professional Tester ®

Top Characteristics of Tester

Communication Understands Users Communication

Understands Understands Users

The Psyche of A Professional Tester ®

Less desired characteristics (what was on the bottom)

19 Intuitive 12.9 Process Centric 15.3

20 Diplomatic 15.2 Flexible 17.0 Diplomatic 17.3 Self Sufficient 16.4

Management ranked programming experience

As part of Rational Services organization helped develop and

Established Software Quality Centers of Excellence for Companies

Experience as Developer, Tester, Architect, Dev Manager, QA

Used to build Air Defense Systems for a living

Father of 6 so prepared for anything