Human Factors Engineering Integrating Human Factors Engineering into the Design Process

Human Factors Engineering Integrating Human Factors Engineering into the Design Process
Design Partners, October 2013, Author: Eugene Canavan.
3

Introduction
Ergonomics has always concerned itself with understanding physical interaction to improve human performance and safety. In order to deal with increasing complexity, the practice has evolved to incorporate both physical and cognitive aspects of human interaction. This developing field has become known as Human Factors Engineering or HFE. The goal of HFE as a discipline is to integrate new practical and actionable methodologies into the formal design process. Design Partners have integrated these techniques into our work and design services. This guide has been written to communicate in a clear way the fundamentals and benefits of this new methodology.

Over 30% of the worlds leading medical device providers have experienced device recalls due to human factors problems. Regulatory bodies, investors and healthcare solution providers are all working toward the same goals of solution optimisation, risk reduction and safety, leading to increased performance and commercial success. Correctly applied, the HFE process will result in the evolution of significantly better products that are safer for professional users and patients alike.
4

Design Partners approach is research based and focused on delivering the best solution to answer user needs. Human factors engineering is integrated into our creative methodology in line with FDA recommendations and training to ensure that we meet the usability and safety requirements that are paramount in medical design. We turn HFE based research into actionable product development programmes resulting in robust and unique products.

Design Partners have evolved a rigorous process of design iteration and verification testing to ensure that our solutions are optimised for safety, usability and manufacture. We provide support and consultation right up to final production specification.

For 30 years we have worked closely with our clients on product development strategy. Our experience and multidisciplinary team provides expert consultancy on all aspects of medical product development.

About Human Factors

What is Human Factors?

According to the International Ergonomics Association (IEA), ergonomics or human factors can be defined as follows: Human factors is the scientific discipline concerned with the understanding of interactions among humans and other elements of a system, and the profession that applies theory, principals, data and methods to design in order to optimise human wellbeing and overall system performance. In this definition, the system represents the physical, cognitive and organisational aspects of human interaction.

The discipline of human factors engineering contributes to performance improvement, risk reduction and safe design by considering the needs, abilities and limitations of the people involved in defined systems. In healthcare, the people involved are diverse and include healthcare providers, patients and patients families. The diverse theories, models, concepts and methods of HFE can be used to address and design for individual needs that can change over time.

10

As a science, human factors engineering is concerned with developing knowledge about human capabilities, limitations, and other characteristics as they relate to the design of the interfaces between humans and other system components. This knowledge, in turn takes the form of design principles, guidelines, specifications, methods and tools.
Hal W. Hendrick, Applied Ergonomics Vol. 39, 2008
11

Human Factors and Medical Product Development

Medical design requires an interdisciplinary approach where complimentary fields of expertise come together to create successful solutions that are tested and verified through an iterative development process.

HFE, correctly applied, maintains a delicate balance between device usability and device safety. The most usable solution may not be the safest solution; while a solution that is too focused on safety may become overly complex to use. Complexity in the mind of the user can lead to device failure, meaning that the medical benefit or safety is compromised. The ultimate goal is to create solutions that deliver great usability while being safe for users and patients.

12

safe

usable

To ensure a highly usable and safe product or truly effective system, the one proven way to accomplish it is through human-centred design.
Hal W. Hendrick, Applied Ergonomics Vol. 39, 2008
13

Regulatory Perspective
Healthcare is a worldwide regulated industry. Medical solutions have a direct impact on health and welfare of millions daily. The American Federal Drug Administration (FDA) has reassessed the importance of HFE relating to device success, risk and safety. Of the worlds leading medical device providers, over 30% have experienced device recalls due to human factors problems in recent years.

With this in mind, the FDA has issued a HFE guidance document incorporating human factors into risk management. The FDA believes that failure to use a product successfully is a source of risk for any product that provides medical treatment. Design history files submitted for PMA are expected to be supported by a HFE report. 510k submissions for devices with human interaction as part of medical treatment delivery are expected to be supported by a HFE report even if product usability has not changed.

(See page 50 for relevant standards)

14

In short, the documents HE74 & HE75 make usability testing a de facto requirement, if not an explicit law. Moreover, medical device manufacturers are practically required to conduct usability tests as a matter of due diligence.
Usability testing of medical devices, M. Wiklund et al 2011
15

Commercial benefits of integrating HFE into the product development process

The application of the HFE process has significant commercial advantages. The discipline provides insight and clarity of direction that can have a profound effect on project outcomes.

These can be summarised as follows: 1. Reduced cost and number of iterations. 2. Mitigation of risk. 3. Reduced time to market. 4. Support of cost containment initiatives without compromising design. 5. Innovative solutions that may create unique IP. 6. A more comprehensively thought through design will inevitably result in superior and more convincing product promotion media.

16

Process informs direction at all phases reducing number of iterations and mitigating the risk of late stage failures.

User centred design enhances brand perception, while research and insight from key stakeholders and users provides market intelligence.

Data collection and testing in each phase supports regulatory compliance and reporting requirements.

17

HFE in Home and Professional Environments

In modern healthcare, patient education and independence are recognised as critically important in returning patients to full health. HFE is an important tool for enabling patient empowerment and the breaking down of barriers such as complexity and use risk. Increasingly the home can be seen as a health care environment. In this space the patient, rather than a trained professional, is the primary user. In this situation the requirement for usability optimisation and safety is even higher.

In the professional environment the safety and performance of medical capital equipment and devices is paramount to care delivery. The requirement for manufacturers to deliver safety and performance can be challenging given the use environment. Typically, users are operating in stressful conditions with a wide series of procedures involving complex interfaces. Worldwide regulatory bodies have identified the practice of applied HFE as a means to address these challenges and as a means of providing primary evidence that each solution is optimised for use, user and use environment.

18

 19

Our Unique Approach

At Design Partners we combine human factors engineering expertise with industrial design creativity and engineering excellence to deliver user centred designs that surpass expectations. While the regulatory perspective is focussed on design optimisation for safety and risk reduction, Design Partners believe that HFE is also a strategic product development tool that delivers competitive advantage by ensuring user needs are met while hidden needs are uncovered and addressed. The outcome of this is delivering unique IP and competitive advantage for our clients.

Integrating a HFE process into product development can also reduce cost and time to market as user needs are defined early and tested throughout the design process, providing research based design guidance to support design decisions and validation planning.

20

The earlier ergonomics is applied in design, the cheaper the cost and the greater the benefit. The cost of catching and solving ergonomic usability design problems early in the design process costs about one-fourth of what the same changes made late would cost.
M. Mantei and T J. Teorey, 1988

Maximum HFE Impact

Start Usability Testing Early

Opportunity to In luence Design

io n/

Va l

id a

tio

n ry e yp sc ov e ic at g si gn ot in Di

Typical Development Cycle

Pr od
21

To ol

Pr ot

Ve ri

De

uc

tio

Te st in

 22

Human Factors Process

Human Factors Process

The following pages summarise a logical step by step sequence for the HFE methodology.

The key steps are:

Preliminary Analysis

Usability Inspection

Usability Testing

Trial Validation Testing

Validation Testing

Human Factors / Usability Report

24

 25

Preliminary Analysis
In order to optimise user performance we must know who the key users are and what critical tasks they need to perform. We need to understand needs, abilities and limitations of typical user populations in the context of the use environment. Understanding the use environment and how it affects task performance and risk are key considerations for system optimisation.

Establishing other key considerations such as other systems and devices in the normal use environment are also critical. Other users and service personal outside the primary use may have different needs that should be considered as part of the overall system or how they effect that system. The needs of maintenance and service technicians are often different to primary user needs.

A review of competitor systems and existing client solutions are important to understand the established interaction paradigms and task flows.

26

Preliminary Analysis

27

Preliminary Analysis

Purpose
Understand user populations, characteristics and limitations Awareness of typical use environments and context of use Define typical use and usability goals Understand context of use in a system

Methods Research and data collection Contextual inquiry Interviews with experts and end users Identify problems with existing or similar products

28

Output Identification of critical tasks Identification of relevant human factors standards and guidelines Definition of typical user populations Definition of typical use environments Definition of other elements in a typical usage system and workflow Competitive analysis

The earlier ergonomics is applied in design, the cheaper the cost and the greater the benefit. The cost of catching and solving ergonomic usability design problems early in the design process costs about one-fourth of what the same changes made late would cost.
M. Mantei and T J.Teorey, 1988

29

Usability Inspection
Usability inspection is a review process that does not necessarily involve physical testing but rather a reference to key principles and the objectives of the project.

Design heuristics are a set of rule of thumb techniques employed to accelerate decisions during the design process. Intuitive judgment combined with experience informs decisions relating to good and bad design solutions. Based primarily on practical experience, these reference principles can usefully expedite the design process by allowing designers to embrace or eliminate proposed design solutions without the need for time and resource intensive empirical testing. Experience can also be supplemented by expert opinion and patient perspectives.

INSPECTION METHODS

USABILITY TESTING

Usability Issue Spectrum

30

Usability Inspection

31

Usability Inspection
Purpose Early identification of possible use error and risks Identify user preferences Optimise workflow and usage within a system Evaluate if usability objectives are attainable Evaluate if usability objectives are being met by design proposals

Methods Contextual inquiry Task analysis Risk analysis (risk related to task analysis) Heuristic, expert and clinical reviews Design standards review

32

Output Human factors engineering usability test plan Definition of essential and critical usability objectives Task analysis broken down into cognitive and physical action Risk matrix related to essential and critical tasks Actionable optimisation recommendations for design development Documented heuristic and expert review

Much of the power of human expertise lies in laid-down experience, gathered over a number of years, and represented as heuristics. Heuristics is defined as a rule of thumb or generally proven method to obtain a result given particular information.

Nigel Shadbolt, Evaluation of Human Work, 2005

33

Usability Testing
Usability testing is an iterative process used to evaluate and document usability design development. Tests are performed using physical assets generated during design development. These can include relatively simple things such as paper and virtual prototypes as well as rigs and physical prototypes. The key to the thinking is to test ideas and form decisions based on subsequent test results. The resultant data then informs design modification which are typically tested again in an iterative way. This is a powerful tool that can have enormous impact on product and project optimisation. Making changes at this stage is a cost effective way to optimise the design by identifying unmet or hidden user needs and by addressing safety and risk issues.

34

Usability Testing

35

Usability Testing
Purpose Gathering of evidence to inform iterative design development decisions Assessment of design with actual users Assessment of design in actual environment and user work flow Identification of unanticipated problems to reduce risk Optimisation of design for user task and workflow Establishment of end user training needs

Methods Simulations and tests using iterative prototypes Cognitive walk through Retrospective analysis interviews with test subjects Confirmation and update of task analysis Determination of task analysis related to risk Risk analysis and effects of failures

36

Output Synthesised and documented test results, interaction problems and adoptive behaviour Updated risk matrix associated with critical tasks Aggregation and prioritisation of problems for risk reduction Final usability design criteria Guidelines for design refinement Summative usability and validation test plan

Usability testing and simulating, particularly early in the design process, provides a practical means for identifying and studying performance issues that would otherwise be ignored.
Ron Laughery, Evaluation of Human Work, 2005

37

Trial Validation Testing

This is the conclusion of formative usability testing. By this stage testing is carried out with formal performance acceptance criteria as a trial run to validation testing. At this point explorative formative testing is complete and the project moves into summative validation testing. Conducting a trial validation test at this stage helps to support the decision to move the project to design freeze.

Trial validation testing may not always be required and the decision depends on project complexity. However, it is highly recommended to carry out these tests when there is a regulatory requirement to carry out usability validation testing in an actual clinical environment. A trial validation tests the proposed HFE validation plan and increases the probability of validation test success at a cost effective point in the project.

The goal is always to make this phase of HFE as conclusive as possible so that validation testing is efficient.

38

Trial Validation Testing

39

Trial Validation Testing

Purpose Trial run of validation test plan Reduction of risk of usability failure at validation test phase Conclusion of formative phase which supports project design freeze Provision of a simulated test in preparation for clinical test Provision of foundations of validation testing

Methods Test simulations to verify essential and critical tasks Test simulations of ideal and compromised scenarios Test simulations with representative or actual users Cognitive walk through Creation of think aloud protocol and retrospective interview/analysis of test subjects

40

Output Usability tests on working prototype verifying use safety Verification that usability objectives are being met Testing and addressing identified risks Documentation of usability failures, analysis and explanation with correction recommendations Addition of recommendations to validation test plan Confirmed and tested HFE validation test plan

Prototyping supports analysis of how a specific design and task allocation will affect human and systems level performance. By developing a prototype designers can conduct experiments and test design concepts with human participants.
Ron Laughery, Evaluation of Human Work, 2005

41

Validation Testing
Validation testing is a mini representation of actual use demonstrating that the device can be used safely where use involves realistic scenarios and conditions established during HFE activities to date. Clinical test environments are used if sufficient testing cannot be performed in a simulated environment.

These tests incorporate worst case users when they are expected to make up a consistent portion of the population or when the illness being managed by the device can impact the ability of users to operate the device. In this case, training level should approximate the training that users will receive.

If problems occur they are documented and counted but also investigated and explained. As users are not entirely predictable, not all use-related problems may have been identified. Problems need to be captured if they occur during validation testing. It is unlikely that new usability issues will occur if usability testing has been as comprehensive as defined by the process in this document.
42

Validation Testing

43

Validation Testing
Subjective assessment is a critical part of validation testing. Subjective assessment provides test participant perspective on all critical and essential tasks as well as identification of either performance difficulty or failure which may not have been recorded in testing. Subjective assessment is a good technique for capturing unanticipated use errors. In a clinical environment observation and performance data collection cannot be intrusive, it may be limited, and therefore subjective assessment is critical.

Purpose The collection of primary evidence for regulatory bodies to demonstrate that the design is safe for use and use environment Validation of critical tasks (tasks that if performed incorrectly can cause harm) Validation of essential tasks (tasks that do not cause harm but affect the medical benefit of the device) Validation of user training level Compliance with HE75 / ISO-62366 HFE Annex to DHF

44

Methods Validation testing using realistic scenarios and conditions with acceptance criteria Creation of test scenarios that include all critical and essential tasks Creation of test scenarios designed to include challenging situations Observational and other objective methods to collect data on performance Systematic collection of user perspective assessment for essential and high risk tasks Directed and interview based subjective assessment by test participants

45

Validation Testing
Output Number and type of test participants with rationale for inclusion Rationale for the tasks included in the testing Test results in table form including number of device uses, success and performance failures Description and evaluation of unanticipated errors based on subjective assessment Statement of mitigations

46

Review

User Needs

Design Input

Design Process

Design Output

Medical Device

Validation

In software design, the goal is to cause the software to function the way its intended users think and process information. In hardware design, a human-centered design approach also enables the development of machines, tools and consumer products having greater usability, maintainability and operational safety than might otherwise occur.
Hal W. Hendrick, Applied Ergonomics Vol. 34, 2003

47

Human Factors / Usability Report

The final output of this process is the creation of a Human Factors / Usability Report. This document records process, methods and the rationale that informs usability design decisions made during product development.

This report addresses regulatory perspective on usability safety and optimisation supporting submission documentation for FDA / CE approval.

The report is part of the design history file (DHF) and contains the following information:

Identification of device use, users, use environments and training

Device user interface Summary of known use problems User task selection, characterisation and prioritisation Summary of formative evaluations Validation testing Conclusion

48

Human Factors/ Usability Report

49

Characteristics of a Good Human Factors Engineering Test Plan

HFE should be applied during the early stages of the product development life cycle. Applying HFE early reduces the risk of problems surfacing during the latter project stages when corrections are costly and often extend time to market.

In Design Partners experience, early application increases the likelihood of meeting user needs with innovative solutions and affords our clients commercial advantage with unique IP and better performing systems.

50

A good HFE test plan will have the following characteristics and benefits:

End user perspective and needs are maintained through the process

Design decisions are informed by end user interaction tasks and needs

HFE activity is tied to the overall device development process

HFE activity is tied to the design controls process and risk management

HFE process provides for an iterative optimisation process and for risk assessment

Builds in ample time to fix problems as they arise, rather than at the end

Calls out actionable risk management activities Documents a separate HFE test plan and links plans to design history file (DHF)

51

Medical Device Standard Relationships

Guidance for Industry and FDA Premarket and Design Control Reviewers (2000)
Medical device use safety: incorporating human factors engineering into risk management.

Draft Guidance for Industry and Food and Drug Administration Sta (2013/2014)
Applying human factors and usability engineering to optimise medical device design.

FDA CR820.3 Design Controls

ANSI/ AAMII/ ISO 14971 : 2007

Medical devices - application of risk management to medical devices.

Note: Standard Relationships was established in consulatation with the Ergonomics Department at the University of Limerick.
52

AAMI/ ANSI HE48 : 1993

Human Factors Engineering guidelines and preferred practices for the design of medical devices

ANSI/ AAMII HE75

Human factors design process for medical devices.

AAMI/ ANSI HE75 : 2009

Human factors engineering design of medical devices.

ISO/ IEC 62366 : 2007

Medical devices - application of usability engineering to medical devices.

PMA Submission of Design History Folder

IEC 62304 : 2006

Medical device software software life cycle processes.

IEC 60601 1 8 : 2006

Medical electrical equipment.

In short, the documents HE74 & HE75 make usability testing a de facto requirement, if not an explicit law. Moreover, medical device manufacturers are practically required to conduct usability tests as a matter of due diligence
Usability Testing of Medical Devices, M. Wiklund et al 2011
53

Conclusion
The human element providing the intelligence behind product performance should not be left to chance. HFE is a recognised methodology ensuring the human component is understood and that this perspective is maintained throughout the product development life cycle. HFE should be applied during the early stages and throughout the product development process.

Applying HFE early in the development cycle reduces the risk of problems surfacing during the later stages when corrections are costly and where delay will extend time to market. As designers we have found that in practice HFE also helps us to identify and meet user needs and to reduce the risk of potential product failure and recall. HFE is a regulatory requirement, but applied well, it is also a strategic product development and cost containment tool essential to commercial success.

54

 55

About the author

Eugene Canavan is a Design Director and leader of the medical device team at Design Partners. He maintains close relationships with clients and the academic and professional medical community.

Eugene is an accomplished designer and ergonomist with over 20 years experience in human factors and mechanical engineering. His work has been recognised by many international awards including the prestigious iF Product Design award in Europe, The Good Design award in Japan and the American IDEA award. He is a tutor at the Centre for Medical Device Design at NCAD and external examiner for Product Design and Innovation at Carlow IT.

56

It would be unheard of to put a mechanical design into production without extensive prototyping, verification and validation. However, the human element providing the intelligence is often left to chance.

57

Contact Us
Europe Design Partners, IDA Business Park, Southern Cross Road, Bray, Co. Wicklow Ireland t: +353 1 282 8913 North America Design Partners, 576 Sacramento Street, Sixth Floor, San Francisco, CA 94111-3022, USA t: +1 415 830 4229 eugene@designpartners.com designpartners.com