Baya, ConceptDesign

INFORMATION TO USERS
This manuscript has been reproduced from the microfilm master. UMI
films the text directly from the original or copy submitted. Thus, some
thesis and dissertation copies are in typewriter face, while others may be
from any type of computer printer.
The quality of this reproduction is dependent upon the quality of the

copy submitted. Broken or indistinct print, colored or poor quality
illustrations and photographs, print bleedthrough, substandard margins,
and improper alignment can adversely affect reproduction.
In the unlikely event that the author did not send UMI a complete
manuscript and there are missing pages, these will be noted.
Also, if
unauthorized copyright material had to be removed, a note will indicate

the deletion.
Oversize materials (e.g., maps, drawings, charts) are reproduced by
sectioning the original, beginning at the upper left-hand comer and
continuing from left to right in equal sections with smalI overlaps. Each
original is also photographed in one exposure and is included in reduced
form at the back of the book.
Photographs included in the original manuscript have been reproduced
xerographically in this copy. Higher quality 6' x 9" black and white
photographic prints are available for any photographs or illustrations
appearing in this copy for an additional charge. Contact UMI directly to
order.
UMI
A Bell & Howell Information Company
300 North Zeeb Road. Ann Arbor MI 48106-1346 USA
313n61-4700 800/521-0600
INFORMATION HANDLING BEHAVIOR OF DESIGNERS DURING

CONCEPTUAL DESIGN: THREE EXPERIMENTS
A DISSERTATION
SUBMITIED TO THE DEPARTMENT OF MECHANICAL ENGINEERING
AND THE COMMITTEE ON GRADUATE STUDIES
OF STANFORD UNIVERSITY
IN PARTIAL FULLFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
IN
MECHANICAL ENGINEERING
Vinod Baya
September 1996
UMI Number: 9714076
Copyright 1996 by
Baya, Vinod
All rights reserved.
UMI Microform 9714076

Copyright 1997, by UMI Company. All rights reserved.
This microform edition is protected against unauthorized
copying under Title 17, United States Code.
UMI
300 North Zeeb Road

Ann Arbor, MI 48103
Copyright by Vinod Baya 1996

All Rights Reserved
ii
I certify that I have read this dissertation and that in my opinion it is fully adequate.
in scope and qUality. as a dissertation for the degree of Doctor of Philosophy.
in scope and quality. as a dissertation for the degree of Doctor ofP osophy.
in scope and qUality. as a dissertation for the degree of Doctor of Philosophy.
~.
Approved for the University Committee on Graduate Studies:
iii
Abstract
This dissertation demonstrates an iterative Observe->Analyze->Intervene, design
research methodology to incrementally improve the understanding and the support of information handling in the conceptual design process. It reports on two
observational studies, based on the verbal protocol method, and an information
management service.
The first observational study is a detailed analysis of the questioning behavior of
designers to understand information needs during a redesign task. This resulted
in the design information framework, which classifies information that should be
captured during a design process for effective reuse at a later date. This framework was used to develop an information management service called Dedal, a
tool for indexing, modeling and retrieving design information. Observations from
deployment and usage of Dedallead to the second observational study. This was
a study of the information handling behavior of individual designers doing short
conceptual design tasks. This study resulted in the Information Handling Framework (lliF).
The lliF is a framework for understanding information handling behavior during
conceptual design. Some key observations from the use of this framework are:
designers move frequently between different types of information (on an
average every 13 seconds),
they handle information about upto 40 concepts in one minute,
their ability to work fluidly and with ease while handling all types of information is essential during the conceptual design process, and
the points of transition between different information types are critical from
the viewpoint of computational support.
Deeper understanding of the information handling behavior is explained by
means of detailed qualitative and quantitative results. The implications of these
results towards the improvement in the understanding of the conceptual design
process and the recommendations on the development of intuitive and integral
information handling services are discussed.
iv
Acknowledgements
I would like to thank Prof. Larry Leifer for his patient, generous and liberal guidance over the course of this dissertation. Larry, I thank you specially for sharpening my communications and management skills and encouraging me to pursue
this line of research.
I would like to thank Prof. Mark Cutkosky, Prof. Sheri Sheppard and Dr. Catherine Baudin for providing valuable feedback on the drafts of this dissertation.
Catherine, Thank you also for giving me an opportunity to work on Dedal.
I would like to acknowledge all the members of the Dead Designer Society: Ade
Mabogunje, David Cannon and Margot Brereton for many things, but specially
for friendship and support over all these years.
I would like to acknowledge students and staff at the Center for Design Research
for providing a very stimulating and amiable research environment. Noelle,
thanks for all the CDR-socials and the numerous other trips.
It would also like to thank my friends in the horse polo world, in and outside of
Stanford, who added a whole new dimension to my experience on the Farm.
Thank you Stanford Polo Club members and horses.
I would like to thank the AI Branch (now the Information Sciences Division) at
NASA Ames Research Center for supporting this research. Thanks also to
TUDelft and Xerox P ARC for providing data for one of experiments used in this
disserta tion.
Finally, I would like to thank my family in India for their patience, understanding
and support while I took my time completing this chapter of my life.
Table of Contents
Chapter 1. Understanding and Improving Engineering Design
Practice .................................................. 1
1.1 Background: Engineering Design ................................. 2
1.1.1 Information Handling in Design ..... '" ...................... 2

1.1.2 Conceptual Design Focus .................................... 4
1.1.3 Research Objectives ......................................... 5
1.2 Research Methodology .......................................... 6
1.2.1 Grounding Research in Reality ............................... 6

1.2.2 Observation by Verbal Protocol .............................. 7
1.2.3 Improving Design Practice: Incremental Support ............... 8
1.3 Analysis of Verbal Protocol Data ................................. 9
1.3.1 Information Handling and Design Information Framework ..... 10

1.3.2 Information Reuse Study: Analysis Example .................. 12
1.3.3 Information Handling Study: Analysis Example ............... 12
1.4 Evolution of the Information Handling Perspective ................ 13
1.5 Key Results ................................................... 16
1.5.1 Key Results from Information Reuse Study ................... 16

1.5.2 Key Results from Dedal Development & Usage ................ 16
1.5.3 Key Results from Information Handling Study .. , ... " ........ 17
1.6 Knowledge Gained ............................................. 17
1.7 Thesis Organization ............................................ 18
Chapter 2. Design Theory & Methodology Research: Literature Revie'W'................................................ 20

2.1 Design Theory and Methodology Research ....................... 21
2.1.1
2.1.2
2.1.3
2.1.4
2.1.5
Prescriptive Studies ........................................ 21

Descriptive (Empirical) Studies .............................. 24
Verbal (or Think-Aloud) Protocol Method .......... " ........ 26
Studies Using Verbal Protocol Method ....................... 28
Studies Using Other (Non-Verbal Protocol) Methods ........... 31
2.2 Design Process: An Information Handling Perspective ............. 32
vi
Chapter 3. Design Information Needs, Capture & Reuse ....... 35

3.1 Design Information ............................................ 36
3.1.1
3.1.2
3.1.3
3.1.4
Information Reuse Essential for Good Design Practice .......... 36

Design Information Capture is Difficult ...................... 37
Information Reuse Study Objectives ......................... 38
Questions Reflect Information Needs ......................... 39
3.2 Experimental Study: Verbal Protocol Method ..................... 39

3.2.1 Experimental Setup and Procedure .......................... 39
3.2.1.1 The Design Problem: Continuously Variable Damper .... 41
3.3 Analysis of Verbal Data ........................................ 42

3.3.1 Transcription ............................................. 42
3.3.2 Question Extraction ........................................ 43
3.3.3 Question Reformulation .................................... 43
3.3.4 Question Classification ..................................... 43
3.4 Question Classification Framework .............................. 44
3.4.1 Design Information Framework (DIF) ........................ 45
3.4.2 Descriptor ................................................ 45
3.4.3 Subject-Class .............................................. 45
3.4.4 Medium .................................................. 46
3.4.5 Level-of-Detail ............................................ 48
3.5 Analysis Examples ............................................. 48
3.6 Results ........................................................ 48
3.6.1
3.6.2
3.6.3
3.6.4
Results on Descriptors ..................................... 50

Results on Subject-Class .................................... 51
Results on Level-of-Detail .................................. 52
Results on Crossing Attributes .............................. 52
3.7 Discussion .................................................... 54
3.8 Recommendations for Information Reuse Service ................. 56

3.9 Summa.ry-...................................................... 56
Chapter 4. Dedal: An Information Management Service ........ 57

4.1 Introduction to Dedal ........................................... 58
4.1.1 Indexing Tool ............................................. 59
4.1.2 Modeling Tool ............................................ 60
4.1.3 Retrieval Tool. ............................................ 60
vii
4.2 Dedal Test Domains and Results ................................ 62

4.2.1 Damper Domain Results ................................... 63
4.2.2 Bioreactor Domain Results .................................. 65
4.3 Discussion on Dedal: Lessons Learned ........................... 66
Chapter 5. Information Handling Behavior Framework ........ 68

S.1 Introduction ................................................... 69
5.1.1 Experimental Setup and Procedure .......................... 69
5.2 Analysis Procedure ............................................. 70
5.2.1 Segmenting into Information Fragments ...................... 70
5.2.2 Classifying Information Fragments .......................... 71
5.3 Information Handling Framework (lHF) .......................... 72
5.3.1 Informational Activity ..................................... 74
5.3.2 Level of Abstraction ....................................... 76
5.3.3 Design Information Measure (dim) . .......................... 77
5.3.3.1 Definition of dim: Assumptions ....................... 78
5.3.3.2 Rules for measuring dims ............................. 79
5.4 Analysis Example .............................................. 80
5.5 Summary...................................................... 82
Chapter 6. Understanding Information Handling Behavior: Quantitative & Qualitative Results ............................. 83
6.1 Introduction ................................................... 84
6.2 Information Fragment Duration (deitat) ...................... 85
6.2.1 deltat Variation over Time .................................. 85
6.2.2 Statistics on deltat . ......................................... 85
6.2.3 Discussion on deltat Results ................................. 86
6.3 Design Information Measure (dim) . .......................... 88
6.3.1 Information Handling Rate ......... , ....................... 88
6.3.2 Discussion on Design Information Measure Results ............ 91
6.4 Distribution of Time and Design Information ..................... 91
6.4.1
6.4.2
6.4.3
6.4.4
Informational Activities Distribution ......................... 91

Information Descriptor Distribution.......................... 92
Information Subject-Class Distribution ....................... 93
Information Level of Abstraction Distribution ................. 94
viii
6.4.5 Discussion on Time and Information Distribution.............. 95

6.5 State Transition Analysis: Statistics and Probabilities .............. 96
6.5.1 Informational Activity State Transition Analysis ............... 97
6.5.2 Information Descriptor State Transition Analysis .............. 97
6.5.3 Information Subject-Class State Transition Analysis ............ 99
6.5.4 Information Level-of-Abstraction State Transition Analysis ..... 99
6.5.5 Discussion on State transition Analysis ...................... 100
6.6 Multiple Attributes Analysis ................................... 101
6.6.1
6.6.2
6.6.3
6.6.4
Informational Activities and Information Descriptor .......... 101

Informational Activities and Information Subject-Class ........ 102
Informational Activities and Level-of-Abstraction ............ 102
Most Frequent Information Fragments ...................... 103
6.7 Summary..................................................... 103
Chapter 7. Conclusions & Future Work ...................... 105

7.1 Concluding Remarks .......................................... 106
7.2 Contributions ................................................. 107
7.3 Discussion ................................................... 107

7.4 Future Work .................................................. 108
Bibliography .............................................. 110
Appendix A. Additional Material on Design Information Reuse

Study................................................. . 117
A.1 Design Experiment Material ................................... 117
A.I.1 Design Experiment Problem Statement ............. '" ..... 117
A.I.2 Description of ME210 ..................................... 120
A.2 Results on individual subjects ................................. 122
A.3 Analysis Examples ............................................ 123
Appendix B. Additional Material on Dedal Development ..... 126

B.1 Dedal Implementation Details ................................. 126
B.2 Dedal Interfaces for Indexing, Querying and Modeling ........... 127
B.3 Examples of Heuristics ........................................ 129
ix
B.4 References on Dedal papers .................................... 133
Appendix C. Additional Material on Design Information Handling

Study................................................. . 134
C.1 Instructions: Overview ........................................ 135
C.2 Design Exercise: Problem Statement Bike Lock 1 ................. 136
C.3 Design Exercise: Problem Statement Bike Lock 2 ................. 139
C.4 Design Expt. Problem Statement: Backpack Harness ........ _..... 142
Appendix D. Additional Results on Information Handling

Study ................................................ 143
0.1 Additional Results on deltat . .................................. 143
0.1.1 deltat vs. ~e ............................................ 143

0.1.2 Distribution of deltat: histograms ........................... 148
0.2 Additional Results on Information Handling Rate ............... 149
0.2.1 Information handling rate over time ........................ 149

0.2.2 Distribution of information handling rate ................... 152
0.2.3 Dim rate vs. Verbal rate: Regression analysis ................ 153
0.3 Information Fragment Attributes Behavior Over Time ............ 153
0.4 Additional Results on Distribution of Time ..................... 158
0.5 Results on State Transition Probabilities ........................ 160
0.5.1
0.5.2
0.5.3
0.5.4
Informational activities state transition probabilities .......... 160

Information descriptor state transition probabilities .......... 161
Information subject-class state transition probabilities ........ 162
Information level-of-abstraction state transition probabilities .. 162
0.6 Additional Results from Multiple Attribute Analysis ............ 162
0.6.1
0.6.2
0.6.3
0.6.4
0.6.5
0.6.6
Information descriptor and informational activities ........... 163

Information subject-class and informational activities ......... 164
Information level-of-abstraction and informational activities ... 164
Information descriptor and subject-class .................... 165
Information descriptor and level-of-abstraction .............. 166
Information subject-class and level-of-abstraction ............ 166
Appendix E. References on Papers from this Dissertation ..... 167

E.1 ASME Conference: DTM-1992 .................................. 167
E.2 ASME Conference: DTM-1994................................. 167

E.3 AS ME Conference: DTM-1995 .................................. 167
E.4 Chapter in Analysing Design Activity: 1996 ...................... 167
xi
List of Figures
Figure 1-1 Strategy for grounding research in reality....................... 7
Figure 1-2 Strategy for incrementally improving design practice............. 8
Figure 1-3 Template for classifying information fragments. . . . . . . . . . . . . . . .. 13
Figure 1-4 Evolution of the work presented in this dissertation.. . . . . . . . . . . .. 14
Figure 3-1 An information perspective on the design process. . . . . . . . . . . . . .. 37
Figure 3-2 Experimental setup for information reuse study.... " ........... 40
Figure 3-3 Analysis procedure for information reuse study................. 42
Figure 3-4 Steps in the development of question classification framework. . .. 44
Figure 3-5 Template for interpreting question and information segment.. . . .. 46
Figure 3-6 Distribution of question deScriptor............................ 50
Figure 3-7 Distribution of question subject-class. ........... . . . . . . . . . . . . .. 51
Figure 3-8 Distribution of question level of detail. .......... . . . . . . . . . . . . .. 52
Figure 4-1 Dedal system architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 58
Figure 4-2 Example of conceptual indexing............................. " 60
Figure 4-3 Example of a model in Dedal. ................................ 61
Figure 4-4 Flow chart of the retrieval algorithm used by Dedal.. . . . . . . . . . . .. 61
Figure 4-5 Retrieval performance of conceptual indexing.. . . . . . . . . . . . . . . . .. 64
Figure 5-1 Analysis procedure for information handling study .............. 71
Figure 5-2 Template for classifying an information fragment. .............. 72
Figure 5-3 Evolution of the informational activity classification. . . . . . . . . . . .. 75
Figure 6-1 Information fragment duration vs. time for S1. ................ " 86
Figure 6-2 Distribution of information fragment duration................ " 87
Figure 6-3 Information handling rate over time for 54.................... " 89
Figure 6-4 Distribution of time and design information across informational activities ...................................................... " 92
Figure 6-5 Distribution of time and design information across descriptor ... " 93
Figure 6-6 Distribution of time and design information across subject-class. " 94
Figure 6-7 Distribution of time and information across level-of-abstraction. " 95
Figure 6-8 Variation in level-of-abstraction over time for 54.............. " 96
Figure 6-9 Informational activity state transition statistics................ " 98
Figure 6-10 Information descriptor state transition statistics .............. " 98
Figure 6-11 Information subject-class state transition statistics ............ " 99
Figure 6-12 Information level-of-abstraction state transition statistics. . . . . .. 100
Figure A-I Distribution of question descriptor for RS1 and RS2. . . . . . . . . . .. 122
xii
Figure A-2 Distribution of question subject-class for RS1 and RS2. . . . . . . . ..

Figure A-3 Distribution of question level-of-detail for RS1 and RS2. . . . . . . ..
Figure B-1 Dedal's indexing and querying environment. .................
Figure B-2 Dedal's modeling interface..................................
Figure B-3 Dedal's answers window. ..................................
Figure D-1 Behavior of deltat over time for 51. . . . . . . . . . . . . . . . . . . . . . . . . ..
Figure 0-2 Behavior of deltat over time for 52. . . . . . . . . . . . . . . . . . . . . . . . . ..
Figure 0-3 Behavior of deltat over time for 53 ...........................
Figure 0-5 Behavior of deltat over time for 55 ...........................
Figure 0-7 deltat behavior over time for all exp-subjects ..................
Figure 0-8 Distribution of the information fragment duration.............
Figure 0-9 Information handling rate over time for 51 ...................
Figure 0-10 Information handling rate over time for 52 ..................
Figure 0-11 Information handling rate over time for 53 ..................
Figure 0-12 Information handling rate over time for 54. . . . . . . . . . . . . . . . . ..
Figure 0-13 Information handling rate over time for 55 ............... . ..
Figure 0-14 Information handling rate over time for 56...................
Figure 0-15 Distribution of dim rate for all exp-subjects. .................
Figure 0-16 Regression analysis between dim rate and verbal rate .........
Figure 0-17 Behavior of informational activity over time for 51. . . . . . . . . . ..
Figure 0-18 Behavior of Information descriptor over time for 52. . . . . . . . . ..
Figure 0-19 Behavior of Information subject-class over time 53. . . . . . . . . . ..
Figure 0-20 Behavior of Information level-of-abstraction over time 54. . . . ..
Figure 0-21 Distribution of time across informational activity. . . . . . . . . . . ..
Figure 0-22 Distribution of time across information descriptor. ...........
Figure 0-23 Distribution of time across information subject-class.. . . . . . . . ..
Figure D-24 Distribution of time across information level-of-abstraction. ...
Figure 0-25 Information activity state transition probabilities.. . . . . . . . . . . ..
Figure 0-26 information descriptor state transition probabilities. . . . . . . . . ..
Figure 0-27 Information subject class state transition probabilities. ..... . ..
Figure D-28 Information level-of-abstraction state transition probabilities. ..
xiii
122
123
127
128
129
144
144
145
145
146
146
147
148
149
150
150
151
151
152
152
153
154
155
156
157
158
158
159
159
160
161
162
163
List of Tables
Table 1-1 Analysis procedure ........................................... 10
Table 1-2 Information handling and design information framework. ......... 11
Tabie 2-1 Empirical design research methods .............................. 25
Table 3-1 Framework to classify questions ................................. 45
Table 3-2 Question descriptors .......................................... 47
Table 3-3 Question subject-class ......................................... 47
Table 3-4 Question level-of-detail........................................ 48
Table 3-5 Example of analysis procedure in information reuse study.......... 49
Table 3-6 Information reuse study: subjects background information .......... 50
Table 3-7 Crossing descriptor with subject-class............................ 53
Table 3-8 Crossing descriptor with level-of-detail.......................... 54
Table 3-9 Crossing subject-class with level-of-detail.. ....................... 54
Table 4-1 Dedal test domains details ...................................... 63
Table 5-1 Details on experiments and experimental subject background....... 70
Table 5-2 Information handling framework (ll-IF) .......................... 73
Table 5-3 Descriptors added to the design information framework ............ 74
Table 5-4 A classification for the Informational activities .................... 76
Table 5-5 Definition and interpretation for level-of-abstraction............... 77
Table 5-6 Examples of dim measurement.................................. 80
Table 5-7 Examples of information handling behavior analysis ............... 80
Table 6-1 Information fragment duration (deltat) statistics................... 87
Table 6-2 Information handling rates and verbal rates for all subjects......... 90
Table 6-3 Distribution of information activities across descriptors. . ......... 101
Table 6-4 Distribution of informational activity across subject-class .......... 102
Table 6-5 Distribution of informational activities across level-of-abstraction... 102
Table 6-6 Frequently encountered info-fragment attribute combinations ...... 103
Table 6-7 Summary of the key findings and recommendations .............. 104
Table A-I Example of analysis procedure in information reuse study......... 124
Table 0-1 Distribution of descriptor across the informational activities ....... 163
Table 0-2 Distribution of subject-cass across the informational activities ..... 164
Table 0-3 Distribution of level-of-abstraction across informational activities .. 164
Table 0-4 Distribution of deScriptor across the subject-classes ............... 165
Table 0-5 Distribution of descriptor across the levels-of-abstraction. . ....... 166
Table 0-6 Distribution of subject-class across the levels-of-abstraction....... 166
xiv
Glossary
Here are some common abbreviations and terms used in this dissertation:
OAl
Observe ->Analyze -> Intervene iterative cycle
OIF
Design Information Framework
IHF
Information handling Framework
EDN
Electronic Design Notebook
ms
Information Reuse Study
IHS
Information Handling Study
DS
[DeScriptor, Subject] pair in information fragment classification
dim
Design Information Measure (quantitative measure)
dpm
dims per minute
wpm
words per minute
deltat
Information fragment duration in seconds
xv
Chapter 1
Understanding and Improving
Engineering Design Practice
Design Research
< >
Deliver High Quality,

Low Cost Products
Goal
I~
TO
TO
,
shortest path
Develop Research
Methods
T~
Understand Design
Practice
Improve Design
Practice
~o
This chapter is a summary of the research work being reported in this

dissertation. It discusses the evolution of this dissertation and the
motivation for pursuing the research. It briefly describes the
experimental approach and analysis procedure, developed to carry
out two observational studies, namely information reuse study and
information handling study. It lists the contributions that this
dissertation is making to the knowledge in the field of design theory
and methodology. It ends with a narration on the content of the
remaining chapters.
Information Handling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 1: Understanding and Improving Engineering Design Practice
1.1 Background: Engineering Design

Engineering design is a complex process involving people, multiple technical
and non-technical disciplines, material and non-material resources, computational and non-computational tools and a variety of cognitive strategies. As a
result, studying, understanding and supporting this process is very challenging.
Given its complexity, there is a potential for substantial savings in cost, labor
and resources by improving engineering design practice. Improvement can be
achieved by supporting design practice with tools and services which accrue savings over the course of the process. Economic pressures to shorten product
development cycles and improve market responsiveness have increased interest
in the study of design theory and methodology both in industry, in academia
and in design education. Developing methods to study and understand engineering design, and using this understanding to build services! to support this
process is essential for developing high quality products at low cost. This thesis
addresses the need for new methods required in design research for improving
design process understanding.
Design process can be studied from many perspectives such as, technical, computational, problem solving, organizational, social and others. Each perspective
has a potential to lead to a better understanding of the process and lead to
improved design practice. I have chosen to approach the engineering design process from an information handling viewpoint.
1.1.1 Information Handling in Design

In this dissertation information handling refers to activities and sub-processes
undertaken by the designer which involve design information2. For instance

1. The term services will be used in this thesis to refer to tools and(or) methods (computational or non-com-
putational) that can support engineering design practice.

2. The terms design information and infonnntion will be used synonymously in this thesis unless stated otherwise.
activities dealing with generating, capturing, accessing, transforming, indexing,

structuring and analyzing information with the intent to create an artifact, are
some examples of information handling. While it is possible to construe all
actions of a designer as information handling, for the purpose of this study I
only consider the actions pertinent to the design task on hand.
Engineering design is an information intensive process. Over the duration of a
design project designers handle a large amount of information. The information
is handled by means of numerous non-computational (paper notebooks, catalogs, handbooks, telephone etc.) as well as computational tools (CAD /CAE/
CAM software, word processors, spreadsheets, e-mail etc.). The Information
exists in a variety of media such as: text, graphic, audio, video, CAD drawings
and others. This diversity imposes considerable burden of information management on designers. As a result, the efficiency and quality of the design process
depends heavily on how capable designers are in handling information.
To find out about the information requirements of engineering designers, [Court
et. a11993] conducted a questionnaire based survey of about 200 designers from
various industries located in the United Kingdom. They found that on average
designers spent 18% of their time searching for information, 23% dealing with
paperwork, 16% in meetings and only 43% of their time designing. Hales [1987]
performed a detailed analysis of all the activities of one project involving the
design of a high-pressure, high-temperature system for testing materials. He
was a participant observer! in this project over a period of 2.8 years and involving 37 different participants. Hales found that only 47% of the total effort was
spent in doing design. The remaining 53% was spent in information retrieval,
planning, cost estimating, reviews, helping others and social contact. These two
studies strongly suggest that there is an opportunity to improve design practice
by supporting information intensive activities such as, information gathering,
1. An observational technique in which the observer is also a member (participant) of the design team.
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
documentation, communication, information management and others. A reduction in information related activities will lead to an improvement in designers
capacity to do better designs.
All over the world, there is a steady movement towards electronic commerce.
Much of information that is available today in paper medium (design notebooks,
libraries, handbook, catalogues etc.) will be available in electronic formats over
computer networks. As a result, many transactions which today are paper based
will become electronic (group communications, interaction with vendors etc.).
This transition will significantly impact design practice in industry. Designers
will have easy access to a large amount of information, resulting in increased
information management burden. We need to make sure that services that are
made available to designers in the electronic workplace integrate smoothly with
their work environment, so that designers can handle information efficiently.
This will be possible if services are built based on a good understanding of the
information handling behavior of designers. Developing this understanding is
the core concern in this dissertation.
1.1.2 Conceptual Design Focus

Among the commonly agreed upon phases of the design process, conceptual, lay-
out and detail [Cross 1990, Page 21], the conceptual phase should deserve special
attention from an information handling perspective. Conceptual phase refers to
the ideational stage of the design process where ideas are generated and analysed, and choices are made. In this stage, ideas and solutions are described
abstractly without quantitative detail. While conceptual activity takes place all
along the design process [Guindon 1990], it occurs more frequently in the early
stage of the process. It is this early stage that is of interest in this dissertation.
The conceptual phase happens to be the one where important decisions are
made [Wood and Antonsson 1987]. Also, it is the most informal, and the least
Infor=tion Handling Behavior of Designers During ConccpruaJ Design: Three Experiments
understood [Rabins et al. 1986]. It is also the stage for which there is little computational support [Rouse and Boff 1987]. Decisions and information generated in
this phase have a big impact on the downstream design process and the overall
cost. It has been reported that upwards of 80% of the final cost are obligated during conceptual design [National Materials Advisory Board 1991]. This means
that understanding and supporting information handling in this phase is very
likely to lead to Significant betterment of design practice. This is why this dissertation focuses on conceptual design.
1.1.3 Research Objectives

The primary objectives of this dissertation are to model the information handling behavior of designers
during conceptual design,
and
develop
requirements to guide the evolution of information handling support services. A

secondary objective is to develop a research methodology for understanding and
improving engineering design practice. Several questions have guided this
inquiry:
1. What is design information?
2. Can design information be classified?

3. Are design information needs during conceptual work special?
4. Are some types of design information more important than others?
5. What are the activities designers perform with design information?
6. vVhat is the temporal distribution of design information processing?
7. Can design information be quantitatively measured?
8. What is the information content in different activities?
9. How does information processing rate change with the design process?
and,
10. What do answers to the above questions inform us about services needed
to support information handling in the design process?
Information Handling Behavior of Designers During Conceplual Design: Three ExperimenlS
1.2 Research Methodology

An underlying concern in this study is to build services which would support
and improve design practice. This required that the research methodology give
adequate attention to, Ca) understanding the design process and (b) development
of computational services. Three research strategies were used in this study to
maintain thls balance,
1. Grounding the research in reality: the research paradigm was grounded in
reality by experimenter's involvement in real design practice.
2. Understanding the design process: design process was examined in detail
by means of observational studies, and
3. Improving design practice: design process was incrementally supported by
a prototype service and the performance of designers using the prototype
service was assessed.
These three strategies are described in detail in the subsections below. In the discussion observational study refers to the method adopted in this dissertation to
understand the design process. Two observational studies were conducted, one
to understand information reuse and the other to understand information handling
behavior.
1.2.1 Grounding Research in Reality

A special feature of the research presented in this dissertation is shown in
Figure 1-1. The figure is meant to indicate that the researcher should have a rich
involvement in real design activity during observational studies and service
development. The involvement in real design activity is motivated by the concern for assuring that formal observations from the experiments are consistent
with experiences from real design activity, and that during service development
there is constant feedback from an ongoing design project.
Results from observational studies are the basis for guiding service development, and feedback form service development to observational studies is only
Information Handling Behavior of Designers During Conccptual Design: Three ExperimcnlS
Service Development
-Dedal
Observational Studies
(formal observation)
- Information Reuse
- Information Handling Behavior
Real Design
(informal observation)
- Variable Damper
- Bioreactor
Figure 1-1 Strategy for grounding research in reality. Double arrow 1 is a means
for assuring that formal observations from the experiments are consistent with
informal observations made from real design activity. Double arrow 2 is a means
for assuring that during service development there is constant feedback from an
ongoing real design project. Single arrow 3 indicates that service development is
based on results of observational studies.
through real design activity (since a service has to be incorporated in real design
activity before its performance can be examined in an observational study). Altogether this strategy provides a mechanism for grounding the research in reality.
During the evolution of this dissertation, which covers two observational studies
and development of a computational service (Dedal), I was a member of the two
design projects, Variable Damper and Bioreactor, as shown in Figure 1-1.
1.2.2 Observation by Verbal Protocol

Observational studies of the design process can be conducted using various
methods. Some of the popular methods are discussed and compared in Chapter
2. This study used the verbal protocol method. This method is popular in single
subject experiments and has been used widely by cognitive scientists [Newell
1968] and design researchers [Eastman 1970; Akin 1979 and Stauffer 1987b]. In
verbal protocol method, experimental subjects are asked to talk out loud while
they work on a design problem. In this manner the actions and intentions of the
subject become explicit in the verbal recording. The transcript of this recording
is the raw data for analysis.
There is debate in the research community regarding the appropriateness of this

method, since it is not understood in what manner and how much the act of verbalizing affects the natural thought process and the behavior of the experimental
subject. Despite these limitations, verbal protocol is a popular method for single
subject studies because of its relative simplicity and tacit assurance that what is
said "is at least subjects own words". Procedures and guidelines have been
developed for collecting and analyzing such data so that the results obtained are
valid and free from bias [Ericsson and Simon 1993]. The appropriateness of this
method is discussed further in Chapter 2.
1.2.3 Improving Design Practice: Incremental Support

A strategy for improving design practice by incrementally supporting the design
process! is shown in Figure 1-2. This approach has been advocated and used by
Analyze
Analyze
Observe
Intervene --.~~ Observe
Intervene --.~~ Observe
Improved Design Practice -----.~~

Figure 1-2 Strategy for incrementally improving deSign practice. The
design
process is cyclically observed, analyzed and intervened while incrementally
developing services to facilitate and improve design practice.
[Tang 1989 and Minneman 1991] in their research in improving collaborative

1. It is an assumption that incremental improvements in understanding and support of the design process
will lead to improved design practice over time. This is not being proved in this thesis.
Information Handling Behavior of Designer.; During Conceptual Design: 1breo: ExperimenlS
design practice. The fundamental assertion of this methodology is that hypothesis free observation of design activity must precede the development of services
intended to support that activityl.
The process starts by conducting observational studies of design activity to collect raw data. The data is analyzed to gain an understanding of the design
activity and to develop frameworks and requirements for services which will
improve the observed design practice. Using these frameworks and requirements, services are built and introduced into the original design process. The
introduction of the service into the design process is an intervention, since it
changes the original process itself. A second cycle of observational study is
needed to understand the changed process. The cycle of observe -> analyze ->
intervene (OAl) goes on, and the design process is incrementally supported,
restructured and improved.
1.3 Analysis of Verbal Protocol Data

A total of eight verbal protocol experiments were conducted. Two were conducted in the information reuse study and six in the information handling study.
The recordings from the experiments were analyzed using the procedure outlined in Table 1-1. The purpose of the analysis was to reduce the raw data into a
form where the design behavior could be quantitatively analyzed.
During the experiments the verbalizations were recorded both on audio and
video tapes2 . The transcription process transformed the audio and video data
into a transcript whi.:h would contain the raw data for analysis. Video data was
useful in numerous situations to resolve ambiguities and establish context during transcription. The segmentation process fragmented the raw data such that
1. The approach of deriving a hypothesis from collected data rather than confirming a pre-formulated
hypothesis.
2. The video tapes had an audio track as well. The audio tapes were used as backup and for facilitating
transcription activity.
Information Handling Behavior of Designers During Conceprual Design: Three Experiments
10
Table 1-1 Analysis procedure for transforming raw data into quantitative descriptions of the
designer behavior.The analysis steps were similar for the two studies. The differences in the
implementation of the steps are compared below
Analysis Steps
Information Reuse Study
Information HandJing Study
conductexper.rrnen~:
to record verbalizations
to record verbalizations
transcribe and time stamp
to get raw data
to get raw data
segment transcription
into questions
into information
rephrase segment
rephrase question
rephrase fragment
classify fragmen~
using design information fraDl e work(D1F)
using information handling

framework (IHF)
quantitative analysis
statistical, first-order
statistical, correlation, regression, state-transition, first-order
fragmen~
each chunk represented a unit of an interesting observation. In the information

reuse study this unit was a question and in the information handling study this
unit was an information fragment. In the next stage of the analysis these units
were classified. The framework used for classification was constructed as the
analysis progressed. It is important to point out here that the analysis was not
approached with a pre-existing framework. The framework evolved iteratively
as an outcome of the analysis procedure. The classifications were quantitatively
analyzed, the results of which are discussed later in this chapter. Each step of the
analysis procedure will be discussed in detail in Chapter 3 and Chapter 5.
1.3.1 Information Handling and Design Information Framework

Table 1-2 shows the information handling GIld the design information framework which resulted from analysis. This framework is a classification of the
activities designers undertake with information and a classification on design
information handled in those activities. Design information was classified using
four attributes: Descriptor, Subject-class, Medium and Level-of Detail. Four more
attributes were added to extend this framework to information handling. These
were, informational-activity, level-of-abstraction, information fragment duration
(deltat) and design information measure (dim). The attributes are briefly defined
below and the values these attributes can take are shown in Table 1-2.
Information Handling Behavior of Designers During Conccplunl Design: Three Experiment'
Chapter 1: Understanding and Improving enginesring Design Practice
11
Table 1-2 Information handling and design information framework.

Information HandIiDg Framework (IHF)
Informational
Activity
Design Information Framework (DIF)
Level of Abstraction
Desaiptor
Subject Class
Medium
Level of Detail
Generate
Unlabeled
Alternative
Assembly
Audio
Conceptual
Access
Labeled
Assumption
Component
Video
Configurational
Analyze
Associative
Comparison
Connection
Text
Detail
Qualitative
Construction
Feature
Graphic
Quantitative
Location
Requirement
Gesture
Operation
Design-concept
Performance
Other
Information Fragment Duration (sec)
Quantitative measure in seconds
Rationale
Design Information Measure (dim)
Quantitative measure: takes integer

values (1,2,3, and so on)
Relation
Requirement
Miscellaneous
Informational-activity. This is the activity (generate, access or analyze) that the

designer performs with information.
Level-of-Abstraction. This represents the level at which the information is in it
evolution from abstract to concrete.
Design information measure. This is a quantitative measure of the amount of
semantic information in an information fragment.
Descriptor. This refers to the basic constitution or the nature of the information
in an information fragment.
Subject-Class. This is the class (assembly, component, connection etc.) to which the
subject1 of the sentence representing the information fragment belongs.

Medium. This is the physical form (text, graphic, audio, video, etc.) in which the
information fragment exists.
1. The word subject is being used in two contexts in this dissertation. One use is in the context of subjectclass and the other in the context of the experimentalSIlbjects. In most cases the context would be sufficient to
impart the right meaning. In other cases the use will be explicitly stated to avoid any possible confusion.
Information Handling Behavior of De.~igners During Conceptual De.~ign: Three ExpcrimcnlS
12
Level of Detail. It is the phase of the design process (conceptual, configuration or
detail) to which the information fragment can be associated with.
1.3.2 Information Reuse Study: Analysis Example

A unit of interesting observation in the information reuse study was a question.
A question is any utterance by the experimental subject expressing a need for
information. Below is an example of a question from the protocol and its
analysis.
QUESTION FROM PROTOCOL:
To find in here where you figured out, or how you figured out your force
requirements. basically the force requirements as they relate to the
diameter of these plates.
QUESTION REPHRASED:
How does the force requirement relate to the diameter of the friction
disks?
INTERPRETATION:
The designer needs DETAIL information regarding the RELATION between the
FEATURES (force requirement) and the (diameter of friction disks) .
CLASSIFICATION:
Descriptor: RELATION
Subject Class: Feature
Level of Detail: DETAIL
Medium: TEXT
1.3.3 Information Handling Study: Analysis Example

A unit of observation in the information handling study is an information frag-
ment. An information fragment is a continuous period during which the

informational activity and, the descriptor of information, remains the same.
Figure 1-3 shows the template that was used for classifying each information
fragment.
An example of an information fragment and its classification is:
INFQRMATION FRAGMENT:
I guess the idea about the strength of these lock is that the lock
should be as strong as the bike ...
CLASSIFIED FRAGMENT:
13
(info-fragment
:activity informational-activity
:time (stan-time end-time)
:descriptor descriptor
:subjclass subject-class
:subjects (subjects-list)
:medium medium
:levels (level-of-detail level-of-abstraction)
:measures (design-information-measures:dim)
)
Figure 1-3 Template for classifying information fragments in the
information
handling study. The figure shows the attributes that are given values to classify an
information fragment. The fields take vcllues from the information handling
framework (Table 1-2).
( info-fragment
:activity GENERATE
:time (00572007 00573300)
:descriptor ASSUMPTION
:subjclass FEATURE
:subjects (Strength-or-Lock)
: medium AUDIO
:levels (CONCEPTUAL ASSOCIATIVE)
: measures (1)
1.4 Evolution of the Information Handling Perspective

The evolution of this study is captured in Figure 1-4. This figure maps the evolution of this dissertation on top of the methodology presented in Figure 1-2. Note
that this figure represents a rationalization of four years of research and while it
closely represent the temporal sequence of the work, many tasks were carried
out concurrently (Figure 1-1).
My early years of doctoral work were devoted towards a project called Genera-
tion and Conservation of Design Knowledge (GCDK) [Leifer 1989]. The major
concerns of this project were to capture and document design knowledge/information so that it could be available for easy reuse at a later time. The concern for
capturing design information in an electronic medium tied this project to the
Information Handling Behavior of Designers During Conceptual Dcsign: Three Experiments
/------( Information Classification \

\
Framework
I
------_/
14
/-----,
,,-----,
{ Dedal Performance \
\.
Results
/
{ Information Handling \
\.
Framework
/
t
t
t
-----------------------------Analyze
1~~'9':f~~~m
Information
Reuse Study
Analyze
Analyze
/9'-'_,,\
Support by
Observe
Dedal
---II~~ Dedal Use
Support by
Dedal
-
/.~-'9~\
Information
......
~.. Handling
Study
Understanding
of Information
Handling
Figure 1-4 Evolution of the work presented in this dissertation. This covers two
observational studies and development of a computational service called Deda!.
development of an Electronic Design Notebook (EDN'fM) . This notebook was

built on top of ~ text-graphic editor called vmacs [Lakin et al. 1989]1. The
hypothesis motivating this research was that design knowledge conservation
(capture, representation, reasoning and retrieval) is far less costly than re-invention of comparable knowledge [Leifer 1991].
The first question faced was: Of all the information designers handle, what information do designers reuse? To find an answer to this question I conducted the
observational study on information reuse. Two experiments were conducted and
questioning behavior of the experimental subjects was analyzed. This resulted in
the design information framework (DIF) which classifies design information that
designers need during redesign2. The experiments and resulting framework are
discussed in Chapter 3.
Based on the results of the information reuse study, an information management
system called Dedal was developed3. Dedal is an information management tool
1. Electronic Design Notebook, EDN and vrnacs are trademarks of The Performance Graphics Company.
2. The experiments were carried out in collaboration with Catherine Baudin and Jody Underwood from
NASA Ames Research Center and Ade Mabogunje from Stanford University.
Information Handling Behavior of ne.,igners During Conceptual Design: Three Experiment'
15
which integrates with an information capture medium such as an Electronic

Design Notebook to provide designers with computational services for indexing,
modeling and retrieval of design information. Dedal emphasizes managing information incrementally and in real time. It uses the framework in Table 1-2 to
implement a conceptual indexing scheme. Development and performance of
Dedal are discussed in detail in Chapter 4.
When evaluating the impact of Dedal on design practice, I learned that the activities of indexing and modeling using Dedal were add-on activities, requiring
additional effort. This was a symptom of lack of adequate integration of Dedal
with the work habits of designers and the design process. I was faced with many
new questions. The critical question was: How can we support information handling so that it is intuitive and integrates smoothly with the design process? I
realized that there was very little understanding of the work habits of designers
relating to the dynamics of information handling, Le. information handling
behavior over time, activities performed with information, nature of information
handled in these activities, amount of information handled and others. I was convinced that knowledge of answers to these questions was critical to developing
services which were intuitive and integrated smoothly with design process.
In search of answers to the above questions I conducted the infonnation handling
study. In this study I conducted six experiments using the verbal protocol
method. Analysis of the experimental data resulted in the information handling
framework (llIF). To determine the information handling rates, I defined a quantitative measure for design information using the framework in Table 1-2. This
measure is called design infonnation measure or simply dim. The experiments, analysis method, dim and results of this study are discussed in Chapter 5 and
Chapter 6.
3. Dedal was built in collaboration with Catherine Baudin and lady Underwood from NASA Ames
Research Center and Ade Mabogunje from Stanford University.
16
1.5 Key Results

Experiments involving human subjects are a challenge to monitor and control,
since results are easily influenced by variables such as human subjects, design
problem and experimenter. Verbal protocol analysis is tedious and time intensive, which makes it impractical to conduct and analyze a statistically significant
number of experiments. The analysis procedure is subjective and there are many
questions regarding the reliability of the results from such an analysis. Being conscious of these shortcomings, I have built guidelines at each stage to maintain
objectivity and rigor in analysis. I have tried to explain some of the results
obtained either from prior understanding of the design process or from peculiarities of the experimental/ analytical method I used. I have used the remaining
results to push the edges of my understanding of the design process. Below I discuss the key results from each of the observational studies and development of
Dedal.
1.5.1 Key Results from Information Reuse Study

The information reuse study resulted in some recommendations for developing
computational support for design information capture. Some key results are:
the design information framework, Table 1-2, classifies the design information that should be captured for effective reuse.
the framework was adequate to classify 80% of the questions encountered in
the experiments
information with descriptor performance, relation, construction and operation
were encountered frequently (57%), and should get special attention during
capture
information with subject-class feature, component and requirement were
encountered frequently (68%), and should get special attention during capture
1.5.2 Key Results from Dedal Development & Usage

Dedal was tested to measure its retrieval performance. Also it was deployed in a
Information Handling Behavior of Dcsigners During Conceptual nc.<ign: Three Experiment<
17
design project to evaluate real time usage. Here are some key results:
retrieval performance of the conceptual indexing scheme implemented in
Dedal (precision = 80%; recall = 60%) was better than keyword based information retrieval systems (precision = 20%; recall = 50%).
information framework was adequate to capture information necessary for
effective reuse
the classification in the DIP was intuitive to designers, hence they were able
to index, model and formulate queries
indexing and modeling integrated poorly with the designer work habits,
requiring designer to spend additional time indexing and modeling
1.5.3 Key Results from Information Handling Study

The information handling study gave a detailed quantitative insight into the
dynamics of information handling behavior of designers. Some key results are:
designers spend between 2 to 35 seconds with one information type. On an
average they changed the information they were handling every 13 seconds.
the global information handling rate (dim rate) for the six experimental subjects ranged from 6.2 to 9.2 dims/minute (average being 8.1 dims/minute)
the local information handling rate (dim rate for each of the information
fragments) ranged between 2 to 40 dims/minute. About 40% of the variation
in dim rate can be ascribed to the variations in verbal rate (words per minute
spoken by experimental subjects)
conceptual tasks are generation intensive. 55% of design time and 53% of
design information are handled while generating new information
more than 65% of time and design information were accounted for in
descriptors [construction, alternative, operation, requirement]; in subject-class
[assembly, component, design-concepts] and in non-quantitative level-of-detail
regarding transitions, more than 82% were to and from generate for informational activity; 29% were to and from construction for descriptors; 54% were
to and from [assembly, component] for subject-class and more than 90% were
to and from non-quantitative level-of-abstraction
1.6 Knowledge Gained

This dissertation adds knowledge to the field of design research. It demonstrates
Information Handling Behavior of Designers During Conceplua[ Design: Three Expcrimcms
18
a quantitative experimental method for studying and understanding the engineering design activity. It illustrates a methodology for incrementally improving
design practice. It is based on observational studies that remain grounded in real
design. This dissertation makes three key contributions, it
demonstrates a framework for characterizing the information handling
behavior of designers. This framework effectively classifies 80% of the
design information that should be captured during the design process for
adequate reuse.
demonstrates a computational implementation of the information framework that is intuitive for designers and works in real time.
provides a quantitative as well as qualitative understanding of the information handling behavior of designers and makes recommendations on how
this understanding can be incorporated in building services to improve
design practice.
1.7 Thesis Organization

Chapter 2 surveys the literature in the field of design research. It looks at the
methods used in studying and understanding the design process. It discusses
the appropriateness of the verbal protocol approach as a design research
method. It also cites studies that have impacted the evolution of this thesis.
Chapter 3 describes the information reuse study. It describes the analysis procedure and the evolution of the design information framework (DIF).
Chapter 4 describes the computational service Dedal. It discusses how Dedal
was used in a design project and what lessons were learned from its deployment
and usage.
Chapter 5 describes the information handling behavior study. It describes the
analysis procedure and the development of the information handling framework
(lliF).
Chapter 6 describes the quantitative and qualitative results of the information
Information H:mdling Behavior of Designers During Conceptual Design: Three Experiments
Chapter 1: Understanding and Improving engineering Design Practice
19
handling study. It discusses the understanding gained and suggests recommendations for the design and implementation of information support services.
Chapter 7 discusses the conclusions of this study. It describes the key contributions of this work and some directions for further research.
The bibliography provides a list of research papers, articles and books that have
influenced the evolution of this dissertation.
Appendix A is a collection of supporting material from the information reuse
study. It includes the design problem used in the experiment, samples of raw
data, examples of analysis and some results.
Appendix B is a collection of supporting material on Dedal implementation and
usage.
Appendix C is a collection of supporting material from the information handling
study. It includes the design problem used in the experiments, samples of raw
data and examples of analysis.
Appendix 0 is a collection of additional results from the information handling
study which are not included in the main body of the thesis.
Appendix E lists the key papers published in the past 4 years. These papers
reflect the evolution of this dissertation.
Chapter 2
Design Theory & Methodology
Research: Literature Review
I..J/"""-~~-'~>
Design Research
\.

Low Cost Products
Goal
~,~--~,"
~
TO
jl
TO
"
Develop Research
Methods
shortest path
Improve Design
______
Practice
Understand Design
Practice
~o
This chapter reviews the literature in the field of design theory and
methodology that has influenced the work reported in this

dissertation. Prescriptive and descriptive (empirical) design research
studies from the past few decades are discussed. Appropriateness of
the verbal protocol method is discussed along with a review of some
studies which have used this method. A few studies using non-verbal
protocol method are also discussed. The chapter ends with a
discussion of research that focuses on information handling issues in
the design process
20
Chapter 2: Design Theory & Methodology Research: Uterature Review
21
2.1 Design Theory and Methodology Research

The objecjve of much of the research carried out in the field of design theory
and methodology is to develop a better understanding of the engineering design
process. This understanding can be used to develop new methods and services
to improve design practice in industry and design education at academic institutions. The complexity! of the design process offers numerous challenges in
accomplishing these objectives. There are many directions possible for conducting design research, as is evident from the number of academic disciplines
involved in it, such as engineering, anthropology, sociology, management and
psychology, among others. As a result, design research is published in a large
variety of literature sources. In this chapter I will discuss the literature which has
impacted the evolution of the work reported in this dissertation.
Looking at design research from the last few decades, the studies that have been
conducted can be grouped under two approaches. They are either prescriptive
or descriptive [Tang 1989, page 25]. Prescriptive studies typically result in recommending a sequential procedure for carrying out design, whereas descriptive
studies aim to provide a description of how design is carried out. In the sections
below I will discuss studies associated with both of these approaches.
2.1.1 Prescriptive Studies

Prescriptive design studies result in laying out a method for doing design. These
methods are typically developed by reflecting and theorizing about the design
process. They suggest a systematic and algorithmic procedure that should be carried out sequentially. A sequential process ensures that decisions are made after
all necessary information has been generated, accessed and considered, so that
no important elements of the design problem are overlooked. These methods
1. The apparent complexity of the design process stems from the fact that it is not just a technical process or
a problem solving process. It is a complex and poorly understood mix of technical, problem-solving, social,
organizational, behavioral and creative processes, just to name a few.
22
emphasize the need for analytical work before generating new design concepts
and moving on to the next stage. Most prescriptive models allow for iterations
and feedback to take place between different stages.
Asimov [1962] prescribed a seven phase procedure to transform needs into physical artifacts. These phases are:
1. feasibility study
2.
3.
4.
5.
6.
preliminary design
detailed design
planning for production
planning for distribution
planning for consumption
7. planning for retirement

The first three phases form the design stage and the last four phases form the
production and marketing stages. Each of these phases involves problem solving
situations which include the stages of analysis, synthesis, evaluation, decision, opti-
mization, revision and implementation. This problem solving pattern carried out
over the seven phases provides a methodology for transforming needs into physical artifacts.
Jones [1963] prescribed a three stage design method. This method was a systematic move from analysis to synthesis to evaluation. These stages are: [Cross, 1990,
page 24]:
Analysis: Listing of all design requirements and the reduction of these to

a complete set of logically related performance specifications.
Synthesis: Finding possible solutions for each individual performance

specification and building up complete design from these with least possible compromise.
Evaluation: Evaluating the accuracy with which alternative designs fulfil

performance requirements for operation, manufacture and sales before
the final design is selected.
Chapter 2: Design Theory & Methodology Research: Literature Review
23
The emphasis in this procedure is on generating numerous alternatives and

choosing the best alternative based on extensive analysis and comparisons.
Alexander [19641 proposed a method of simplifying and structuring design problems by listing all the requirements of the problems and analyzing the
interactions between them. These requirements could then be grouped into sets,
which lead to mostly independent sub-problems, which were easy to solve. Solving these sub-problems and integrating their solution would lead to a solution
for the bigger problem. His method is usually seen as an iterative analysis - syn-
thesis model of design. Analysis is the stage of decomposing the problem into
mostly independent smaller problems and synthesis is the stage of solving these
smaller problems and integrating the resulting solutions.
Archer [1963-641 prescribed a three phase design process based on the following
six activities:
1. Programming: establish crucial issues, propose a course of action.

2. Data Collection: collect, classify and store data.
3. Analysis: identify sub-problems; prepare performance specifications; reappraise proposed program and estimate.
4. Synthesis: prepare outline design proposals.

5. Development: develop prototype designs; conduct validation studies
6. Communication: prepare manufacturing documentation.
The three phases based on the above activities are: analytical (activities of programming and data collection), creative (activities of analysis and synthesis) and
executive (activities of development and communication). He suggests:

"One of the special features of the process of designing is that the
analytical phase with which it begins requires objective observation and
inductive reasoning, while the creative phase at the heart of it requires
involvement, subjective judgement and deductive reasoning. Once the
crucial decisions are made, the design process continues with the execution of working drawings, schedules etc., again in an objective and
Information Handling Behavior of Designers During Concepturu Design: Three Experiments
24
descriptive mood. The design process is thus a creative sandwich. The

bread of objective and systematic analysis may be thick or thin, but the
creative act is always there in the middle. [Archer, 1964]
The most recent prescriptive design model is by Pahl and Beitz [1977 and 1984].
This model presents design process as iteratively going through four stages.
These stages are: clarification of task, conceptual design, embodiment design and detail
design. Each stage is accomplished by the following seven step procedure:

1. Clarify and define the task: Specifications
2. Determine functions and their structures: Function structure

3. Search for solution principles and their combinations: Principal structure
4. Divide into realizable modules: Module structure
5. Develop layouts of key modules: Preliminary layouts
6. Complete overall layout Definitive layouts
7. Prepare production and operating instructions: Product documents
This mode1lays out a systematic procedure for developing a good and complete
understanding of the problem, breaking it up into smaller problems, solving

these smaller problems and combining the solutions to obtain the overall solution. This model has wide acceptance in the European design community.
It is not uncommon to see criticism for prescriptive methods. They are seen as
focusing on the problem rather than on the solution and therefore not reflecting
designer's manner of thinking. They also structure the design process into a set
of compartments with well defined boundaries. This is very contrary to how
most designers work and what many researchers have observed as will be evident from the descriptive studies below.
2.1.2 Descriptive (Empirical) Studies

Unlike prescriptive studies which layout a sequential procedure, descriptive
studies aim to provide a description and understanding of what goes on during
the design process as a basis for improving design practice. These are typically
Information Handling Behllvior of Designers During Conceptual Design: Three ExperimcnL'
25
based on formal observations and an in-depth analysis of design activity. Much

of the recent design research falls under this category. Methods have been developed to conduct experiments to collect raw data. The data is analyzed2 to gain
an understanding of certain design behavior. This understanding is subsequently used to develop services to improve design practice. Waldron and
Waldron [1996] have presented a survey of the various methods that are in use
today to study design. Definitions, advantages and disadvantages of these methods are summarized in Table 2-1.
Table 2-1 Empirical design research methods. The table lists brief definitions of some of the
popular empirical methods and the design scenarios for which they are appropriate. Advantages
and disadvantages for each of the methods is also discussed.
Empiric:aI Method
Appropriate
Scenario
Disadvantages
Advantages
Deposition: timely intersingle

views during the process
designer
(can also be used with verbal protocol)
- targeted .:lata capture

- easier analysis
- can study long tasks
- data is chronological
- interferes with design process

- incomplete data capture
Verbal Protocol: designer single

designer
talks out loud during
design to externalize thinking process
- access to designers thoughts

- comprehensive data capture
- difficult and time consuming

analysis procedure
- can only study short tasks
- verbalizing may interfere
with design process
Discussion protocol: natu- small team
- no interference with design

process
- can study communication.
negotiation and such

analysis
- can only study short tasks
Retrospection: interviews single or

or questionnaire after the
small team
design process

process
- targeted data capture
- easier analysis
- data collected is not chronological or real-time

- partial data capture
Participant observer:
observer is also a member
of the design team
- can study very long tasks

process

analysis
raI communication among

team members
small or
large team
The studies reported in this dissertation in Chapter 3 and Chapter 6 were conducted using the verbal protocol method. Section 2.1.3 is devoted to describing
and discussing the appropriateness of the verbal protocol method. In subsequent
2. An analysis procedure is usually a by-product of these research techniques.
Information HllOdling Behavior of Designers During Conceptual Design: Three ExperimenlS
26
sections studies conducted using verbal protocol method and some studies conducted using other methods will be discussed.
2.1.3 Verbal (or Think-Aloud) Protocol Method

The verbal protocol method was developed in early sixties to study human problem solving behavior [Newell and Simon 19721. In this method the experimental
subjects are requested to verbalize their thinking (think-aloud) as they work on
the task given to them. The verbalizations are recorded on audio and video
medium.3 These provides the raw data for analysis. This method was and is
most commonly used in cognitive science research. However in the last decade
this method has gained noticeable popularity within the design research commu-
nity as is evident from the number of researchers, publications and workshops

focusing on this method [Stauffer, 1987; Ullman et al., 1988; Guindon,1990; Christiaans et al., 1993; Cross et al., 19961.
The appropriateness of the verbal protocol method has been debated extensively
for and against by Ericsson and Simon [19931 and Nisbett and Wilson [1977]
respectively. Crutches [1994] emphasizes the value of the verbal protocol
method as a means for making data explicit for capture. He reports:
"Beyond the increased acceptance of verbal reports as behavioral data,
the unique advantages of verbal report data are increasingly apparent to
many people. In particular, researchers have emphasized the advantage
of verbal reports as protocol data, providing a sequence of observations
over time rather than just a single observation at the end of the process.
......... In addition, verbal protocols provide many more observations of a
phenomenon over a given time period than other methodologies, increasing the information yield of studies.......... Verbal reports can provide
3. In its early usage, the verbal protocol was usually recorded on audio medium. Now days, because of
advancements in video technologies, the protocol is also recorded on video medium. For studying the
design process, video data is an essential ingredient for analysis since many actions which are non-verbal,
and therefore cannot be recorded on an audio medium, get recorded on video medium resulting in a more
complete capture of data.
Information HllIIdling Behavior of Designers Ouring Conceptual Design: Three Experiments
27
information difficult to obtain by other means- in particular, information

about types of knowledge accessed in task processing..... " [Crutcher,
1994, In Psychological Science, page 242]
The important issues concerning verbal report data are:

Validity: Does the information in the verbal report reflect the actual thinking process?
Interference: Does the act of verbalizing alter the normal thinking process?
Completeness: Does the method capture all relevant data?
From a cognitive viewpoint Ericsson and Simon [1993] have argued that:
"subjects can generate verbalizations, subordinate to task-driven cognitive processes (think aloud), without changing the sequence of their
thoughts, and slowing down only moderately due to the additional verbalization." [Ericsson and Simon, 1993, page xxxii].
This means that as long as thinking is assumed to be a sequential process4, the
verbalizations are a valid reflection of the natural thought process. From the
viewpoint of information handling activity during a design process, the evolution of the design is reflected in the sequence of actions performed by the
designers. Verbal protocol reports (captured on audio as well as video medium)
will provide an accurate capture of these sequences.
As stated by Ericsson and Simon above, verbalizing does cause the thought pro-
cess to slow down moderately. However, the interference to the natural thought
process can be minimized by providing adequate instructions to the experimental subject. Payne [1994] suggests that while it is not necessary for subjects to be
extensively traineds, they should not be requested to verbalize a particular type
of information. He reports:
" ...... Extensive training in verbalization is seldom needed or really
4. Verbal protocol analysis assumes that cOgnition, specially for tasks such as problem solving, is serial.
5. It is a common experimental procedure to train the subjects on a short trial problem to get them comfortable with thinking out loud.
28
desired ... The subject must remain focused on the task, with verbalization clearly subordinate.... Researchers should also avoid temptation of
trying to get the subject to give a coherent sequence of verbal reports by,
for instance, asking for verbalizations of only selected information (e.g.
reasons for behavior). Such efforts frequently compromise concurrent
verbalizations. The subject should be asked to verbalize all thoughts that
occur to him or her during the performance of the task without emphasis
on any particular type of information." [payne, 1994, in Psychological Science, page 246]
From the viewpoint of completeness, the verbal protocol method has a drawback. Many representations and information that designers deal with during
design is non-verbal. Much of this information is not captured in the verbal portion of the protocol data. However video recordings can be used, as is common
now days, to capture some of the non-verbal information.
From the accounts above, we can conclude that verbal protocol method is appropriate for collecting data for single designer working on short duration design
tasks. Experimental subjects should verbalize all of their thinking and should
not be requested to verbalize any specific kind of information. This is specially
critical if the focus is on observing the information related behavior as was the
case in the studies presented in this dissertation.
2.1.4 Studies Using Verbal Protocol Method

Akin [1978, 1979] used the verbal protocol method to study the architectural
design process. He was interested in understanding and analyzing the information processing activities of the designers and finding the relationships between
the design problem and the representations used. He observed that designers
shifted attention among different issues frequently, and that there was a relation
between the representations used and the issue being attended to. From his analysis he concluded that designers exhibit eight recurring behavior patterns which
he called schemata. These schemata are:
29
1. instantiation: creating a new item or symbol

2. generalization: grouping related symbols together into a common category
3. inquiry: seeking new information, internally or externally
4. inference: generating new information based on internal knowledge
5. representation: making a visual image of an item
6. goal-definition: defining goals to pursue solutions
7. specification: specifying a partial solutions
8. integration: combining partial solutions
He also observed that design is a top-down, breadth-first process and that

designers seek solutions that adequately satisfy the requirements rather than
develop optimal solutions.
Ullman [1988] and Stauffer [1987, 1991] used protocol analysis method to study
five mechanical designers doing non-routine tasks. Based on their analysis, they
propose an information processing system model which describes the problem
solving process of a mechanical designer. It is called task/episode accumulation
model (TEA model). They state:
" ........ The key features of the model are (a) the design is constructed by
incrementally refining and patching the initial conceptual design, (b)
design alternatives are not considered outside the boundaries of design
episodes (which are short stretches of problem solving aimed at specific
goals), (c) the design process is controlled locally, primarily at the level of
individual episodes."[U11man et al. 1988 page 33]
The TEA model contains ten operators which are identified as the finest information operator. The design is accumulated gradually by the application of
operators in a meaningful sequences called episodes. Hence the name task/
episode accumulation model. The episodes are on an average 56 seconds long.
Guindon [1990] studied early stages of the software design process using verbal
protocol method. She analysed protocols from three experienced designers and
observed that, designers deviate from a strictly top-down approach in early
Information Handling Behavior of Designers During ConceplUal Design: Three Experiments
30
stages of the design and display an opportunistic behavior. She states:

"........ Designers were observed interleaving decisions at various levels of
abstraction in the solution decomposition....... The sudden discovery of
new requirements and partial solutions triggered by data-driven rules
and associations, the immediate developments of solutions for newly discovered requirements , and drifting through partial solutions are
important causes of opportunistic design.
II
Using the results of her study she suggests implications for training, methods
and computational environments to support early stages of the design process.
The interest in using protocol analysis to study design process has been growing. However, much of the effort is isolated among many research groups. It is
therefore difficult to generalize, validate or compare results across researchers
and therefore incrementally grow the knowledge about the design process. To
address these concerns a protocol analysis workshop was organized at Delft,
The Netherlands in September 1994. All participants of the workshop were
requested to analyze data resulting from the same experiments so that analysis
procedure and conclusions could be compared. Cross et al. have summarized
the results of this workshop in a book [1996]. They concluded that use of protocol method for studying the engineering design process is in its very early
stages. They state:
fl............
Protocol analysis is clearly not a universal cure for design

research problems. Nevertheless, we feel that we can say that protocol
analysis as a research technique for design has been 'validated' with
some qualifications.....
The qualifications were concerned with the completeness of the data collected
and the process of carrying out the analysis. They caution that, (a) verbal protocol method does not capture the non-verbal thought process going on in design
work and therefore should not be treated as capturing complete data, and (b) the
bias of experimental setup, researcher and experimental subject on the analysis
Information Handling Behavior of Designers Ouring Conceptual Design: Three ExperimenlS
31
procedure should be minimized.
2.1.5 Studies Using Other (Non-Verbal Protocol) Methods

Waldron and Waldron [1988] conducted a study using the retrospective method.
They studied a large design project on the design of an experimental walking
machine involving several designers over a few years. They constructed the
design process sequence retrospectively after the design process was complete.
This was based on interviews with the designers and records from different
stages of the design project. They found that

experience of the design team members had a significant influence on the
development of concepts.
competing design concepts were pursued in parallel
important decisions were based mainly on qualitative information and simpie models
the separations between conceptual, layout and detailed design phases was
not distinct, rather all phases were encountered all along
the design team members changed roles among different conceptual, layout
and detailed design throughout the design project
Tang [1989] used the discussion protocol method to study the use of shared
workspaces by a group doing conceptual design. Using techniques borrowed
from video interaction analysis he studied 8 groups of 3 or 4 designers work on
short term design tasks. He observed that the actions of the designers could be
grouped under the categories of list, draw and gesture. These actions were used
to accomplish functions of storing information, expressing ideas and mediating interaction. He observed that a large portion of the actions are gestures which are not
adequately handled by the existing computational support tools. He recommends that tools to support workspace activity of teams working on conceptual
design tasks should consider:
providing ways of conveying and supporting gestural information
minimize the overhead encountered in storing information
32
- allowing intermixing of workspace actions and functions

-enabling all participants to share a common view of the workspace
while providing simultaneous access and a sense of close proximity to it
- facilitating the participants' ability to coordinate their collaboration
[Tang 1989, page 101]
Minneman [1991] used discussion protocol method to study group engineering

design practice. He collected data from an industrial team working on design of
a photocopier sub-system as well as conducted short experiments to simulate
design communication between different divisions of a design firm (engineering, manufacturing and marketing). He expressed his understanding in terms of
a framework and designers practices. These are:
1. Framework: a perspective for thinking about the content of design work.,
has two dimensions- facets (the things being worked on) and trajectories (a
temporal sense to the work being done), and
2. Practices: actions for getting design work done. Four practices were
observed to be crucial. These are negotiating understanding, conserving
ambiguity, tailoring engineering communication for recipients, and
manipulating mundane representations.
The framework highlights that participants have a personal views of the ongoing
activity and at any given time, design activity is doing communicative work on
multiple projects. The practices signify that, designers' activity is not something
shaped by externally imposed context, but rather as attending to and creating a
recognizable order in the ongoing social interaction. He argues that there is no
innate structure to designing, the structure in the activity is a product of the
designers taking a series of local actions to produce a result.
2.2 Design Process: An Information Handling Perspective

In the previous sections we looked at many studies which have attempted to
understand either the individual or the team design process. These studies
examined the process from a variety of perspectives such as: problem solving,
information processing, workspace activity and collaboration. The studies in this
33
section examine the design process from an information handling perspective.

They are aimed at understanding the information needs and information management burden on designers.
Hales [1987] performed a detailed analysis of all the activities from one project
involving the design of a high-pressure, high-temperature system for testing
materials. He was a participant observer in this project over a period of 2.8 years
which involved 37 different participants. He found that only 47% of the total
observed effort was spent in doing design. The remaining 53% was spent in
information retrieval, planning, cost estimating, reviews, social contact and helping others. He observed that structured or prescriptive methods accounted for
less than 25% of the design effort. Also, over 50% of observed effort involved
either working alone or in pairs and about 30% in meetings of 2, 3 or 4 people.
This emphasizes that large portion of the design happens by individuals or
small teams. Based on a quantitative analysis he found that design process can
be characterized by a set of overlapping phases, each consisting of a particular
mix of procedural steps and other general activities.
Kuffner et al. [1991] conducted a verbal protocol study over six experiments to
determine the information requests of mechanical design engineers doing redesign. He developed a taxonomy of design information, based on the questions
and conjectures raised by the designers. Design information was categorized by
its nature (construction, location, operation, purpose) and topic (assembly, component,
interface, feature). These nature and topic are identified as the information that
should be captured, structured and made available in CAD environments for
facilitating reuse.
Court et al. [1993] conducted a questionnaire based survey of over 200 engineers
and designers from various industries (power generation, automotive, construction, manufacturing, agriculture and utilities) to determine their information
requirements. This comprehensive survey covered industries of all sizes (small,
34
medium, large) and targeted designers in variety of roles (management, creative

design, routine design, etc.). They found that only about 43% of designers time is
spent doing design. The rest of the time is spent in meetings (16%), searching for
information (18%) and paperwork (23%). They also found that 40% of the
respondents did not use any formal operating procedures or methods. Their
study underscores the fact that designers spend a large portions of their time
handling and managing information, and that there is a lack of adequate computational tools to alleviate the information management burden.
The above studies provide some understanding of information related behavior
of designers. More so, they bring out the need for developing a richer understanding of the information handling behavior so that it can be supported by
novel services and methods. The following chapters will describe the studies
that were conducted using the verbal protocol method to develop an understanding of the information handling behavior of designers as a basis for
developing requirements for information management frameworks and services.
Chapter 3
Design Information Needs,
Capture & Reuse
/------( Information Classification \

Framework
J
\
-------_/
/-----,
/-----,
( Dedal Performance \
\.
Results
I
( Information Handling \
\.
Framework
I
___ 1__________l __________ l __ _

Analyze
Is
Development
of Dedal
19901991
Information
Reuse Study
Analyze
;.
Analyze
~eVise
Dedal
/.
1991-1992
Support by
Observe
Dedal
---~~ Dedal Use
1992-1995
Support by ___
~~ Information
Dedal
Handling
Study
Understanding
of Information
Handling
This chapter describes the experimental procedure, analysis method

and results of an observational study conducted to understand the
information needs of designers during the design process. Questions
asked by the designers while engaged in design activity are captured
to comprehend their information needs. A classification for these
questions is developed. This classification furnishes the framework
for design information that should be captured and structured during
the design process to enable effective reuse in the future.
35
Chapter 3: Design Information Needs, Capture & Reuse
36
3.1 Design Information

Engineering design is an information intensive process. All along the design process, designers need information to complete their tasks. They fulfill these
information needs by retrieving information from a variety of sources such as:
personal information databases, colleagues, handbooks, technical reports, vendors, suppliers and in some cases by library/information retrieval services.
From a survey of more than 200 industry designers from various industries
Court et. al. [19931 found that designers spend in excess of 18% of their time
searching for information. In a participant observer! study [Hales 1987] found
that 53% of the design effort was spent on non-design related activities. In his
study Hales considered information retrieval a non-design related activity.
These studies corroborate the understanding that timely availability of appropriate information in a usable format will significantly improve design practice.
3.1.1 Information Reuse Essential for Good Design Practice

It is well known that much of the design activity taking place in industry is rede-
sign. A new design is either an improved version of an older design, or an

adaptation of a design from a different domain, or a novel combination of features from many other designs. In all these scenarios knowledge and experience
from previous designs is used in designing a new product. Figure 3-1 shows
how the process is cost and time effective if designers reuse design information
from past designs rather than re-engineering it. But, a prerequisite to effective
reuse is capturing information in reusable form. The design process does not necessarily lend itself naturally to capturing information in a reusable form. Allen
[1977] reported that, during the design process designers use information to produce physical changes in the world. They consume and transform information to
produce a product which is information bearing, but this information is no
1. A study where the observer is a member of the design team being studied.
.-
- past experience
- technical documents
-colleagues
~'7
Information Capture
allows Reuse
r=
Old
and is
Cost and Time Effective
Satisfy
'- Information
Design
Information
37
-/
Needs
'-
Design Process
,
-"'7
- reusable info
- structured info
j~
New
Design
Information
o.
~
- physical artifacts
- CAD drawings
- past designs
No Information Capture
requires Re-Engineering
which is
Expensive and Time Consuming
..)
- non-verbal info
- non-reusable info
Figure 3-1 An information perspective on the design process. The figure shows
that old design information can be used in satisfying information needs during a
design process resulting in new design information. Reusing old information will
be cost and time effective compared to re-inventing it. Current design practices
have less reuse (shaded region) and more re-engineering (unshaded region).
Reuse can be increased by capturing information in reusable form.
longer in verbal form. Information in the form of physical or even software product artifact is difficult to reuse, since it does not capture decisions and the
rationale behind those decisions. Therefore, capturing information in a reusable
form is necessary to avoid expensive and time intensive re-engmeering.
3.1.2 Design Information Capture is Difficult

A good methodology for capturing design information is a prerequisite for effective reuse. The capabilities of current design tools in capturing information is far
less than what is needed to realize the benefits of reuse. Information capture services available today integrate poorly with the design process, burdening
designers with additional tasks. Consequently these services are under-utilized
or not used. This is specially true for the conceptual design phase. Design inforInformation Handling Behavior of Designers During Conceptual Design: Three Experiments
38
mation capture is difficult because:

design information is often informal, fragmented or incomplete and therefore difficult to represent in a computational medium
the design process most often results in non-verbal information bearing
product
design information exists in many media
effort is required to structure information for reuse
it is not practical or wise to capture all information, therefore capture must
be selective
capture services cannot keep pace with the design process, i.e. rate of information capture is less than the rate of information generation (by a wide
margin).
There is a need to develop innovative information capture services which integrate smoothly with the design process and capture information in a form
suitable for reuse. We can build such services if we have a better understanding
of the information needs of a designer. This chapter will focus on developing
this understanding.
3.1.3 Information Reuse Study Objectives

The objective of the information reuse study was to develop an understanding of
designer information needs and develop a framework and guidelines for developing information capture and reuse services. Understanding provides
knowledge of the information that should be captured during design. It will also
inform the development of guidelines for structuring this information for reuse.
Kuffner [1991] conducted a study to analyze the information requests of mechanical engineers. He came up with a taxonomy which characterized information
based on its nature, topic and category. There are some similarities in the objectives and the approaches of his study and this study. These studies will be
compared in the discussion at the end of this chapter.
39
3.1.4 Questions Reflect Information Needs

It is natural for humans to express a need for information in the form of a question. For instance, if you needed to find the torque developed by a motor
manufactured by a vendor, you will most likely express your need in the form of
a question such as "How much torque does this motor develop when hooked up
to a car battery?" An answer to this question would fulfill your information
need. If we were to capture the questions that designers ask while engaged in
design, we would have a view into their information needs. The information
reuse study was designed to capture the questions that designers ask while
engaged in design.
3.2 Experimental Study: Verbal Protocol Method

I designed an observational study using the verbal protocol method to capture
the questions designers have when engaged in design activity. Experimental subjects were asked to redesign an existing product to a new set of requirements. I
chose the experimental method since it allowed me to:
look at the design process from a reuse viewpoint
deal with the complexity of the process by exercising adequate control
simulate real design activity
study design activity at a fine level of granularity
make designers actions explicit for observation
I used the redesign scenario in the experiments since, redesign is a very common
scenario in industry and information reuse is prevalent during redesign.
3.2.1 Experimental Setup and Procedure

Figure 3-2 shows the experimental setup used to collect data. The subject
worked in a room fitted with two video cameras. One camera captured a wide
angle view of the workspace and the other captured a close-up of the desk. The
40
data was independently recorded on the two cameras and an audio recorder.
~"
fi
Workspace
...
./
Documents
"
Figure 3-2 Experimental setup for information reuse study. Two video cameras
capture activity in the workspace (camera 2) and the wide angle view of the
subjects work area (camera 1). The subject has access to documentation from
the first generation design, and to a member of the first generation design team.
(expert). The expert sits in a separate room and is available on request.
At the start of an experiment the subject was given a small problem, to get him
comfortable with talking out loud and the experimental setup. After this he was
given the experiment's redesign problem. The subject was asked to complete the
redesign task in about six hours. The experimenter was out of the room monitoring the subject on video. If the subject was silent for an extended duration (15-20
seconds), the experimenter would remind the subject to talk out loud. The experiment was conducted over two sessions with an hour long break in between. The
subject was asked to summarize the status of his work before and after the break
to capture any thoughts that may have occurred during the break.
Subjects had access to design notebooks and technical reports from the first gen-
41
eration design. These contained detailed description of the first generation

functional prototype and its performance. One member of the original design
team, to be referred to as the expert, was also available to answer questions and
serve as an index to information in the documentation which totaled more than
a thousand pages. The subject using the expert as an index into the documentation provided a natural way of making the questions explicit without being
obtrusive to the subject. The expert stayed outside the room but could be called
in by the subject any time. The expert stayed out of the room, so as to be unaware
of the information that the subject was encountering. This setup assured that the
expert answered any questions objectively without volunteering additional information. The instructions and the redesign problem used in the experiment are
attached in Appendix A.
3.2.1.1 The Design Problem: Continuously Variable Damper
The problem chosen was the redesign of a continuously variable car damper
(popularly known as shock absorber) to meet a new and revised set of requirements. The design was complex enough that the subjects needed to develop a
good understanding of the first generation design before coming to a solution
and simple enough that a conceptual solution could be reached in about six
hours. Such a trade-off was essential to conduct an experiment in a laboratory.
The first generation design was done in a real world academic setting by a three
member design team. This project was done for a leading USA automobile manufacturer over a period of seven months [Baya 1990]. The design was carried out
from concept to laboratory testing of a working prototype. Three documents
spaced equally over that period were produced by the design team. Each document summarized the status of design at the time it was written. These
documents captured the evolution of the first generation design from early ideation stage to final machine drawings and test results of a functional prototype.
Information Handling Behavior of Designers During Conceptual Desigo: Three Experiments
42
3.3 Analysis of Verbal Data

The analysis required a procedure for transforming verbal data into a framework which reflects design information needs. This procedure is shown in
Figure 3-3. It involves six steps. The form of the data at input and output of each
Design
Problem
Conduct
Experiment
Q uestions with
Context
Reformulate
Questions
Classify
Questions
Recorded Data
Transcribe
Questions
Classification
Written Data
Extract
Questions
Statistical
Analysis
Quantitative
Results
Figure 3-3 Analysis procedure for information reuse study. The steps in the
analysis reduce recorded data into quantitative description of the designer
information needs.
of the steps is indicated. Altogether these steps reduce recorded data into a quantitative description of the information activity of designer in this scenario. Each
of the steps is described below. The examples for each of the step are discussed
in section 3.5.
3.3.1 Transcription
The process of transcription yields written protocol data. The transcript captures
the words spoken by the subject as well as information about pauses, intonation,
focus of attention and context. Video tapes are used in this step to add action
and gesture data to the record. The resulting transcript has the details necessary
for carrying out the rest of the analysis. However there were occasions when
raw data was used in later stages of analysis to resolve ambiguities.
Information Handling Behavior of Designers During Conceptual Design: Three ExperimcnlS
43
3.3.2 Question Extraction

The written data resulting from transcription is used for extracting questions.
For the purpose of this study a question is an interrogative expression uttered by
the subject to seek information (e.g.. what is the diameter or X?) , or an utterance
which suggests a need for information, although it may not have been explicitly
asked (e.g.. I should find the diameter of X). In ambiguous cases, the subject was
consulted (after the fact) to determine if an utterance was a question or not.
Some key scenarios and rules in extracting questions were:
explicit ask to expert e.g.. "How heavy is this device?"
searching of documentation by self e.g.. "Where are the test results?"
utterance to self e.g.. "I will need to find the thickness of this part."
conjecture, e.g.. "This must be 12 inches."
a question uttered twice within the same activity focus! was counted once.
a question repeated in an unrelated context or in a different activity focus
was counted again.
a complex question was broken into a set of simple questions.
3.3.3 Question Reformulation

Questions as spoken by the subject in the transcript are often dependent on
nearby information to be intelligible. i.e. they have meaning within the context
of their location in the transcript. Reformulation was the required to fill in contextual and missing information so that question classification could be done
objectively without consulting the transcript.
3.3.4 Question Classification

Before questions could be classified, there was a need to develop a framework
for doing so. Figure 3-4 shows some of the important steps involved in the development of the question classification framework. This framework is described in
1. An activity focus was defined by closeness in context, time or design goal of the designer.
Information Handling Behavior of Designers During Conccprual Ocsign: Three ExpcrimcnlS
44
Questions
step 1
Characterize ~
"1
/
/
\"
____
//"
\"
\
No
""
/ ,
"
" step2
Collection of Question Characteristics
~"'/~
Group into Categories
step 4
Label Categories - -~
step 6
Can all questions be classified?
Yes
Classify New
Questions
Rgure 3-4 Steps in the development of question classification framework. First

step is divergent and generates a collections of terms to characterize questions.
This collection is grouped, based on similarity in characteristics or meaning, into
categories. These categories were labeled. If this set of categories were suitable
to classify any question encountered in the data then it was the classification
framework.
detail in section 3.4. Question classification used this framework on the reformulated questions. The classification was based on interpreting the questions with
respect to a template as a method of maintaining rigor and objectivity in classification. The template is shown in Figure 3-5.
3.4 Question Classification Framework

Table 3-1 shows the framework for classifying questions. In this framework a
question is characterized by four attributes namely
MEDIUM
and
LEVEL-oF-DETAIL. It
DESCRIPTOR, SUBJECI'-cr.ASS,
is classified by being given a label from each of
the four attributes. The values for each of the attributes are shown in Table 3-l.
They are not a list of all possible values. They represent what is observed in the
experimental data and suggest values significant for describing design informationneeds ..
45
Table 3-1 Framework to classify questions. The same framework can be used to
classify design information. This will also be called the Design Information Framework
(DIF).
subject.c:Jass
Descriptor
Medium
Level of Detail
Alternative
Assembly
Audio
Conceptual
Construction
Component
Video
Configurational
Location
Connection
Text
Detail
Operation
Feature
Graphic
Performance
Requirement
Gesture
Rationale
Design-concept
Relation
Other
Requirement
Miscellaneous
3.4.1 Design Information Framework (DIF)

The framework that is used to classify design information is the same as the one
used to classify questions. This is because any segment of design information
can be considered to be an answer to an implicit question. Thus it is possible to
interpret design information segments in a template identical to that of a question as shown in Figure 3-5.
3.4.2 Descriptor
The descriptor of a question refers to the inherent character, the basic constitution or the nature of design information that is sought by the question. The
descriptors are independent of the design problem or the domain of design.
Table 3-2 shows the descriptors, their definitions and examples.
3.4.3 Subject-Class
The subject-class of a question refers to the class to which the subject of the question belongs. The subject of a question is the subject of the sentence or the clause
making up the question. Like descriptors the subject-classes are independent of
the design domain. The subjects of the question are dependent on the design
46
Question Interpretation Template:
the subject is
requesting,
information
OOll)
(LEVEL-QF-DETAIL)
regarding the
0'IlS
of the question-subject which
(DESCRIPTOR)
belongs to
OSC
(SUBJEcr-CLASS)
r---------------------------,
Design Information Interpretation Template:
the
design
information
segment
is
lOll)
(LEVEL-OF-DETAIL)
information regarding the
l'lYES
(DESCRIPTOR)
of the information-
subject which belongs to --r=~l=:SC~'=--=-=.

(SUBJEcr-CLASS)
L ___________________________
Questionlinformatjon classification based on above interpretation

Question Descriptor: Q!JYES
Question Subject-Class: QSC
Question Level-of-Detail: QOIl)
Information Descriptor: l'lYES

Information Subject-Class: lSC
Information Level-of-Detail: l.OlD
,,-----------------------------------------------------------------_.
I
Figure 3-5 Template for interpreting question and information segment.

Question-subject and information-subject in the above templates refers to the
subject of the sentence forming the question or the segment of design information
respectively.
domain. Table 3-3 shows the different subject-classes, their definitions and
examples.
3.4.4 Medium
The medium of a question refers to the form of the information which would
adequately answer the question. While this is an important category for classifying design information, from a computational viewpoint, the verbal protocol
47
Table 3-2 Question descriptors refer to the basic constitution of the information sought by the
question. It can take the values defined below in the deSCriptor column.
Descriptor
Question seeks information regarding
Example
Alternative
choices or decisions
What are the other ways of changing

linear motion into rotaxy motion?
What does the damper design look
like?
Construction
topology, shape, dimensions, arrangement

properties
Location
position or site
Operation
manner of functioning
Performance
behavior, efficiency, evaluation/test

results
Where do you want the maximum

resistive force in the full ON state?
How is the resistive force varied in the
hydraulic concept?
What was the maximum temperature
at the solenoid during test?
Rationale
purpose, reason, basis, explanation of

choices
Why did they choose the rotaxy friction concept?
Relation
inter-dependence, cause-effect, interaction
What is the effect of temperature on

the solenOid?
Requirement
specifications, needs, constraints
Miscellaneous
process details, plans, strategies, and anything not included in the other descriptors
What was the limit on solenoid stroke

length in the previous design?
What are the characteristics of the
suspension system?
Table 3-3 Question subject-class refers to the class to which the subject of a question
belongs.The seven classes below cover the range of question subjects encountered in the data.
Subject-Class
Subject or the question refers to
Example
Subject
damper
Assembly
an assembly, sub-assembly or a
collection of components
Is the shape of the volume of

the damper a cylinder?
Component
a part of the assembly, a single

item
Does this end of the arm go top

to bottom?
arm
Connection
interface between two or more

components
How is the disk attached to the

hub?
attachment
Feature
attribute of a assembly or component, physical properties
Why should stroke length be

less than 6 inches?
Requirement
specification or constraint of
the design
How heavy was the last design?
strokelength of
solenoid
weight
Design-concept
design concept, partiallyformed ideas
How does rotaxy friction concept develop resistive force?
rotaxy-frictlon-concept
Other
anything not included above
What are these four lines in the

graph of the solenoid?
lines
method skews the data heavily towards the audio medium. As a result I have
not made an attempt to draw any conclusions regarding the medium. Table 3-1
Information Handling Behavior of Designers During Conceptwll Design: Three Experiments
48
lists some possible values that medium can take. This is not a complete list. It is
a representative sample of the dominating media observed in the study.
3.4.5 Level-of-Detail
If we were to classify the information about a design on how detailed it is, then
the level-of-detail of the design information which answers a question is the

level-of-detail of the question. It is common perception that design requires
information at the following levels-of-detail: conceptual, configurational and detail.
These levels differ in the extent to which design information is specified. Table 34 defines the levels-of-detail with examples.
Table 3-4 Question level-of-detail. refers to the specificity of information sought by the question.
Level of Detail
Question seeks information at a
Conceptual
qualitative, conceptual level
Configurational
layout, arrangements of components
Detail
specific, quantitative
Example
What are the other design concept for

the force generation mechanism?
Where does the solenoid fit in in the
damper?
How much resistive force should the
damper have when the arm is here?
3.5 Analysis Examples

A few examples of analysis from transcription to classification will clarify the
procedure. Table 3-5 shows these examples. Note that the original verbalizations
are rarely complete sentences or grammatically correct. Question segments are
underlined.
3.6 Results
Table 3-6 summarizes the background of the experimental subjectsl . The first
experiment lasted about five and a half hours and the second about three and a
half hours. 160 question were extracted from the first and 80 from the second
1. The designers in the two experiments will be referred to as RSl and RS2 respectively.
Information Handling Behavior of Designers During ConceplUaJ Design: Three ExperimenlS
49
Table 3-5 Example of analysis procedure in information reuse study. The segments 7,8,9 and 10
cover a continuous period of about 50 seconds. The question from protocol is underlined and
shown again in the second column. The rephrased question is also shown in the second column
under the question from the protocol.
Question from Protocol
No.
7
After 'framcription
All right, let'srz see.
(pause} This is a, that
was design a continuously
variable damper. 1a tb1a
[d~aign g~gbl~l A
~gDtiDUg~alx ~~1abl~
rumroer' Yeah, okay. Okay,

let's see.
S
Now, I don't see why, do
I really have to go, or

gg 1 baxe tg ~g ~1tb Kbat
tb!iD{;
!~r~gya
team} came
up
aeaign
with"
It depends on what their,

on what they found out
about the other
possibilities. tibae dZ;:~
tb~ g~A~~~ka
gf
tb~
other Systems?
10
I think they had that in

here. (opening d3,
flipping pages} When they
were considering other
systems, like, um. Okay,
the following section
points out approaches,
(reading} each of which
we believe has the
potential to develop into
a reliable prototype.
Each design should
mmble (inaudible} into a
whole class of possible
solutions,frictional,elect
rorheological, and moving
fulcrum concept. (stop
reading} (pause} Okay,
wllat lrli:Z;:~ eD!: a~DtAg:Ii::i
Question Rephrased
Is this a
continuously variable
damper?
Is it a continuously
variable damper in
the design problem?
00 I have to go with
what they came up
with?
00 I have to go with
the deSign concept
the first design team
came up wi th?
What were the
drawbacks of the
other systems?
What were the
drawbacJcs of the
other design concepts
the first design team
looked at?
What were the
advantages to this?
Wha t were the
advantages of the
hydraulic concept?
After Reformulation
Question Classification
Is it a
continuously
variable damper in
the design problem?
Desaiptor. Requirement
Subjclass: Assembly
leveL oJDetaH: Conceptual.
00 I have to go
with the design
concept the first
design team came
up with?
Descriptor. Miscellaneous
Subjclass: Design-Concept
leveL ofDetail: Conceptual.
What were the

drawbacks of the
design concepts
the first design
team looked at?
Desaiptor. Rattonaie
Subjclass: Design-Concept
leveL oj Detail: Detail
What were the

advantages of the
hydraulic concept?
Desaiptor. Performance
Subjclass: DesignConcept
leveL oj Detail: Detail
tg tbh'
one. A possible reason for this difference is the familiarity of the subjects with
the expert. RSI was familiar with the expert and felt comfortable asking question
whereas RS2 was not that familiar and preferred to look through documentation, therefore not externalizing some questions. Another possibility is that RS2
being experienced, asked fewer but pertinent questions and also took about 2
hours less than RSI in coming to a solution.
Information Handling Behavior of Designers During Conceptual Design: Three Experiment'
50
Table 3-6 Information reuse study: subjects background information.

Design
Experience
Experiment
Subject
Duration
Familiar with
Expert?
No. of Questions
Extracted
RSl
2 years
5.5 hours
Yes
160
RS2
10 years
3.5 hours
No
80
The distribution of questions, in the classification, in the two experiments is not

significantly different. The results below are for questions from both the experiments. Appendix A has results for individual subjects, RSI and RS2, and a
comparison of any differences.
3.6.1 Results on Descriptors

Figure 3-6 shows the distribution of questions across the descriptors. Information regarding performance, relation and construction was frequently accessed. All
other descriptors are well represented. About one fifth (21%) of questions were
miscellaneous. Thus this rather sparse classification scheme dealt effectively with
about 80% of the questions encountered. However 21% is a large enough num-
Performance
Relation
Construction
Operation
Alternative
Rationale
Requirement
Location
Miscellaneous
21
30
Percent of Questions
Figure 3-6 Distribution of question deSCriptor. Information

with
descriptors
performance, relation and construction was frequently needed. 21 % of the
questions needed miscellaneous information, i.e. the classification works well for
about 80% of the questions encountered.
ber and suggests that there is considerable interest in accessing information
51
about design process strategies, plans, organization, design methods, evaluation

methods and others. These are some of the characteristics of information
spanned by the miscellaneous descriptor.
The information in Figure 3-6 can guide decisions about what type of information to capture during the design process. The relative frequency of occurrence
of these descriptors can be used to set priorities in developing capture strategies.
3.6.2 Results on Subject-Class

Figure 3-7 shows the distribution of questions across the subject-classes. Information about feature (35%), components (20%) and assembly (12%) were frequently
asked for. We know that there are more features in a design than components,
more components then assemblies. While it is reasonable to see more questions
35
40
Figure 3-7 Distribution of question subject-class. Information on features and

components is asked for frequently, but other classes are also well represented.
addressed to the class which has more members, the breakdown emphasizes the
concern for adequately representing all subject-classes in a capture tool.
There were a significant number of questions on the subject-class requirements
(13%) and design-concept (9%). This suggests that adequate attention should be
given to capturing information about requirements and design-concepts.
Information Handling Behavior of Designers During Conceplual Design: Three Experiments
52
3.6.3 Results on Level-of-Detail

Figure 3-8 indicates that more than half the questions (57%) were asked about
Detail
57
Conceptual
Configurational
10
20
30
40
50
60
70
Figure 3-8 Distribution of question level of detail. The nature of the design
problem used in the experiment may have focused the designers attention to
detail information.
the details of the design. This could partly be due to the nature of the design
problem used in the experiment. Since the design problem was reasonably complex and time available to the subjects limited, the subjects decided to go with
the original concept quite early in the design process and tried to achieve the
new requirements by making parametric changes. This channeled their interest
to the details of the design for the most part. However, about 28% of the questions are addressed to the conceptual level-of-detail. This asserts the need for
capturing information from the conceptual stage.
3.6.4 Results on Crossing Attributes

Interesting results are obtained by crossing two attributes, descriptor with subject-
class, descriptor with level-of-detail and subject-class with level-of-detail.

Descriptor with Subject-Class. Crossing descriptor with subject-class gives us
the knowledge of what type of information about the different subject-classes is
frequently needed. Table 3-7 highlights the combinations of descriptor and subInfonnation Handling Behavior of Desigocrs During Conccptlllll Design: Three Experiments
53
ject-class encountered frequently. Information about relations of features, operation

Table 3-7 Crossing descriptor with subject-class. Highlighted
frequently asked for descriptor for a particular subject-class.
numbers
signify
o
o
o
o
the
most
Subject-Class
Descriptor
Assem.
Comp.
Conn.
Feature
Alternative
Construction
Operation
o
o
Performance
Location
Rationale
Relation
33
5
22
43
20
38
10
o
o
14
10
11
51
49
83
31
22
17
240
Requirement
Miscellaneous
SCL-Total
29
of components and assemblies was frequently asked for. Information about perfor-
mance and construction of features was also significant.

Descriptor with Level of Detail: Table 3-8 crosses
descriptor with level-of-
detail. It can be seen that detailed information regarding the performance, relation
and construction was most frequently asked for. Configurational information was
frequently asked about operations, and conceptual information was often asked
about the alternatives.
Subject-Class vs. Level of Detail: Table 3-9 crosses subject-class against level-ofdetail. It shows that detailed information about features and components was
needed often. Configurational information was frequently asked about assemblies
and components and conceptual information regarding the design-concepts and
requirements was needed.

The information such as in the table above can be used in"filtering and prioritizing information during capture. It can also be used in ordering responses to
54
Table 3-8 Crossing descriptor with level-of-detail. Highlighted numbers show what types of
deSCriptors were most needed at a level of detail.
Level of Detail
Descriptor
Configuratiooal
Detail
Descriptor Total
Alternative
18
Construction
11
33
Location
Operation
22
Performance
43
Rationale
20
Relation
38
Requirement
10
Miscellaneous
25
24
51
LOD-Total
66
37
137
240
Table 3-9 Crossing subject-class with level-at-detail. The highlighted numbers show what
subject-class was most frequently asked for at each level-at-detail.
Level of Detail
Conceptual
SCL-Total
Assembly
Subject-Class
29
Component
14
49
Connection
Feature
83
31
Requirement
Design-concept
10
Other
LOD-Total
22
17
37
137
240
queries during retrieval.
3.7 Discussion
The objective of this study was to develop an understanding of the information
needs of designers before developing services for supporting information capture and reuse. The goal was not to generalize the results but to utilize the
design information framework, developed in this study, in implementing an
Information Handling Behavior of Designers During Conceptual Design: Three ExpcrimenlS
55
information management service. The next chapter will go into the details of the
development of this service, called Dedal.
The design information framework has been adequate in classifying information
that deSigners ask for while engaged in design. While it cannot be used to classify all the information designers deal with, it provides an example of
information that should be captured for reuse. This framework will be applicable to a wide range of design domains.
A sample of two subjects is quite small to draw any generic conclusions. However, this study does demonstrate experimentation as a powerful method to
develop design process understanding and provides an analysis method for
doing so. Quantitative results obtained in this study have been explained based
on the nature of the experiment, the design problem and current understanding
of the design process. What is unexplained has been used to improve understanding and refine technique of conducting and analyzing protocol
experiments.
There are differences in the classification developed in this study and the taxonomy of questions reported in [Kuffner 1991]. The taxonomy in [Kuffner 1991]
classifies a question (and a conjecture) based on its nature (construction, location,
operation, purpose) and topic (assembly, component, interface, feature). Some reasons
for the differences in the outcome of the studies can be attributed to the
following:
The size of the design problem will influence the range of the question one
would encounter. In [Kuffner 1991] the first generation design was done in
an experimental setting over a period of 6 hours. In this case the first generation design was a real world design. The bigger scale of the design problem
exposed new types of questions which may not have occurred in a simpler
problem.
Designers do not carry out design with a pre-determined set of questions.
They raise questions as new information is needed. The questions are often
triggered by exposure to information accessed by them. In [Kuffner 1991]
Information Handling Behavior of Designers During Conceptual Design: Three ExperimenL~
56
the documentation available to the subjects was in the form of machine

drawings and specifications. In this study subjects had access to a more
complete and extensive documentation. The completeness in documentation provides an opportunity to collect a more complete set of questions that
designers can raise.
The questioning behavior is not random. New questions are asked after
reflecting on information received in answer to a question. So the type of
questions asked will depend on the form in which the information is available to the designer. For instance, if a designer was given a machine drawing it would be very likely for him to ask questions about location and
connections, however if the same information was given in a descriptive
form, the line of questioning could be different. In this experiment the information was available to designers in many forms, making it possible to collect a more complete set of questions.
3.8 Recommendations for Information Reuse Service

Based on the results of the information reuse study, here are a few recommendations for information capture and reuse service:
it should be able to capture and represent the information described by the
design information framework
it should keep pace with the generation of information. i.e. information capture and indexing should happen during the design process and should not
be a post-facto add on activity.
it should be capable of dealing with information at various levels of detail.
it should be capable of dealing with multi-media information.
3.9 Summary
I have discussed quantitative and qualitative results from an empirical study of
design information reuse during design. I have developed a method for analyzing the questioning behavior of designers and used it to develop a framework
for classifying and understanding design information needs. The framework
characterizes design information based on its descriptor, subject-class, level-ofdetail and medium. This framework is the foundation for the development of a
service for information capture and reuse discussed in Chapter 4.
Chapter 4
Dedal: An Information
Management Service
/-----,
,.-----,
/--------
{ Dedal Performance ,
Results
\
/
( Information Classification \
\
Framework
/ I
{ Information Handling,
\
Framework
. t
t
.
t
--------- ----------- --------------
..... _ - - - - Analyze
Analyze
D~m
of DadaI
/.
Analyze
;.~
Dadal
1991-1992
19901991
Information
Reuse Study
------
Support by
DadaJ
Observe
DadaJUsa
Support by
DadaJ
).2-'~\
Information
Handling
Study
Understanding
of Information
Handling
This chapter describes the information management utility called

Dedal which embodies services for indexing, modeling and retrieving
design information. It uses the design information framework
described in Table 3-1 to implement a conceptual indexing method.
This chapter will report on Dedal's retrieval performance and results
obtained when it was deployed to support design activity in real
time. Details regarding implementation of Dedal are discussed in
AppendixB.
57
Chapter 4: Deda/: An Information Management Service
58
4.1 Introduction to Deda11

Using the requirements extracted in the information reuse study (section 3.8), a
service for managing design information was developed. This service is called
Dedal2. Dedal provides tools for indexing, modeling and retrieving design information in real time. Figure 4-1 shows the different components of Dedal and the
flow of information among these components.
CD
en
cu
.0
cu
cu
USER
Graphical User Interface
en
"C
~
()
Modeling Tool
CD
a:
c
C)
en
CD
Retrieval
Heuristics
cu
"0
CD
: :l
::iE
---- ---- ------ ---- - - - -- -- -- --- ---------- ---------- --- - -

Figure 4-1 Dedal system architecture. The dark shaded area represents the
information capture environment and the light shaded area show the components
which make up Dedal. Dashed arrows indicate information transfer. Indexing tool
is used to create the indices database, modeling tool to create a domain model,
and retrieval tool to answer queries. The indices database and the domain model
are used by all the tools. The information capture environment is the tool
designers use to document design information. It is integrated with Dedal via the
graphical user interface. The multimedia design records database represents
design information captured by the information capture environment.
1. Dedal has been developed in collaboration with Catherine Baudin and Jody Underwood from NASA
Ames Research Center and Ade Mabogunje from Stanford University.
2. The name Dedal comes from the greek mythological character Daedalus, the legendary builder of the
Cretan labyrinth who makes wings to enable himself and his son Icarus to escape imprisonment. It is the
characteristics of finding way in complex labyrinth that is a parallel to the function of Dedal.
59
The tools for indexing, modeling and retrieval are accessed via a graphical user
interface which integrates Dedal with an information capture environment. The
screenshots of these interfaces are shown in Appendix B. The information capture environment would typically be a collection of tools that designers use to
carry out their day to day work such as: word processor, drawing/sketching programs, spreadsheet, CAD /CAE tools, electronic mail and others.
Dedal deals with multimedia information and enables indexing and modeling to
take place in real time. It deals with information at all levels of detail and formality. For more details on the development of Dedal and its architecture please see
[Baudin et.al. 1993a]. A copy of this paper is included in Appendix B. Dedal is
written in common lisp. The interfaces for dealing with audio and video information are written in the language C.
4.1.1 Indexing Tool

The indexing tool is used to generate the indices .database using a conceptual
indexing scheme. In conceptual indexing, an information segment is indexed by

describing the concept represented in it. Concepts are domain dependent entities which embody the semantics of the domain. In Dedal the concept is
described by using the design information framework in Table 3-1, Le. information is indexed by the classification of the question it answers. Figure 4-2 shows
an example of an information segment and its conceptual index.
The size and granularity of an information segment can vary depending on the
type of document and the medium of information. Typically a segment is a paragraph of text or a single graphic. It is possible, and often necessary, to index the
same information segment with more than one index. This is specially true of
cases where the segment is rich enough to answer multiple questions. Since
Dedal is integrated with the information capture environment the designer only
specifies the descriptor, subject, and level-oj-detail of the information. The medium,
60
document, segment and paragraph-no. are automatically picked up through the

interface.
Indexing pattern on the information segment on
page 18 of document DRD-spring-1993:
Descriptor: Rationale
Subject: Solenoid
Medium: text
Level of Detail: Conceptual
Document: DRD-spring-1993
Segment: 18
Paragraph no: 2
'C'he solenoid was chosen

since It Is able to generate
upto 30fbs of force with the
given power and
A question which will return the above index

as answer:
Subject: Solenoid
18
Figure 4-2 Example of conceptual indexing. Paragraph 2 on page 18 of the

document DRD-spring-1993 answers the question: What was the rationale of the
solenoid? Therefore the indexing pattern for this segment is as shown. The
concept this index represents is solenoid, a domain dependent entity.
4.1.2 Modeling Tool

The modeling tool is used to generate and maintain a model of the design information. A model in Dedal is a means of defining relationships among the
domain concepts in a hierarchical manner as shown in Figure 4-3. These relationships are typically of the form: part-of, is-a, depends-on and others. The model
includes a representation of the artifact structure, requirements, alternatives and
decision points and it evolves incrementally with the design process. The simple
hierarchical structure of the model assures that designers themselves can create
them in real time. The modeling tool also helps maintain consistency of concept
names and relations and provides browsing and editing facilities ..
4.1.3 Retrieval Tool

The retrieval tool is used to get answers to queries. It uses the indices database,
solenoid
disk-stack
part-of
outer-clisks
61
part-of
"
inner-clisks
Rgure 4-3 Example of a model in Dedal. The figure

shows the simple
hierarchical representation which relates domain concepts.part-of and is-a
relations can be seen in this model.
the domain model and heuristics to direct user to appropriate locations in the
design records database. The algorithm used by the retrieval tool is shown in
Return Answers
to User
Figure 4-4 Flow chart of the retrieval algorithm used by Dedal. The answer for a
query is first searched in the model .
Figure 4-4. A query is phrased by specifying the descriptor and subject(s)

(Figure 4-2). The retrieval algorithm first tries to answer the query from the
model. The model has information about decisions made and alternatives considInformation Handling Behavior of Designers During ConceplUal Design: Three ExperimenL~
62
ered. Simple questions about them can be answered from information in the
model. If this fails to find a relevant answer then the indices database is searched
for an index which matches the query. Failing this heuristics are used to expand
the search. Dedal uses three types of heuristics. They are:
1. Related descriptor heuristic: This heuristic uses the fact that the bound-
aries between the descriptors often overlap. Therefore, sometimes information about a descriptor of a subject can be found by searching for a
information about a different descriptor of the same subject. An example of
such a heuristic is:
if (question is rationale of X)
then
(search for decision of X)
2. Proximity heuristic: This heuristic uses the fact that a document is a continuum of information. Often a certain type of information can be found
near another type of information. An example of such a heuristic is:
if (question is function of X) and (X is part-of Y)
then
(search for operation of Y)
3. Causal relation heuristic: This heuristic uses the knowledge of certain relations which are causal in nature to change the subject of the query. For
instance:
if (question is rationale of X) and (X depends-on YJ
then
(search for rationale of Y)
Many examples of the heuristics are included in Appendix B.
4.2 Dedal Test Domains and Results

To measure retrieval performance of conceptual indexing scheme and the real
time performance of Dedal, we deployed it to support design information management in three domains listed in Table 4-1. Results obtained in each of the
domains are discussed below.
Information Handling Behavior of De..igners During Conceptual De..ign: Three Experiment.
Chapter 4: Dedal: An Information Management Service
63
Table 4-1 Dedal test domains details. I =indexing, M =modeling and R =retrieval.
Project
Duration
Designer
Damper
7mon
R perfonnance
Symbolics
Capture
Medium
vmacsa
Bioreactor-I
7mon
Real-time I & M
Unix
Framemaker
No
Bioreactor-II
2mon
Real-time R
Unix
Framemaker
No
No.
Domain
Dedal Feature Thsted
Platform
Indexes?
Yes
a. vmacs is a trademark of perfonnance graphics company.
4.2.1 Damper Domain Results

The first testbed domain was the design of a continuously variable damper
[Baya et al. 1990}. This domain was used to assess the retrieval performance of
the conceptual indexing scheme implemented in Dedal. Design information was
captured in an electronic design notebook called vmacs [Lakin et al. 1989}. A
member of the design team used Dedal to index about 400 pages of documentation with 340 indices. Forty distinct questions, the answers to which were
available in the documentation, were selected from the 240 questions extracted
in the information reuse study (Table 3-6). Note that questions to which the
answer is not in the documentation cannot be used to assess retrieval performance since no answers retrieved would be relevant. This will cause the results
to be biased towards poor performance.
Figure 4-5 shows the retrieval performance measured is this study. A comparison was done with full-text boolean retrieval scheme, a popular information
retrieval method. Full-text boolean retrieval is a baseline reference point for comparison. The retrieval performance is measured by two parameters, recall and
precision. Recall is the ratio of the relevant answers retrieved to the total number
of relevant answers present in the information database. Precision is the ratio of
the number of relevant answers retrieved to the total number of answers
retrieved. A perfect retrieval system would have recall and precision of 100%.
As is clear from Figure 4-5, conceptual indexing scheme of Dedal out performed
Chapter 4: Deda!: An Information Management Service
Recall
# of right answers retrieved

total right answers present
64
..
Precision
of right answers retrieved

= #total
answers retrieved
100~------------------------------------~
~
- - - - - - - - - - - - . . . Oedalwith
, retrieval heuristics
80
as
0
CD
a:
60
Boolean retrieval
_-_0 - - - - - - - - - - _ - - - I
40
I
I
I
20
I
I
00
I
I
_. _ - -
-B
Dedal with Jut
: retrieval h uristics
20
40
60
Precision
80
100
Figure 4-5 Retrieval performance of conceptual indexing. Oedal's

retrieval
engine based on conceptual indexing performed far better than full-text boolean
retrieval. Oedal's recall can be improved by using heuristics to widen the search
space. However, it does lead to some loss in precision.
the boolean retrieval scheme by a wide margin. The precision of boolean

retrieval scheme was measured at 10% and that of Dedal at 80% (for exact
match, i.e. when not using retrieval heuristics). The recall was comparable at
around 52%. Dedal's recall performance can be improved by using heuristics to
loosen the search but at the expense of some loss in precision. Nevertheless, the
gain in recall, about 40%, is far more than the loss in precision, about 17%. It is
important to note that boolean or full-text search requires no indexing effort
unlike conceptual indexing which requires effort in indexing. Clearly the gain in
performance justifies the effort required in conceptual indexing, but can this
effort be expected of designers is a separate question. Another question which
rises is what techniques can be used to automate/semi-automate conceptual
indexing?
Information Handling Behavior of Designers During ConceplUai Design: Three Experiments
Chapter 4: Deda!: An Information Management Service
65
4.2.2 Bioreactor Domain Results

The bioreactor domain was used to assess the real-time performance of Dedal.
Dedal was deployed to support indexing and modeling of design information in
real time, i.e design information was indexed and modeled during the design
process. The objective of the bioreactor project was to design an apparatus for
interactive control of a life sustaining environment for bacteria. The bioreactor
project took place in two phases. In the first phase (I) a bioreactor was designed
over a period of 7 months by a team of three designers. This phase resulted in a
working prototype. In the second phase (IT) a team of three different designers
redesigned a subsystem of the bioreactor to improve on its design. The indexed
and modeled documents from bioreactor phase I were made available in bioreactor phase II. This was an ideal situation to determine if the effort put into
capturing design information from bioreactor phase I would payoff during
phase II. The indexing and modeling activities in phase I were done by a participant designer (not one of the three principal designers). Following are some of
the observations that were made:
Bioreactor Phase I:
information framework was adequate for indexing information for reuse
the information framework was intuitive since the designer was able to
index, model and formulate queries without formal training
it was possible to index and model incrementally with the design process
(about 2 hrs per week of additional effort was required)
Bioreactor Phase II:
information reuse was effective, the design team redesigned and prototyped
a new design in two months, relying only on Dedal to learn about the previous design
on-line access to information was very effective. It contributed to improved
design efficiency and practice
Information Handling Behavior of Ocsigners During Conceplual Ocsign: Three Experiment'
Chapter 4: Oeda/: An Information Management Service
66
4.3 Discussion on Dedal: Lessons Learned

Development and deployment of Dedal was an important learning experience. It
convinced me that design information capture has to be an integral part of the
design process and cannot be demanded as an add-on activity. Some how, the
process of structuring and archiving design information for reuse has to be seen
as a requirement of the design process so that it happens incrementally and in
real time.
Dedal has a framework for classifying design information. This framework can
be used for modeling and indexing design information in real time. It also
allows for accurate retrieval. Nevertheless, it has lessons. Some of them are:
the activity of indexing and modeling are add-on activities.
the modeling process, although incremental, is static in nature. The user has
the burden of monitoring and reorganizing concept names as they evolve,
which is a considerable cognitive load.
the payoff of indexing and modeling was seen mostly by the redesign team
and not by the team which did the indexing and modeling. It is my belief
that information capture would always remain an add-on activity unless the
rewards of these activities were immediate.
the integration with the information capture environment is only a rough
draft. It should be carefully redesigned to empower user indexing and modeling.
the definition of level-ofdetail was not very useful. Although it was specified
during indexing, users did not use it during retrieval. Also the retrieval
algorithm did not use this information to any significant advantage.
It was evident from these limitations that development of integral and intuitive
services for capturing and managing design information require a deeper understanding of the dynamics of information handling behavior. We need to
understand the various activities designers perform with information and how
they change over time in characteristics and content. This understanding would
provide the clues necessary for implementing intuitive and integral services/
interfaces. To my knowledge no one has reported such an understanding. To
Information Handling Behavior of Designers During Conceptual De.~ign: Three Experiments
Chapter 4: Dedal: An Information Management Service
67
develop this deeper understanding I conducted the information handling behavior study, discussed in the next chapter.
Chapter 5
Information Handling Behavior
Framework
/-------{ Information Classification \

\.
Framework
}
------_/
/-----'"
{ Dadal Performance \
\.
Results
/
,...-----,
\.
Framework
/
i
i
t
-----------------------------Analyze
Analyze
Analyze
1~~I00'f:;r.rmt !~1_19~~
Information
Reuse Study
Support by
Dadal
-
Observe
......
~~ Dadal Usa
Support by
Dedal
-
1--
Information
......~ Handling
Study
995
Understanding
of Information
Handling
This chapter describes the experimental procedure and the analysis
method of an observational study conducted to understand the

information handling behavior of designers engaged in conceptual
design. Information handling is studied in the framework of activities
designers perform with information and the characteristics of
information handled in those activities. A quantitative measure for
design information is defined and used to measure information
handling rate. Many examples of the analysis procedure are included
in this chapter.
68
Chapter 5: Information Handling Behavior Framework
69
5.1 Introduction
The experience of developing and implementing Dedal (Chapter 4) resulted in
many unanswered questions regarding information handling. Some of the key
questions were:
1. What are the activities designers perform with design information?
2. What is the temporal distribution of design information processing?

3. Can design information be measured quantitatively?
4. What is the information content in different activities?
5. How does information processing rate change during the design process?
and,
6. What do answers to the above questions inform us about developing intuitive and integral tools to support information handling?
To find answers to these questions I conducted the information handling behavior
study using the verbal protocol method. The experimental approach was similar
to the one used in the information reuse study, reported in Chapter 3. The differences were in the design problems used in the experiments and the analysis
method. To avoid repetition I would only bring out aspects of the experimental
approach and analysis procedure which are different from the description in sections 3.2 and 3.3.
5.1.1 Experimental Setup and Procedure

A total of six experiments were conducted in this study. The experimental setup
is same as the one used in information handling study (shown in Figure 3-2).
Table 5-1 lists some details on the experimental subjects and the design problems used. Three different design problems were used. They are included in
Appendix C. Two of the design problems dealt with design of a bike-lock and
the third one was about designing a harness for attaching a backpack to mountain bikes.
70
Table 5-1 Details on experiments and experimental subject background. The table lists the
design time, design problem and the design experience of each of the experimental subjects.
Design Experience (yrs)
Subject
Design TIme
(minutes)
Design Problem
Academic
Industrial
Sl
78.8
bike-Iockl
S2
67.5
bike-Iockl
S3
71.4
bike-Iock2
S4
74.4
bike-Iock2
S5
51.2
bike-Iock2
S6
119.5
back-pack
20
The first two design problems were based on the everyday activity of locking
and unlocking a bike. In short the design problem was to design an off the shelf
lock for bicycles which could be integrated with the bike and operated without
having to take it off the bike (as opposed to kryptonite locks or U locks which
are not integrated with the bike and are operated as an independent unit). This
design problem was chosen because: it required no special technical background, it enabled the designers to work within a space of familiar vocabulary
and technical jargon and it was suitable for an hour long conceptual design exercise. The designers had access to a bike and a kryptonite lock.
5.2 Analysis Procedure

The objective of the analysis procedure was to transform raw data from the
experiments into quantitative descriptions of the information handling behavior.
The procedure is shown in Figure 5-1. Some of the steps, conduct experiment and
transcribe, are same as described in section 3.3. The other steps, segmentation and
classification are described in sub-sections below.
5.2.1 Segmenting into Information Fragments

An information fragment is a continuous period during which the informational
Design
Problem
F
Information
Fragments
Conduct
Experiment
71
Recorded Data
Transcribe
Transcript
Segment into Information Fragments
Classify
Fragments
Classification
Statistical
Analysis
Quantitative
Results
Figure 5-1 Analysis procedure for information handling study. This

procedure
transforms raw data into quantitative description of the information handling
behavior.
activity (activity performed with information) and the descriptor of information

(defined in section 3.4.2) remains the same. Segmentation of the transcript into
information fragments was done by developing a set of rules to identify points
of transitions between fragments. These rules are:
1. Typical points of transition could be semantic such as: change in designer's
train of thought, change in focus of attention or change of action.
2. Typical points of transition could also be syntactic such as: significant

pause, change in the medium of information or utterance of certain phrases
such as "and", "ok", "alright", "umm".
3. The level-ofabstraction during an information fragment remains the same.
4. The level-ofdetail during an information fragment remains the same.
5.2.2 Classifying Information Fragments

Before an information fragment could be classified, a framework for describing
the information handling behavior was needed. Section 5.3 goes into the details
of the development of this framework. For now it is sufficient to note that an
information fragment is classified by
1. specifying the informational activity associated with it (section 5.3.1),
Information Handling Behavior of Designers During ConceplUai Design: Three ExperimenlS
72
2. classifying the information in the fragment using a revised design information framework (Table 5-2), and
3. quantitatively measuring the design information content using design information measure (section 5.3.3).
Figure 5-2 shows the template used for this classification. Note that for frag(info-fragment
:activity infonnational-activity
:time (start-time end-time)
:descriptor descriptor
:subjclass subject-class
:subjects (subject-list)
:medium medium
:levels (level-of-detaillevel-of-abstraction)
:measures (design-information-measure )
)
Figure 5-2 Template for classifying an information fragment. This
template
captures values for nine attributes of each information fragment. These attributes
describe the informational activity and the classification and content of design
information in that fragment.
ments rich in content it may be necessary to encode it with two or more

classification.
5.3 Information Handling Framework (IHF)

Table 5-2 shows the information handling framework (lliF) which is used for
describing information handling behavior. This framework is an extension of the
design information framework (DIP) developed in the information reuse study (Table
3-1). Three new attributes were added to the DIP to create the lliF. These are
informational activity,
lev~l-of-abstraction
and design information measure. Each of
these are defined in detail in sub-sections below. The lliF adequately describes
the activities designers perform with information, and the characteristics and
content of the information handled in those activities. Monitoring design activity
73
within this framework over time will improve our understanding of the dynamics of information handling behavior.
Table 5-2 Information handling framework (IHF). This includes a revised design information
framework (DIF) to classify design information. Three new attributes namely, informational activity,
level-of-abstraction and design information measure are added to DIF to create the IHF. The
values these attributes can take are shown below.
Information Handling Framework (lHF)
Design Information Framework (DIF)
Informational
Activity
Level of
Abstraction
Descriptor
SubJect-Class
Medium
Level of Detail
Generate
Unlabeled
Alternative
Assembly
Audio
Conceptual
Access
Labeled
Assumption
Component
Video
Configurational
Analyze
Associative
Comparison
Connection
Text
Detail
Qualitative
Construction
Feature
Graphic
Quantitative
Location
Requirement
Gesture
Operation
Design-concept
Performance
Other
Design Information Measure (dim)
Rationale
Quantitative measure: takes

integer values (1,2,3, and so on)
Relation
Requirement
Miscellaneous
Since the scenario of experiments and the focus of observation in this study was
different from the information reuse study it was necessary to revisit the development of the OIF using the approach described in Figure 3-4. It is encouraging
to report that the DIF of Table 3-1 required only minor modifications. Two new
values were added for the attribute descriptor. The other attributes remained
unchanged. All attributes making up the DIF are redefined below in the context
of information handling behavior.
Descriptor. This refers to the basic constitution or the nature of the information
in an information fragment. Two new descriptors were added. They are defined
74
in Table 5-3. (details on other descriptors are in Table 3-2).
Table 5-3 Descriptors added to the design information framework. Other values are defined in
Table 3-2.
Descriptor
Information regarding
Example Information Fragment

from Protocol
Assumption
supposing the truth or validity

of a fact or taking for granted
in general there is probably

enough room around here that you
could,inside the frame, inside
this tube attach this pivot,
slash sliding mechanism,
Comparison
examining two or more conceplS to determine similarities

and dissimilarities
(task of locking} maybe a little

bi t more than 1IIIIIIIIIIl , maybe a
little bit more than changing a
tire is permissible
Explanation
Designer assumes
information regarding
space constraint
Designer compares
the task of locking to
that of changing tires
Subject-Oass. This is the class to which the subject of an information fragment
belongs (details in Table 3-3).

Medium. This is the physical form in which the information fragment exists
(details in section 3.4.4).
Level of Detail. It is the phase of the design process to which the information
fragment can be associated with. (details in section 3.4.5).
5.3.1 Informational Activity

The activities designers perform with information will be called informational
activities, in this thesis. It is possible to view all actions as being performed with
information. However, for the purpose of this study, only actions which are
explicitly captured in the protocol data and which need to be supported by computational services are of interest. Abstract and unobservable actions such as
thinking, observing, speculating, ideating are not relevant to this study. Figure 53 shows the process of developing the informational activities classification. In
the initial stages this process was tightly coupled with the development of rules
for segmentation of the transcript. This is because some transition points are at
the start and end of an informational activity, and therefore without a classification on activities it is not possible to identify transition points consistently and
objectively.
Information Handling Behavior of Designers During Conceptual DC5ign: Three Experiments
< >
Information Fragment
step 1
Characterize informational action

, I
/
,
'-------/
No
""
....
~""-/~
Group inlo Infannational Activities
step 4
~
step 6
step 2
Collection of all informational actions
-- ----
"-.....
\\ "
..-
" Segmentation:
\
~
Identify transition points I
,_ .......
\....
//"
/
75
Can all info-fragments be classified?
/.tep 5
P
Classify N.:,te 5
Yes
Label Activities - -~ Info-Fragments
Figure 5-3 Evolution of the informational activity classification. The cloud on the
top right hand side signifies that the process of segmentation was coupled with
the evolution process (in the initial stages).
The resulting classification of informational activities is shown in Table 5-4. It

shows that informational actions can be grouped into three classes namely Gener-
ate, Access and Analyze. This is a minimal view of the informational behavior of
designers. However, it was possible to classify all actions that were observed in
the experiment in these three activities. This classification is a suitable level of
granularity for the purpose of this study. Note that this classification is most suitable for a single designer scenario. A team design scenario would require a
different classification.
To better understand this classification, assume that at any time in the design
process there is an information space associated with the design. At the beginning, this information space may be thought of as containing the design
requirements. As the designer works on the deSign, he/she makes changes to
the information space, either by adding information to it by generating/accessing
76
it or they transform information in the space by analyzing it.

Table 5-4 A classification for the Informational activities. Their definitions, associated actions and
example. The list of associated actions is a representative list, not an exhaustive list.
Informational
Activity
Generate
Defioition/lnterpretation and
Associated actioM
Analyze
Explanation
so umm I wonder if we
could do a combination
of a the 0 lock type
arrangement and the
doughnut type
arrangement
Actiqn: Talk
An action which references information within or outside the information space from an identifiable
source. Actions: read, listen, recall,
request
{reading from dl}

Locking mechanism
should be an off the
shelf item
Action; Bead
An action which changes an

attribute of the information fragment Action: interpret, organize,
calculate, evaluate
than you really have to

work it {apply force}
bending this tube to
get this thing out,
ActioQi ~11I1Iti:
An action which adds new information to the information space from

an unidentified source. Actions:
~ting,dr.awing,tUUdng
Access
Example Information
Fragment from Protocol
Designer generates an
alternative of combining two
arrangements
Designer accesses a
requirement by reading it.
Designer analyzes the

performance on strength by
making a qualitative
evaluation of strength
5.3.2 Level of Abstraction

Information in design evolves from abstract to concrete. Thus the ability to represent and monitor this evolution is essential to studying information handling.
While level-ofdetail was defined in the DIF to address this concern, its granularity and definition was not adequate to empower structuring or retrieval of
information (section 4.3). Level-ofabstraction has been defined as a substitute for
level-ofdetail. It represents the evolution of concepts in the design process.

Table 5-5 shows the different level-ofabstractions used in this study along with
their definitions and interpretations. The definition and granularity of these levels is based on observations from the experiments and therefore are most
suitable to represent the information encountered in the conceptual design pro-
77
cess. The levels in the table are arranged in a decreasing order of abstractness.
Table 5-5 Definition and interpretation for level-at-abstraction. The levels are arranged in
decreasing order of abstractness. going from top to bottom. Subject" in the table refers to the
subject of the information fragment.
Level of
Abstraction
DefinitionlInterpretation
Example Information
Explanation
unlabeled
Referring to the subjectof an inforrnation fragment without a name, as in a

idea or a new concept
so its not really a

wedge its a what do
we call it. okay.
what I know what it
looks like. is chat
right?
The designer makes no

mention of the name for
the subject (Kreally not a
wedge") that helshe is
describing
labeled
Referring to the subject of an inforrnation fragment by a name, either by generating a new name or reusing a previous
name
umm but the reason

they are chains. I
think is that you can
have any size bike to
lock.
The subject under

consideration is
mentioned by name i.e.
Kchains"
associative
Referring to the subject of an infonnation fragment in relations to or by associating with other concepts/subjects
where aaa it kind of

hooked underneath.
hooked underneath the
wheel or its also
underneath the frame.
The subject (Kit") of the

fragment is associated
with the -Wheel- and
Kframe-
qualitative
When a subject or features of a subject

are qualitatively addressed,
Citadels are probably

a lot stronger right
here in the middle
The feature Kstrength of

citadel- is qualitatively
described as stronger
or
or
or
When operation or motion is qualitatively described
(unlocking oper) and

then you put the key
in turn it its just
spring loaded and its
just opens up .
The operation of
unlocking is
qualitatively described
as a sequence of actions
When a subject or the features of a subject are quantitatively described
it looks just by
looking at that bike
it would be limited
on a mountain bike to
6 to B inches in
length .
The feature "length- of a

component is
quantitatively
described, " 6 to
8 inches-
quantitative
5.3.3 Design Information Measure (dim)

Measuring information is an old concept. It originated in the field of communications engineering, wherein the quantity of an information message is inversely
related to the probability of receiving that message from a source [Shannon and
Weaver, 1949). This measure is related to the statistical properties of a communication system and does not depend on the semantic content of the message.
[Kim, 1990 and Suh, 1990) suggests a definition of information content from a
cybernetic perspective. Here the quantity of an information message is related to

the effect of the message on a system's ability to achieve a goal. So the measure
of information content is with respect to a certain goal. To use this measure one
Information Haodling Behavior of Designers During Conceptual Design: Three Experimenrs
78
needs to define a goal and a probability distribution on information regarding

its potential to bring the system closer to the goal. Neither of these two measures are relevant to the nature of analysis done in this study. This is because the
analysis aims at measuring semantic information and it is not possible to put a
probability distribution on semantic information with regard to a goal.
The design information measure (dim) is a quantitative measure of the amount of
semantic design information contained in an information fragment. The definition is based on the information fragment classification shown in Figure 5-2.
Every unique [descriptor, subject] pair (to be referred to as a DS pair from now
on) in the classification constitute 1 dim of design information. Thus, if a fragment is classified by 1 descriptor and 3 subjects (in the subject-list), it forms 3
unique DS pairs and its content is 3 dims. Note that DS pairs represent the semantic content of an information fragment. A DS pair is interpreted as; the fragment
contains information regarding the D of S (Figure 3-5). For instance, if a fragment was classified as [operation, solenoid], the interpretation suggests that the
fragment has information regarding the operation of solenoid.
5.3.3.1 Definition of dim: Assumptions
Here are a few assumptions made in defining the measure dim. This will help
understand the rules for calculating this measure.
1. The content of an information fragment is encoded in its classification.
2. The measure takes contributions from two attributes in the classification

namely: descriptor and subjects (refer to Figure 5-2).
3. Assumptions pertaining to descriptors:
All descriptors encode equal amount of semantic information. i.e. [Dl, 5]
has the same measure as [D2, 5], where Dl and D2 are two distinct
descriptors and S is a subject.
A fragment classified by more descriptors has more content, and the relationship is linear. i.e. all else remaining same, a fragment classified by 2
descriptors has twice the content of a fragment classified by 1 descriptor.
79
4. Assumptions pertaining to subjects:
Subjects from all subject-classes encode equal amount of semantic information. i.e. [D, SI] where SI is a SCI is equal in content to [D, S2] where S2
is a SC2. SC stands for subject-class and SCI and SC2 are two distinct subject-classes (for e.g.. assembly and component).
A fragment which refers more subjects has more semantic content, and
the relationship is linear. i.e. all else remaining same a fragment classified by 2 subjects has twice the content of a fragment classified by 1 sub-
ject.
5. All values of level-ofabstraction encode equal amount of information, i.e. a
labeled OS has the same content as a qualitative OS. Therefore, since each
fragment has one level-ofabstraction (by definition), the dim measure is
independent of the level-oj-abstraction.
6. All values of level-ofdetail encode equal amount of information, i.e. a conceptual OS has the same dim measure as a detail OS. Therefore, since each
fragment has one level-ofdetail (by definition), the dim measure is independent of the level-ofdetail.
7. Since information in any medium can be represented as a OS pair, the measure is independent of the medium of information. i.e. A graphical fragment
and a textual fragment with the same OS classification have equal content.
5.3.3.2 Rules for measuring dims
Once a fragment has been classified the rules below help us determine the dim
measure:
1. For fragments with 1 descriptor and 1 subject the measure is 1 dim.
2. For fragments with 1 descriptor and n subjects the measure is n dims.

3. For fragments with n descriptors, the measure depends on how many subjects are associated with each of the descriptor. If descriptor i has Si subjects
then the measure is sum of all Si ' s dims.
Table 5-6 shows some examples on how the above rules were used in counting
Information Handling Behavior of Designers During Conceptual Design: Three Expcrimcnl5
80
dims for information fragments.

Table 5-6 Examples of dim measurement. Three examples are listed, each using a different rule
to measure the information content.
Example Information
Information Fragment
CIassification
Number
Descriptor
Subject-list
oCdims
Explanation
than you really have to

work it (apply force)
bending this tube to get
this thing out
Performance
strength-of-tube
one unique DS pair:

1. [perfomuuu:e, strength-of-tube]
so the measure is I dim
so like the lets assume we

are gonoa attach it (lock)
to the down tube.
Alternative
lock
down-tube
two unique DS pairs:

1. [alternatiue, lock] and
2. [alternatiue, downtube]
so the measure is 2 dims
I am trying to decide if
there is any apparent
advantage to locking right
near this wheel verses
locking on this part
here (down-tube)
Location
bike-lock
Comparison
near-wheel
down-tube
1+2=3
This fragment has two classifications

which make three unique DS pairs:
l.llocation. bikelock]
2. [comparison, near-wheel] and
3. [comparison. down-tube]
so the measure is 3 dims.
5.4 Analysis Example

In Table 5-7 below is an example of the analysis procedure from transcription to
the classification of the information fragment, for about 1 minute of verbal protocol data. The data is shown in three different stages: after transcription, after
segmentation and after classification.
Table 5-7 Examples of information handling behavior analysis_ In the table below protocol data is
shown at different stages of the analysis. The first and the last row show brief transcript before
and after the segment analyzed below_ The rows form a continuum from the protocol, so only the
start time for each fragment is shown. The end time is the start time of the next fragment.
After
Transcription
Start1ime
Stamp
After Segmentation
After Classification
Transcript Sefore: (OO:56:47:22)_: OK I have spent er forty-five minutes or so now forty

minutes doing that er so em ... (mutter) going to check every possible location here
Information Handling Behavior of D.:signers During Conceptual Design: Three Experiments
81
Table 5-7 Examples of information handling behavior analysis. In the table below protocol data is
and after the segment analyzed below. The rows form a continuum from the protocol, so only the
After
1hmscription
alright so
we do have
one that
comes up
front em
really a
little bit
wary about
backpacks
on em
fronts of
bikes em
let's see
if we can
SfartThne
Stamp
00:56:56:20
After Segmentation
alright so we do have
one that comes up front
you know
from the
aesthetic
standpoint
everybody
likes
things
symmetric
and er this
is not that
big a pack
em my
initial
tests
indicated I
probably
couldn't
have one
right like
that
(info-.fragment
:activity ACCESS
:time (00565620 005659241
:descrtptDr AL7ERNA'IlVE
:subjclass DESIGN-CONCEPI'
:subjects (front-position)
:medium VIDEO
:levels (CONCEP1UAL lABELED1
:measures (1]1
00:56:59:24
em really a little bit

wary about backpacks on
em fronts of bikes em
(info-.fragment
let's see if we can em

and there is that issue
of it being off the
side __
(info-.fragment
you know from the

aesthetics standpoint
everybody likes things
symmetric
(info-.fragment
:acttvity GENERA7E
:tfme (00572326 005730051
:descrtptDr REQUIREMENr
:subjclass ASSEMBLY
:subjects (syrnmetry-for-lDoks1
:medium AUDIO
em and
there is
that issue
of it being
off the
side __
:acttvity ANALY:ZE
:tfme (00565924005710061
:subjclass DESIGN-CONCEPT
:subjects (front-position)
:mediumAUD10
:measures (111
00:57:10:06
00:57:23:26
:acttvity ACCESS
:tfme (00571006 005723261
:descrtptDr AL1ERNA1TVE
:subjclass DESIGN-CONCEPI'
:subjects (side-position)
:medium VIDEO
:measures (1]]
:measures (111
00:57:30:05
and er this is not that

big a pack
(info-.fragment
:acttvily ACCESS
:tfme (00573005 005736001
:descrtptDr CONSTRUC7!ON
:subjclass FEA1tTRE
:subjects (size..of-padc}
:medium AUDIO
:levels (CONFIGURATIONAL QUALITATIVE1
00:57:36:00
em my initial tests
indicated I probably
couldn't have one right
like that (horizontal
on the back!
(info-.fragment
:measures (1]1
:activity ACCESS
:time (00573600 005742141
:subjclass DESIGN-CONCEPT
:subjects (badc-horlzDntal]
:medium VIDEO
:levels (CONCEP1UAL UNlABELED1
:measures (1]1
82
Table 5-7 Examples of information handling behavior analysis. In the table below protocol data is
and after the segment analyzed below. The rows form a continuum from the protocol, so only the
After
1hmscription
certainly I
would not
do it this
way I would
do it that
way em in
fact I
would em
maybe do it
this way
StartThne
Stamp
00:57:42:14
After Segmentation
certainly I would not
do it this way
{vertical in the back}
(fnfo-.fragment
:acttutty ANALY2E
:time (00574214 00574822)
:clescriptDr AL1ERNAnvE
:subjclass DESIGN-CONCEPr
:subjects (back.vertical}
:medlum VIDEO
:levels (CONCEPItJAL UNLABELED)
:measures (I})
00:57:48:22
I would do it that way

{horizontal in the
back} em in fact I
would em maybe do it
this way {horizontal in
the back with opening
on the outside}
(tnfo-.fragment
:acttutty GENERA1E
:time (00574822 00575124)
:clescriptDr COMPARISON
:subjclass DESIGN-CONCEPr
:subjects (backhorizontal back-horlzontal2)
:medium VIDEO
:levels (CONCEPItJAL UNLABELED)
:measures (2})
Transcript After: (00:57:5l:24) : em __ and em er so we have that as an option here em there

would be a standard ca=ier ....
5.5 Summary
In this chapter I have described the observation method for conducting experi-
ments and analysis method for transforming protocol data into a classification
which can be quantitatively analyzed. In the next chapter I will describe the
results of the quantitative analysis and the understanding gained from the information handling study.
Information Handling Beha\ior of Designers During Conceptual Design: Three Experiments
Chapter 6
Understanding Information
Handling Behavior:
Quantitative & Qualitative
Results
/-------.
/-----""
( Dedal Performance \
\.
Results
/
( Information Classification \
\
Fmmewo~
I
------_/
/-----""
\.
Fmmewo~
/
t
t
J---~
------------------------------
- - ,
Analyze
I
/
Analyze
\~rb~~~f"ent
1990-1991 \
Information
Reuse Study
Support by
Dedal
-
;.
Ana !yze
9~~~F
/.
1991-1992 \
Observe
.....
~~ Dedal Use
Support by....._
Dedal
1992 1995\
..
~~ Information
Handling
Study
Understanding
of Information
Handling
This chapter reports on the quantitative and qualitative results of the
information handling behavior study. Results on information

fragment duration (deltat), design information measure (dim),
distribution of time & information in the information fragment
attributes (informational activity, descriptor, subject-class, level-ofabstraction), state transition analysis and multiple attribute analysis
are reported. Implications of these results improve the understanding
of the design process and the development of intuitive and integral
information handling services.
[nfonnadon Handling Behavior of Designers During Conccprua/ Design: Three ExperimenlS
83
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Results
84
6.1 Introduction
Analysis of protocol data is a long and arduous process. In this study it took an
average of about one hour to analyze one minute of raw data. This makes it
impractical to conduct a large number of experiments. However, the six experiments used in this study are a representative sample of design activity involving
an individual designer doing conceptual design. Therefore, although the results
may not generalize statistically they do raise our level of understanding about
the conceptual design process.
In this chapter, information fragment attributes (or simply attributes) will refer to
the categories used in classifying information fragments, which are: informational
activity, descriptor, subject-class and level-ofabstraction.l In the following sections I

will discuss results on information fragment duration (deltat), design information measure (dim), distribution of time & information in the attributes, state
transition analysis and multiple attribute analysis. At the end of each section the
gain in understanding about conceptual design and the implications for developing information management too1s2 will be discussed.
Out of the six subjects the results for S33 differ considerably from that of the others. S3 appeared to be uncomfortable with the talking aloud protocol and had
little design experience. This could have contributed to the differences. However, since the objective is not to generalize the results, I have chosen to include
the results from S3 in all the analysis. The results reported include contributions
from all six subjects. When reporting graphical results, in many cases results for
one subject chosen randomly will be shown in this chapter. Results for other subjects and some supporting data will be included in Appendix D.
1. Medium and level-of-detail are intentionally not included in this list. Results on medium are not considered
since the nature of protocol experiment methodology biases the data towards the audio medium. Results on
level-of-detar1 are not considered since it has been replaced by level-of-abstraction.
2. From the information management viewpoint the functionality of interest is similar to what Dedal was
designed for, such as: information capture, information structuring (indexing) and information retrieval.
3. The six experimental subjects in this study will be referred to as 51-56 respectively.
85
6.2 Information Fragment Duration (deitat)

An information fragment is the unit of measurement in this study as described in
section 5.2.1 and 5.2.2. The duration of an information fragment, to be referred to
by deltat, is the difference between the end-time and the start-time for an information fragment. deltat is a measure of how frequently designers move between
different kinds of design information1, and therefore informs us regarding the
dynamics of information handling.
6.2.1 deltat Variation over Time

Figure 6-1 shows the variation of deltat over time for 51. This graph for other subjects is similar in characteristics.
Altogether, they show that designers
frequently and fluidly change the type of information they are handling, spending between 2 to 35 seconds with one kind of information. The smooth curve in
Figure 6-1 is a polynomial fit to the raw data. It illustrates movements between
small and large information fragments at a level higher than the raw data. It
shows that the behavior is oscillatory, i.e. deltat is neither steadily increasing nor
decreasing.
6.2.2 Statistics on deltat

Table 6-1 shows the statistics on deltat for each of the subject. It lists the design
activity time and the number of information fragments handled by each of the
subject. A total of 1605 information fragments were extracted from 347 minutes
of data. The average deltat ranged from 11 to 19 seconds and the average over all
subjects was about 13 seconds. The distribution of deltat is shown in Figure 6-2.
The figure shows that the mode (most frequent) of deltat is about 6 seconds. The
variation in the mode across the subjects is small.
1. By definition of information fragment, when designers move from one fragment to another they change
the information type (section 52.1).
Information Handling Behavior of Ocsigners During Conccplual Design: Three ExpcrimcnlS
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Resuffs
86
f\
rn 30
c:
"'0
0
(,)
Q)
rn
c:
;:
:::J
c:
Q)
E
C)
u.
0
as
E
....
"'" ~~
10
~
11:1
c:
;:
20
th
!It:.t
~~
.E
c:
50
30
70
90
TIme (minutes)
Figure 6-1 Information fragment duration vs. time for 81. Note that deltat ranges from
about 2 seconds to 35 seconds. The thick curve is a polynomial fit to the raw data showing
the movements between short and long information fragments at a higher level than
shown by the raw data. It shows an oscillatory behavior.
6.2.3 Discussion on deltat Results

The results on deltat inform us that designers move from one type of design
information to another on an average every 13 seconds. Most frequently they
spend about 6 seconds with anyone kind of information. These numbers provide us with an understanding of the pace at which information handling is
taking place during conceptual design. They also explain why paper is the
medium for most conceptual work, as it poses no hindrance to designers in moving from one information type to another. As was the case during the
experiments, using paper as a working medium, designers worked fluidly and
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative Resufts
87
-S1
-52
-53
---54
- - 55
-S6
~--"'~-:"': ...
: ...: ...'"
...
00~--~--~10~--~-~2~0~--~~==~3~0~~~
Duration of Information Fragment (deltat seconds)
Figure 6-2 Distribution of information fragment duration. The smooth curves
shown here are 10 degree polynomial fit to histograms with bin width of 0.5. For
clarity sake, the histograms are not shown here, they are shown in Appendix D.
Table 6-1 Information fragment duration (delta~ statistics. The design activity time listed
below only includes the sum of the time for all information fragments and hence is less than
the total design time listed in Table 5-1. Average deltat=(design time)/(# of info-fragments).
Most frequent deltat is read from Figure 6-2 (the pOint of maxima on the curves).
Subject
Design1iJDe
(minutes)
Number oCinfofragments
Average dellat
(seconds)
Most Crequent
dellat (seconds)
Sl
S2
S3
S4
S5
S6
67.5
49.9
62.9
44.7
42.4
79.6
13
TotallAvg.
347.0
323
262
200
220
190
410
1605
5
6
6
5
5
6
6
11
19
12
13
12
13
with ease. Akin [1978] also found that designers shift attention among different
design issues frequently, and that the design issue being attended to and the representation used are related.
Being able to work fluidly and with ease is essential during the conceptual
Information Handling Behavior of Designers During Conceptual Design: Three Experimcnl~
Chapter 6: Understanding Information Handling Behavior. Quantitative & Qualitative Results
88
design process. A service aiming to provide support for information management during the conceptual process should allow designers to work at a pace
suggested by the results on deltat. Next generation design tools should realize
this and build interfaces which are responsive and adapt to the designers pace.
A measure of deltat for emerging media may be useful index for estimating their
power or improvement.
6.3 Design Information Measure (dim)

The quantitative measure for the design information used in this study is called
design information measure. The unit of this measure is dim. Dim has been
defined in section 5.3.3. The knowledge of how much information designers handle will provide an insight into the burden faced by designers in managing this
information. Table 6-2 lists in column 2 & 3 the design activity time and the number of dims handled by each of the subjects. A total of 2802 dims were handled in
347 minutes.
6.3.1 Information Handling Rate

The ratio of the amount of information handled (in dims) to the design time (in
minutes) is being called the information handling rate to be measured in dims
per minute (dpm). Dim rate can be calculated globally (total number of dims
divided by total design time) as well as locally for each information fragment
(number of dims in the info-fragment divided by deltat for the information fragment). Since the number of dims and deltat vary with the information fragments,
so does the local dim rate. Figure 6-3 shows the variation of dim rate over time
for 54. This graph for other subjects is similar in character. They show that the
local dim rate ranges from 2 to 40 dpm. This means that, based on the definition
of dim, the designers handle anywhere between 2 to 40 [descriptor, subject-class]
pairs in one minute. Table 6-2 shows, in column 4 & 5 the global dim rate and
the average local dim rate for each of the subjects.
Information Handling Behavior of Designers During Conceptual Design: Three Experimenl~
89
40
30
E
%
.cg
a:CIS 20
E
20
o
o
~~ ... I
,~
U
20
CI
~
~
IIUIII
I~
If
"II
V
40
60
TIme (minutes)
80
Figure 6-3 Information handling rate over time for 54. The dim rate ranges
between 2 and 40 dpm. The thick curve is a polynomial fit and illustrates the
variation in dim rate at a level higher than the raw data.
The global dim rate ranged from 6.2 to 9.2 dpm (average being 8.1 dpm) and the
average of the local dim rate ranged from 10.5 to 13.8 dpm (average being 12.6
dpm)l. The small range on the observed dim rate is an encouraging sign for the
definition of the dim measure. It suggests that the dim rate is measuring something that is within a small range for most humans. It is a hypothesis that it most
closely relates to the information processing ability of humans. The dim rate
may correlate with some of the results from studies in cognitive psychology on
human information processing. However, before these correlations are investigated further it is important to measure the correlation with some obvious
factors.
1. The global average and the average of the local averages differ since the duration of information fragments is a variable. If all deltat were the same, i.e. the segmentation process fragmented the data at equal
time intervals, than the global and the local averages would be equal.
lnfonnalion Handling Behavior of Designers During Conceptual Design: Three Experiments
90
Table 6-2 Information handling rates and verbal rates for all subjects.
dim = design information measure; dpm dims per minute; wpm = words per minute;
Dt = total design activity time in minutes; N =total number of dims handled;
Global Dim Rate N/Dt dpm; Global Verbal Rate (Total words spoken)! Dt wpm;
C r = Coefficient of correlation between local dim rate and local verbal rate;
Cd=Coefficient of determination
Dim Rate (dpm)
Verbal rate (wpm)
Subject
Dt(min)
N(dim)
Global
Iocalav.
Global
Iocalav.
Cr
Cd
Sl
67.5
572
8.5
13.3
130.0
15l.2
0.51
0.26
S2
49.9
406
8.1
U.8
97.7
122.4
0.49
0.24
S3
62.9
387
6.2
10.5
77.6
104.4
0.88
0.77
S4
44.7
411
9.2
14.3
88.5
116.4
0.7
0.49
S5
42.4
337
8.0
11.6
91.1
105.6
0.52
0.27
S6
79.6
689
8.7
13.8
117.3
141.6
0.55
0.30
TotallAvg.
347.0
2802
8.1
12.55
100.4
123.6
0.69
0.39
Besides the information processing ability of humans, the dim rate can be a function of variables such as the verbal rate (measured in words per minute, wpm),
design experience, design problem and possibly others. There certainly is some
correlation between the dim rate and the verbal rate of subject, since in the limiting case when there is no verbalization the dim rate would be zero since there
would be no basis to measure dim. Also humans talk at a rate within a range
when they need to be comprehended by others. This would suggest that the verbal rate would be within a small range. In that case, is the dim rate merely
measuring the verbal rate of the subjects?
Regression analysis was performed between dim rate and verbal rate to determine the extent of correlation between them. Table 6-2 lists the global verbal rate
and average local verbal rate for each of the subject. It also lists the coefficient of
correlation! and the coefficient of determination2 between the local dim rate and
local verbal rate. On an average the coefficient of correlation was about 0.7 and
1. Coefficient of correlation (C f ) provides the extent to which two variables are related [Runyon 1988. page
166]. Details on how Cf is calculated is given in Appendix D.
2. Coefficient of determination (CdJ provides an estimate on the proportion of variance in one variable due to the
other. It indicates the proportion of total variation that is explained by the coefficient of correlation [Runyon 1988.
page 210]. Details on how Cd is calculated is given in Appendix D.
Chapter 6: Understanding Information Handling Behavior. Quantitative & Qualitative Resuns
91
the coefficient of determination was about 0.4. This means that about 40% of the
variation in dim rate is accounted for by the variation in verbal rate. This level of
correlation is consistent with expectations.
6.3.2 Discussion on Design Information Measure Results

We now understand that designers in the experiments handled anywhere
between 2 to 40 [descriptor, subject-class] pairs in one minute. The average information handling rate was about 13 dims per minute (dpm). As was expected, the
information handling rate is correlated well with the verbal rate. However, verbal rate only accounts for about 40% of the variation in the dim rate. It is a
hypothesis that a share of the remaining contribution to dim rate can be correlated with the human information processing abilities [reference1. This is
suggested as possible future work.
Design tool creators can use the information about dim rate to design interfaces
which enable designers to deal with an adequate amount of design information
in a given time. A service should support information generation, access and
analysis at a pace suggested by the observed range on information handling rate.
6.4 .Distribution of Time and Design Information

Knowledge of deltat and dim rate gives an insight into the dynamics of information handling. However, knowledge of time spent and design information
handled among the different values of the information fragment attributes (the
values are listed in Table 5-2), provides an understanding of where designers
spend the major share of their time and information. This understanding can be
used to focus the development of support services.
6.4.1 Informational Activities Distribution

Figure 6-4 shows the distribution of time and design information across the
92
informational activities. It shows that subjects spent 55% of their time generating
information, 24% of their time accessing information and 21% analyzing information. Also, 53% of the design information was generated, while 25% was
accessed and 22% was analyzed. This suggests that conceptual design is mainly
a generative activity. The relative dominance of generative activity indicates that
unless support services can keep pace with generation of new information, they
will fail to support the conceptual design process.
"Ndim
Dnme
Analyze
Access
Generate
55
20
40
60
Proportion (%)
Figure 6-4 Distribution of time and design information across informational
activities. Ndim is the number of dims. More than half the time and information is
dealt with in generate, suggesting that conceptual design is generation intensive.
6.4.2 Information Descriptor Distribution

Figure 6-5 shows the distribution of time and design information across the
information descriptors. Most of the time (65%) is shared by four descriptors:
construction (22%), alternative (15.5%), operation (15.1%), and requirement (12%).

Also, about 60% of the total information handled belonged to these four descriptors. This suggests that developers of information handling services should
identify the characteristics of these descriptors in the context of computational
support (representations, media and interfaces) and give adequate attention to
Chapter 6: Understanding Information Handling Behavior: Quantitative & Qualitative ResuUs
93
~Ndim
Miscellaneous ..,
c::J TIme
.iiiiiiL---,
Relation .iiiiiiiiiiiiii------J
Requirement
Rationale
~iiiiiiiii.
.iii
Operation .iiii
Performance
ii~------.....J
Location
Construction ---,
ijil------------~
Comparison __
Assumption
.iiii~
Altemative - - - .
20
10
30
Proportion (%)
Figure 6-5 Distribution of time and design information across descriptor. Ndim
is the number of dims. Major share of time and design information is grouped
under the categories: construction, alternative, operation and requirement.
Representations, media and interfaces which facilitate handling of these
information types would cover a significant share of conceptual activity.
integrating these characteristics. For instance information about operation is often

best captured and described by a visual medium (3-D geometric model or video)
and information about alternatives by a decision tree or hypertext tool. Identifying a collection of such characteristics so that they can be made available to
designers in an integrated seamless fashion will go a long way in supporting
information handling.
6.4.3 Information Subject-Class Distribution

Figure 6-6 shows the distribution of time and design information across the sub-
ject-classes. A major share of the time is spent in handling information associated
Information Handling Behavior of DesignetS During Conceptual Design: Three Experiments
94
Chapter 6: Understanding Information Handling Behawor: Quantitative & Qualitative Resufts
with assembly (29%). The distribution is quite even among component (17.4%),
design-concept (15.7%) andfeature (13.8%). About 75% of the information handled

belonged to these four subject-classes. Again information handling tools should
give adequate attention to representations, media and interfaces which make
handling information belonging to these subject-classes easier.
"Ndim
Other.--..,
L-,
Time
Design-Concept
Requirement
Feature ...,
Connection
component
Assembly ...,
20
10
30
Proportion (%)
Figure 6-6 Distribution of time and design information across subject-class.

Ndim is the number of dims. Assembly, component, design-concepts and feature
account for a large share of time and information.
6.4.4 Information Level of Abstraction Distribution

Figure 6-7 shows the distribution of time and design information across the levelof-abstraction. All levels except quantitative are well represented. The distribution of time is unlabeled (21%), labeled (28%), associative (22%), qualitative (22%)
and quantitative (7%). We know that computational tools are very good at representing and manipulating quantitative information. However, 93% of the time
and 94% of the design information during conceptual design is dealt with in nonquantitative levels of abstraction.
Over the past few years there has been
progress in dealing with associative and qualitative information [references].

However, support services are yet to become adept at manipulating unlabeled
95
"Ndim
c:::J TIme
Quantitative
Qualitative
Associative
Labeled
Unlabeled
Proportion (%)
Figure 6-7 Distribution of time and information across level-of-abstraction.
Ndim is the number of dims. The plot indicates that, most of the time and
information is dealt with in non-quantitative levels of abstraction.
information and capturing its evolution over time.

Since level-of-abstraction can be ordered by the extent of detail represented
(abstract to concrete), it is valuable to look at the behavior of level-of-abstraction
over time. Figure 6-8 shows the movement of S4 between the levels of abstraction over time. The graphs for other subjects are similar in characteristics.
They show that designers move from one level to another often without any
apparent pattern. It was also observed that in most cases a new concept started
out as unlabeled and matured to other levels, although not in any particular
order. This means that support for easily transitioning between the different
level-of-abstraction is essential to supporting the conceptual design process.
6.4.5 Discussion on Time and Information Distribution

The distribution of time and design information in each of the information frag-
Infonnation Handling Behavior of Designers During Conceptual Design: Three ExperimenlS
96
Quantitative
Qualitative
Associative
Labeled
Unlabeled
10
20
30
40
50
60
70
80
Time (minutes)
Figure 6-8 Variation in level-at-abstraction over time tor 84. It was observed that
concepts often started out at unlabeled level and matured to other levels as they
became more concrete. Movement between the different levels happens often.
However no pattern is obvious.
ment attributes, informs us of the types of design information which are

frequently dealt with during conceptual design. This information can be used to
prioritize the functionalities of support services and in selecting representations,
media and interfaces for manipulating information. The challenge lies in identifying the characteristics of services to support the behavior observed, and
providing simple and well integrated mechanisms for moving between these
behaviors.
6.5 State Transition Analysis: Statistics and Probabilities

In the previous sections we have developed an understanding of how frequently
designers transition from one information type to another. To further this understanding, it is important to look deeper at the points of transition. Points of
transition can be defined as the points where the designer changes from one
97
state! to another for any of the information fragment attributes. State transition
statistics2 will inform us about transitions which are more frequent and therefore more critical from the viewpoint of developing support. On the other hand
state transition probabilities3 inform us about which transition is most likely
from a given state and can be used in developing adaptive interfaces. In this section results on state transition statistics will be reported. Results on state
transition probabilities are included in Appendix D. fu the figures below the
arrows represent a transition from and to the states connected by the arrow.
6.5.1 Informational Activity State Transition Analysis

Figure 6-9 shows the state transition statistics for informational activities. With
three distinct states, there are six possible combinations for transitions. Transitions between generate and analyze make up 51% and transitions between
generate and access make up 31%, of all the informational activity transitions.
Clearly a tool which makes it easier to move to and from generate will be better
at supporting information handling.
6.5.2 Information Descriptor State Transition Analysis

Figure 6-10 shows the state transition statistics for information descriptor. There are 11
distinct states, therefore 110 types of transitions are possible. For clarity sake, only the
frequent transitions are shown in Figure 6-10. It is evident that transitions to and from
construction are more common than others and altogether they make up 29% of the total
descriptor transitions. Transitions to and from alternative (11 %), operation (9%) and
1. State refers to a particular instance of each of the information fragment attributes as listed in Table 5-2.
2. State transition statistics are calculated by counting the number of transitions of a particular type. For
comparison purposes results reported are normalized. So results for each transition type is reported as a
ratio (expressed in percentage) of the number of the transition type to the total number of transitions
observed.
3. State transition probabilities are calculated under the assumption that a next state is determined by only
the current state and not any states prior to it (i.e. the memory of anything but one previous state is erased.
This assumption is commonly made in developing Markov chain for a process with discrete states. Such an
assumption is valid for a first order analysis.
98
Figure 6-9 Informational activity state transition statistics. All

six
possible
transitions are shown. Transitions to and from generate make up 82% of all the
observed transitions.
relation (6%) are also significant. An information handling tool should make it easier to
transition to and form construction, alternative, operation and relation type information.
Most transitions between any two states are between construction and alternative (6.6%).
Figure 6-10 Information descriptor state transition statistics Only

the
most
frequent transitions of the 110 possible combinations are shown here. Transitions
to and from construction, alternative, operation and relation make up bulk of the
transitions. Transition between construction and alternative are most likely {6.6%}.
99
6.5.3 Information Subject-Class State Transition Analysis

Figure 6-11 shows the state transition statistics for infonnation subject-class. There are
six distinct states, therefore 30 types of transitions are possible. Once again for clarity
sake only the frequent transitions are shown in Figure 6-11. Transitions to and from
assembly and component make up 54% of all the subject-class transitions. Transitions to
and from feature (19%) are also frequent. Most transitions between any two states are
between assembly and component (13.7%).
Figure 6-11 Information subject-class state transition statistics Only the most
Significant of the 30 possible combinations are shown here. Transitions to and
from assembly and component make up a large share of the total transitions.
Transitions between assembly and component are most likely (13.7%).
6.5.4 Information Level-of-Abstraction State Transition Analysis

Figure 6-12 shows the state transition statistics for information level-of-abstraction. There are 20 types of transitions possible for the five distinct states. Top 10
frequent transitions are shown in Figure 6-12. Transitions among the states unla-
beled, labeled, associative and qualitative make up 93.5% of all the level-ofabstraction transitions. The frequency of transitions to and from each of the
states is unlabeled (25%), labeled (53%), associative (34%), qualitative (37%) and quan-
titative (6.5%). A critical need for easy transitions between non-quantitative levels
Information Handling Behavior of Designers During ConceplUaI Design: Three ExperimenlS
100
of abstraction is evident. Most transitions between any two states are between
labeled and associative (19.5%).

8.1
Figure 6-12 Information level-of-abstraction state transition statistics Only

the
most significant transitions of the 20 possible combinations are shown here.
Transitions to and from unlabeled, labeled, associative and qualitative make up a
large share of the transitions.
6.5.5 Discussion on State transition Analysis

From this section and the previous sections we have come to understand that frequent transitions between different information types is fundamental to the
conceptual design process. State transition statistics deepen our knowledge of
the designers movements between different states for each of the information
fragment attributes. They identify transitions which frequently take place and
therefore are critical to support.
From the viewpoint of most tools and services, mechanisms for supporting transitions are the common places contributing to difficulty in usability. Since most
computation tools are syntax driven a user has to explicitly specify a transition.
For instance if a user wants to draw a circle in the middle of drawing a line, he
or she has to select circle from the menu, then draw the circle and select line to
come back to drawing a line. Paper is popular in conceptual design since it offers
no hindrance in making such transitions. It is anticipated that the knowledge of
Information Handling Behavior of Designers During Conceprual De.ign: Three Experiments
101
the state transition statistics can be used in developing well integrated services,
which support the frequent transitions well and therefore will be user friendly.
6.6 Multiple Attributes Analysis

The behavior of one attribute at a time informed us about the relative importance of different values of the attributes. However, deeper insight can be gained
by looking at behavior over multiple (two or more) attributes at a time. First
results over two attributes combined together are described followed by results
over all four attributes. Because of space considerations, only some of the combinations are discussed here. Appendix D has results for some of the other
combinations.
6.6.1 Informational Activities and Information Descriptor

Table 6-3 shows the distribution of informational activities across the information descriptors. The information generated most often was of descriptor
alternative (19%) and construction (17%), information accessed most often was of
descriptor construction (25%) and requirement (20%), and information analyzed
most often was of descriptor construction (26%) and performance (21%). Looking
at these combinations one can recommend that a successful support service
should make it easy to generate [alternative, construction] information, access [con-
struction, requirement] information and analyze [construction, performance]

information.
Table 6-3 Distribution of information activities across deSCriptors. The numbers below show the
proportion of the different information descriptor in each of the informational activities. Two of the
most significant combinations for each informational activity are highlighted.
Gen
Assn
Comp
Loca
Oper
Perf
Rati
Rela
Requ
Mise
12
10
11
Ace
11
Ana
13
13
Information Handling Behavior of Designers During ConceptUllI Design: Three Experiments
102
6.6.2 Informational Activities and Information Subject-Class

Table 6-4 shows the distribution of informational activity across the different
subject-classes. It can be observed that most of the information generated was
about component (24%), assembly (17%), design-concept (17%) and feature (16%),
most information accessed and analyzed was about assembly (41% and 26%
respectively). Thus an information handling support service should make it easy
to generate [component, assembly, design-concept, feature] information, access assem-
bly information and analyze assembly information.

Table 6-4 Distribution of informational activity across subject-class. The
numbers
below
represent the proportion of the different subject-classes in each of the informational activity.
The most frequent combination for each informational activity is highlighted.
Conn.
Feature
Reqmt.
D-conc.
Other
Generate
13
16
17
Access
12
13
10
Analyze
10
17
14
16
Activity
6.6.3 Informational Activities and Level-of-Abstraction

Table 6-5 shows the distribution of informational activities across the information level-of-abstraction. Information is well distributed across all but
quantitative level-of-abstraction. Therefore a support service for conceptual

design should make it easy to generate, access and analyze information at unla-
beled, labeled, associative and qualitative level-of-abstraction.

Table 6-5 Distribution of informational activities across level-of-abstraction. The
numbers
below represent the proportion of the different level-of-abstraction in each of the
informational activity. The most frequent combinations for each informational activity are
highlighted. It is clear that much of the concentration is in non-quantitative levels.
Quantitative
Activity
Generate
Access
10
Analyze
Infonnalion Handling Behavior of Designel'5 During Conceptual Design: Three ExperimenlS
103
6.6.4 Most Frequent Information Fragments

Table 6-6 shows the top nine most frequently encountered combination of the
information fragment attributes. The listing is in the order of the time spent in
each of the combinations. The table also lists the number of dims handled and
the number of information fragments in each of the combinations. About 4.4% of
the total time and 4.8% of the design information is handled in generating qualita-
tive operations of an assembly. However the most number of information

fragments were in generating unlabeled alternative design-concepts. Looking at combinations in this manner can be used in prioritizing the functionalities of support
services.
Table 6-6 Frequently encountered info-fragment attribute combinations. The number of fragments,
time and the number of dims of information in each combination are shown along with their
proportion form the total.
InfoActivity
mon* %*
TIme
Level of
Subject-clas
Descriptor
ndim
# of fragments
%
4.8
44
2.7
Generate
Operation
Assembly
A ......
Qualitative
Generate
Alternative
Design-concept
Unlabeled
12.9
3.7
61
2.2
Access
Requirement
Assembly
Labeled
Access
Requirement
Design-concept
Unlabeled
Generate
Relation
Component
Associative
Generate
Operation
Component
Qualitative
2.1
1.8
1.5
1.3
Construction
Component
Labeled
Generate
Construction
Component
Associative
Access
Performance
RI
Labeled
59
35
62
51
31
40
40
2.1
1.2
2.2
1.8
Generate
7.4
6.1
5.1
4.6
3.8
3.6
3.5
4.4
1.1
1.0
l.0
l.l
1.4
1.4
3.2
25
32
27
26
22
20
21
1.6
2.0
1.7
1.6
1.4
1.2
1.3
6.7 Summary
Table 6-7 summarizes the quantitative and qualitative findings presented in this
chapter. Altogether this raises our understanding of the information handling
behavior of designers during conceptual design tasks. This understanding has
made it possible to formulate some recommendations for developing successful
information handling support services.
104
Table 6-7 Summary of the key findings and recommendations.

IA=lnformational activity; D=Descriptor; SC=Subject-Class; loA=level of abstraction
ditn=design information measure; dptn=dims per minute;
RecoDUDendation~)
Observation(s)
Results on:
deltat
average=13 sec; mode=6 sec;

range={2 to 35} seconds
useful index for estimating power of

support
Dim Rate
about 8 [D.SC] pairs per minute

mode=6 dpm; range={2 to 40} dpm
another measure to estimate power of

support
IA
generate (53%)
emphasis on ease of generating new

information
construction. alternative. operation

(51%)
emphasis on ease of handling these

types of information
SC
assembly, component. design-concept(62%)
emphasis on ease of handling these

types of information
LoA
all levels except quantitative are well

represented
emphasis on ease of handling all levels of abstraction
Distribution
of dims
State Transition Analysis
IA
82% of all transitions are to and from
should be easy to move to and from
generate
generate
construction (29%). alternative

(11 %). operation (9%)
movement to and from these should

be the easiest
SC
assembly and component (54%)
movement to and from these should

be the easiest
LoA
non-quantitative levels (96%)
easy transitions between non-quantitative levels
Most information is generated about

qualitative operation of an assembly
Brings out combinations which provide recommendations at a deeper

level
Multiple Attribute
Analysis
and similar observations
Information Handling Behavior of Designers During Conceprual Design: Three ExperimenlS
Chapter 7
Conclusions & Future Work
Design Research
< >

Low Cost Products
Goal
~~
TO
TO
,r
shortest path
Develop Research
Methods
T~
Understand Design
Practice
Improve Design
Practice
~o
This chapter revisits the salient points of this thesis. It presents

concluding remarks on the information reuse study, the information
handling study, and deployment of Dedal. It states the contributions
that this thesis is making to the field of design research and presents
some opportunities for further research.
105
Chapter 7: Conclusions & Future Work
106
7.1 Concluding Remarks

This thesis demonstrates a methodology for incrementally understanding and
supporting the conceptual design process. It does so by reporting on two observational studies and one information management service. It shows that the
verbal protocol method is good for studying individual designers and describes
a procedure for analyzing protocol data to extract understanding of the design
activity and requirements for support services.
By developing and deploying Dedal, this ~sertation demonstrates how requirements emerging from a protocol study can be used to develop successful
support services. It also presents an information handling framework for studying
the information handling behavior of designers. Using this framework it
describes the gain in understanding about the information handling behavior.
This dissertation has found that the ability to transition frequently between different types of information is fundamental to the conceptual design process.
Information handling services available today are mainly syntax driven1 and fail
to support transitions effectively. In such services, a transition has to be explicitly specified by the designer, which results in breaking his or her rhythm. On
the other hand, paper presents no hindrance in making transitions, and therefore is used extensively during conceptual design. This suggests that there is a
need to develop well integrated representations and interfaces which allow the
designer to transition easily between different information types. I hope that the
quantitative measures presented in this dissertation will be used to evaluate services in their ability to support smooth transitions.
Design is an information intensive activity. The rewards for capturing and structuring information during a design process for efficient reuse during or after the
1. An example of syntax driven is: if one wants to change from drawing a line to drawing a circle, he/she
has to explicitly pick circle from a menu. When using paper no such change is needed.
107
design process are well known. The work reported in this dissertation, my own
design experience and collaboration with other researchers at the Center for
Design Research [Leifer 1990; Lakin et al. 1989; Mabogunje 19931 has resulted in
the following realizations:
design information capture and structuring should happen during the
design process as opposed to after the process,
design information capture and structuring should be a natural extension of
the design process instead of being an add on activity, and
the rewards of doing information capture and structuring should be available as a natural feedback to the designers doing the capture and structuring, as opposed to only the next generation of designers or re-designers.
7.2 Contributions
This dissertation adds knowledge to the field of engineering design research. It
corroborates verbal protocol approach as a valuable method for studying, understanding
and
incrementally
improving
engineering
design
activity.
Additionally, this dissertation makes three key contributions, it

demonstrates a framework for characterizing the information handling
behavior of designers. This framework effectively classifies information that
should be captured during the design process for effective reuse,
demonstrates a computational implementation of the design information
framework that is intuitive for designE'fs and works in real time, and
rigorously describes the information handling behavior during conceptual
design using quantitative measures and makes recommendations on how
these measures can be incorporated in building successful services to support information handling.
7.3 Discussion
The complexity of the design process presents many challenges in understanding and supporting it. Although this dissertation shows that protocol studies
have the potential for providing a detail and rich understanding, it also illustrates that the process of conducting the experiments and analyzing the protocol
108
data is a long, tedious and a subjective process. As a result, it is often not practical to conduct experiments on a statistically large sample for each of the
scenarios and perspectives that are observed in a design process. However,
because of the success of protocol studies the interest in using this approach is
growing. Therefore, like in any new research field, the design research community faces the challenge of establishing mechanisms for validation and
comparing results across researchers. This will enable new research to build on
top of results already available.
While observational studies provide a rich understanding and lead to requirements for support services, embodying these requirements in a usable
computational implementation is a challenge and probably a subject of research
as well. I found being involved in real design during the course of conducting
this research very empowering in shaping my judgement and my ability to pick
representations, media and interfaces for computational implementations.
7.4 Future Work

This dissertation raises numerous opportu.'1ities for further research. A natural
extension, in the interest of the iterative methodology presented, would be to

use the recommendations stated in the Chapter 6 to develop support services for
information management. These services could be evaluated using the observational methodology presented in this dissertation, to further refine and improve
information handling support. Few other directions for further research are:
study information handling in the other phases of the design process, so that
support for information management can be continued all along the design
process
investigate correlation between design information measure and cognitive
models of human information processing
investigate relationship between information media and information handling behavior
109
use the iterative methodology to study, understand and support the design
process from a viewpoint other then information handling behavior.
use the information handling framework to analyze information handling
behavior in other design scenarios such as team design
The work presented in this dissertation is only a beginning towards studying,
understanding and supporting information handling behavior during conceptual design. I hope that the recommendations made in this dissertation will be
useful in building support services. I also hope that the frameworks and the
methodology reported will be employed to study other design scenarios and
other phases of the design process.
Bibliography
110
Bibliography
Akin Orner, ''How do Architects Design?", Artificial Intelligence and Pattern
Recognition in Computer Aided Design, J. Latombe (Ed.), IFIP, North-Holland
Publishing Company, 1978.
Akin, Orner, An Exploration of the Design Process", Design Methods and
Theories, Vol. 13, No. 3-4, pp 115-119, 1979. Also published in Developments in
Design Methodology, Nigel Cross (Ed.), Chichester, John Wiley & Sons, pp 33-56,
1984.
/I
Alexander, Christopher, Notes on the Synthesis of Form, Harvard University Press,

Cambridge, MA, 1964.
Antonsson, Erik K., "Development and Testing of Hypothesis in Engineering
Design Research", Journal of Mechanisms, Transmissions, and Automation in Design,
Vol. 109, pp. 153-154,1987.
Archer, L.B., "Systematic Methods for Designers", Design, Vols. 172-174, 176,
179, 181, 185, 188, April 1963 - August 1964. Also in Developments in Design
Methodology, N. Cross (Ed.), Wiley, Chichester, 1984.
Asimov, Morris, Introduction to Design, Prentice Hall, Engelwood Cliffs, NJ, 1962.
Babbie, Earl R.,"The Practice of Social Research", Wadsworth Publishing Co.,
Belmont, CA, Fourth Edition, 1986.
Baudin, Catherine; Underwood, Jody G. and Baya, Vinod, "Using Device
Models to Facilitate the Retrieval of Multimedia Design Information", In
Proceedings of the 13th International Joint Conference on Artificial Intelligence,
Chambery, France, pp. 1237-1243, August 29-September 2,1993a.
Baudin, Catherine; Kedar, Smadar; Underwood, Jody G. and Baya, Vinod;
"Question-based Acquisition of Conceptual Indices for Multimedia Design
Documentation", In Proceedings of the 11th National Conference on Artificial
Intelligence AAAI-93, Washington D.C., pp 452-458, July 1993b.
Baya, Vinod; Stout, Greg and Vaska, Matt, "Continuously Variable Damper
Project", Design Requirement Document, ME210 Fall, Winter and Spring quarter
1989-90, Technical Report, Design Division, Department of Mechanical
Engineering, Stanford University, 1990.
Baya, Vinod; Gevins, Jody; Baudin, Catherine; Mabogunje, Ade; Toye, George
and Leifer, Larry, "An Experimental Study of Design Information Reuse", In
Proceedings of the 4th International Conference on Design Theory and Methodology,
Bibliography
111
ASME, Scottsdale, Arizona, pp.141-147, Sept. 13-16, 1992.

Baya, Vinod and Leifer, Larry J., IIA Study of the Information Handling Behavior
of Designers During Conceptual Design", In Proceedings of the 6th International
Conference on Design Theory and Methodology, ASME, Minneapolis, Minnesota, pp.
153-160, Sept. 11-14, 1994a.
Baya, Vinod and Leifer, Larry J., ''Understanding Information Management in
Conceptual Design", Chapter in Analysing Design Activity, Cross, N.; Christiaans,
H. and Dorst, K. (Eds.), John Wiley & Sons, pp. 151-168, 1996.
Baya, Vinod and Leifer, Larry J., ''Understanding Design Information Handling
Behavior Using Time and Information Measure", In Proceedings of the 9th
International Conference on Design Theory and Methodology, ASME, Boston,
Massachusetts, DE-Vol 83, Volume 2, pp. 555-562, Sept. 17-20, 1995.
Bemers-Lee, T.J. and Cailliau, R. ''World-Wide Web", Computing in High Energy
Physics 1992 Conference, Anneey, France, 23-27 September 1992.
Blair, D.C. and Maron, M.E., "An Evaluation of Retrieval Effectiveness for a FuUText Document Retrieval System", Communications of ACM, 28,3, pp. 289-299,
March 1985.
Blessing, L.T.M., IIA Process-Based Approach to Computer Supported
Engineering Design", In Proceedings of the International Conference on Engineering
Design (lCED-93), The Hague, pp.1393-1400, August 1993.
Chovan, John D. and Waldron, Majula B., liThe Use of Function and Form When
Reading Mechanical Engineering Drawings by Experts and Non-experts", In
Proceedings of the International Conference on Design Theory and Methodology (DTM90), Chicago, illinois, pp. 137-144, Sept. 1990.
Conklin, Jeff and Begeman, M. L., " g ffiIS: A Hypertext Tool for Exploratory
Policy Discussion", In Proceedings of the 1988 Conference on Computer Supported
Cooperative Work, Portland, Oregon 1988.
Court, Andrew W.; Culley, Stephen J. and McMahon, Christopher A.; "The
Information Requirements of Engineering Designers", In Proceedings of the
International Conference on Engineering Design, The Hague, pp. 1708-1716, August
17-19,1993.
Cross, Nigel, Engineering Design Methods, John Wiley & Sons, July 1990.
Crutcher, Robert J., "Telling What We Know: The Use of Verbal Report
MethodolOgies in Psychological Research", In Psychological Science, Vol. 5, No.5,
pp. 241-244, Sept. 1994.
Dixon, John, R., "On Research Methodology Towards a Scientific Theory of
Bibliography
112
Engineering Design", In Artificial Intelligence for Engineering Design, Analysis and

Manufacturing (AIEDAM), Volume 1, Number 3, pp. 145-157, June 1988.
Dorst, Kees; Christiaans, H. and Cross, Nigel, (Eds.). Analyzing Design Activity,
VViley,Clrichesrex,InK,1996.
Eastman, Olarles M., "On the Analysis of Intuitive Design Processes", Emerging
Methods in Environmental Design and Planning, Gary T. Moore (Ed.),
Cambridge, MIT Press, pp. 21-37, 1970.
Eder, Ernest, VV., ''Design Science - Meta-Science to Engineering Design", In
Proceedings of the 2nd International Conference on Design Theory and Methodology,
Chicago, lllinois, pp. 327-335, Sept. 1990.
Ericsson, K. A. and Simon, H. A., Protocol Analysis: Verbal Reports as Data, 2nd
edition, Cambridge, MA, MIT Press, 1993.
Fulton, Robert E. and Craig, James, I., Report from a Workshop on "Information
Framework Technology for Integrated Design/Engineering Systems", Sponsored by
NSF and Georgia Institute of Technology, Callaway Gardens, Georgia, March 1215,1989.
Greeno, James G., ''Understanding Concepts in Activity", Chapter in Discourse
Comprehension, Edited by C.A. VVeaver, ill, S. Mannes and C.R. Fletcher,
Lawrence Erlbaum Associates, Publishers, Hillsdale, NJ, pp. 65-95, 1995.
Gruber, Thomas; Baudin, Catherine; Boose, John and VVeber, Jay, "Design
Rationale Capture as Knowledge Acquisition: Tradeoffs in the Design of
interactive Tools", In Machine Learning: Proceedings of the Eighth International
Workshop, San Mateo, CA, Birnbaum, L. and Collins, G (Ed.), Morgan Kaufman,
pp.3-12,1991.
Guindon Raymonde, "Designing the Design Process: Exploiting Opportunistic
Thoughts", Human Computer Interaction, Vol 5, pp. 305-344,1990.
Hales, Crispin, "Analysis of the Engineering Design Process in an Industrial
Context", Ph.D. Thesis, Department of Engineering, University of Cambridge, 1987.
Hundal, M.S., "A Computer-Based Approach to Conceptual Design of
Engineering Systems", In Preprints of NSF Engineering Design Research Conference,
Amherst, Massachusetts, pp. 251-265, June 1989.
Kim, Steven, H., Designing Intelligence: A Framework for Smart Systems, Oxford
University Press, 1990.
Kuffner, Tom A. and Ullman, David G., "The Information Requests of

Mechanical Design Engineers", Design Studies, Vol. 12, No.1, pp. 42-50, 1991;
also in Proceedings of 1990 Design Theory and Methodology Conference, ASME, pp.
Information Handling Behavior of Designers During Concepllllll Design: Thn:e Experiments
Bibliography
113
167-174, Chicago, September 1990.

Lakin, Fred, "Spatial Parsing for Visual Languages", Chapter in VISUAL
LANGUAGES, edited by Shi-Kuo Chang, Tadao Ichikawa and Panos. A.,
Ugomenides Plenum Press, 233 Spring Street, New York, NY, 1986.
Lakin, Fred; Wambaugh, John; Leifer, Larry; Cannon, Dave and Sivard, Cecilia,
"The Electronic Design Notebook: Performing Medium And Processing
Medium", THE VISUAL COMPUTER: International Journal of Computer Graphics,
edited by Makkuni, Ranjit, Springer Verlag publishers, 5(4),1989.
Leifer, Larry J., "Generation and Conservation of Design Knowledge", Original
Proposal to the AI Branch, NASA Ames Research Center, Moffett Field, CA, 1989.
Also Center for Design Research Technical Report CDR 1990.
Leifer, Larry J., "Concurrent Engineering Design and Management Knowledge
Capture", Technical Report CDR-NASA-90-3, Center for Design Research,
Stanford University, March 1990.
Leifer, Larry J., "Instrumenting the Design Process: For Real Time Text-Graphic
Design Process Records", In Proceedings of the 1991 International Conference on
Engineering Design, ICED-91, Zurich, pp. 314-321, August 27-29, 1991.
Mabogunje, Ade, "Harnessing Questions in Mechanical Design", Engineers
Thesis, Department of Mechanical Engineering, Stanford University, July 1993.
Mauldin, Michael L.,"Retreival Performance in FERRET: A Conceptual
Information Retrieval System", In Communications of ACM, 448,1, pp. 347-355,
1991.
Minneman, Scott L., "The Social Construction of a Technical Reality: Empirical
Studies of Group Engineering Design", Ph.D. Thesis, Department of Mechanical
Engineering, Stanford University, December 1991.
Minneman, Scott L. and Bly, Sara, "Experiences in the development of a Multiuser Drawing tool", Proceedings of the third Guelph Symposium on Computer
Mediated Communication, Guelph, Ontario, pp. 154-167, May 1990.
National Materials Advisory Board, NMAB-455, "Enabling Technologies for Unified
Life-Cycle Engineering of Structural Components", National Academy Press,
Washington, D.C., 1991.
Newell, Allen, "On the Analysis of Human Problem Solving Protocols", Calcul et
Formalization dans les Sciences de L 'Homme, J. c. Gardin and B. Joulin (Eds.),
Paris, pp. 145-185, 1968.
Newell, Allen and Simon, Herbert A., Human Problem Solving, Englewoods
Cliffs, NJ, Prentice Hall, 1972.
Infonnalion Handling Bchavior of Designcrs During Conceptual Design: Three Experiments
Bibliography
114
Newsome, Sandra L. and Spillers, William R., "Tools for Expert Designers:
Supporting Conceptual Design", Design Theory '88, S. L. Newsome (Eds.),
Springer-Verlag, pp. 49-55, 1989.
O'Shanghnessy, K. and Sturges, R. H. Jr., "A Systematic Approach to Conceptual
Engineering Design", In Proceedings of the 4th International Conference on Design
Theory and Methodology, ASME, Scottsdale, Arizona, pp. 283-291, Sept. 1992.
Pahl, G. and Beitz, W., Engineering Design, Design Council, London, 1984.
(Original German version published in 1977)
Payne, John W., "Thinking Aloud: Insights Into Information Processing", In
Psychological Science, Vol. 5, No.5, pp. 241-248, Sept. 1994.
Perlman, Gary, "Descriptive Models of Cognitive Aspects of the Engineering
Design Process", Design Theory '88, S. L. Newsome (Eds.), Springer-Verlag, pp.
10-17, 1989.
Rabins, Michael; Ardayfio, David; Fenves, Steven; Seireg, Ali; Nadler, Gerald;
Richardson, Herbert and Clark Howard, "Design Theory and Methodology: A
New Discipline", Mechanical Engineering, pp. 23-27, August 1986.
Regan, Melissa and Sheppard, Sheri, "Interactive Multimedia Courseware and
the Hands-on Learning Experience: An Assessment Study", Journal of
Engineering Education, pp. 1-9, April 1996.
Roozenburg, N. and Cross, N., "Models of the Design Process - Integrating
Across the Disciplines", In Proceedings of the International Conference on
Engineering Design (lCED-91), Zurich pp. 186-193, August 1991.
Rouse, William B. and Boff, Kenneth R., "Designers, Tools and Environments:
State of Knowledge, Unresolved Issues, and Potential Directions", System
Design: Behavioral Perspectives on Designers, Tools, and Organizations, William B.
Rouse and Kenneth R. Boff (Eds.), New York, North-Holland, pp. 43-63, 1987.
Runyon, Richard P. and Haber, Audrey, Fundamentals of Behavioral Statistics,
Sixth edition, Random house, New York, 1988.
Schon, Donald A. and Bucciarelli, Louis L., "Design Theory and ~fethods - An
Interdisciplinary Approach", Design Theory '88, S. L. Newsome (Eds.), SpringerVerlag, pp. 29-35, 1989.
Shannon, C.E. and Weaver, W, The Mathematical Theory of Communication,
Urbana: University of lllinois, 1949.
Simon, Herbert A., The Sciences of the Artificial, 2nd Edition, MIT Press,
Cambridge, Massachusetts, 1992 (originally published in 1969).
Bibliography
115
Sivard, Cecilia; Zweben, Monte; Cannon, David M.; Lakin, Fred; Leifer, Larry J.
and Wambaugh, John, "Conservation of Design Knowledge", In Proceedings of
the 1989 American Institute of Aeronautics and Astronautics, AJAA-89, Washington
DC,1989.
Spillers, W.R. and Newsome, S.L.,"Engineering Design, Conceptual Design, and
Design Theory: A Report", In Design Methodology and Relationships with Science,
M. J. de Vries et al. (Eds.), Kluwer Academic Publisher, Netherlands, pp. 103120,1993.
Stauffer, Larry A.,"An Empirical Study on the Process of Mechanical Design";
Ph.D. thesis, Oregon State University, September 1987.
Stauffer, Larry A. and Ullman, David G.," A Comparison of The Results of
Empirical Studies into the Mechanical Design Process", Design Studies, Vol 9, No
2, pp. 107-114, April 1988.
Stauffer, Larry A. and Ullman, David G.,"FundamentaI Processes of Mechanical
Designers Based on Empirical Data", Journal of Engineering Design, Vol. 2, No.2,
1991.
Takeda, Hideaki; Hamada, Susumu; Tomiyama, Tetsuo and Yoshikawa,
Hiroyuki, "A Cognitive Approach to the Analysis of Design Process", In
Proceedings of the International Conference on Design Theory and Methodology (DTM90), Chicago, lllinois, pp. 153-160, Sept. 1990.
Tang, John C. "Listing, Drawing, and Gesturing in Design: A Study of the Use of
Shared Workspaces by Design Teams", Ph.D. thesis, Department of Mechanical
Engineering, Stanford University, April 1989.
Tang, John, C. and Minneman, Scott L.,''VideoWhiteboard: Video Shadows to
support remote collaboration", Proceedings of the Conference on Computer Human
Interaction (CHI) '91, New Orleans, LA, pp. 315-322, April 28 - May 2, 1991.
Tong, Richard M.; Appelbaum, Lee A. and Askman, Victor N.,"A Knowledge
Representation for Conceptual Information Retrieval", In International Journal of
Intelligent Systems, Vol. 4, pp. 259-283, 1989.
Toye, George; Cutkosky, Mark R.; Leifer, Larry J.;Tenenbaum, J. M. and
Glicksman, Jay, "SHARE: A Methodology and Environment for Collaborative
Product Development", In Post-Proceedings of the IEEE Infrastructure for
Collaborative Enterprise, 1993,
Ullman, David G.; Dietterich, Thomas G. and Stauffer, Larry A., "A Model of the
Mechanical Design Process Based on Empirical Data", AI EDAM, 2(1), pp. 33-52,
1988.
Bibliography
116
Ullman, David G., IIDesign Histories: Archiving the Evolution of Products", In

Proceedings of the DARPA WOrkshop on Manufacturing, Salt Lake City, Utah,
February 5-6, 1991.
Valsiner, Jaan (Ed.), The Individual Subject and Scientific Psychology, Plenum Press,
New York, 1986.
Waldron, Kenneth J. and Waldron, Manjula B.,"A time Sequence Study of
Complex Mechanical System design", Design Studies, Vol. 9, No.2, April 1988,
pp.95-106.
Waldron, M.B. and Waldron, K.J., ''Methods of Studying Mechanical Design", In
Mechanical Design Theory and Methodology, Editors: Waldron, M.B. and Waldron,
K.J., Springer-Verlag, New York, pp.21-34, 1996.
Wallace, K. M. and Hales, Crispin, "Detailed Analysis of an Engineering Design
Project", In Proceedings of the 1987 International Conference on Engineering Design,
ICED-87, Boston, MA, pp. 94-102, August 17-20,1987.
Wilson, Timothy D., "The Proper Protocol: Validity and Completeness of Verbal
Reports", In Psychological Science, Vol. 5, No.5, pp. 249-252, Sept. 1994.
Wood, Kristin L. and Antonsson, Erik K., "Engineering Design - Computational
Tools in the Synthesis Domain", The Study of the Design Process: In Proceedings of
NSF Workshop on Design Theory and Methodology, Oakland, CA pp. 337-360,
February 1987.
Information Handling Behavior of Designers During ConcePlual Design: Three ExperimcnlS
Appendix A: Additional Material on Design Information Reuse Study
117
Appendix A
Additional Material on
Design Information Reuse
Study
A.I Design Experiment Material
A.I.I Design Experiment Problem Statement
A.I.2 Description of ME2IO
A.2 Results on individual subjects
A.3 Analysis Examples
A.1 Design Experiment Material

Enclosed is the design problem and a description of first generation design team
(ME2IO), used in conducting the information reuse study experiments.
A.l.l Design Experiment Problem Statement
118
Generation & Conservation of Design Knowledge

CDR-Stanford University, NASA-ARC,
Information Reuse Experiment
Programmable Continuously Variable Damper
BACKGROUND:
The suspension system in cars determines ride handling and comfort
characteristics of the car. The suspension system is a vibrational system with the
three essential vibrational components, mass (of the car), spring (per wheel) and
damper (per wheel). Softer suspension systems provide a comfortable ride as in
luxury cars, whereas stiffer suspensions provide better handling of the road as
in racing cars.
The softness/stiffness of a suspension system can be controlled by the design of
the damper. Conventional dampers, which are based on gas or hydraulic fluids,
produce a damping force (non conservative force) which is a function of the
actuation velocity and the damping coefficient. Thus, for a given damper design
and road conditions there is no way of controlling the damping force. Therefore,
a trade-off is made between the comfort and handling capability of the car,
when designing the damper.
The concept of semi-active suspensions, which has been around for over 20
years, is an attempt at improving the above mentioned trade-off. The semi-active
nature is achieved by having the ability to continuously vary the damping force
developed by the damper, and removing its dependence on the actuation
velocity. A computer onboard the car gets inputs from sensors about the speed,
steering, braking, road condition etc. and it computes the damping force for
dampers at each of the wheel which would result in the best trade-off between
comfort and handling. It then sends appropriate signals to the dampers for
developing the required force.
One ME210 group in 1989-90 designed and tested a working prototype of a semiactive damper based on the rotary friction-disk concept.
The requirements that the design was to meet were the following:
Maximum resistive force in full ON state: greater than 500 lbs
Minimum residual force in OFF state: less than Sibs
Draw a maximum power of 72 watts (12 volts at 6 amps available from car
battery).
Force response with less than 30-degree phase lag for a 10Hz sine wave input.
Damper should have a stroke length of 6 inches (+- 3).
Weight less than 12 lbs.
Information Handling Behavior of Designers During Conceptual Design: Three ExperimeDls
119
Size less than 66 cubic in.

Cost comparable to conventional gas shocks.
Should be able to operate in harsh environments seen by external car parts.
- temperature range (-40 to 120 degree F)
- water proof
- withstand dust/dirt.
Should be maintenance free.
Should last longer than 100,000 miles.
REDESIGN PROBLEM:
Design firm A is designing a vehicle for use on a rough desert terrain as well as
on regular tar roads. This requires the vehicle to have the ability to change its
suspension system characteristics so as to adapt to a particular terrain. The
adaptability can be achieved by having a continuously variable damper in the
suspension system.
Based on some analysis it has been determined that the damper should develop
damping force in the range of 10 lbs to 750 lbs. It has also been determined that
the higher limit on the stoke length of the damper should be at least 10 (+-5 from
mean position) inches. The vehicle will have a ground clearance more than what
is designed in compact cars. This will allow more room and freedom in
integrating the damper with the suspension system. The maximum volume
available for the damper is 120 cubic inches. The desert has an ambient
temperature in the range of 20 to 120 F.
The power available from the onboard battery is 12volts with a peak current of
10 amperes. The damper should be maintenance free.
Design a continuously variable damper which can be used on this vehicle.
Information Handling Behavior of Designers During Conccptual Design: Three Experiments
120
A.I.2 Description of ME210

Generation & Conservation of Design Knowledge
CDR-Stanford University, NASA-ARC
Design Experiment
Description of ME210
This sheet of paper contains background information on the first generation
design of a continuously variable damper which needs to be redesigned. The

design was performed in the academic year 1989-90 at Stanford University by a
group of three masters level graduate students. The project was part of the
requirement for the depth series ME210 A, B and C. The project was sponsored
by the electronics division of Ford Motor Corporation. At the end of the
academic year the design team delivered a working prototype of a continuously
variable damper along with complete documentation of the design and its
performance as evaluated by laboratory experiments. The design team worked
with a budget of about $13,750.
ME210 is a three quarter depth series in Automation in Machine Design.
Approximately 30 students work in groups of two or three through out the
series. In the beginning of ME210 A, students are given two short 2 week long
projects. These projects are typically run as competition (eg. paper bike race) and
students work on these projects from design to fabrication. These projects serve
as warm-up exercises and means by which students can make appropriate
choice of teammates for the corporate sponsored project which span the rest of
A, B andC.
There are about 13-15 sponsored project in any year, and they vary in focus from
process design to product design, automobiles to space applications etc. Each
team has a coach who guides the team over the course of the project in issues of
design, group dynamics, consultation etc. Most coaches are old ME210 students
or industry designers with a good deal of design experience.
Design teams report their progress every two weeks to the teaching staff and the
coaches. Guidelines for themes of the presentation are provided by the teaching
staff to ensure that design teams cover all grounds. The presentation also serve
as a means of getting feedback from other students in the class. Students are
encouraged to use structured methods such as trade-off analysis, value analysis,
how-why analysis as aids in making important decisions and scoping their
problem. They are also advised to rapid prototype their ideas to get early
answers to important questions and discover any surprises.
At the end of each quarter the design team summarizes its progress in the
design requirement document. Thus, the three requirement document capture a
121
good overview of the design process undergone by the design team. In ME210 A
the emphasis is on scoping the design problem and identifying the specifications
(functional, physical and external) of the design. Teams also list a few promising
concepts.
ME210B emphasizes selecting a final concept for the design and developing
configurations and other details by rapid prototyping and other structured
methods. The report contains documentation on any structured methods used,
choices made and reasons for the choices etc.
ME210C deals with detail design, fabrication and testing of the prototype
design. The performance of the design is evaluated under laboratory conditions.
The teams are encouraged to make recommendation to their sponsors about
ways in which the design can be improved.
The above breakdown is only qualitative and serves as a guideline for the teams,
and is not strictly imposed. Each of the report has extensive supporting material
in the appendix.
Besides the three reports, you will also have access to the personal design
notebooks of the three designers, pictures of the prototype and a video of the
laboratory setup for testing.
The goal of our experiment is to observe the reuse of design information and
subsequently develop tools which would facilitate the reuse.
Information Handling Behavior of Designers During ConceplUaI Design: Three Experiments
122
A.2 Results on individual subj eets
Performance
Relation
Construction
Operation
~"~~""'-~"""""'""
."""""""".,
[=:J
~,~'-,~
Alternative
RS2
RS1
""""
Rationale
Requirement
Location ~
Miscellaneous
""""""""'"
o
Figure A-1
10
20
Proportion of Questions (%)
30
Distribution of question descriptor for RS1 and RS2. a
Requirement k-oo~~'""C""O'-+~""'Il
Assernblyk-oo~~'""C""O~~
[=:J RS2
~
RS1
Design-concept k-oo~~------I
Connection Io.:-...--..J
10
20
30
Figure A-2 Distribution of question subject-class for RS1 and RS2.
Information Handliog Behavior of Designers During Conceprual Design: Three Experiments
40
Conceptual t-c-~"T"'O~~-T-o~~
123
c:=J
RS2
ISS:S]
RS1
Configurational ~~~...,.--"'"
10
20
30
40
50
60
70
Figure A-3 Distribution of question level-of-detail for RS1 and RS2.
A.3 Analysis Examples

Below is an analysis example from about 17 minutes of the protocol data.
51: What have I, I made two comments just now. The lever and the heat. If I do
the lever. then I may get a better force. And if I put the bub. if I separate the
bub and the solenoid. then I may be able to deal with the heat problem. There's
some reference material here on the solenoid. Now I'm looking for effect of
temperature on the solenoid. Okay, there's some graphs here showing the
relationships between the force and the stroke. so part of what I was looking
for, force and stroke. Heat.
R2: what page is it?
51: This is E-2 in the appendix. Heat and current. There's current here. but I'ye
not yet seen them put it together Temperature and force resistance factor.
Okay ...
{a033} {vSO: 4B}
There are two ways to become aware of
force and stroke. The recommendations
sure if I'll have been more sensitive
sure stands out now .. What do you use
these dependencies between the heat and the

has a lot of that detail there. I'm not
to it without the initial questions, but it
a transducer for?
R2: Is the transducer a part of the damper?

51: yeah, but I think you didn't do that.
El: we used it only for testing.
51: for testing, okay ... {aOS3} {reading} Ah. This is your bearing. I think I
read they had some problem with bearings. But I can't remember. but at least it's
good to know this document is there. No problem, it operates in good temperature.
What's the C-card?? Adyanced technology. composite. what do you use it for.
Probably for testing too. Composite materials.
El: Some other frictional material we looked at.
124
51: Ah. Looking for your index.

El: Index starts on like al bl cl.
{v54:22}
51: Okay. Now I'm thinking to myself,
that's another way of um, they had .3
high. If I can go higher. like 5 or
~ I know what they used There's
what was their friction material. Because

coefficient of friction. That's pretty
4. that'll be good Get some force Qut of
carbon, carbon there's asbestos,
R2: what page is that?

51: This is e23. Terrible coefficients of friction. point .14. 5tatic to dynamic
ratio. I don't quite see where we'll be getting all those jerky motions El spoke
~. Oh okay. This funny looking torque curves .. So we want a, yeah. When the
ratio is good, then the torque curve is very beautiful. Oh yes, El's a very smart
guy ..
R2: What are you looking at now?

51: right now I'm still in the appendix. And I'm looking at the different
frictional surfaces that they used. One thing I'd like to establish. was the key
information that I want there really is something that has a high coefficient of
friction and a good dynamic to static friction ratio. I'm not sure how critical
that is, but if I can push up my coefficient of friction, that'll be good. So
what else ..
{aI09} {v58:21}
I'm now looking at the original cost proposal. Just to see if there's something,
I may have missed that ...
{aI25} {v59:49}
Right now I'm just looking at other materials. I don't think they will be ...
{a13?} Okay, there's some test data here. It talks about levers. This is page P56 ...
{a16l} {vl:03:0?}
Ah, good information here. They have something on the potential friction
materials. Which says the carbon fiber will be a good option, but it's very
expensive. It's 135 dol~ars and the kevlars are like 35 bucks ...
{al?5} {vl:04:1?}
Here's some information about the electrorheological fluid .. It also shows ~
examples of mono tube that uses electrorheological fluids. I wonder why it was
eliminated ... Okay. So let's see .. At this point I'm, I know a bit much more
about the problem ... {a205} So maybe I'll call El. Let's talk a little bit .. ~
happens if my leyers had a longer arm?
Table A-1 Example of analysis procedure in information reuse study.

QNo.
After Reformulation
Descr/pror: Relation
Subjclass: Component
b52
I f I do the lever, then I may

get a better force.
I f I use the lever will I get

a better force?
b53
And i f I put the

separate the hub
solenoid, then I
to deal with the
I f I separate the hub and the
Descrlpror.Relation
solenoid, will that impact

positively on the heat problem?
Subjc/ass: Assembly
hub, if I
and the
may be able
heat problem.
Information Handling Behavior of Designers Ouring Conceptual Design: Three ExperimenlS
125
Table A-1 Example of analysis procedure in information reuse study.

QNo.
After Reformulation
bs4
Now I'm looking for effect ?f

temperature on the solenoid.
What is the effect of

temperature on thesolenoid?
Descriptor: RelatiDn
Level ofDetail: Conceptual
bss
Heat and current. There's

current here, but I've not yet
seen them put it together.
What is the relation between

heat and current to the
solenoid?
Descriptor: RelatiDn
Subjclass: Feature
bs6
What do you use a transducer

for?
What do you use a transducer

for?
Subjclass: Componellt
bs7
I think I read they had some

problem with bearings. But I
Did they have a problem with

bearings?
Descriptor: Performance
Subjclass: Cornponertt
bs8
Advanced technology,
composite, what do you use it
for.
What do you use composite for?
bs9
what was their friction

material.
what was their friction

material?
Descriptor: Construction
Subjclass: Feature
Level ofDetoil: Detail
b60
If I can go higher, like .5 or

.4, that'll be good. Get some
force out of that.
If I go higher, like .5 or .4,

will I get some force out of
that?
Subjclass: Feature
Level of Detail: Detail
b61
I don't quite see where we'll

be getting all those jerky
motions El spoke about.
Where will we be getting all

those jerky motions E1 spoke
about?
Descriptor: OperatiDn
Subjclass: Assembly
Level ofDetail: Configtua
b62
I'm looking at the different

frictional surfaces that they
used.
What different frictional

surfaces did they use?
Descriptor: Altemative
Subjclass: component
b63
One thing I'd like to

establish, was the key
information that I want there
really is something that has a
high coefficient of friction
and a good dynamic to static
friction ratio.
Is it very important that the

coeffiCient of friction be
high and dynamic to static
friction ratio be equal?
Descriptor: Requirement
b64
two examples of mono tube that

uses electrorheological
fluids. I wonder why it was
eliminated ..
Why did they eliminate using

electro rheological fluids?
Subjclass: Design-concept
b6s
What happens if my levers had

a longer arm.
What happens if my levers had

a longer arm?
Subjclass: Feature
LevelofDetoil: Detail
can't remember,
ttona!
Appendix 8: Additional Material on Deda! Development
126
Appendix B
Dedal Development
B.1 Dedal Implementation Details

B.2 Dedal Interfaces for Indexing, Querying and Modeling
B.3 Examples of Heuristics
B.4 References on Dedal Papers
B.l Dedal Implementation Details

Dedal runs in the Unix environment under Sun as 4.1.3
Framemaker is the information capture environment
It is mostly written in Common Lisp
Heuristics are represented in MRS (Meta Reasoning System). MRS is also is
used to do the reasoning and backward chaining during retrieval
Interface to audio and video players is written in C
Dedal has also been ported to the Macintosh but has not been tested yet.
A web based implementation of Dedal is under development. With this, the
tasks of indexing, modeling and retrieval will be available over the world
wide web.
Information Handling Behavior of Designers During Conceptual Design: Three ExpcrimcDlS
Appendix B: Additional Material on Dedal Development
127
B.2
Dedal Interfaces for Indexing, Querying and
Modeling
All the interfaces shown in figures below are from the Unix implementation of
DedaI.
Figure 8-1 Oedal's indexing and querying environment. The same interface is
used to indexing (this adds indices to the database) and to query (this searches
the indices database).This interface allows users to perform concept based
indexing, the scheme resulting from the information reuse study.
Figure 8-2 Dedal's modeling interface. Users can relate concepts in a model by
using this interface. New concepts can either be created here or during indexing.
Information Handling Behavior of Ocsigners During Conceptual Ocsign: Three Experiments
128
Appendix 8: Additional Materia! on Deda! Development
129
Figure 8-3 Dedal's answers window. Answers to a query are returned in a

window shown above. The name of the document is linked to the document.
Users can give feedback on whether a answer is relevant or not.
B.3 Examples of Heuristics

Here are some examples of heuristics encoded as rules in a lisp like syntax. A
symbol preceeded by a $ sign stands for a variable. $q is the variable for the
descriptor. The : if section of the rule checks for some conditions. If these conditions are true then it generates a new-question represented in the : then
part of the rule. The text following rule is the name of the rule.
(rule look-isa
Information Handling Behavior ot Designers During Conccptlllll Design: Thn:c Experiments
Appendix 8: Additional Material on Dedal Development
:if (and
(question $q)
(object $0)
(a-kind-of $0 $p)
(unprovable(is-generic-term $p)
(new-que.tion $q $p nil look-isa
(rule look-for-specializations
:i (and
(question $q)
(object $0)
(a-kind-of $p $0)
(choice $dec $p
: then
(new-que.tion $q $p nil look-for-specializations

(rule look-isa-for-attribute
:i (and
(question $q)
(attribute $s $0)
(isa $0 $p)
(unprovable (is-generic-term $p)
: then
(new-question $q $p $s look-isa-for-attribute
(rule 100k-for-dependent-attributes-1
:i (and
(question $q)
(or (question relation) (question rationale
(attribute $s $0)
(dependent-on ($s of $0) ($sl of $01)
: then
(new-que.tion $q $01 $sl 100k-for-dependent-attributes-1

(rule from-attribute-to-component-operation :weight 2
:i (and
(question relation)
(attribute $s $0
: then
(new-que.tion operation $0 nil from-attribute-to-component-operation

(rule from-attribute-to-component :weight -5
:if (and
(question $q)
(attribute $s $0
: then
(new-que.tion $q $0 nil from-attribute-to-component

(rule look-for-common-computed-from
:if (and
(question $q)
(attribute $s $0)
(attribute $s2 $02)
(attribute $s3 $03)
(notsame $s $s2)
(is-computed-from ($sl of $01) ($s of $0
(is-computed-from ($sl of $01) ($s2 of $02
(is-computed-from ($sl of $01) ($s2 of $02)
: then
(new-que.tion $q $01 $sl look-for-common-computed-from

(rule look-for-common-computed-from
:if (and
(question $q)
(attribute $s $0)
(attribute $s2 $02)
130
(notsame $s $s2)
(is-computed-from ($sl of $01) ($s of $0
(is-computed-from ($sl of $01) ($s2 of $02)
: then
(new-qae.t!oD $q $01 $sl look-for-common-computed-from

(rale look-relation-<
:if (and
(question $q)
(attribute $s $0)
(is-computed-from ($sl of $01) ($s of $0)
: then
(Dew-qae.t!oD $q $01 $sl look-relation-<

(rale look-related-attribute
: i f (and
(question $q)
(attribute $s $0)
(isa $s $p
: then
(new-qae.tion $q $0 $p look-related-attribute
(rale look-related-value
:if (and
(question $q)
(attribute $s $0)
(val ($s of $0) $v)
(is-symbol $v
: then
(new-qae.tion $q $v nil look-related-value

(rale look-parent-of-related-value
: i f (and
(question $q)
(attribute $s $0)
(val ($s of $0) $v)
(is-symbol $v)
(isa $v $p
: then
(new-qae.tion $q $p nil look-parent-of-related-value

(rale part-of
:if (and
(question location)
(object $0)
(subpart $p $0
: then
(new-qae.t!on description $p nil part-of configuration schematic

(rale part-of-2
: i f (and
(question location)
(object $0)
(subpart $p $0
: then
(new-qae.t!on construction $p nil part-of configuration schematic

(rale find-function-component
:if (and
(question function)
(object $0)
(subpart $p $0
: then
(new-qae.t!on operation $p nil find-function-component

(rale find-function-connectorl
:if (and
(question function)
Information Handling Behavior of Designers During ConcepruaJ Design: Three ExperimenlS
131
Appendix B: Additional Material on Deda/ Development
(object $0)
(connects $0 $p1 $p2)
: then
(aew-qge.tion construction $p1 nil find-function-connector1)

(rule find-function-connector2
:if (and
(question function)
(object $0)
: then
(new-qge.tion construction $p2 nil find-function-connector2)
:if (and
(question function)
(object $0)
: then
(aew-qge.tion operation $p1 nil find-function-connector3))

:if (and
(question function)
(object $0)
(connects $0 $p1 $p2))
: then
(new-qge.tion operation $p2 nil find-function-connector4))
(rule basic-part-of :weight-1
:if (and
(or (question description)
(question operation)
(question construction)
(question analysis)
(question performance)
(question $q)
(object $0)
(part-of $p $0)
: then
(new-qge.tion $q $p nil extended-part-of ))
(rule find-alternatives
:if (and
(question alternative)
(object $0)
(alternative $d $0)
(alternative $d Salt)
(notsame $0 $alt)
: then
(aew-qge.tion alternative Salt nil find-alternatives)
(rule find-alternatives-from-decision-point
: i f (and
(object $0)
(alternative $0 $alt))
: then
(new-qge.tion alternative Salt nil find-alt-from-decision-point)
(rule find-alternatives-for-attribute-value
:if (and
(attribute $att $0)
(decision $d $0 $att)
(alternative $d $alt))
: then
(new-qge.tion alternative Salt nil find-alt-from-attribute-value)
Information Handling Behavior of Designers During ConcepltlaI Design: Three Experimenls
132
133
(rule describe-connectors-l
:if (and
(question description I
(object $01
(connects $0 $pl $p211
: then
(new-qa tioD description $pl nil describe-connectors-lll
(rul. describe-connectors-2
:if (and
(object $01
(connects $0 $pl $p211
: then
(Dew-qa tioD description $p2 nil describe-connectors-211
(rule from-attribute-to-requirement-l
:if (and
(attribute $s $01
(r-relate ($s of $01 $req $ type I I
:th.D
(Dew-qa tioD requirement $0 $s from-attribute-to-reqmt-lll
(rul. from-attribute-to-requirement-2
:if (and
(question descriptionl
(attribute $s $01
(r-relate ($s of $01 $req $ type I I
: then
(new-qa tion requirement $req nil from-attribute-to-reqmt-211
B.4 References on Dedal Papers

1. Baudin, Catherine; Underwood, Jody G. and Baya, Vmod, "Using Device
Models to Facilitate the Retrieval of Multimedia Design Information", In
Proceedings of the 13th International Joint Conference on Artificial Intelligence,

Chambery, France, pp 1237-1243, August 29-September 2,1993a.
2. Baudin, Catherine; Kedar, Smadar; Underwood, Jody G. and Baya, Vmod;
"Question-based Acquisition of Conceptual Indices for Multimedia Design
Documentation", In Proceedings of the 11th National Conference on Artificial
Intelligence AAAI-93, Washington D.C., pp 452-458, July 1993b.
3. Baudin, Catherine; Underwood, Jody G.; Baya, Vmod and Mabogunje,
Ade, "Using Domain Concepts to Index Engineering Design Information",
In Proceedings of the meeting of the Cognitive Science Society, Bloomington,
Indiana, 1992.
Appendix C: Additional Material on Design Information Handling Study
134
Appendix C
Design Information
Handling Study
C.1 Instructions and Overview

C.2 Design Exercise Problem Statement: Bike Lock 1
C.3 Design Exercise Problem Statement: Bike Lock 2
C.4 Design Exercise Problem Statement: Backpack Harness
Information Handling Behavior of Designers Ouring Conceptual Design: Three Experiments
135
Center for Design Research

Stanford University
Design Experiment
Date: May 26, 1993
e.l
Instructions: Overview
Dear Subject:
Thank you for willing to participate in the design experience you are now
starting. The whole experience is anticipated to take about 1.5 hours. The
experience will be recorded on audio and video mediums.
I am interested in the manner in which you would be handling information
during your design exercise. To get an explicit recording of your activities with
information, I request you to talk out loud while you are doing the design. If
there are noticeable periods of silence, I would very likely tap you on your
shoulder to remind you to talk out loud.
We will be following the procedure below.
1. Warm-up Exercise: 10 min, This is a short exercise to give you an experi-
ence in talking out loud.

2. Design Exercise: 1 hour, This is the meat of the experience.
3. Debriefing: 10 min, Answering a small questionnaire.
Some stationery is provided for your use. Feel free to ask me for any
clarifications or information during the course of the experience.
Thanking you,
Vinod Baya
Appendbc C: Additional Material on Design Information Handling Study
136

Stanford University
Design Experiment
Date: May 26, 1993
C-2 Design Exercise: Problem Statement Bike Lock 1

Background: As you might have experienced, an everyday activity of locking/
unlocking a bicycle with a kryptonite lock is quite an arduous task. It involves
seven steps (believe me I have counted) and takes a fairly long time (It takes me
on an average 15 seconds to lock or unlock my bike). Imagine it taking you so
long to lock/unlock your car! Wouldn't it be nice if there was a more userfriendly bike lock.
Problell1: Design a user-friendly locking mechanism for a bicycle which meets

the following requirements.
1. The locking mechanism should be an off the shelf item which can be
installed on a bicycle without much difficulty.

2. Once installed on a bike the mechanism should become an integral part of
the bike.
3.
It should be possible to perform the locking/unlocking operations quickly

and in few steps.
4. 'the lock should have the strength equal to or better than that of a krypto-
r\ite lock.
5. 'the locking mechanism should have a reasonable cost.
6. 'the locking mechanism should be safe to use.
The following assumptions can apply to the design:

1. A. bike is considered locked if either the front or the rear wheel is not free to
rotate.
2. 'the lock can be mounted anywhere on the frame of bike.
3. Use the bike provided as the sample of what the lock could be mounted on.
4. '!'he U lock provided can be assumed to be a representative of a kryptonite
lock.
137
5. Design of an appropriate lock/key mechanism can be assumed to be available.

6. While it is sufficient if the bike can be locked in isolation, an ability to lock
the bike to a fixed external frame (shown below) would be an added feature of the lock. If bike is locked to a fixed external frame, then the wheels
can be free to rotate.
7. One possible solution that is widely used in India is sketched in an accompanying sheet. This is meant only to explain the requirements.
A detail quantitative design solution is not expected. An adequate solution

would qualitatively describe the configuration and the operation of the
mechanism.
3.5 feet
2.5 feet
Fixed external frame to which the bike should be locked
138
unlocked
position ---II~ /
locked ~(
position
locking
mechanism
rearfork
rearwheel
Sample Solution of a Bike Lock which is

installed on a Bike, commonly used in India
139

Stanford University
Design Experiment
Date: March 17, 1994
C.3 Design Exercise: Problem Statement Bike Lock 2

Background: As you might have experienced, an everyday activity of locking/
unlocking a bicycle with a kryptonite lock is quite an arduous task. It involves
seven steps (believe me I have counted) and takes a fairly long time (It takes me
on an average 15 seconds to lock or unlock my bike). Imagine it taking you so
long to lock/unlock your car! Wouldn't it be nice if there was a more userfriendly bike lock.
Problem: Design a user-friendly locking mechanism for a bicycle which meets

the following requirements.
1. The locking mechanism should be an off the shelf item which can be
installed on a bicycle without much difficulty.

2. Once installed, the mechanism should become an integral part of the bike.
3. It should be possible to lock the bike in isolation as well as to a stationary
frame. The shape of a typical frame is at the end of this statement. Being
able to lock to just this frame would be sufficient.
4. It should not be possible to detach the locking mechaniSm from the bike
when it is in the locked position (otherwise it wouldn't be a lock would it?)
5. It should be possible to perform the locking/unlocking operations quickly
and in fewer steps.
6. Strength of the lock should be better than or equal to that of a kryptonite
lock.
7. The mechanism should have a reasonable cost and be safe to use.
The following assumptions can apply to the design:

1. A bike is considered locked if either the front or the rear wheel is not free to
rotate, or if the frame of the bike is not free to move.
2. The lock can be mounted anywhere on the frame of bike.

3. Use the bike provided as the sample of what the lock could be mounted on.
Information Handling Behavior of Designers During Conceptual Design: Three Experir!1C'nls
140
4. The U lock provided can be assumed to be a representative of a kryptonite

lock.
5. Design of an appropriate lock/key mechanism can be assumed to be available.
6. One possible solution that is widely used in India is sketched in an accompanying sheet. This is meant only to explain the requirements. Note that
this solution only locks the bike in isolation.
A detail quantitative design solution is not expected. An adequate solution

would qualitatively describe the configuration and the operation of the
mechanism.
3.5 feet
2.5 feet
Fixed external frame to which the bike should be locked
The sketch for possible solution is same as the one used in problem statement for
bike lock 1 (page C-S).
Informnlion Handling Behavior of Designers During Conceptual Design: Three Experiments
141
A n a i y sin g Des i g n Act i v i t Y - The Delft Protocols Workshop

Instructions: Protocol Study Individual Design
Thank you for agreeing to participate in this research project. We are interested in the ways that
designers work on conceptual design projects. This session is being videotaped for later analysis.
I am going to ask you to undertake a short design project. Let me assure you that no commercial
advantage will be taken of any design work or ideas that you produce.
Because we want to understand what you are thinking during the task, we want you to think
aloud continually during the session. This would be as if you are talking to yourself but loud
enough for me to hear. In order for you to get used to thinking aloud, I will give you a small
problem which has nothing to do with the design task. Try to solve this problem while thinking
aloud. This is only a training exercise, which last 10 minutes: so please do not worry if you do
not solve the problem. Here is the problem I want you to work on.
Three missionaries and three cannibals are together on one side of a river. They have one rowing-boat,
which can hold up to two people. They all know how to row.
How can they all reach the other side of the river, given that for obvious reasons there must never be more
cannibals than missionaries on either river-bank?
I am now going to ask you to undertake the design project. The time available is limited to two
hours. In a moment I will give you a short written design brief for the project. Additional
information which you may require during the project is available in an information file which I
keep and have ac.:ess to. So, if you think that you need additional information during the project,
please ask me for what you need to know. Please be specific about what you ask for. The
information available in the file includes both technical and client information. Although access
to the file is only available through me, please do not feel constrained about asking for any
information that you feel you need during the project. I also have an outside helper who can give
me additional information if it is not available in the file. This book TITLE is also available if you
need it please ask me for it when you want it and say why you want it. Apart from providing
you with information when you ask for it I will not interact with you in any other way. You
should try to ignore my presence in the room.
Drawing materials - paper, pens and markers are available for your use. There is also a white
board which you may use if you wish. The particular design project which we are asking you to
undertake is concerned with a new product related to mountain bikes and backpacking.
Therefore we also have in the room a mOlIDtain bike and a backpack, which you may use or refer
to as you wish. Please work as you normally would on such a design assignment as this. But
remember to keep thinking aloud.
Before I give you the design brief, are there any questions about the procedure? If there are no
further questions we will start the session now. I remind you that the maximum time available to
you is two hours. I will remind you of the time at 30 minutes and 15 minutes before the end of
the session, if you are still working.
(After the designer has read the assignment the experimenter says:)
The bicycle here in in the room is not the same as the Batavus Buster.
142
C.4 Design Expt. Problem Statement: Backpack Harness

Assignment
HiAdventure Inc. is a fairly large US firm (some 2000 employees) making backpacks and other
hiking gear. They have been very successful over the last ten years, and are well known
nationwide for making some of the best external-frame backpacks around. Their best selling
backpack, the midrange Hi Star, is also sold in Europe. In the last one and a half years, this
European activity has suffered some setbacks in the market; in Europe internal-frame backpacks
are gaining a larger and larger market share.
As a response, HiAdventure has hired a marketin-firin to look for new trends and opportunities
for the European markeL On the basis of this marketing report, fiiAdventure has decided to
develop an accessory for the HiStar:
A special carrying/fastening device that would enable you to fasten and carry the
backpack on mountain bikes.
The device would have to fit on most touring- and mountain bikes, and should fold
down, or at any rate be stacked away easily.
A quick survey has shown that there is nothing like this on the european market.
This idea is particularly interesting for HiAdventure, because the director, Mr. Christiansen, has
a long-standing private association with one of the chief product managers at the Batavus bicycle
company (one of the larger bicycle manufacturers in northern Europe, based in Holland). Mr.
Christiansen sees this as a good opportunity to strike up a cooperation and profit from the
European marketing capacities of Batavus.
The Batavus product manager, Mr. Lemmens, is very enthusiastic about putting a combinationproduct on the market, a mountain bike and a backpack that can be fastened to it. The idea is to
base the combination-product on the Batavus Buster (a midrange rtlountain bike), and to sell it
under the name Batavus HikeStar. The design department at Batavus has made a preliminary
design for the carrying I fastening device, but both Mr. Christiansen and Mr. Lemmens are not
very content with it. The user's test performed on a prototype also showed some serious
shortcomings.
That is why they have hired you as a consultant to make a different proposal. Tomorrow there is
going to be a meeting between Mr. Christiansen and Mr. Lemmens, scheduled as the last one
before presenting the idea to the board of Bataeus. Before then, they need to have a clearer idea
of the kind of product it is going to be, its feasibility and price.
You are hired by HiAdventure to make a concept design for the device, determining the layout
of the product, concentrating on
- ease of use
- a sporty, appealing form
- demonstrate the technical feasibility of the design
- make sure to stay within a reasonable price range
You are asked to produce annotated sketches explaining your concept design. Good Luck.
Information Handling Behavior of Designers During Conceptual Design: 11IRC ExperirncDls
Appendix 0: Additional Results on Information Handling Study
143
Appendix D
Additional Results on
Information Handling Study
This appendix has results on the information handling study, in addition to
those discussed in Chapter 6. Some supporting data is included as well. In many
cases graphs for each of the exp-subjects are shown to illustrate the nature of the
raw data across all the exp-subjects and for comparison purposes. However, in
cases where the characteristics of the raw data is similar, data for only one exp-
subject is shown because of space considerations. Results included are on:

information fragment duration (deltat)
information handling rate (dpm)
information fragment attributes behavior over time
distribution of time for each attribute
state transition probabilities
multiple attribute analysis
0.1 Additional Results on deltat

0.1.1 deltat vs. time
Figure 0-1 to Figure 0-6 show the variation of information fragment duration
(deltat) over time for all exp-subjects. The curve in each of the figures is a polynomial fit to the raw data. It shows the movement of the exp-subjects between small
and large information fragments at a level higher than shown by the raw data.
This curve for all subjects on a normalized time scale is shown in Figure 0-7.
Information Handling Behavior of Designers During Conceptual Design: 1luec Experiments
Appendix D: Additional Results on Information Handling Study
144
I II
"C
C
a(J
Q)
1a
....
:J
20
cQ)
\ ~"
I.....
E
C)
'/V'
LL
a
:;:;
as
E
....
a
:-.~
r-.
"y.
:U
"
,pi
r-..
010
Figure D-1
90
70
. 50_ .
Time (minutes)
30
Behavior of deltat over time for 81
I II
"C
C
a
(J
Q)
a
Cii
:520
c
cQ)
Cl
e
u.
c
a
Cii
E
....
a
rr
f'-1
)
11
'J
~ V
~~
!\
N
i'.,
..
35
Time (minutes)
55
Figure D-2 Behavior of deltat over time for 82.
Information Handling Behavior of Designers During Cooceprual Desigo: 1luec Experiments
75
145
40.-----------------------------------------------------,
( J)
"C
c:
~30
~
c:
e
:J
c
E20
CD
C)
e!
u.
c:
o
iii 10
:s
5~~--~------~25~----~----~45~----~------~6~5------~~
Time (minutes)
Figure 0-3 Behavior of deltat over time for 83.
(i)30
"tJ
c:
0
CD
(J)
c:
0
:;20
c
c:
CD
C)
CIS
....
u.
c:
0210
iii
E
....
1:
010
~ 1'\
1\
~~
L../1
r"30
~\
50
Time (minutes)
1\//
:~~
\
70
Figure 0-4 Behavior of de/tat over time for 84.
Information Handling Behavior of Designers During ConccplUai Design: Tbrcc Expcriments
146
40
i)
.J
30
2
ii
5
)
J
20
:nIS
-....,
L.
~ 10
2
0~------~------~2~0--------~------~4~0--------~------~60
Time (minutes)
Figure 0-5 Behavior of de/tat over time for S5.
en
"C
30
c:
Q)
i\
(J)
c:
:e
:::J
Cl
'E
Q)
E
C)
~
u.
c:
0
:a;
E
....
:s
20
10
1
I"-
~,
.~l.~
~.
"
I
.
}
65Ti(me (minutes
115
Figure 0-6 Behavior of de/tat over time for S6.
Information Handling Behavior of Designers During Conceptual Design: Three Experimcn~
147
..... ~ .
S5 /'
/'
l'
______
_~_-
,.-~;
---....... S2
______ /
I,
/----
I
I
\...,/
'J
S1
0.6
0.8
Normalized Time
Figure D-7 de/tat behavior over time for all exp-subjects. The
curves
are
polynomial fit to the raw data. The time on the x-axis is normalized to neutralize
the effects of varying design times. The oscillations are more pronounced in some
cases than others but clearly de/tat behavior is oscillatory, i.e. it is neither steadily
increasing or decreasing. As one moved along the time axis, three regions stand
out. Region 1 (O to 0.2): this is the region where designers developed an
understanding of the design problem and a strategy to solving it. Region 2 (0.2 to
0.9): this is where the bulk of the problem solving occurred. Region 3 (0.9 to 1.0):
this region started soon after exp-subjects were informed of the remaining time
and it involved wrapping up the design and describing a solution.
Information Handling Behavior of Designers During ConceplWll Design: Three ExpcrimcnlS
1.0
148
0.1.2 Distribution of deltat: histograms

20
20
81
15
10
82
15
10
10
20
~ -llmllJL
TTTn!
rm
30
10
20
~
c: 15
15
E
C)
as
....
83
E
Q)
10
c:
as
10
~
E
....
c:
.Q
30
84
Q)
II)
E
C)
as
....
20
a
'E
a
E
....
-....
a
c:
Q)
'0
....
Q)
10
20
.c
E
:::s
30
.c
E
:::s
15
15
85
10
10
o~wwwwww~uuuwww~~~ww~~~ww~
10
20
30
10
86
il ~~
~.rrufl
Ilnllflll
20
deltat (seconds)
Figure D-8 Distribution of the information fragment duration. The curves are polynomial
fit to the histograms. Histograms are drawn using a bin width of 0.5.
Infonnation Handling Behavior of Designers During Conceptual Design: Three Experiments
nn
111I1Im-tt
30
149
0.2 Additional Results on Information Handling Rate

0.2.1 Information handling rate over time
Figure 0-9 to Figure 0-14 show the variation of information handling rate (dpm)
over time for all exp-subjects. The curve in each of the figures is a polynomial fit
to the raw data. It shows the movements between small and large information
fragments at a level higher than shown by the raw data.
40~--------------------------------------------------~
E
a.
"C
's20
as
II:
r\
--
V--'-~-<1"Ir
~U i~
~~O----~-----'3~O----~-----5~IQ~----~----~7~O----~----~90
Time (minutes)
Figure 0-9 Information handling rate over time for 51
Infonnation Handling Behavior of Designers During Concepwal Design: 1'It= ExperimenlS
150
40
Ea.
"C
. !20
ta
II:
"\ .
",
"
I~
~
25
""
/
~
"....
r'
~~
'A
.45
TIme (minutes)
I~.
1A rl
.~
65
Figure 0-10 Information handling rate over time for 52
40r-------,-----------------~----------------------------_,
E
a.
"C
.!20
ta
II:
I~
o~------~----~~------~------~------~------~~----~--~
Time (minutes)
Figure 0-11
Information handling rate over time for 53
Infonnation Handling Behavior of Designers During Conccpwal Design: Three Experiments
151
40
Ec..
-ca
"0
Q)
20
a:
'f
(5
~4~'~o,l
V~
oI
2U
., '>,
1",
I/
r-
V-
r. "'\
('-
~.
ij
60
4U
Time (minutes)
80
Figure D-12 Information handling rate over time for 54.
40~------------------------------------~------------------~
ECo
"0
ok---------~-------~20--------~------~4nrU--------~-------~tO
Time (minutes)
Figure D-13 Information handling rate over time for 55
Information Handling Behavior of Designers During Conc:eplWll Design: Three Experiments
152
40r-------------------~------------------------r_~------_,
Ea.
1:1
' s20
as
a:
C5
Figure 0-14 Information handling rate over time for 56.
0.2.2 Distribution of information handling rate

15
-51
-52
-53
54
. 55
-56
en
E
Q)
E
C)
...
as
u.. 10
c:
.Q
Cii
E
...
0
:s
"0 5
...
Q)
.c
E
:::J
Dim Rate (dpm)
Figure 0-15 Distribution of dim rate for all exp-subjects. The

curves
are
polynoml~1 fit to histograms now shown here. The average most frequent
information hdndling rate (the average of the maxima point of each curve) is 6.1
dpm.
Infomuuion Handling Behavior of Designers During Conceprual Design: Three ExperimenlS
153
0.2.3 Dim rate vs. Verbal rate: Regression analysis

28
~24
Ec..
~20
CD
16
a: 16
(ij
~S1
-e
-S2
b:--AS3
~ 12
"'--"54
+- +S5
-S6
Ok---~~--~~--------~~------~~--~----~----~~
dim rate (dims/min)
Figure 0-16 Regression analysis between dim rate and verbal rate. Only a few
data points are shown for clarity sake.The lines are linear regression fit to the raw
data. Except for 53 the coefficient of correlation is within a narrow range
D.3 Information Fragment Attributes Behavior Over Time

Figure 0-17, Figure 0-18, Figure 0-19 and Figure 0-20 show the behavior of
informational activity, information descriptor, information subject-class and
information level-of-abstraction over time respectively.
154
CD
,...o
.....
rn
CD
0_
III
~
CD
E
oi=
...
t')
""
CI)
~
"iii
c:
<
rJ)
rJ)
CI)
CJ
CJ
<
CI)
1ii
"CI)
c:
CI)
(!)
Figure 0-17 Behavior of informational activity over time for 81. The
figure
illustrates that designers move frequently between the different informational
activities all along the conceptual design process. No predictable pattern is
obvious. The behavior for other exp-subjects is similar in characteristics.
Information Handling Behavior of Designers During Conc:epruw Design: Three Experiments
155
- g
-~
156
::
"
..,0
(i)
Q)
:l
c:
g
Q)
E
i=
g
....
Q)
~
a.
o
c:
()
Q)
o
I
c:
C)
en
Q)
Q)
0Q)
ex:
c:
.2
'0
Q)
c:
c:
o
U
cQ):
c:
o
a.
E
o
>.
:c
E
Q)
en
en
Cl
Figure 0-19 Behavior of Information subject-class over time 53. The

figure
illustrates that designers frequently move between the different information
subject classes all along the conceptual design process. No predictable pattern is
obvious. The behavior for other exp-subjects is similar in characteristics.
Information Handling Behavior of Designers During Concepnml Design: Three Experiments
157
r-----------------------------~----------~~=======r--------_.~
,..o
tn
::
I
;=:
:
Ul
(J)
'S
c
I
...
0
t')
C\I
5!
(J)
>
E
:;=
tU
:l
(J)
>
E
(ij
:l
(J)
"0
"0
>
~
.0
(\l
.c
(\l
--l
Ul
Ul
(J)
CD
(J)
(J)
:::>
Figure D-20 Behavior ot Intormation level-ot-abstraction over time 54. The figure
illustrates that designers move between the different levels ot abstraction often all
along the conceptual design process. It was also observed that concepts start at
the unlabeled level and mature to other levels as they become more concrete. No
predictable pattern is obvious.
(J)
i=
158
D.4 Additional Results on Distribution of Time

G--E>S1
G-EJS2
1!r--6.S3
Analyze
x---~S4
+-+55
~S6
Access
Generate
20
40
60
Proportion of Total Design Time (%)
Figure 0-21 Distribution of time across informational activity. The

distribution
across the different exp-subjects is quite consistent. For all exp-subjects, more
than half the time is spent generating information.
Miscellaneous
G--E>S1
G-EJS2
1!r---6.S3
X_J( S4
Requirement
Relation
+- +55
~S6
Rationale
Performance
---::::::
Operation
Location
Construction
Comparison
Assumption
Altemative
20
Proportion of Total Design Time (%)
Figure 0-22 Distribution of time across information descriptor. The distribution

is quite comparable across the exp-subjects. For all exp-subjects, information
regarding I construction, operation, alternative and requirement dominate.
40
159
Other
G---eSl
G-lS2
/!r--AS3
Design-Concept
X---i<S4
+-
Requirement
+S5
~S6
Feature
Connection
Component
...
- ... -:~---::-:-:---:-:---~
Assembly
-. :=1'"-)(
40
20
Proportion of Total Design TIme (%)
Figure 0-23 Distribution of time across information subject-class. The

figure
shows that the distribution is reasonably consistent for most classes across the
exp-subjects. It differs for classes connection and requirement.
Quantitative
G---e51
G-l52
/!r--A53
X---K 54
+ +55
Qualitative
~56
Associative
Labeled
Unlabeled
20
40
Proportion of Total Design TIme (%)
Figure 0-24 Distribution of time across information level-af-abstraction. The

distribution differs across the exp-subjects.The differences are well distributed
across all levels of abstraction.
Information Handling Behavior of Designers During Conceptual Design: Thn:c Experiments
160
D.5 Results on State Transition Probabilities

State transition probabilities inform. us about the likelihood of a transition from
one state to another. These probabilities are calculated under the assumption
that the next state is determined by only the current state and not any states
prior to it (i.e. the memory of anything but one previous state is erased. This
assumption is commonly made in developing Markov chain for a process with
discrete states. This assumption is being made in obtaining the results below.
0.5.1 Informational activities state transition probabilities
22
Figure D-25 Information activity state transition probabilities. The figure shows
that from generate, transition to analyze is most likely, and from access or analyze
transition to generate is most likely.
D.5.2 Information descriptor state transition probabilities
Figure 0-26 information descriptor state transition probabilities. For clarity sake
only the transitions with probability greater than 0.19 are shown here. From most
states transition to construction is most likely. From construction, transition to
alternative or operation is most likely.
Information Handling Behavior of Designers During Conceprual Design: Three ExperiDll:ms
161
Appendix 0: Additional Resufts on Information Handling Study
162
0.5.3 Information subject-class state transition probabilities
Figure 0-27 Information subject class state transition probabilities. For

clarity
sake only transition with probability greater than 0.2 are shown here. Transition to
assembly are most likely from most other states.
0.5.4 Information level-of-abstraction state transition probabilities

Shown in Figure 0-28.
D.6 Additional Results from Multiple Attribute Analysis

Table 0-4 to Table 0-6 show various combinations resulting from multiple
attribute analysis. In all the tables, significant combinations are highlighted. The
numbers in the table are the fraction (in percentage) of information fragments in
the particular combination to the total number of information fragments.
163
Figure 0-28 Information level-of-abstraction state transition probabilities.

Transitions with probabilities greater than 0.25 are shown here. No particular
transition particularly stands out here.
D.6.1 Information descriptor and informational activities

Table 0-1 Distribution of descriptor across the informational activities.
The numbers below represents the proportion of informational activities in each of
the descriptors. Much of the information regarding the descriptors is generated. In
case of performance most of the information is analyzed and most of the
requirements are accessed.
Descriptor
Access
Alternative
12
Assumption
17
Comparison
11
20
Construction
28
27
Location
32
II
Operation
21
Perfonnance
30
Rationale
24
_. --. ,.'. - ,. - 1
: ____ X, _.
Relation
16
Requirement
25
Miscellaneous
12
~J
164
0.6.2 Information subject-class and informational activities

Table 0-2 Distribution of subject-cass across the informational activities. The
numbers below represent the proportion of informational activities in each of the
subject classes. For all classes, much of the concentration is in the activity
generate. However, a significant number of info-fragments in assembly were
accessed and a good number of info-fragments in requirement were accessed and
analyzed.
Analyze
Subject Class
Assembly
23
Component
18
Connection
20
Feature
Requirement
Design concept
15
Other
15
0.6.3 Information level-of-abstraction and informational activities

Table 0-3 Distribution of level-of-abstraction across informational activities. The
numbers below represent the proportion of informational activities in each of the
level of abstraction. For all levels, large share of information is generated and
nearly equal amount of quantitative information fragments were in each of the
informational activities.
Level of Abstraction
Access
Analyze
Unlabeled
24
20
Labeled
29
23
Associative
16
20
Qualitative
Quantitative
Infnrmation Handling Behavior of Designers During ConcepwaI Design: Three Experiments
165
0.6.4 Information descriptor and subject-class

Table 0-4 Distribution of descriptor across the subject-classes. The
numbers
below
represent the proportion of subject-classes in each of the information descriptor. Most
alternatives were about design-concepts. Most assumptions were about assembly, feature
and requirements. Most performance were about requirements. Most relations were about
components. Most requirements were about assembly.
Descriptor
Other
Comp.
Conn.
Alternative
16
Assumption
Assem.
Feature
Reqmt.
Comparison
14
Construction
Location
Operation
-' ...
2
Performance
11
. ." ,
Rationale
17
'.
- -
'
. .
'-
~"I
Relation
19
Requirement
11
15
13
Miscellaneous
10
.-
166
0.6.5 Information descriptor and level-of-abstraction

Table 0-5 Distribution of descriptor across the levels-of-abstraction. The
numbers
below
represent the proportion of level-of-abstractions in each of the information descriptor. most
alternatives were unlabeled or labeled. Most assumptions, comparison, performance were
labeled. most operations were qualitative.
Descriptor
Associative
Qualitative
Quantitative
Alternative
14
Assumption
18
13
Comparison
15
20
Construction
12
16
Location
Operation
14
Performance
Rationale
13
Relation
20
Requirement
Miscellaneous
0.6.6 Information subject-class and level-of-abstraction

Table 0-6 Distribution of subject-class across the levels-of-abstraction. The numbers below
represent the proportion of level-of-abstractions in each of the information subject-class. Most
assembly information is labeled. Most components and connections are associated. Most
requirements are labeled and most design-concepts were unlabeled.
Subject Class
Quantitative
Associative
Assembly
Component
Connection
18
Feature
Requirement
Design concept
IO
13
Other
Information Handling Behavior of Designers During ConceplUlll Design: Three Experiments
Appendix IE: References on Papers from this Dissertation
167
Appendix E
References on Papers from
this Dissertation
E.l ASME Conference: DTM-1992
Baya, Vinod; Gevins, Jody; Baudin, Catherine; Mabogunje, Ade; Toye,
George and Leifer, Leifer; "An Experimental Study of Design Information
Reuse", In Proceedings of the 4th International Conference on Design Theory and
Methodology, ASME, Scottsdale, Arizona, pp. 141-147, Sept. 13-16, 1992.
E.2 ASME Conference: DTM-1994

Baya, Vinod and Leifer, Larry J.; "A Study of the Information Handling
Behavior of Designers During Conceptual Design", In Proceedings of the 6th
International Conference on Design Theory and Methodology, ASME, Minneapolis, Minnesota, pp. 153-160, Sept. 11-14,1994.
E.3 ASME Conference: DTM-1995

Baya, Vmod and Leifer, Larry J.; "Understanding Design Information Handling Behavior Using TIme and Information Measure", In Proceedings of the
7th International Conference on Design Theory and Methodology, ASME, Boston, Massachusetts, Sept. 1995.
E.4 Chapter in Analysing Design Activity: 1996

Baya, Vmod and Leifer, Larry J., "Understanding Information Management
in Conceptual Design", Chapter in Analysing Design Activity, Cross, N.;
Christiaans, H. and Dorst, K. (Eds.), John Wiley & Sons, pp. 151-168, 1996.
Information Handling Bebavior of Desigoers During Conceptual Design: Three Experiments

Baya, ConceptDesign

Enviado por

Dados do documento

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Baya, ConceptDesign

Enviado por

Direitos autorais:

Formatos disponíveis

INFORMATION TO USERS

The quality of this reproduction is dependent upon the quality of the

manuscript and there are missing pages, these will be noted.

unauthorized copyright material had to be removed, a note will indicate

INFORMATION HANDLING BEHAVIOR OF DESIGNERS DURING

UMI Number: 9714076

UMI Microform 9714076

300 North Zeeb Road

Copyright by Vinod Baya 1996

Approved for the University Committee on Graduate Studies:

1.1.1 Information Handling in Design ..... '" ...................... 2

1.2.1 Grounding Research in Reality ............................... 6

1.3.1 Information Handling and Design Information Framework ..... 10

1.5 Key Results ................................................... 16

1.5.1 Key Results from Information Reuse Study ................... 16

Chapter 2. Design Theory & Methodology Research: Literature Revie'W'................................................ 20

Prescriptive Studies ........................................ 21

2.2 Design Process: An Information Handling Perspective ............. 32

Chapter 3. Design Information Needs, Capture & Reuse ....... 35

Information Reuse Essential for Good Design Practice .......... 36

3.2 Experimental Study: Verbal Protocol Method ..................... 39

3.3 Analysis of Verbal Data ........................................ 42

Results on Descriptors ..................................... 50

3.7 Discussion .................................................... 54

3.8 Recommendations for Information Reuse Service ................. 56

Chapter 4. Dedal: An Information Management Service ........ 57

4.2 Dedal Test Domains and Results ................................ 62

Chapter 5. Information Handling Behavior Framework ........ 68

Informational Activities Distribution ......................... 91

6.4.5 Discussion on Time and Information Distribution.............. 95

Informational Activities and Information Descriptor .......... 101

6.7 Summary..................................................... 103

Chapter 7. Conclusions & Future Work ...................... 105

7.3 Discussion ................................................... 107

Bibliography .............................................. 110

Appendix A. Additional Material on Design Information Reuse

Appendix B. Additional Material on Dedal Development ..... 126

B.4 References on Dedal papers .................................... 133

Appendix C. Additional Material on Design Information Handling

Appendix D. Additional Results on Information Handling

0.1.1 deltat vs. ~e ............................................ 143

0.2.1 Information handling rate over time ........................ 149

Informational activities state transition probabilities .......... 160

0.6 Additional Results from Multiple Attribute Analysis ............ 162

Information descriptor and informational activities ........... 163

Appendix E. References on Papers from this Dissertation ..... 167

E.2 ASME Conference: DTM-1994................................. 167

Figure A-2 Distribution of question subject-class for RS1 and RS2. . . . . . . . ..

Observe ->Analyze -> Intervene iterative cycle

Design Information Framework

Information handling Framework

Electronic Design Notebook

Information Reuse Study

Information Handling Study

[DeScriptor, Subject] pair in information fragment classification

Design Information Measure (quantitative measure)

dims per minute

words per minute

Information fragment duration in seconds

Deliver High Quality,

This chapter is a summary of the research work being reported in this

Information Handling Behavior of Designers During Conceptual Design: Three Experiments

Chapter 1: Understanding and Improving Engineering Design Practice

1.1 Background: Engineering Design