Using Machine Vision and Hand-Motion Control to Improve

Crane Operator Performance

Transactions on Systems, Man, and Cybernetics--Part A: Systems

Journal:
and Humans

Manuscript ID: SMCA-10-09-0357

Manuscript Type: Regular Paper

Date Submitted by the

26-Sep-2010
Author:

Complete List of Authors: Peng, Kelvin Chen Chih; Georgia Institute of Technology
Singhose, William; Georgia Institute of Technology, Mechanical
Engineering
Bhaumik, Purnajyoti; Georgia Institute of Technology

Key Words: Control system human factors, Machine vision, Vibration control

Note: The following files were submitted by the author for peer review, but cannot be converted
to PDF. You must view these files (e.g. movies) online.

Glove crane.mov
Wand crane.mov
Page 1 of 11
SUBMITTED TO TSMC 1

1
2
3
4
Using Machine Vision and Hand-Motion Control to
5
6
Improve Crane Operator Performance
7 Kelvin Chen Chih Peng, William Singhose, Purnajyoti Bhaumik.
8
9
10
11
12 Abstract—The payload oscillation inherent to all cranes makes Trolley
13 it challenging for human operators to manipulate payloads
14 quickly, accurately, and safely. Manipulation difficulty is also
15 increased by non-intuitive crane-control interfaces. This paper
16 describes two new crane-control interfaces that allow an operator
17 to drive a crane by moving his or her hand freely in space. An
18 image-processing system is used to track the movement of the
19 hand-held device, which is then used to drive the crane. Computer
20 simulations and experimental data show that a combination of
21 aggressive feedback control to position the trolley, and an input Payload
22
shaper to suppress payload swing, generates a fast crane response Operator w/
23
and low residual oscillation. Studies involving novice operators Pendent
24
further demonstrate the benefits of hand-motion crane control. (a) Pendent (b) Schematic
25
26 Index Terms—Control interface; machine vision; cranes; os- Fig. 1. Crane Control Using a Push-Button Pendent.
27 cillation; input shaping.
28
29 In addition to facing the challenges of controlling large-
30 I. I NTRODUCTION
31 amplitude, lightly-damped payload swing, operators must also
32 Cranes play a key role in maintaining the economic vi- master non-intuitive machine interfaces. Examples of typical
33 tality of modern-day industry. Their importance can be seen crane-control interfaces include push-button pendents, joy-
34 sticks, and control levers. Figure 1 illustrates crane control
35 at shipyards, construction sites, warehouses, and in a wide
variety of material-handling applications. The effectiveness of using the pendent. The operator must be adept in the cognitive
36
37 crane manipulation is an important contributor to industrial process of transferring the desired manipulation path into
38 productivity, low production costs, and worker safety. a sequence of button presses or lever deflections that will
39 produce the desired crane motion. For example, if the operator
40 One inherent property of cranes that is detrimental to
efficient operation is the natural tendency for the payload wants to drive the crane through a cluttered workspace using
41
42 to oscillate like a pendulum, a double pendulum [1], or a push-button pendent, then the desired path must be mapped
43 with even more complex oscillatory dynamics [2]. Significant into a sequence of events where the “Forward”, “Backward”,
44 “Left”, and “Right” buttons are pushed for the correct time
effort has been made to develop control schemes to reduce
45 duration and in the correct sequence.
46 the oscillatory response from both issued commands and
47 external disturbances [3]–[9]. There has also been research Furthermore, as operators move through the workspace to
48 in controlling cranes that contain rotational joints, which adds drive the crane and monitor its progress, they may rotate
49 their bodies and change the direction they are facing. In such
an extra level of complexity due to their nonlinear dynamics
50 cases, the “Forward” button causes motion to the left, right,
51 [10]–[13]. Abdel-Rahman provides a review of crane control
52 strategies developed during the second half of the twentieth or even backward. As an additional challenge, the operator
53 century [14]. Operators who manipulate a crane utilizing can only directly drive the overhead trolley, not the payload.
54 appropriate oscillation-suppression technology generate safer Therefore, the operator must account for the time lag between
55 the commanded motion of the trolley, which can be many
56 and more efficient crane motions than operators without such
compensation [10], [15]–[17]. meters overhead, and the delayed oscillatory response of the
57
58 payload.
59 K.C.C. Peng, W. Singhose, and P. Bhaumik are with the Woodruff School While significant strides have been made to improve the
of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA,
60 30332 USA e-mail: Singhose@gatech.edu operational efficiency of cranes by controlling the dynamic
 Page 2 of 11
SUBMITTED TO TSMC 2

1
2 response to issued commands, relatively little consideration Bridge
3 has been given to the way in which operators issue those
4 commands [18]. Interfaces that are tailored to the cognitive

}
5
6 processes associated with specific control system have been Runway
7 shown to be beneficial [19]–[21]. Trolley
8 This paper presents a novel control interface that allows an Suspension
9 operator to drive a crane simply by moving his or her hand Cables
10
in space. Machine vision is used to track the location of a
11
12 hand-held device (a wand or a glove) that is moved by the Payload
13 operator. The position of the hand-held device is then used as Hook
14 the reference command signal to drive the crane. The hand-
15
motion control interface is well tailored to the task of driving a Fig. 2. 10-Ton Industrial Bridge Crane
16
17 crane through a cluttered workspace because it eliminates the
18 cognitive mapping process that is necessary with traditional
19 runway, and a trolley that suspends the hook and payload
control interfaces. Because of this, operators no longer need
20 by cables. The trolley travels along the bridge. Laser range
to account for the direction in which they are facing. The
21 sensors measure the trolley position along the runway and the
22 manual dexterity required for safe and efficient operation is
bridge. A Siemens programmable logic controller (PLC) is
23 also reduced.
used to control the motor drives and acts as the central control
24 Additionally, the control algorithm minimizes payload
25 unit. Commands to the crane can be issued with a push-button
swing without significantly slowing the system response.
26 control pendent, a laptop, or other devices [18].
27 Therefore, the burden of manually reducing payload oscillation
A downward-pointing Siemens Simatic VS723-2 camera
28 is removed, allowing the operator to concentrate solely on the
mounted on the trolley measures the relative position of the
29 path planning and final positioning of the payload. Further-
30 hook [26]. Reflective material, arranged in a hexagonal pattern,
more, because a non-oscillating payload will always come to
31 is attached to the top of the hook. The reflectors aid image
rest directly beneath the overhead trolley, final positioning of
32 segmentation and blob detection. Using the known geometry
33 the trolley is equivalent to the final positioning of the payload.
of the reflective markers, the position of the hook can be
34 Hand-motion control also offers other cognitive advantages
35 determined, and the length of the suspension cables can be
over traditional interfaces. There are two primary divisions
36 estimated. The advantage of using multiple reflectors is two-
of cognitive control: analytic problem solving and perceptual
37 fold: 1) robustness against changes in ambient light and noise,
38 processing [22]. Perceptual processing tends to be faster and
and 2) robustness against occlusion by the suspension cables.
39 can be performed in parallel, while analytic processing takes
40 longer and typically progresses serially. Analytic problem III. H AND -M OTION C ONTROL
41 solving also tends to be more prone to error [22], [23]. The
42 In order to provide a more intuitive control interface, a
results of many studies also suggest that people prefer, and
43 system was developed to drive the crane by simply moving
44 adopt, perceptual processing when possible [17], [22], [24],
a wand or a glove. The wand, shown in Figure 3, is a retro-
45 [25].
46 reflective ball mounted to the end of a hand-held pole. The
From this perspective, hand-motion control helps operators
47 glove, shown in Figure 4, is monotonically black, with a
by lowering the cognition level required to drive the crane.
48 circular reflective marker attached to the top. After image
49 Operators no longer need to think analytically about the
segmentation and processing on the camera, the wand/glove
50 sequence of buttons to push, or to account for the swinging
appear as a single contiguous blob of high pixel intensity. To
51 payload; they only need to move the hand-held device to the
52 distinguish the wand/glove blob from the blobs formed by
desired position, or along a desired path. This allows them to
53 the hook reflectors, a K-means clustering algorithm is used
54 perform simpler perceptual processing.
[27]. The camera calculates the positions of the hook and
55
wand/glove approximately every 140 ms. Figure 5 shows the
56 II. 10-T ON I NDUSTRIAL B RIDGE C RANE
57 difference in position between the wand/glove and the trolley
58 The work detailed in this paper uses the 10-ton industrial (e) that is used as the error feedback signal.
59 bridge crane shown in Figure 2. A bridge crane consists Three different control architectures are investigated in this
60 of a fixed overhead runway, a bridge that travels along the paper. The standard pendent controller is used as the baseline
Page 3 of 11
SUBMITTED TO TSMC 3

1
2 Camera
Trolley
3
4
5
6
7
8
9
10 Wand
11
Wand/Glove
12 Hook
13
14
15
16 e
17 Hook
18 Fig. 5. Schematic Diagram of Hand-Motion Crane Control.
19
20 Fig. 3. Wand Control
21 Motor, Hook
22 Pendent
Trolley Dynamics
23
24
25 Fig. 6. Standard
Standard Pendent Pendent
Control Control
Block Diagram
26
27 Hand-Held
28 Glove Device Position
Hand-Held
29 Device Position
+ e1
30 + PDe Motor, Hook
Saturator Motor, Hook
31 PD Trolley Dynamics
- Trolley Dynamics
32 -
33 Trolley1/S
Position
Trolley Position
34
35 Fig. 7. PD Hand-Motion Control Block Diagram
36 Hook PD Wand/Glove Control
37
38 Hand-Held
39 Device Position
Hand-Held
40 Device Position
+ e
Fig. 4. Glove Control Input Motor, Hook
41 PD
+ Shaper Trolley Dynamics
42 - e1 Input Motor, Hook
PD Saturator
Trolley Position
43 Shaper Trolley Dynamics
-
44 for comparisons. Its control block diagram is shown in Figure
45 Fig. 8. PD with Input Shaper1/S
Hand-Motion Control Block Diagram
6. The pendent produces velocity commands for crane motors. Trolley Position
46
47 Trolley movement then induces hook dynamics.
48 A Proportional-Derivative (PD) feedback controller is used PD with Input Shaper Wand/Glove Control
49 for hand-motion control. Its block diagram is shown in Figure hand-motion controller with the addition of an input shaper
50 Hand-Held
7. The difference between the horizontalDevicepositions of the after the saturator. The linear input shaper is placed after
Position
51
52 trolley and the hand-held device is used as the error signal the nonlinear saturator (rather than before) to preserve the
53 (e from Figure 5) in negative feedback with PD control. The oscillation-reducing
Feedback loop relevant to theproperties of the shaped
stability analysis usingsignal.
root locus
54 output of the PD controller is passed through a saturator
+ e1 before Note
V that from theVPD perspective of control
VZV theory, the Vwand
55 Linearized R K (PD) Input Motor, T Hook
it is sent to the trolley motors. and
Converter the glove
PD are identical. However,
Shaper in terms of ergonomics
Trolley Dynamics
56
57 To mitigate the oscillatory hook response that- results from during operation, the wand has a greater reach and can drive
58 using the PD hand-motion controller, an architecture that the crane towards tight spaces,
1/S such as corners. On the other
Trolley Position
59 combines PD feedback with an input shaper (which will be hand, the glove sacrifices range of reach for a smaller size and
60 explained in Section IV) is introduced. Figure 8 shows the PD ease of use.
Linearized PD with Input Shaper Wand/Glove Con-
trol for Root Locus Analysis
 Tokyo Institute of Technology
Ookayama, Meguro-ku, Tokyo, Japan Page 4 of 11
Email: vaughanje@gatech.edu
SUBMITTED TO TSMC 4

1
2 30

Percent Residual Vibration

3 ZV
a wide 25
ZVD
4 EI
ations.
5
ayload
6
s that
7
ties8 of
* 20

15 0.29
otion.
9
0 !
namic10 10 0.06
he 11 dif-
ts 12ini- 5 Vtol
boom13
d 14and 0
0.7 0.8 0.9 1 1.1 1.2 1.3
of 15 the Normalized Frequency (ω/ωm)
se 16un-
17 ! Fig. 10. Sensitivity Curve for the ZV, ZVD, and EI Shapers
18
19
20 Fig. 9. Figure 1. The
The Input-Shaping Input-Shaping
Process ProcessStep Input.
Applied to a Reference
To form a non-dimensional vibration amplitude, (1) is
21
aterial
22 has been shown to improve crane operator performance divided by the amplitude of residual vibration from a single
ge 23 va- [15, 16, 17, 18]. IV. I NPUT S HAPING impulse of unity magnitude. The resulting expression gives
sites,
24 A second significant limitation of using feedback con- the ratio of vibration with input shaping to that without input
uscep-25 Inputonshaping
trol cranes is is the
a technique
difficulty used for negating
of measuring the flexible
the motion of shaping. This percentage residual vibration (PRV) is given by
motion26 modes of a system
the payload. and does
Therefore, not feedback
some require thecontrol
feedback mech-
methods
[35]:
oscil-
27 are constructed
anisms to avoid
of closed-loop the need to[28]–[32].
controllers measure the Thelocation of
reference
P RV = e−ζωtn [C(ω, ζ)]2 + [S(ω, ζ)]2 .
p
uttered
28 the actual payload. For example, if the hook swing angle (4)
command is modified in such a way that the resonant modes
scilla-
29 is measured, then a feedback controller could be designed
are not excited [31], [33], [34]. Equation (4) represents the level of vibration induced by an
ut 30and to eliminate the swing by assuming that driving the hook
Figure angle
9 illustrates impulse sequence given at any frequency and any damping ra-
31 swing to zerothe willinput-shaping
stop the payload process.
swing.When a ref-
Figure
dback32 erence step command,
2 illustrates the problem represented
with thisbyapproach
the dotted line,the
when is pay-
issued tio less than one. A constraint on residual vibration amplitude
uccess33 load creates a double-pendulum effect. Figure
to a flexible system, the response is oscillatory. If however, the 2(a) shows can be formed by setting (4) less than or equal to a tolerable
34
onflict the location of theis payload when
35 reference command convolved withthe swing angle
a sequence of the
of impulses, level of residual vibration at the modeled natural frequency
human hook is 10◦ . The payload is to the left of the hook. Figure and damping ratio.
36 known as an input shaper, then a shaped command is produced.
; they 2(b) shows the location of the payload for the same hook
When For the simplest, Zero Vibration (ZV) shaper, the tolerable
37
esired anglethe
of shaped
10◦ , butcommand
in this case, is the
issued to theis flexible
payload system,
to the right of
38
ontrol thethe
amplitude
payload.of residual
These two vibration
photographs is reduced,
demonstrateas shown at the
that mea- amount of vibration is set to zero. This results in a shaper of
39 suringofthe suspension cable angle does not provide reliable the form [28], [29]:
bottom Figure 9.
40 information about the payload location. Double-pendulum
e con- The amplitudes and time locations of the impulses in the " #  1 K

41
erators payloads commonly arise in many types of crane handling Ai 1+K 1+K
(5)
42 shaper can be obtained from closed-form solutions or by ZV = = ,
ations. processes. A feedback controller designed simply based on ti 0 √π 2
43 solving a setangle
of constraint
would not equations. Theimproperly,
primary design ω 1−ζ
nce of the swing only function but
44 where,
refer- constraint is atolimit
could lead on theresults.
unstable amplitude of vibration caused by the
45 √−ζπ
Input
46 shaper. InTheaddition
vibration to payload
amplitude oscillation, lack of mobility
of an under-damped, is
second- K=e 1−ζ 2 . (6)
7,47 8], another limiting factor of crane performance.
order system from a sequence of n-impulses is given by [29]: Most cranes,
4] 48 and like the one shown in Figure 3, have little or no base mo- To gain more insight, one can analyze the performance of
ω
e−ζωtn [C(ω, ζ)]2 + [S(ω, ζ)]2 , (1)
p
49 AP = p the ZV shaper with the use of a sensitivity curve, shown in
50 1 − ζ2
Figure 10. The vertical axis is the Percent Residual Vibration
51 where,
52 n
X p (P RV ) and the horizontal axis is the actual natural frequency,
53 C(ω, ζ) = Ai eζωti cos(ωti 1 − ζ 2) (2) ω, normalized by the modeled frequency, ωm (which was used
54 i=1
n to design the input shaper). The sensitivity curve for a ZV
55 X p
shaper is shown by the solid line. The curve indicates how
S(ω, ζ) = Ai eζωti sin(ωti 1 − ζ 2 ), (3)
56
57
i=1 residual vibration amplitude changes as a function of modeling
58 and ω is the natural frequency of the system, ζ is the damping errors in frequency. The sensitivity curves for the Zero-
59 ratio, and Ai and ti are the amplitude and time of the ith - Vibration-Derivative (ZVD) [29] and the Extra-Insensitive (EI)
60 impulse, respectively. [36] shapers are also shown. These more robust shapers
Page 5 of 11
SUBMITTED TO TSMC 5

1
3.5 2.5
2
3 3
4 2
5 2.5

Position (m)
Position (m)

6 1.5
2
7
8 1.5 1 Hand-Held Device
9
1 Trolley1 Trolley
10
Hook1 0.5 Hook
11 0.5 Trolley2
12 Hook2
13 0 0
0 5 10 15 20 25 30 35 40 0 5 10 15 20 25 30 35 40
14
15 Time (sec) Time (s)
16
17 Fig. 11. Simulated Point-To-Point Responses Using Pendent Control Fig. 12. Simulation Result of the PD Hand-Motion Controller with Low
18 Gains
19
20 contain three impulses and are longer in duration than the 2.5
21
22 relatively non-robust ZV shaper.
2
23
24 V. S IMULATION R ESULTS Position (m)
25 1.5
Computer models were constructed to simulate the control
26
27 architectures in Figures 6, 7, and 8. Hand-motion trajectories
1 Hand-Held Device
28 were specified as ramps in position with gradients equivalent Trolley
29 to the maximum velocity of the industrial crane (0.3577 m/s), 0.5 Hook
30 which is approximately the speed of a slow walk. This also
31
mimics the typical hand-motion trajectories from a human 0
32 0 5 10 15 20 25 30 35 40
33 operator.
Time (s)
34
35 A. Pendent Control Fig. 13. Simulation Result of the PD Hand-Motion Controller with High
36
37 Some typical responses using the pendent controller are Gains

38 shown in Figure 11, using a cable length of 5 m. Using the

39 pendent, if a move button is depressed for a certain duration,
40 Under this set of gains, the crane is more responsive; the
then the trolley will move at constant velocity for a certain
41 10% to 90% rise time is reduced by 53% to 4.4 s. However,
42 distance. The position of the trolley plotted against time is
hook oscillations are now more significant. The maximum
43 a ramp that plateaus after a certain distance. Due to the
percentage overshoot has increased by eight-fold to 16.6%.
44 pendulum-like nature of the hook, this type of movement
45 Due to the large overshoot and lightly damped nature of the
will, in general, induce oscillations. Figure 11 shows the hook
46 hook dynamics, the 2% settling time has increased by more
responses to move distances of approximately 2 and 3 m.
47 than 100% to 119 s.
48
49 B. PD Hand-Motion Control
50 C. PD with ZV Shaper Hand-Motion Control
Figure 12 shows the simulation results for the PD hand-
51
52 motion controller with low feedback gains. The response is There is an inherent trade-off using the PD hand-motion
53 slow, with a 10% to 90% rise time of nearly 9.3 s and a 2% controller. Low gains do not induce large hook oscillation
54 settling time of 55 s. The maximum percentage overshoot is amplitudes, but the trolley response is slow. High gains pro-
55 duce a faster trolley response at the cost of high amplitude
just over 2%, indicating the residual oscillation is also low.
56
57 To increase the responsiveness of the hand-motion con- hook oscillations. The goal of combining high gains PD
58 troller, the gains must be increased so that the trolley is able to feedback with a ZV input shaper is to create a controller with
59 rapidly accelerate to its maximum velocity. Figure 13 shows the desirable performance features of fast response and low
60 the response of the hand-motion control with increased gains. residual oscillations. The ZV shaper was designed using the
 Page 6 of 11
SUBMITTED TO TSMC 6

1
2 2.5
3
4 2
5
Position (m)

6 1.5
7
8 e e e=0 e=0
1 Hand-Held Device
9 Starting Moving Decelerating Stopped
Trolley
10
11 0.5 Hook Fig. 15. Hand-Motion Control: Starting and Stopping
12
13 0
0 5 10 15 20 25 30 35 40 2.5
14
15 Time (s)
16 2
17 Fig. 14. Simulation Result of the PD with ZV Shaper Hand-Motion

Position (m)
18 Controller 1.5
19
20 1 Hand-Held Device
21 natural frequency (determined by the 5m cable length) and Trolley
22 damping ratio corresponding to the industrial crane discussed 0.5 Hook
23
in Section II.
24
25 Figure 14 clearly shows the improved response of the com- 0
0 5 10 15 20 25 30 35 40
26 bined controller. The 10% to 90% rise time is only 4.6 s. The
Time (s)
27 2% settling time is reduced to 18 s. The maximum percentage
28 overshoot is only 7.8%. Note that the small oscillation in the
29 Fig. 16. Experimental Result of the PD Hand-Motion Controller With Low
30 response results from the PD control of the trolley, not from Gains
31 a deficiency in the input shaper.
32
33 VI. E XPERIMENTAL R ESULTS to 90% rise time is around 10 s, the maximum percentage
34
The hand-motion control system was implemented on the overshoot is about 5%, and the 2% settling time is 50 s. Note
35 that the wand/glove was not moved in a perfect ramp, as in
36 10-ton bridge crane. The wand/glove trajectories produced
37 by the human operators were similar to those used in the the simulations.
38 simulations. The ramp gradient was approximately equivalent Figure 17 shows the experimental response of PD hand-
39 to the maximum velocity of the crane, and the move distance motion control with increased gains. The 10% to 90% rise
40 time was reduced by over 40% to 5.8 s; however, the maximum
41 was approximately 2 m for the tests reported here.
Figure 15 illustrates an operator using hand-motion control percentage overshoot doubled to 10%. The hook took longer
42
43 to start and stop the crane. To start moving the crane, the than 53 s to settle within 2% of the stop distance. Note that
44 operator can expose the wand/glove to the camera at any with the higher gains, the hook tracks the wand/glove position
45 much more closely than with the low gains.
distance away from the trolley. The result is that the value of e
46
47 jumps instantaneously when the wand/glove is detected. When
48 the crane approaches the desired stopping location, the crane B. PD with ZV Shaper Hand-Motion Control
49 operator lowers the wand/glove, which becomes undetectable The experimental results using the PD hand-motion con-
50 by the camera. As a result, the control software reacts by trollers clearly demonstrate the trade-off between low and high
51
52 setting e to zero. Due to these operational effects, experimental gains. Using high gains will reduce rise time at the expense
53 results contain spurious and discontinuous artifacts for the of increased overshoot and settling time.
54 wand/glove position during the starting and stopping stages Figure 18 shows the experimental response of the combined
55 of the motion. PD and ZV input-shaping controller. The 10% to 90% rise
56
57 time is around 4.8 s, and there is virtually no overshoot or
58 A. PD Hand-Motion Control residual oscillation. For this reason, the 2% settling time is
59 Figure 16 shows the experimental results for the PD hand- approximately 8 s (an 84% improvement over PD with low
60 motion controller with a low proportional gain. The 10% gains).
Page 7 of 11
SUBMITTED TO TSMC 7

1
2 2.5 2.5
3
4 2 2
5
Position (m)

Position (m)
6 1.5 1.5
7
8
1 Hand-Held Device 1 Hand-Held Device
9
Trolley Trolley
10
11 0.5 Hook 0.5 Hook
12
13 0 0
0 5 10 15 20 25 30 35 40 0 5 10 15 20 25 30 35 40
14
15 Time (s) Time (s)
16
17 Fig. 17. Experimental Result of the PD Hand-Motion Controller With High Fig. 19. PD With ZV Shaper Hand-Motion Controller With 4 m Suspension
18 Gains Cable Length
19
20 2.5
2.5
21
22
23 2 2
24 Position (m)
Position (m)

25 1.5 1.5
26
27 1 Hand-Held Device 1 Hand-Held Device
28 Trolley Trolley
29 Hook 0.5 Hook
0.5
30
31
32 0 0
0 5 10 15 20 25 30 35 40 0 5 10 15 20 25 30 35 40
33 Time (s)
34
Time (s)
35
36 Fig. 18. Experimental Result of the PD With ZV Shaper Hand-Motion Fig. 20. PD With ZV Shaper Hand-Motion Controller With 6 m Suspension
37 Controller Cable Length
38
39
40 The experimental and simulation results demonstrate that changed to 4 m and 6 m, the real natural frequency, ω, is
41 modified to 1.57rad/s and 1.28rad/s, respectively. These
42 the hand-motion controller with high PD gains and ZV input
43 shaping is able to produce a fast hook response without correspond to a normalized frequency of 1.12rad/s and
44 significant overshoot and residual vibration. 0.91rad/s, respectively. Referring to the ZV sensitivity curve
45
To investigate the robustness of the controller, the same in Figure 10, it can be seen that this is still reasonably good
46 performance, as the P RV for both cases is under 20%. The
47 movement was repeated for suspension cable lengths of 4 m
and 6 m, without redesigning the PD controller or the input case of the 4 m cable length has a slightly higher P RV ,
48
49 shaper. These correspond to a 1 m decrease and increase from which explains the presence of the visible, but small amount
50 the original 5 m cable length. The experimental results for of residual oscillations in Figure 19. If the crane was expected
51 to undergo large changes in cable length, then the more
52 these cases are shown in Figures 19 and 20, respectively.
The controller still shows effective suppression of residual robust shapers shown in Figure 10 could be used instead [29],
53
54 oscillations, even when the cable lengths were changed. [36]. Alternatively, adaptive input shapers could also provide
55 robustness to parameter variations [37]–[39].
To gain more insight, one can analyze the ZV shaper’s
56
57 performance with the use of a sensitivity curve, which was
shown in Figure 10. For this specific application, the ZV VII. O PERATOR S TUDIES
58
59 shaper designed for the 5 m cable length has a modeled This section presents the results from two studies that were
60 frequency of ωm = 1.4rad/s. When the cable length is conducted to compare the operating efficiency of pendent
 Page 8 of 11
SUBMITTED TO TSMC 8

1
2 200
Start Pendent
3

Completion Time (sec)

Wand
4 150
5 1.2 m Finish
6
100
7
0.7 m
8
9 Trolley 50
10 Axis
11 Bridge 0
12 Axis 1 2 3 4 5 6 7 8 9 10 11 12
13 Operator
14 Fig. 21. Obstacle Course 1 Fig. 22. Obstacle Course 1 Completion Times.
15
16
17
18 control versus hand-motion control. In each study, the goal 14
19 was to move the crane hook from start to finish as quickly Pendent
20 12 Wand

Number of collisions
as possible without collisions with the obstacles. Operators
21 10
22 were each given five minutes to familiarize themselves with
23 the control interfaces before commencing the study. 8
24 6
25
4
26 A. PD Hand-Motion Control
27 2
28 The obstacle course used in this study is shown in Figure
0
29 21. The obstacles were arranged such that the fastest route to 1 2 3 4 5 6 7 8 9 10 11 12
30 the finish required diagonal crane movements (simultaneous Operator
31 movement in both the trolley and bridge axes). Twelve novice Fig. 23. Obstacle Course 1 Collisions.
32
33 operators completed the obstacle course using the following
34 control interfaces:
35 1) Pendent control with the block diagram shown in Figure 0
36
6.
Trolley Axis (m)

37 -0.5
38 2) Wand control using PD hand-motion control with low
39 gains and no input-shaping. The block diagram is shown -1
40 in Figure 7 and the gains were the same as the controller
41 -1.5
that produced Figure 16. Low gains were used so that
42 Pendent
43 the crane would not excite large amplitude oscillations. -2
Wand
44 Figure 22 shows the course completion times for each -2.5
45 -0.5 0 0.5 1 1.5 2 2.5 3 3.5
operator. The average completion time using the pendent was
46 Bridge Axis (m)
47 97 s. The average completion time using the wand was only
48 46 s, a 53% improvement. A paired t-test indicated that the Fig. 24. Obstacle Course 1 - Overhead View of Hook Response.
49 improvement in completion time was statistically significant,
50
t(11) = 4.20, p < 0.002.
51
52 Figure 23 plots the number of collisions that occurred during
53 each trial. Using pendent control, all operators suffered at least Figure 24 shows a typical two-dimensional hook response
54 one collision, and the average number of collisions was 5.1. for a single operator using both pendent and wand control. It
55 Using wand control, most operators had few or no obstacle is evident that wand control allows the hook to be controlled
56
57 collisions, and the average number of collisions was 0.92, an more precisely. The reduction in the amplitude of hook swing
58 81% improvement. A paired t-test indicated that the reduction and overall smoothness of hook travel are major contributors to
59 in collisions was statistically significant, t(11) = 4.45, p < fewer obstacle collisions and more efficient navigation through
60 0.001. the obstacle course.
Page 9 of 11
SUBMITTED TO TSMC 9

1
2 3
Pendent
3 0.5 m

Number of Collisions
Glove
4 1.3 m
5 Finish 2
Start 1.2 m
6 0.7 m
7
8 Trolley 1
9 Axis
10
Bridge
11 Axis
12 0
1 2 3 4 5 6 7 8 9 10
13 Fig. 25. Obstacle Course 2 Operator
14
15 Fig. 27. Obstacle Course 2 Collisions.
16 120
17 Pendent
Completion Time (sec)

100 Glove 0
18
19 80 -0.5

Trolley Axis (m)

20
21 60
-1
22 40
23 -1.5
24 20
Pendent
25 -2 Glove
0
26 1 2 3 4 5 6 7 8 9 10
27 Operator -2.5
7.5 8 8.5 9 9.5 10 10.5 11
28
Fig. 26. Obstacle Course 2 Completion Times. Bridge Axis (m)
29
30
Fig. 28. Obstacle Course 2 - Overhead View of Hook Response.
31
32 B. PD with ZV Shaper Hand-Motion Control
33
34 The obstacle course used in this study is shown in Figure Figure 28 shows a typical two-dimensional hook response
35 25. Ten novice operators completed the obstacle course using for a single operator using both the pendent and glove. It is
36 the following control interfaces: evident that glove control reduces hook swing and allows the
37
38 1) Pendent control with the block diagram shown in Figure operator to control the hook more precisely. For this reason,
39 6. operators were able to navigate the obstacle course with more
40 2) Glove control using PD feedback with high gains and efficiency and without colliding with the obstacles.
41
ZV shaper hand-motion control. The block diagram is
42
43 shown in Figure 8 and the gains selected were the same VIII. C ONCLUSIONS
44 as the controller that produced Figure 18. Crane controllers based on operator hand-motion have been
45
Figure 26 shows the course completion times for each successfully installed on an industrial bridge crane. An over-
46
47 operator. The average completion time using the pendent was head camera tracks the position of a hand-held device (a
48 77 s. The average completion time using the glove was only wand or a glove), which is moved by the operator through
49 24 s, a 69% improvement. A paired t-test indicated that the the desired trajectory. The crane then follows the wand/glove.
50
improvements in completion time were statistically significant, Three types of hand-motion controllers were simulated. It
51
52 t(9) = 9.71, p < 0.0001. was found that aggressive PD feedback gains to position the
53 Figure 27 plots the number of collisions that occurred during overhead trolley, combined with ZV input shaping to limit
54 each trial. Using pendent control, many operators collided the hook swing, produced the desired performance characteristics
55 crane with the obstacle. The average number of collisions was of fast response, short settling time, low amplitude overshoot,
56
57 0.9. However, all operators were able to avoid the obstacle and low residual oscillations. Experiments conducted on an
58 using the glove. A paired t-test indicated that the reduction industrial crane verified the usefulness of the proposed control
59 in number of collisions was statistically significant, t(9) = architectures. Furthermore, the results from operator studies
60 3.25, p < 0.01. provide evidence that hand-motion control is more effective
 Page 10 of 11
SUBMITTED TO TSMC 10

1
2 than traditional control interfaces such as a push-button pen- [17] J. Vaughan, A. Smith, S. Kang, and W. Singhose, “Predictive graph-
3 dent. ical user interface elements to improve crane operator performance,”
4 Accepted to IEEE Trans. on Systems, Man, and Cybernetics–Part A:
5 Systems and Humans, 2010.
IX. ACKNOWLEDGMENTS [18] K. L. Sorensen, J. B. Spiers, and W. E. Singhose, “Operational effects of
6
The authors would like to thank Siemens Industrial Automa- crane interface devices,” in IEEE Conference on Industrial Electronics
7
and Applications, Harbin, China, 2007.
8 tion, the Manufacturing Research Center at Georgia Tech, and [19] S. Guerlain, G. Jamieson, P. Bullemer, and R. Blair, “The mpc elucida-
9 Boeing Research and Technology (BR&T) for their support of tor: a case study in the design for human-automation interaction,” IEEE
10
this work. Transactions on Systems, Man and Cybernetics, Part A: Systems and
11 Humans, vol. 32, no. 1, pp. 25–40, January 2002.
12 [20] G. Jamieson, “Ecological interface design for petrochemical process
13 R EFERENCES control: An empirical assessment,” IEEE Transactions on Systems, Man
14 [1] W. Singhose, D. Kim, and M. Kenison, “Input shaping control of double- and Cybernetics, Part A: Systems and Humans, vol. 37, no. 6, pp. 906–
15 pendulum bridge crane oscillations,” ASME J. of Dynamic Systems, 920, 2007.
16 Measurement, and Control, vol. 130, no. 034504, May 2008. [21] S. V. Dam, M. Mulder, and M. van Paassen, “Ecological interface design
17 [2] J. Yoon, W. Singhose, J. Vaughan, G. Ramirez, M. Kim, and S. Tawde, of a tactical airborne separation assistance tool,” IEEE Transactions on
18 “Dynamics and control of crane payloads that bounce and pitch during Systems, Man and Cybernetics, Part A: Systems and Humans, vol. 38,
19 hoisting,” in IDETC/CIE 2009, San Diego, CA, 2009. no. 6, pp. 1–1233, 2008.
20 [3] Y. Sakawa and Y. Shindo, “Optimal control of container cranes,” [22] J. Reason, Human Errors. Cambridge, UK: Cambridge University
21 Automatica, vol. 18, no. 3A, pp. 257–266, 1982. Press, 1990.
[4] G. P. Starr, “Swing-free transport of suspended objects with a path- [23] E. Brunswick, Perception and the Representative Desing of Experiments.
22
controlled robot manipulator,” J. of Dynamic Systems, Measurement and Berkeley, CA: University of California Press, 1956.
23
Control, vol. 107, pp. 97–100, 1985. [24] A. Kirlik, “The organization of perception and action in complex control
24 [5] D. Strip, “Swing-free transport of suspended objects: a general treat- skills,” Ph.D. dissertation, The Ohio State University, Columbus, OH,
25 ment,” IEEE Trans. Robotics and Automation, vol. 5, no. 2, pp. 234 – 1989.
26 6, 1989. [25] K. J. Vicente and J. Rasmussen, “Ecological interface design: Theoreti-
27 [6] N. Singer, W. Singhose, and E. Kriikku, “An input shaping controller cal foundations,” IEEE Transactions on Systems, Man and Cybernetics,
28 enabling cranes to move without sway,” in ANS 7th Topical Meeting on vol. 22, no. 4, pp. 589–606, 1992.
29 Robotics and Remote Systems, vol. 1, Augusta, GA, 1997, pp. 225–31. [26] K. Hekman, K. Sorensen, C. Stäheli, and W. Singhose, “Measuring
30 [7] K.-S. Hong, J. H. Kim, and K.-I. Lee, “Control of a container crane: Fast crane payload swing through computer vision,” in Int. Symp. on Flexible
31 traversing, and residual sway control from the perspective of controlling Automation, Osaka, Japan, 2006.
32 an underactuated system,” in Proceedings of the 1998 American Control [27] S. Lloyd, “Least squares quantization in pcm,” IEEE Transactions on
33 Conference, vol. 2, 1998, pp. 1294–1298. Information Theory, vol. 28, no. 2, pp. 129–137, 1982.
[8] Y.-S. Kim, H.-S. Seo, and S.-K. Sul, “A new anti-sway control scheme [28] O. J. M. Smith, “Posicast control of damped oscillatory systems,”
34
for trolley crane system,” in Thiry-Sixth IAS Annual Meeting. Conference Proceedings of the IRE, vol. 45, no. September, pp. 1249–1255, 1957.
35
Record of Industry Applications Conference, vol. 1, 2001, pp. 548–552.
36 [29] N. C. Singer and W. P. Seering, “Preshaping command inputs to reduce
[9] Z. Masoud and M. Daqaq, “A graphical approach to input-shaping
37 system vibration,” J. of Dynamic Sys., Measurement, and Control, vol.
control design for container cranes with hoist,” IEEE Transactions on
38 112, pp. 76–82, 1990.
Control Systems Technology, vol. 14, no. 6, pp. 1070–1077, 2006.
39 [30] W. Singhose and W. Seering, Command Generation for Dynamic Sys-
[10] G. Parker, K. Groom, J. Hurtado, J. Feddema, R. Robinett, and F. Leban,
tems. ISBN 978-0-9842210-0-4, www.lulu.com/content/621219, 2007.
40 “Experimental verification of a command shaping boom crane control
[31] W. Singhose, “Command shaping for flexible systems: A review of
41 system,” in American Control Conference, San Diego, CA, 1999, pp.
the first 50 years,” International Journal of Precision Engineering and
42 86–90.
Manufacturing, vol. 10, no. 4, pp. 153–168, 2009.
43 [11] O. Sawodny, A. Hildebrandt, and K. Schneider, “Control design for
[32] T. Singh and G. R. Heppler, “Shaped input control of a system
44 the rotation of crane loads for boom cranes,” in IEEE International
with multiple modes,” Journal of Dynamic Systems, Measurement, and
45 Conference on Robotics and Automation, 2003. Proceedings. ICRA’03,
Control, vol. 115, pp. 341–347, September 1993.
vol. 2, 2003.
46 [33] P. Meckl and W. Seering, “Experimental evaluation of shaped inputs
[12] J. Neupert, T. Mahl, O. Sawodny, and K. Schneider, “A Nonlinear
47 to reduce vibration for a cartesian robot,” J. of Dynamic Systems,
Control Strategy for Boom Cranes in Radial Direction,” in American
48 Control Conference, 2007. ACC’07, 2007, pp. 25–30. Measurement, and Controls, vol. 112, pp. 159–165, June 1990.
49 [13] C. Ko, J. Chiou, C. Y. Gau, S. Lee, and W. Chang, “Tower crane [34] P. H. Meckl and W. P. Seering, “Minimizing residual vibration for point-
50 vibration supression using generalized input shaping,” International J. to-point motion,” Journal of Vibration, Acoustics, Stress and Reliability
51 of Fuzzy Systems, vol. 6, no. 1, pp. 38–43, 2004. in Design, vol. 107, pp. 378–382, October 1985.
52 [14] E. Abdel-Rahman, A. Nayfeh, and Z. Masoud, “Dynamics and Control [35] K. Kozak, W. Singhose, and I. Ebert-Uphoff, “Performance measures for
53 of Cranes: A Review,” Journal of Vibration and Control, vol. 9, no. 7, input shaping and command generation,” Journal of Dynamic Systems,
54 p. 863, 2003. Measurement, and Control, vol. 128, no. 3, pp. 731–736, 2006.
55 [15] A. Khalid, J. Huey, W. Singhose, and J. Lawrence, “Human operator [36] W. Singhose, W. Seering, and N. Singer, “Residual vibration reduction
performance testing using an input-shaped bridge crane,” ASME J. of using vector diagrams to generate shaped inputs,” ASME J. of Mechan-
56
Dynamic Systems, Measurement, and Control, vol. 128, no. 4, pp. 835– ical Design, vol. 116, no. June, pp. 654–659, 1994.
57
841, 2006. [37] S. Rhim and W. Book, “Noise effect on adaptive command shaping
58
[16] D. Kim and W. Singhose, “Performance studies of human operators methods for flexible manipulator control,” IEEE Transactions on Control
59 driving double-pendulum bridge cranes,” Control Engineering Practice, Systems Technology, vol. 9, no. 1, pp. 84 –92, jan 2001.
60 vol. 18, no. 6, pp. 567 – 576, 2010. [38] A. Tzes and S. Yurkovich, “An adaptive input shaping control scheme for
Page 11 of 11
SUBMITTED TO TSMC 11

1
2 vibration suppression in slewing flexible structures,” IEEE Transactions
3 on Control Systems Technology, vol. 1, no. 2, pp. 114 –121, jun 1993.
4 [39] E. Pereira, J. Trapero, I. Daz, and V. Feliu, “Adaptive input shaping
5 for manoeuvring flexible structures using an algebraic identification
technique,” Automatica, vol. 45, no. 4, pp. 1046 – 1051, 2009.
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60