Proceedings of the 2001 IEEE

International Conference on Robotics & Automation

Seoul, Korea May 21-26, 2001

A Virtual Collaborative World Simulator for Underwater Robots

using Multi-Dimensional, Synthetic Environment,
S.K. Choi and J. Yuh
Autonomous Systems Laboratory
Department of Mechanical Engineering
University of Hawaii at Manoa
2540 Dole Street #302
Honolulu, HI 96822 USA
schoi@hawaii.edu & yuh@eng.hawaii.edu
extreme conditions in a controlled laboratory
environment before operational or sea-trial
deployment. In addition, since many military,
scientific, and commercial tasks in open-oceans
often require multi-national participation, it
becomes a necessity to rehearse these operations
before the actual operation, thus establishing
operational strategy and ensuring the success of
the operation without releasing proprietary or
secured materials. Therefore, the Distributed
Virtual Environment Collaborative Simulator
(DVECS) becomes the ultimate tool for these
needs.

Abstract
DVECS (Distributed Virtual Environment
Collaborative Simulator) is used for testing of
unmanned underwater vehicles (UUVs) where
both real and simulated vehicles can interact
and cooperate in a hybrid, synthetic, virtual
environment. This virtual reality system can be
used to determine: (1) the optimal performance
and criteria for the cooperating vehicles and its
relative application; (2) the determination of the
advantages and disadvantages of collaborative
application tasks between multiple UUVs; and
(3) the optimal communication links between the
cooperating vehicles and its remote control
stations.
It also utilizes a virtual reality
projection and polarized eyewear systems,
which immerses the user for optimal
visualization.

Several universities have conducted research in

the graphic simulator arena. To mention a few,
they are: (a) the Naval Postgraduate School and
their NPS AUV Integrated Simulator for their
NPS AUV [1]; (b) the University of Tokyo and
their Multi-Vehicle Simulator for their TwinBurger AUV [2]; and (c) the Autonomous
Undersea Systems Institute and their
Cooperative AUV Development Concept [3].
Both the NPS and UT systems were developed
on the IRIX environment of the Silicon Graphics
workstation, while the AUSI system runs on the
Win32 environment on an Intel based system.
All systems, however, are running OpenGL
graphic
protocols,
which
is
platform
independent.

Introduction
Even with the increased interest in the
development of underwater robotic technology,
the design, fabrication and analysis of
autonomous underwater vehicles (AUVs) are
still very complex, expensive, and timeconsuming. The unpredictable and hazardous
underwater
environment
is
extremely
unforgiving and remote. With stark limitations
in communication, an AUV must operate in a
fully autonomous or near autonomous modes.
This requirement immensely complicates the
development, diagnosis, and evaluation of
AUVs many subsystems. In order to ensure
reliability in these systems, it is imperative to
obtain and maintain accurate software and
hardware data. For this purpose, it is necessary
to test and re-test these systems under severe or
0-7803-6475-9/01/$10.00 2001 IEEE

DVECS was developed with the sole objective

of reducing the lead-time required for (1) the
tedious aspects of pre-testing software and
hardware before deployment and (2) the
collaboration of various AUVs without having
to consider the transportation of these vehicles.
Much of DVECS is based and developed on the
graphic test platform architecture for AUVs
926

uncontrolled environment is considerably safer

and cost-effective in a controlled synthetic
environment than a real environment, since the
research vehicles are not placed at risk of loss or
damage. Mission planning, monitoring and
analysis can also benefit from an interactive, 3D
virtual environment since its performance can be
tested prior to actual sea-trials. For these
reasons, the DVECS is used in hybrid synthetic
simulations for testing real and virtual vehicles
in a common environment and, for mission
collaboration, planning, monitoring and analysis
of existing UUVs. (Figure 2)

by Yuh, Adivi and Choi [4], and the SGI GL

based 3-dimensional graphics by Choi, Yuh and
Takashige [5].
DVECS utilizes a similar architecture of a
combined hierarchical and heterarchical
structure of the previous test platforms along
with UTs MVS simulation system; but DVECS
differs by utilizing a variety of different wireless
communications methods RF links,
commercial cellular telephones, wireless
Ethernet, wireless LAN, and asynchronous
transfer mode for data transfer. Finally,
DVECS incorporates a projection VR system
consisting of a RGB high-resolution, highrefresh-rate projector and polarized eyewear
with an emitter, and creates an immersion effect.

Since DVECS and MVS are based on a similar

concept and to achieve this collaboration effort,
DVECS architecture is designed to operate in a
networked environment such that each
component of the simulation can be run on a
separate system, processor or virtual machine
within a single computer; thus, distributing the
computation load.

ODIN
The Omni-Directional Intelligent Navigator
(ODIN) is the University of Hawaiis
experimental AUV as shown in Figure 1. The
electro-mechanical components were designed
for easy expandability and are in Table 1. [6]

This feature offers several advantages over a

single system layout. First, it is possible to
modify or create new components with minimal
restrictions on internal architecture. As long as
each component adheres to specifically
prescribed requirements for communication,
language and operating system, the design and
implementation of each component is irrelevant
to the rest of the system.

Table 1: ODIN Specifications (condensed list)

Component
Pressure Vessel
Main Computer
Com. Bus
Ext. Memory
RTOS
Power Supply
Thrusters
Stepper Motors
Manipulator
Communication
Sonar Ranger
Inertia System
Pressure Sensor

Specification(s)
Near-sphere shape; Al 6061-T6.
Motorola MC68040/33; 4MB DRAM.
3U-VME Bus; 12 slot chassis.
8 MB with 512 KB EPROM.
VxWorks 5.2 OS software.
24 Lead Gel batteries.
8 Cylindrical shaped with hood; Al
6061-T6;
Brushless
motor;
bidirectional propeller; torque-controlled.
X, Y & Z weight compensation control.
1-dof (90 motion); potentiometer.
RF Modem; open; RS232 connection.
8 channel range data (0.1-14.4 m).
INS 3-axis angle and heading rates, 3axis linear acceleration; RS232 input.
Calibrate between 1.5 to 8.2 V .

Second, users can design and test their AUV

simulations across the Internet using common
servers. This allows computationally intensive
3D simulation of interaction between multiple
objects in the virtual world to be simulated on
one or more centralized high performance
computers, while the processing requirements
for the users AUV simulations are no more than
what their physical AUVs would normally
require. Thus, this optimizes the computation
time and allows users to accurately evaluate
their vehicles computation performance and
requirements.

DVECS
The design, testing and operation of AUVs and
their control systems can benefit immensely
from interactive, 3-dimensional (3D) computer
simulations.
In particular, developing and
testing complex systems that involve multiple
autonomous underwater robots operating in an

Finally, multiple simulated or physical entities

can interact over a networked environment
without requiring them to share code or
knowledge of each others capabilities so
927

| Pitch | Yaw (x,y,z,,,) and an ENTITY ID,

which is an integer ID for the entity within the
DVECS universe for example, ODIN typically
uses 21. Messages can include things like
resize, translate, rotate, position and orientation,
range (for sonar), depth, etc.

proprietary or secured algorithms can be tested

in a common environment without making them
public. This allows for collaboration from many
different sectors of the underwater community
that wish to evaluate their AUV in conjunction
with pre-tested AUVs, as shown in Figure 3.
Communication

Wireless communication between a vehicle in

the field (ODIN) and DVECS, for use as a
monitoring or testing system, can thus be
accomplished either of two methods. First, it is
possible by a serial data link that connects to an
intermediate interface that relays data between
our TCP packets and the serial link, or second,
by a direct network connection over a wireless
link with an interface either on board the AUV
or at the test site. Either system is completely
transparent to the DVECS and the
communications delay created by adding
additional reflectors is typically negligible.

One key feature of DVECS is that multiple

simulated and real systems must be able to
interact with one another.
In order to
accomplish this, communication between the
various components is essential and critical.
Within the DVECS environment, UDP socket
DGRAMs or TCP socket streams convey
messages from one component to another.
These messages or data are used in the various
sub-systems vehicle graphic module,
background graphic module, numerical module,
navigation module, control module, &
communication module to exactly and
accurately determine the position of the AUV
within the virtual environment. In addition, a
simple network reflector interface that allows a
simulated entity within the DVECS to be
controlled by telemetry data from an external
component is used to garner data from ODIN,
the UH AUV. This interface can be used to
monitor a physical or simulated vehicle either
directly over a network or with an additional
software reflector if the communications system
does not include network support.

Typically, for ODIN, the transfer of data

between the test vehicle and the monitoring
laptop is via a RF modem and limited to 9600
baud. The transfer of data from the monitoring
laptop to the DVECS is via a cellular phone,
microwave connection, or wireless LAN. The
limitation of the wireless LAN is sight-to-sight.
Sensor Fusion
For interactive vehicle testing in a hybrid,
simulated synthetic environment, combining the
actual and the virtual sensor measurements
generates the synthetic sensor data. This is
easiest for range data, such as from sonar, in
which the synthetic range is simply the
minimum of the actual and virtual sensor data.

Currently, messages are converted to the use of

DGRAMs, since they offer a more optimal
solution where the loss of data packets is not as
critical as the delay such as wait till lost
packet is recovered or the suspension of a
stream causing noticeable latencies.
The
communication data, in DGRAMs or TCP
packets, contains the following information:

UUVs with even a modest number of sensors

can generate an enormous amount of data that
can be difficult for an operator or observer to
easily interpret. For this reason, it is desirable to
combine data from a variety of sensors and
present it in a clear, easily understandable way.
Within DVECS, it includes a sensor fusion
system that can apply sensor data to surfaces in
the virtual world that represents actual objects in
the real environment of the AUV.

< DESTINATION ADDRESS | TIME STAMP | ENTITY

ID | MESSAGE TYPE | OPTIONAL DATA >

The OPTIONAL DATA, as indicated here, depends

on message type and is usually the information
about the vehicle itself. For example, the
information to DVECS will be the 6 position
and orientation data that includes X | Y | Z | Roll

928

An obvious way to apply sensor data to the

surfaces in the virtual world is to use
tomographic methods, solving the inverse
problem for the sensors. Unfortunately, even for
sensors such as cameras under non-reflective
and non-refractive conditions where the light
paths are all straight lines between the sensor
and the object being sensed, directly solving the
inverse problem can be computationally
intensive and require a significant amount of
processing time. For other sensors, such as
sonar in water, where multi-path effects become
important and with refractive media, the paths
are not straight lines and directly solving the
inverse problem at reasonable resolution within
a reasonable amount of time is impractical on
current computers.

oriented with positive y up, horizontal left is

positive x. Given a point, p, on geometry G, if
the corresponding view plane then is
parameterized by the coordinates [u,v,L], for
fixed L and point p has coordinates [x,y,z] then
the parameterization of p has the simple form,

[u, v]T

L
[x, y ]T .
z

Although the actual parameterization will take

on a more complicated form for more
complicated sensor problems, in many cases,
this approximation can be used quite effectively.
Unfortunately, obvious problems arise with
discontinuities and multiple representation of a
given area in multi-path problems or in media
with variable refractive index. In principle,
however, many of these problems may be dealt
with by using image-processing techniques at
the stage of generating the textures. The sensor
fusion for ODIN within DVECS can be seen in
Figure 4.

In order to overcome these problems, the use of

texture mapping of images generated from the
sensor data was selected. DVECS achieves this
by generating a sensor space coordinate system
on the geometries, combining sensor data into
texture images and applying the textures to the
surfaces of the geometries.

Current Simulation
The water current simulation in DVECS is very
simple. It contains no turbulence and no curl,
and is totally isotropic. Basically, it takes a
fixed vector - an amplitude and a frequency and generates an acceleration based on:

There are several benefits of using texture

mapping over directly solving the inverse
problem. First, generating and manipulating the
texture images can be done with commonly
available image processing routines. This has
the advantages that these routines can be highly
optimized so processing time is minimal and that
these routines are well tested, existing code
considerably simplifying and shortening design
and implementation.

FD = F0 + Sin(t )v ,
where FD is the total disturbance, F0 is the
current direction, and v is the constant direction
vector that specifies perturbation direction (+/-).
This simplified assumption can be easily
modified to include cross-sections. Random
current simulation is being implemented;
however, accurate vortices and turbulences may
be difficult to generate.

Second,
computing
the
sensor
space
parameterization of the geometries typically
involves solving relatively simple equations at a
small number of points on the geometry so does
not require a significant amount of processing
time. For a simple optical camera in nonrefractive media imaging a non-reflective
surface, the form of this parameterization is very
simple. For simplicity, adopt a sensor space
coordinate system where the origin is the focal
point, forward distance from the focal point
away from the vehicle is the positive z direction,
up is positive y and when facing positive z and

Vehicle Dynamics
DVECS, when in a monitoring mode, does not
consider vehicle dynamics since transmitted data
directly reflects the motions of the vehicle in the
real-world environment. However, when in the
simulation mode, the interacting system must

929

The Main Control Panel allows access to

different environments or simulations, mission
controls (start, stop, pause), placement of grid
layout, modification of object labels, and control
of the sensor and thruster data. It also allows
monitoring of system messages, warnings, etc.

consider the basic underwater vehicle dynamics.

The following vector equation:

MV& + A(V )V + h = F
where V R 6 is the linear and angular
velocities in the vehicle coordinates; M R 6 x 6
is the inertia matrix; A R 6 x 6 includes all the
nonlinear dynamic terms with velocity terms;
and h R 6 is a vector representing other forces
and torques except F R 6 , which represents
the forces and torques generated by the thruster
forces. [7]

Finally, most commonly available high

performance graphics hardware includes texturemapping optimization, so scenes with complex
geometries and high resolution, high detail
textures are rendered in real time.
Conclusion
This paper presents a brief description of the
Distributed Virtual Environment Collaborative
Simulator (DVECS) developed at the
Autonomous Systems Laboratory of the
University of Hawaii. Various tests have shown
that the system can greatly help reduce the
development time of underwater vehicle
hardware and software testing and verification
as well as issues on multi-agent problems.
Further tests are scheduled to refine the program
to allow higher degrees of robustness.

Virtual User Interface

As mentioned before, DVECS uses a Silicon
Graphics workstation setup.
This dual
workstation setup comprises of an Onyx and an
Indy. It interfaces with an Electrohome Virtual
Reality Projection unit and Stereographics
CrystalEyes eyewear and emitter system.
DVECS software is a multi-layered C++
program modularized by its subsystems and
utilizes the inheritance properties.
It uses
OpenGL graphics libraries to generate the
background, vehicle and obstacles, and uses
Open Inventor 3D toolkit protocols to create the
3-dimensional, virtual images.

Acknowledgement
This project is partially supported by the
National Science Foundation Grant Number
INT-9603043 and partially supported by the
Office of Naval Research Grant Numbers
N00014-97-1-0961 and N00014-00-0629.

The DVECS consists of three windows. They

are the Main View Window, Main Menu Bar
Window, and Main Control Panel Window.

References

The Main View displays the virtual environment

with the vehicle being tested. The background
environment can be modified to represent an
area being used, such as the UH dive well, or
can used pre-mapped seafloor data to represent a
specific deployment area. The window allows
instantaneous change in viewpoints and
magnification.

[1]

[2]

The Main Menu Bar allows access to the

background, multiple vehicles or obstacles. This
is a simple, pull-down menu layout allowing for
access to a specific objects properties
dimension, location and attributes.

[3]

930

D.P. Brutzman, Y. Kanayama & M.J.

Zyda, Integrated Simulation for Rapid
Development of Autonomous Underwater
Vehicles, Proc. of the IEEE Oceanic
Engineering Society AUV92 Conference,
Jun. 1992.
Y. Kuroda, K. Aramaki, T. Fujii & T. Ura,
A Hybrid Environment for the
Development of Underwater Mechatronic
Systems, Proc. of the 1995 IEEE 21st
International Conference on Industrial
Electronics, Control and Instrumentation,
Nov. 1995.
S.G. Chappell, R.J. Komerska, L. Peng &
Y. Lu, Cooperative AUV Development

[4]

[5]

[6]

[7]

Concept (CADCON) An Environment

for
High-Level
Multiple
AUV
Simulation, Proc. of the 11th International
Symposium on Unmanned Untethered
Submersible Technology, Aug. 1999.
J. Yuh, V. Adivi & S.K. Choi,
Development of a 3D Graphic Test
Platform
for
Underwater
Robotic
Vehicles, Proc. of the 2nd International
Offshore
and
Polar
Engineering
Conference, Jun. 1992.
S.K. Choi, J. Yuh & G.Y. Takashige,
Omni-Directional Intelligent Navigator,
Underwater Robotic Vehicle: Design and
Control, TSI Press, NM, 1995.
K. Kawaguchi, C. Ikehara, S.K. Choi, M.
Fujita, W.C. Lee & J. Yuh, Design of an
Autonomous Underwater Robot: ODIN
II, The Proc. of the First International
Symposium on Intelligent Automation and
Control, FRANCE, May 1996.
S.K. Choi & J. Yuh, Experimental Study
of a Learning Control System with Bound
Estimation for Underwater Robots, J. of
Autonomous Robots, Mar. 1996.

Figure 2: DVECS development environment

Figure 3: DVECS and MVS Collaboration

Figure 1: ODIN in UH Pool

Figure 4: ODINs sonar representation in
DVECS.

931