Real-Time

Applications

KEY POINTS OF THE CHAPTER

l. Real-timesystemsare comPlex.
2. The first real-timeapplicationmay have been Project Whirlwind.
3. Real-timedatabasesare specialapplications.
4. Real-timeimaging is an importantbut challengingapplicationarea.
5. Real-timelanguagesmay not be suitablefor "real" applications.

It is appropriateto closethe text with a discussionof severalinterestingspecialty

areasfor real-time systemsdesignersand somereal-timeapplications.

AS COMPLEXSYSTEMS
SYSTEIvIS
14.1 REAL.TIME

Although tremendous problems are involved in building real-time multi-

processing systems (all the problems of uniprocessingsystems plus inter-
communication,resourcesharirg, load balancing,etc.), most real-time systems
are usually distributed heterogeneousmultiprocessingsystems.This is true in
most military applications and in industrial applications where standard
componentssuchas RISC workstationsconnectedby local areanetworks(LANs)
and wide area networks (WANs) are the implementations of choice. Special-
purpose architectures,such as those discussedin Chapter 72, are increasingly
found to be noncompetitive over the life of a system when compared to
approachesthat use off-the-shelfhardware.

315
316 Chap. 14 I Real-TimeApplications

It is increasinglybeing recognizedthat the constructionof large-scale,real-

time systemsis not just about real-timesystemsand scheduling.The fact is that
most large, modern,practical computer applicationsembody many propertiesthat
arecurrentlybeing studiedlargelyin isolation.For example,most combatsystems
are real-time and distributed, seekto be reliable, and are fault-tolerant.Although
it is important to study these issuesindependently,collectively these properties
have a complexinterrelationship.Globally,thereis increasingactivity in the area
of "complex systems"as a new, multidisciplinaryfield of study.In a sense,this
book is an attemptto discusssomeof the areasthat the complexsystemsdesigner
(no longerthe "real-time" systemsdesigner)will needto understand.Throughout
this text it has been argued that a holistic approach is the only way to build
complex, distributed,real-timeapplications.

14.2 THE FIRST REAL.TIMEAPPLICATION?

In 1947MIT, in conjunctionwith the U.S. Navy, begandevelopmentof Project
Whirlwind (see Figure 14.1). This projept was the first high-speedelectronic
digital computer able to operatein "real-time" and to be put to use as a practical
device. The goal of the project was to develop an airplane trainer/analyzerthat
would simulate aerodynamicforces acting on the pilot's controls. The pilot's
reactionswouid have to take effect as promptly in the simulator as they would in
a real airplane. Thus, the equipment had to operate within these "real response
times" [32]. Whirlwind was begun by Jay W. Forrester,who soughtto develop

Figure 14.1 Photograph of the U.S. Navy's Whirlwind computer. (Courtesy

of Jay Forrester and the MIT Museum.)
Sec. I Real-Time Imase Databases 317

an airplane trainer/analyzerthat would simulate aerodynamicforces acting on the

pilot's controls.The pilot's reactionswould have to take effect as promptly in the
simulator as they would in a real airplane. Thus, the equipment had to operate
within these "real responsetimes." We might also argue that Whirlwind was
"complex" becausein addition to speed,reliability was also of greatimportance,
and it was during this project that the military conceptof "command and control"
was introduced|32).
Whirlwind's features were made possible through the use of ferrite cores,
which were faster than the cathode-ray tube memories of previous systems.
Ferrite coresof the samekind were also used for computer memoriesin the 1950s
in the United Statessuch as the 1953 UNIVAC 1003 model, the 1954IBM 704,
and others[75]. Another distinctivefeatureof Whirlwind was the inclusion of an
"algebraic compiler," an early pseudoprogramminglanguageresulting from the
work of Laning andZierler [751.
During the project's final stages,the Whirlwind team became assimilated
into the Air Force's Semiautomatic Ground Environment (SAGE) air defense
system project [132]. As a result, the basic ideas of Project Whirlwind were
incorporated into SAGE. Thus, while it is widely held that SAGE was the first
real-time application, Project Whirlwind seemsto deservethis credit.

14.3 REAL.TIME
DATABASES

An important application area for real-time systemsis in databases.A real-time

databasesystemis a transactionprocessingsystemin which transactionsand data
have explicit timing constraints (deadlines). Real-time databasesare used in a
variety of applications, including process control, radar systems, computer-
integrated manufacturing, traffic control, virtual reality, and multimedia.
Designing real-time databasesinvolves

1. Analyzing the system requirementsto identify data

2. Determining the hardware capabilities
3. Identifying the different data criticalities and integrity constraints

Ordinary database management systems provide concurrency control

techniquesthat seek to preserve data consistency (e.g., read and write locking)
and logical consistency(e.g.,two-phaselocking). Real-timedatabasesmust also
impose concurrency control that preserves the schedule and data temporal
consistencyrequirements.As with most real-time systems,minimizing average
response is less important than guaranteeingthe timing constraints of critical
transactions.In the caseof databasesystems,serializableexecution of concurrent
transactionsmay also be of less interest.
A rcal-time databasesystem must operate within the context of the bsr
operating system and use existing services. Hence, construction of a real-time
318 Chap. 14 I Real-TimeApplications

databaseis nearly impossiblewith non-real-time.operating systems,and most

real-timedatabases are constructedon top of specialreal-timeoperatingsystems.
Consequently,commercialreal-timedatabasesystemshave yet to emerge.Most
of the available "hard" real-time databasesuse proprietary algorithms and are
suitablechiefly for softer real-timeapplicationswhere transactiondeadlinesare
"negotiable."
As an exampleof a real-time database,the architecturefor the U.S. space
station"Freedom" is shown in Figure l4.2.The primary role of this systemis to
provide integrateddataprocessingand communicationservicesfor both the core
function and payloads.The systemneedsto be robust in that it must support30
yearsof continually changingdata processingand networking needs.

Figure 14,2 Data managementsystem of spacefreedom.

The system consists of a Fiber Distributed Data Irfterface (FDDI) ring.

Sel eral standard data processorsare attached to the core network, along with
specializedhardware such as communication and tracking hardware, defined as
orbital replaceableunits (ORU). Thesedevicesare then interfacesto other devices
via the Mil-Std-15538 bus.architecture or via multiplexer/demultiplexer (MDM)
units. The standarddata processoris actually a multiprocessingsystemconnected
via Multi Bus II and containsa network interfaceunit (NIU) that runs the network
operating systemand a bus interface unit (BIU) that interfaceswith the MDMs or
the ORU hardwarethrough the 15538'bus.The embeddeddata processor(EDP)
runs a version of real-time UNIX (LynxOS), which is POSlX-compatlble.This
complex, multiprocessingreal-time system is mission critical, and simulations
have shown that it must handle approximately 5000 databasequeries in a period
Sec. 14.,4I Real-TimeImase Processins 319

of 6 seconds.Variousschedulingdisciplines,including rate-monotonicare being

investigatedto optimizeperformance.A detaileddescnptionof the s1'stemcan be
found in [28].

14.4 REAL.TIME IMAGE PROCESSING

Real-timeimage processingdiffers from "ordinary" image processingin that the

logical validity of the systemrequiresnot only correctbut also timely outputs.It
is often said that image processingin real-timeis no more than a minor variation
on imageprocessingwithout regardto time; fastermachinerywill makeany speed
problems eventually go away. But real-time imaging is not just about speedy
hardware.It involves at least three fundamentaltradeoffs:performanceversus
image resolution,performanceversus storageand input/outputbandwidth, and
number of tasks versus synchronization.Of theseproblems,only the first and
possibly the second might be solved by faster machines. Furthermore,the
problems of expressing image algorithms (especially for multiprocessing
architectures),of finding appropriateprogramming languages,of testing and
reliability, and of practicalsoftwareengineeringtechniqueare not readily solved
simply with fasterhardware.
Becauseof its nature,thereare both supportsfor and obstaclesto real-time
imageproeessing.On the positiveside,many imagingapplicationsarewell-suited
for parallelization and hence faster, parallel architectures.Furthermore, many
imaging applicationscan be constructedwithout using languageconstructsthat
destroydeterminism.Finally, specialreal-timeimaging architectures are available
or can theoreticallybe constructed.
On the down side, many imaging applicationsare time critical and are
computationally intensive or data intensive. And there are no standard
programminglanguagesavailablefor real-time image processing.Finally, real-
time processing science itself is still struggling to produce usable results,
especiallyfor parallel processingmachines.
Real-time image processingcovers a multidisciplinary range of research
areas,including (but not limited to) image compression,target acquisitionand
tracking, remote contrel and sensing,image enhancementand filtering, network-
ing for real-time imaging, advancedcomputer architectures,computer vision.
optical measurementand inspection,and simulation.These researchar€asare
critical in such applicationsas robotics, virtual reality, multimedia, industrial
inspection,high-definition television,advancedsimulators,computer-integrated
manufacturing, and intelligent vehicles.
In summary,real-time systemsare unique in that semanticvaliditl'entails
not only functionalcorrectnessbut also deadlinesatisfaction.A generalizedreal-
time image processingsystemis shown in Figure 14.3.It functionsby taking an
analogvideo input from a Qameraor sensingdevice and digitizing it. The image
is then enhancedor filtered to remove noise. If obiect or motion detectionis
320 Chap. 14 I Real-TimeApplications

Figure 14.3 A generalizedreal-time image processingsystem.

required,thatprocessis applied.If appropriate,an object recognition algorithm or

otherdecisiorilogic is appliedto the modified image.In most cases,someson of
image display is also required. For many applications,these steps must be
executedin real-time-for,example, at the 30 frames per secondrate neededfor
continuous motion PercePtion.

14.4.1VirtualReality :
To investigateone real-time imaging application briefly, consider virtual reality.
Virtual reality (VR) systemsare complex computer simulations involving visual,
audio, tactile, and other feedbackqo entice a person'sperceptualmechanismsinto
believing they are actually in an \rtificial world. Although virtual reality has
obvious applications in combat simulation and training, its most promising
applicationsare civilian, including exerciseand recreation,physical rehabilitation
and therapy,occupationaltraining, and psychological diagnosistraining. In these
types of systems, high performance and deadline guaranteesare essential to
ensurethat the illusion of reality is maintained'
Virtual reality systemscome in at least three paradigms:

I Telepresence-where a human operator can remotely control robits or

other devices as if the operatorwere physically present.For example, an
individual can don a special suit that would pefmit the repair of an
underwaterstructurevia a robot whose movernentsreflected those of the
operator. The operator "thinks" that they are underwater and are working
on the real sffucture.
I Immersion:-where a simulator is implementedthat can actually fool the
sensesof the human subject into believing that the simulated scenariois
 14.4 I Real-TimeImage Processing 321

real. Fansof televisionsciencefiction programs,suchas "Star Trek: The

Next Generation,"are familiar with this type of virtual reality.
r Augmentedreality-where a displayof live informationis supplemented
with simulatedinformation.A simulatedX-ray view of a patient'sbody
superimposedon a monitor display of the live patient for diagnostic
purposesis such an examPle.

A generalizedarchitecturefor eachparadigmis shownin Figure I4.4.In case(a),

telepresence,a human operatorinteractswith a helmet-mounteddisplay, hand
controls,and foot leversconnectedto the VR processor.The processorthenrelays
the control information via satellite to a remote ship. The ship is attachedto a
tetheredrobot that movesin accordancewith the human'smovements.The human
also receives feedback information from the robot in the form of visual images
and possibly force fqedbackon the hand or foot levers.
In case (b), immersion, a specially designed room provides a human
participant with simulated images (criminals and bystanders)through a helmet-
mounted display. Motion sensorsin the ceiling and light reflectors on the floor
help to track the participant's movement in order to aid in generatingthe correct
perspectivefor the 3-D images in the room.
Finally, in case (c), augmentedreality, a diver is equippedwith a goggle-
mounted display that augments the real-world view of the submarine with
computer-generatedoverlays that give the diver information about the inner
workings of the ship for the purpose of repair.
In each case, extensive run-time support is needed for high-resolution
graphics,helmet-mounteddisplays,digital cameras,motion sensors,force and
tactile feedbackperipherals,and other devices.
BecauseVR systemsrepresentcomplex,multitasking,graphicssimulations
that involve control and tracking problems,they are perhapsthe most challenging
of all real-time systems; using today's technology, perfect results are not
obtainable. Virtual reality also involves concurrent programs and processors,
shared peripherals, and the important notion that synchronization is at least as
important as timeliness. For example, in virtual-reality-type flight simulators,
even a slight skew in the synchronizationof a pilot's commandsand the resultant
display update(e.g.,a turn is made) can causenausea'

14.4.2 Multimedia
Advances in multimedia computers have significantly affected research in
multimedia computing. Multimedia systems can be collectively viewed as a
combination of voice, images, animation, full-motion video and audio. The
prim4ry area. of research in multimedia computing deals with multimedia
hardwareand software systems,as well as tools developmentfor computer-aided
leaming (e.g., encyclopedias,interactive games). In particular, much research
focuses on distributed multimedia systems. Distributed multimedia systems
1)') Chap. 14 I Real-TimeApplications

Figure 14.4 Virtual reality paradigms: (a) Telepresence,(b) immersion, (c) augmentedrealr-u.
I Real-TimeImage Processing 323

require synchronization,large storage, indexing, and retrieval of data. For

example,as with VR, in multimedia applications,it is clear that audio speech
output must be in synchronywith the image of a personspeaking.
Multimedia systems are complex systems that need to deal with high
processingpower, high-speednetworks,large storagedevices,video and audio
compressiontechniques.real-time operating systems, software development,
storageand retrievalof data,paralleltechniques,and architecturesfor distributed
environments.Large-scalemultimedia storagewill be an integral part of the
emergingdistributed multimedia cofilputing infrastructure.Work has beendone to
provide high bus bandrvidth afrd a high-performance multimedia real-time
operating system that supports real-time scheduling. In the software tools
development,researchhas beencenteredarounda new object-orientedlanguage
paradigm.A major concernin multimediasystemis audioand video compression.
An importdnt consideration in constructing deterministic, predictable, and
synchronizedmultimedia systemsis whether the compressiontechniquesare
deterministic.
A generalizedmultimediaarchitectureis given in Figure l4.5.In this design,
compressgdand uncompressedaudio and video (and control inputs such as
timing) are input into the system.A local databasemay be accessedfor the
purpose of decompression(such as in vector quantizationtechniquqs)or to
provide local information to supplementthe system(e.g.,backgroundsOgnery).

Controlinputs

Multimedia
processrng
sysrem

Figure 14,5 A generalizedmultimedia architecture.

324 Chap. 14 I Real-TimeApplications

The processingsystemmanipulatesthe data in a variety of ways (for example,

considerthe foreign film systemshown in Chapter5), but ultimately the system
outputsvideo and audio.

14.5 REAL.TIME UNIX

An important real-time "application" is not really an application at all but an

operating system.However, becauseof its commercial importance,the UNIX
operatingsystemand its real-time relativesare of great interestto applications
developers.
Real-time UNIX solutions seek to guarantee response times thrt -:r
considerationsin <ineof the following areas:kemel features,memor) rrrj,i-lr-
ment, real-time I/O, and user tools.
The standardUNIX kernel is not consideredamenableto real-timebecause
it doesnot provide for preemptiveschedulingand becausethe kernel servicesare
not re-entrant.Real-time UNIX kernels seek to guaranteeresponsetimes by
providing a re-entrantand interruptiblekernel and preemptivepriority scheduling.
Good interprocesscommunicationand synchronizationare also necessary.
Real-timeUNIX alsoprovidesreal-timememory managementby providing
data integrity for all data structures.Some real-time kernels use over 3000
semaphoresto protect as many data structures.(Is there a potentialfor deadlock
here?)Also availableare preallocationof files to preventdisk fragmentation(and
its insidiousproblems)and processlocking in memory to reduceswap times.
Many real-time UNIX versions also provide user tools in the form of
programmercontrol of systemresources(not afforded by ordinary UNIX) and
rate-monotonicschedulingtools basedon spreadsheets. An architecturefor one
real-time UNIX is shown in Figure 14.6. For an excellent discussionof this
implementationof real-timeUNIX, see[48].

14.6 BUILDING REAL.TIME APPLICATIONS

WITH REAL.TIME PROGRAMMING LANGUAGES

Over the last 20 yearp,and especiallyin the last 10, severalspecializedreal-time

languageshave been developed.It is probablytrue, however,that most real-time
code is written in C, Ada, FORTRAN, and assembly language, with C++ and
Ada95 gaining rapidly. Nevertheless,certain applicationsare built using specialty
languages,and lheir suitability for the task merits some discussion.
As we mentioned briefly in Chapter 3, real-time programming languages
seek to provide for schedulability analysis in one of three ways:

T To eliminate nondeterminancy
I To augment the languageto provide explicit timing and control
Sec. 14 4 B Building Real-l-irneApplicationswith Real-TimeProgrammingLanguages 325

user level
Kernel level

Kernel level
hardware level

Figure 14.6 One architecturetbr real-time UNIX see [48].

r To constructthe languagejointly with the operatingsystem(e"g.,POSX

compliant languages)

Someof the languagesthat have been deveiopedover the last 20 yearsto handle
real-timeapplicationsinclude

r Ada95: A language that seeks to addressthe real-time problem b1

augmentation(in this case of the standardAda lan-sua-eeiThis uas
discussedin Chapter3.
ts occam-2:A programminglanguagedesignedfor use on translrutersthat
uses augmentationto provide some explicit timin,s and control. This
languagehas seencontmercialuse in the UK.
326 Chap. 14 I Real-TimeApplications

PEARL: The Processand ExperimentAutomation Realtime Language

developedin the early 1970sby a group of German researchers.Pearl
uses the augmentation strategy and has fairly wide application in
Germany,especiallyin industrial control settings.
Real-timeEuclid:An experimentallanguagethat enjoysthe distinctionof
beingone of the only languagesto be cornpletelysuitedfor schedulability
analysis.This is achievedthroughlanguagerestriction.Unfortunately,the
languagehas not found its way into mainsfrainapplication.
Real-timeC: Actually a genericname for a variety of C macroextension
packages.These macroextensionstypically provide timing and control
constructsthat.arenot found in standardC.
r Real-time C++: A generic name for several object class libraries
specificallydevelopedfor C++. TheselibrariesaugmentstandardC++ to
provide an increasedlevel of timing and control.
r Java: A languageused frequently in World Wide Web pages.It was
originally intendedfor embeddedsystemslike consumerelectronicsand
is still gaining acceptancein that domain. Java combines a virtual
machine (an interpreter)with a C++-like languageand an associated
compiler that convertsthe sourcecode into interpreterdirectivescalled
byte codes.The advantageof this approachis that the code is machine
independent(i.e., it is targetedfor the virtual machine)-hence its use in
the heterogeneous world of the Web. In someways, Java is a greatreal-
time language.It allows for multithreading (with POSIX compliance
when running on UNIX platforms), provides explicit synchronization
rnechanismssuch as spinlocks,and is object-oriented,but without such
drawbacksas garbagecollectton.

Thereare,of course,many otherreal-timelanguages/operating environmentswith

nameslike MACH, EIFFEL, MARUTI and ESTEREL that are widely referenced
in the literature. Many of these languagesare used for highly specialized
applicationsor in researchonly. For a thoroughdiscussionof real-timelanguages,
see[156].

14.7 EXERCTSES

l. Researchthe real-time programming languagesdiscussedin this chapter and write a report

comparingand contrastingtheselanguageswith respectto the languagef'eaturesdiscussed
in Chapter3.
2. From the research literature, identify and discuss at least three real-time languages not
mcntionedin this book.
3. Using any of the languagesabove,constructa softwaremodel for the systemdescribedin
Section 13.5.Many compilersfor theselanguagesare availablefor fiee on the Intemet.