Avionics sensor simulation and prototype design workstation James Falasco, GE Fanuc Embedded Systems

1240 Campbell Road

using COTS reconfigurable computing technology Richardson, Texas 75081

Abstract
This paper reviews hardware and software solutions that allow for rapid
prototyping of new or modified embedded avionics sensor designs, mission
payloads and functional sub assemblies. We define reconfigurable
computing in the context of being able to place various PMC modules
depending upon mission scenarios onto a base SBC (Single Board
Computer). This SBC could be either a distributed or shared memory archi-
tecture concept and have either two or four PPC7447 A/7448 processor
clusters. In certain scenarios, various combinations of boards could be
combined in order to provide a heterogeneous computing environment.
Keywords: Sensor simulation, reconfigurable computing, video compres-
sion, video, reflective memory, payload integration, sensor fusion, high
speed data acquisition, video surveillance
1. Introduction
The design of avionics sensors and mission payloads can be a complex and
time-consuming process. However, the design cycle can be significantly
shortened if the system designer has access to a flexible, reconfigurable
development environment that closely mimics the capabilities and tech-
nologies of the deployed system.
By integrating various PMC modules with a scalable, reconfigurable multi-
processor architecture, it is possible to create a development tool that will
allow the system designer to rather quickly and accurately simulate and
of embedded development at this early stage. The COTS SBC shown has the
ability to host two PMC modules per card. The Video card can efficiently
manage real sensor input coming in from an E/O sensor/camera, and also
display that data in a fused fashion. This approach allows the system
builder to work in the lab and the field using the same hardware/software
environment. This system configuration also provides the developer with a
test bed to define/design new hardware requirements and processing
streams.
3. Prototyping System Structure
Modern embedded avionics sensor design is primarily driven by the goal of
providing a net-centric flow of data from platform to platform. The achieve-
ment of this vision depends on transferring and processing vast amounts of
data from multiple sources. Let’s examine how one could use this approach
to control and manage various sensors in a rapid prototyping environment.
As depicted in Figure 1, the host server for the embedded avionics sensor
design workstation is a COTS Dual 7447A PowerPC VMEbus SBC. The board’s
architecture provides a distributed processing environment that allows
scaling of multiple processing nodes to achieve high performance for the
most demanding signal and imaging applications associated with various
sensor and mission payload design requirements.
RS170 PMC
Up to Up to
for payload packages as well as the sensor itself and in its actual deploy-
ment integration. Most importantly it will allow for cost-effectively
maintaining and extending the system when new technologies are avail-
able in the future.
3.1 An Example: Infrared (IR) scene projection
Increased sensor performance and bandwidth has begun to overwhelm
existing backend processing capability. Current test and evaluation
methods are not adequate for fully assessing the operational performance
of imaging infrared sensors while they are installed on the weapon system
platform. However, the use of infrared (IR) scene projection in test and
evaluation can augment and redefine test methodologies currently being
used to test and evaluate forward looking infrared (FLIR) and imaging
IR sensors.
One could project accurate, dynamic and realistic IR imagery into the
entrance aperture of the sensor, such that the sensor would perceive and
respond to the imagery as it would to the real-world scenario. This
approach includes development, analysis, integration, exploitation, training,
and test and evaluation of ground and aviation based imaging IR
sensors/subsystems/systems. This applies to FLIR systems, imaging IR
missile seekers/guidance sections, as well as non-imaging thermal sensors.
The systems approach proposed in this paper has the scalability to accom-
plish this type of reconfigurable sensor mix and match. Algorithms such as
FPGA PMC the one depicted in Figure 2 could be transformed into a Mathlab®
Simulink® Blockset for easy execution in a reconfigurable system utilizing
3.2 The advantages of FPGA Computing 3.3 The ‘Mix and Match’ Platform Advantage 4. Conclusions
As the development tools have evolved, the core-processing platform has Today’s soldier, who is the ultimate customer of the embedded systems The goal of integrating a COTS system such as the one depicted here is to
also been enhanced. These improved platforms are based on dynamically designer, is faced with a continuously fluid chain of world events. These allow designers to use the same environment in the lab that they could
reconfigurable computing, utilizing FPGA technologies and parallel changing events are closely mapped into the deployment of various air, sea then take to the field for live data collection activity. This is of particular
processing methods that more than double the performance and data and land platforms which contain the reconfigurable embedded systems value to a system engineer who could use such an environment to perform
bandwidth capabilities. This offers support for processing of images in architecture under discussion in this paper. For example, based on new development activities. Because of these efficiencies, the
Infrared Scene Projectors with 1024 X 1024 resolutions at 400 Hz frame changing mission requirements, any of the following three different areas outlined prototyping system would pay off in accelerated product develop-
rates. Simulink Blocksets could add the programming involved to organize might be needed by the soldier: sonar processing; SIGINT; or ment time.
algorithms that could then be partitioned to operate on a multiprocessor video surveillance.
Systems designers are traditionally faced with the challenge that today’s
based FPGA/PPC based configuration.
The system designer addressing these defined application areas could sensors are generating data at a rate far faster than the backend end
Another key component of reconfigurable scalable embedded avionics place various PMC modules together in the prototyping system with relative
systems are configured to process. Combine this fact with the reality that a
sensor design and prototyping capability is access to FPGA based ease. Specifically, these applications might collectively require the following
sensor fusion “paradigm” is now mandatory for designers wishing to turn
processing. FPGA (Field Programmable Gate Array) is defined as an array of list of PMC modules: PMC #1—Graphics; PMC#2—Video Compression;
concepts into reality rapidly. A key component to a flexible hardware
logic blocks that can be ‘glued’ together (or configured) to produce higher PMC#3—Reflective Memory; PMC#4—1553; PMC#5—High Speed Data Acqui-
system, of course, is a software structure that enables designers to go from
level functions in hardware. Based on SRAM technology, i.e., configurations sition; PMC#6—Race++. A key cornerstone of the ‘mix and match’ strategy is
their ideas and algorithmic concepts to code or HDL.
are defined on power up and when power is removed the configuration is that while the PMC module may change from application to application, the
Packaging of the system is largely dependent on the user’s requirements
lost – until it is ‘reconfigured’ again. Since an FPGA is a hardware device, it is core SBC ‘multiprocessor’ engine and its associated software tool chain
for flexibility. Should the user desire a system that can be scaled up by
faster than software. remains constant. The same can be said for the VME enclosure that
adding additional cards, then a larger slot chassis could be configured to
contains the processing cards.
The FPGA can best be described as a parallel device that makes it faster allow for the addition of other cards. The key point is that the core
than software. FPGAs as programmable “ASICs” can be configured for high Let’s now examine the application areas and understand how we could hardware outlined not only has the potential for scalability by adding addi-
performance processing, excelling at continuous, high bandwidth applica- place various PMC combinations together to address the processing
tional modules to the base processing units, but the entire system has
tions. FPGAs can provide inputs from digital and analog sensors —LVDS, requirements of each application. scalability as well.
Camerink, RS170 — with which the designer can interactively apply filters,
3.4 Sonar Processing: PMC #1—Graphics, PMC #5
do processing ,compression, image reconstruction and encryption time of
—High Speed Data Acquisition

test with real data sets the various sensor designs and mission compo- 1 GHz
7448 or
320 MB/s
Peak
1064 MB/s
Peak 1 GHz
7448 or

applications. Examples of the flexibility of this approach using COTS PMC

32 MB VME Bus 32 MB
7447A PMC 1 7447A
User Flash P1 User Flash
PowerPC

FPGA- and PPC-based computing elements. Embedded avionics In this example the PMC Graphics card would be utilized to display sonar
PowerPC

nents to be fielded. 64-bit/ 2eSST PCI-X

64-bit/

Modules hosted by a COTS multiprocessing base platform are shown in

133 MHz 133 MHz
VME Bridge Bridge

NVRAM
PPC to 64-bit/133 MHz PPC to
NVRAM
HWIL Simulation demands include low latency, high data rates and waterfall display data perhaps with an overlay of tactical positions. The
Specifically, we present a rapid prototyping and rapid evaluation system PCI-X Bridge
(64360)
64-bit/133 MHz
PCI-X Bridge
(64360)
Figures 3 & 4.
interfacing. It is essential to have a capable platform for handling and High Speed Data Acquisition PMC could be used to facilitate the information
256 MB DDR 256 MB DDR

that will simplify the establishment of performance requirements, and allow Gigabit Multifunction
PCI-X
Bridge Gigabit Multifunction

processing of the data streams. Tools must also complement this so that a coming in from a high speed sensor interface. The combination of the two
Ethernet Serial Ethernet Serial
Up to

the quick evaluation of hardware and software components being

1064 MB/s PMC 2
Peak

Video Graphics PMC systems designer is able to construct the final system leveraging design modules hosted by the multiprocessor-enabled SBC could move
considered for inclusion in a new sensor system design or a legacy Compression
PMC Video Input
Input 1 MPEG-4
MPEG-4 Compression Bit-stream
tools, such as Mathlab and Simulink, with a reconfigurable Processing from this application area to another type of sonar processing, triggered by
platform upgrade.
Input 2
MPEG-4 Compression
1553 Technology
computing platform. TS-PMC with Stratix FPGA WAVE
Software
software or the base modules could be switched to those shown in the
Chassis Technology Drivers
(PCI Bus)
SIGINT example below.
2. An Open Systems, COTS Platform
Video Server CPU
Display Technology
Video Server Application

The system hardware and software used to evaluate sensor designs and
Server Configuration via TCP/IP
3.5 SIGINT: PMC #6—Race ++, PMC #3—Reflective memory
Fig. 1. FPGA vision and graphics platform RTP Encapsulated MPEG-4 Bit-stream

mission payload components and algorithms should be open and recon- In a SIGINT application scenario a RACE ++ PMC module is tied to other
FPGA PMC + Dual RS-170
Mezzanine + Video Compression Encoder
with Host Decode = Video Compression PMC
Integrated Bundle
Race++ peripherals allowing for interfacing while the reflective memory
figurable to allow for the mixing and matching of various vendor offerings.
Provision for hardware independence is critical, since the hardware is very module stores real time acquired data that could be used for post
The COTS SBC architecture combined with the FPGA/Video cards allows for Target

likely to rapidly evolve at the pace of new computer technology. The

Window processing data analysis.
Average
seamless mapping of imaging applications oriented toward change detec-
software infrastructure should be scalable and flexible allowing the algo- Multiprocessor SBC
3.6 Video Surveillance: PMC #2—Video Compression, PMC #4—1553
tion and sensor fusion which will allow the systems designer to view Band
Demean Cross Background/
Interleaved Input Matrix RX Image
rithm developers the ability to spend their time and budget addressing the Image Format Window Product Guard
Processor Output In the video surveillance application area, the video compression PMC
multiple data streams in a simulations real time display environment. Input Averager Generator
Window
Averager Fig. 3. Example: FPGA application. PMC FPGA processor for capture and
important functionality and usability aspects of the systems design. The module would pre-process and reduce the incoming data stream and pass
compression. PMC module for graphics and display.
system proposed here, a test and evaluation workstation built around The SBC implements processing nodes using the latest Motorola
it to the multiprocessor base system for potential change detection
reconfigurable hardware and a component-based software toolset, 7447A/7448 PowerPC® processors running at up to a 1.4 GHz clock rate.
analysis. Then using the PMC 1553 module, one could communicate to an
provides the necessary tools to ensure the success and cost effectiveness Using the distributed processing architecture of the SBC, one bridge chip
external avionics platform to perhaps control or guide ordnance
of initial sensor design and payload development. per node allows the Front Side Bus (FSB) of each PowerPC to run in MPX TS-PMC GPU PMC
Fig. 2. Example: sensor images Video Camera Capture Thin Pipe Network Overlay being placed upon the target under surveillance.
Audio Input and and
mode up to its maximum rate of 133 MHz. The high performance data Compress Display

At the center of any scalable prototyping system is a reconfigurable multi- In each of the three examples above, the modules are interchangeable
transfer mechanism facilitated by the built-in 64-bit full duplex crossbar in • Capture
• Filter, Format
• Graphic Generation
• Overlay
processing CPU engine with associated memory. The system depicted in • Encrypt • Display
depending upon the specific mission. This approach would allow the soldier
the Discovery II bridge permits concurrent data transfers between different • Compress

Fig. 1 provides developers a scalable environment for application design This approach allows one to demonstrate how algorithms can be imple- to move module packages between platforms achieving different mission
interfaces as well as transaction pipelining for same source and
and test. A COTS single board computer (SBC) tightly integrated to a PMC mented and simulated in a familiar rapid application development Multiprocessor SBC
scenarios through software loading while maintaining the core base multi-
destination transactions. Video Camera
TS-PMC
Capture
TS-PMC
Capture
FPGA card for design experimentation forms the core system. Other PMC environment before they are automatically transposed for downloading Audio Input and
Compress
and
Compress processing and packaged environment.
modules can then be selected depending on the type of sensor input that The prototyping architecture outlined here is based on a combination of Encrypter Wireless
directly to the distributed multiprocessing computing platform. This • Capture
Network
• Graphic Generation
• Filter, Format • Overlay
needs to be processed. tightly integrated reconfigurable computing, video and graphics. It will • Encrypt • Display
complements the established control tools, which usually handle the • Compress

place the embedded avionics sensor design community in position to utilize

One of the main advantages of this approach is the ability to rapidly proto- configuration and control of the processing systems leading to a tool suite
the proposed system architecture in development of the actual controller Fig. 4. RS-170/MPEG-4 video PMC. Integrated bundle of PMC, mezzanine and
type mockups for test and evaluation without a concern for the limitations for system development and implementation.
MPEG-4.

Representative Sensor Solving The Bandwidth Problem Mix & Match Platform Advantage Example 1 Mix & Match Platform Advantage Example 2
Sonar Processing SIGINT
MICROCONTROLLER
ELECTRONICS
Race ++ PMC Module
CCD IMAGING CHIP PMC Module Reflective Memory PMC
AND VIDEO ELECTRONICS

Module
INTENSIFIER
HIGH VOLTAGE Collect & Process Data from Multiprocessor BSP
POWER SUPPLY suspension subassembly
INTENSIFIER
Adding Applications Layer
FOCAL BACKPLANE Data Acquisition Sensor
FILTER WHEEL
OPTICAL FILTER Step 1 Step 2

Data acquired in real time Data is passed to PMC module to be

processed & on Nexus Single Board
Computer for backend processing &
matching for decisions (mission driven
scenarios)
Video Rate Camera Up to
320 MB/s
Up to
1064 MB/s
1 GHz Peak Peak 1 GHz
• 30 frames per second 7448 or 7448 or
32 MB 32 MB
• Average image size (after registration):
7447A
PowerPC
User Flash
VME Bus
P1
PMC 1 User Flash
7447A
PowerPC
Step 3
642 x 343 pixels (8 bit) 64-bit/ 2eSST PCI-X
64-bit/
The process of the first two steps allows
133 MHz 133 MHz
VME Bridge Bridge
the system developer to combine real
Spinning Filter Wheel NVRAM
PPC to 64-bit/133 MHz PPC to
NVRAM world data with simulated scenarios and Graphics PMC Module
• Creates 6-band multispectral image
PCI-X Bridge PCI-X Bridge compare and contrast in order to hone
256 MB DDR
(64360)
64-bit/133 MHz
(64360)
256 MB DDR overall system design efficiency High Speed Data Acquisition PMC Module
PCI-X Multiprocessor BSP
UAV Application Examples
Gigabit Multifunction Bridge Gigabit Multifunction
Ethernet Serial
Up to
Ethernet Serial
Graphics SW Layer (OpenGL)
1064 MB/s PMC 2
Peak
Adding Applications Layer
Battle Damage Assessment LADAR
Electronic Surveillance Data Link Compression
Cross-cueing Sensors All Weather Target Acquisition
FOPEN Radar Sensor Fusion
Minefield Detection Mission Planning

SIGINT Reconfigurable System Environment Mix & Match Platform Advantage Example 3 Overcoming Data Compression Bottlenecks
Video Surveillance
Pentium enabled SBC’s
Signal Analysis Function (SAF) 1553 Data link
VMIC Product Line
Real-Time Capabilities 1553 PMC
Real-time Signal Detection
Signal Tracking
Digital Capture of
Signal Data
Near-Real-Time Capabilities
Automatic Detection Video Compression PMC PPC enabled SBC’s single,dual
& quad configurations
Conclusions
Operator Event Marking Data Tapes Signal Localization
Display and Analyze Results
Multiprocessor BSP Richardson Product Line
Signal Classification Existing Platforms & New Designs Will Continue Flying For Decades
Fusion of Data Operator in the Loop Adding Applications Layer
Display Capability Incremental Mission Payload Responsibilities Will Be Added
Incoming Information Pre-Processed- CDI Product Line
Processed & Post Processed and stored Requirements For Interoperability Continue To Increase
RF Receiver for future analysis Processing & analysis may be performed on the ground at near-real-time
Storage Capability Developers Will Need To Shrink Payload Processing System
rates by transfer of information from platform to ground station.
Camarillo Product Line