Internet-Based

Distributed
Measurement
and Control Applications
ndustries that develop and use distributed measure-
ment-and-control (DMC) systems are migrating away
from proprietary hardware and software platforms in fa-
\'or of open systems and standardized apprmches. Some
of the key components of '1 DMC system are sensors, actua-
tors, controllers, and control networks. Large DMC systems
often consist of many sensors and actuators from various
manufacturers and multiple sensors or control networks sup-
plied by differellt \renders. Component interfacing and sys-
tem integration are major issues that require significant efforts
from multiple dxsciplines with technical backgrounds. Thus,
standardized trmsclucer (sc'I1sc)r and actuator) interfaces,
June 1999 IEEE Instrumentation & Measurement Magazine
1094-6969/99/$10.00019991EEE
23
high-level programming languages, object-oriented plat-
forms, and intranet and Internet technologies serve to shape
the next-generation DMC landscape. They all are key to eas-
ing system integration. This article describes a framework for
standardized DMC integration and a demonstration J une
1999dhering to the framework designed at the National Insti-
tute of Standards and Technology (NIST).
Based on our effort, we have found three major areas
where open standards will play a critical role in future DMC
development processes. These areas are:
B Transducer interfaces
B Open network communications
B Distributed application development
rk
Fig. 1 illustrates the distributed control framework for an in-
dustrial automation environment. Information and data sup-
plied by sensors in the process connection level-the lowest
level in the control hierarchy-are sent to the control network
nodes where distributed intelligence can be executed. Sensor
data received by the processors in the network nodes is broad-
cast onto the network. Other nodes in the network use the sen-
sor information to make control decisions that manipulate the
actuators or communicate with other nodes. Closed-loop con-
trol may occur either at an individual network node or among
multiple nodes. Distributed control schemes allow decisions
to be made in the process level in a distributed manner based
on simple commands from the enterprise applications.
This distributed framework has less network traffic than
the traditional centralized control design. Also, computing is
distributed over the entire system, as opposed to being cen-
tralized in the traditional manner. Status is reported to the ap-
plication and enterprise levels for system-type control,
condition-based monitoring, diagnostic, or database archiv-
ing. In order to provide an environment that supports
interoperability as shown in Fig. 1, a framework for standard-
ization must be included in the overall distributed
measurement and control design.
We describe a demonstration project using this NET,
Internet-based DMC framework in the following sections. The
hardware, software, and standards used during the develop-
ment are described. The standards used to implement the con-
nectivity of transducers to the network nodes are also addressed.
Wenext focus on how Ethernet is used as a control network me-
dium, and we describe the software NIST developed to support
open-network communications. Finally, we describe the soft-
ware used to implement the distributed control applications on
both the network nodes as well as in the client.
At the lowest level of a distributed measurement and control
hierarchy, sensors and actuators are needed to sense environ-
mental conditions and control physical entities, respectively.
Transducer interfacing refers to the process of physically and
electrically connecting the transducer to a microprocessor in
the network node. A key reason for standardizing the interface
at the hardware interconnection level is the current compatibil-
ity problems transducer manufacturers face when integrating
their devices into multi-vendor networks [l], [2], [3].
Transducer interfacing also requires standard software in-
terfaces to provide application and network interoperability
at the network nodes. Because the network and the transducer
must expo:;e their interfaces directly to transducer applica-
tions on each node, any attempt to migrate the application, the
sensor hardware, or the network node to another platform re-
quires a time-consuming and costly redesign of the applica-
tions interface to the new environment.
When standard interfaces such as IEEE P1451 are used in
the DMC or any closed-loop manufacturing system, sen-
sor-to-sensor interchangeability and sensor-to-network
interoperability can be realized. Some of the key features of
the IEEE P1451 smart transducer interfaces are that the stan-
dards specify digital c
Enterprise Level Enterprise i~~~
Transaction Aml i cati ons
. .
Process Mapping
Specifications
Application Level
- ~ ~ ~ ~ - ~ ~
I
CORBA, DCCM
Application
Frameworks
L
. .
Distributed Intelligence Level
*- - - * i L%VC_i *W
IEEE P1451 1
Process Connection Level
IEEE 1451 2
Fig. I . Industrial automation framework.
24 IEEE Instrumentation & Measurement Magazine
nmunication protocols from sensor to
network, and self-identification of
sensor. This means that a sensor
can describe itself to the network,
facilitating an automatic system
configuration. The 1451-based sen-
sors or actuators achieve self-
identification via a memory chip
physically attached to the sensor.
The chip stores information such as
manufacturer name, identification
number, type of device, and serial
number, as well as calibration data.
This information is called the trans-
ducer electronic data sheet (TEDS).
The TEDS can upload to the system
upon power up or upon request.
The incorporation of IEEE 1451
standards into transducer hard-
ware will simplify the tasks of sen-
June 1999
To
Machine
Location of
Transducers
Fig. 2. The components of the DMC system.
sor and device maintenance by enabling simple "plug and
play" for replacement and upgrade. The TEDS also serves as
documentation. Over time, this electronic documentation is
more accurate and cost-effective than traditional paper docu-
mentation methods.
Transducer networking is similar to networking the personal
computer (PC). There is a network-capable application proces-
sor (NCAP), equivalent to a small computer, that resides in ev-
ery transducer network node or control network node. An
NCAP could be an 8-bit microprocessor for a DeviceNet control
network, or a 32-bit microprocessor for an Ethemet-based con-
trol network. Presently, many DMC-based control networks
use proprietary hardware and software interfaces that limit the
availability of data to higher-level networks and repositories.
Ethernet has long been established as a data communication
network used widely in office and factory automation.
The use of Ethernet as the preferred medium for
DMC-based control networks is rapidly gaining in momentum
due to its speed, cost-effectiveness, and the ability to leverage
off-the-shelf application components to facilitate building dis-
tributed systems [4]. Changes in the IEEE 802 Ethernet specifi-
cation are making Ethernet quite powerful as a network for
device-level control. For example, the IEEE 802.1~standard for
message prioritization or quality of service (QoS) was initially
developed for streaming live video, audio, and other multime-
dia content. Indeed, those in the control-network arena can le-
verage the real-time deterministic capabilities in the Ethemet
standard. The standard effectively guarantees that messages can
be delivered and/or acknowledged in less than 4 ms. This range
is within most tolerances for providing adequate response times
for process-control applications.
Opponents of using Ethernet as a control-network me-
dium have argued that Ethernet will never be deterministic
enough for real-time, closed-loop control applications. How-
ever, Hewlett-Packard (HP) engineers have recently shown
that timing in many critical applications can be maintained to
an accuracy of better than 500 ns using Ethernet as the control
network [5]. Also, the Foundation FieldBus organization has
selected Ethernet for their control network draft specifica-
tions. The FieldBus specifications define an H2 network,
which is a standard for higher-speed networks with data rates
around 100 megabits per second (Mbps) [6]. High-speed
Ethernet was chosen above several other competing propri-
etary buses.
An implementation at NET based on the DMC framework
merges the newly emerging IEEE 1451 smart transducer inter-
face standards and the de facto networking standards from
the Internet community to remotely monitor and control an
industrial process via the Internet. State-of-the-art hardware
and software technology for DMC deployment is described in
terms of a real-world, Internet-based application. The specific
scenario addressed in the NIST DMC demonstration project is
as follows. In the NIST machine shop, a high-speed, precision
milling machine is used as the testbed. During the millingpro-
cess, a means of maintaining a relatively constant temperature
of the material is required. Large temperature differentials in
an open machme shop environment can result in dimensional
variation of the finished parts. A coolant tank, shown in Fig. 2,
serves as the coolant source to the part being milled.
We designed an Internet-based, DMC closed-loop control
system to regulate the temperature in the coolant tank. Fig. 2
illustrates the mechanical components of the DMC-based con-
trol system. The goal is to keep the coolant in the tank within a
specified temperature range. During the machining, the cool-
ant temperature tends to rise due to the heat generated in the
cutting process. The increase in ambient air temperature
around the machine also contributes to the rise in coolant tem-
perature. A chiller maintains a reservoir of cold water that is
constantly circulated. The coolant from the chiller is directed
into the coolant tank when demanded, via a three-way valve.
Through this control process, a consistent coolant tempera-
ture will be maintained in the coolant tank throughout the
part-machining cycle.
The goal of the NET Internet-based DMC demonstration
project was to provide a means, via standardized interfaces, to
access, view, monitor, and control this DMC application in
real time. In order to provide this capability, several key hard-
ware and software technologies were developed. The overall
NIST DMC demonstration project topology is shown in Fig. 3.
A J ava applet shown in Fig. 4 executes from within a web
browser like Internet Explorer 4.0 or Netscape Communicator
4.5.' It provides an information delivery mechanism to the
shop floor or operation manager's desk across the country.
Any web browser that supports J ava can be used to view this
application. A web server located at NIST provides the J ava
and hypertext markup language (HTML) data file repository
capabilities for the browser in this demonstration. The actual
DMC system was developed using two, state-of-the-art, HP
Ethernet-based, prototype network nodes.
These nodes support a reference implementation of emerg-
ing IEEE P1451 specifications that provide standardized ap-
plication programming interfa& (API) to the transducer and
the network, and standardized hardware interfaces for con-
J une 1999 IEEE Instrumentation &? Measurement Magazine 25
+ I
Ethernet Control Network 8
I
60 Temperature Sensors @Val ve Actuator
Fig. 3. The overall Internet-based DMC demonstration topology
Fig. 4. Remote monitoring and control applet
necting transducers at the digital level to the Ethernet nodes
[7], [SI. The temperature sensors and the valve actuator were
all directly controlled in this distributed settink using C lan-
guage applications developed at NIST for the embedded
Ethernet-based P1451 compatible HP nodes. Higher-level dis-
tributed control and monitoring capabilities were imple-
mented in the J ava programming language.
Thenodes communicate information between themselves us-
ing a TCP/IP publish-subscribe messaging protocol. The
messaging technology dows a node to publish a topic, a specific
piece of information in ASCII string, for example, "node2.sen-
sor.funktemp," which represents the temperature in the coolant tank,
on the network. Another node would subscribe to thistopic to re-
ceiveupdates on the temperature of the tank. TheHP nodes shown
in Fig. 3 were programmed to provide many other services. These
services can be controlled and monitored viathe web-based Java
client applets. TheJava applets do not communicate directly with
the Hp nodes. Theapplets must connect with a NIST-developed
TCP/IP gateway to request services fromthe nodes.
Fig. 4 illustrates the NIST-developed J ava applet for re-
mote monitoring and control using the Microsoft Internet Ex-
plorer 4.01 browser. This demonstration was featured at the
1451 booth at the International Society for Measurement and
Control (ISA) Expo98 exhibition in Houston, TX, October
1998. This application can also be viewed at: http://ino-
tion.aptd.nist.gov/P1451 /ISADemo.htm.
The research conducted on the Internet-based DMC project in-
fluences several other areas of research in other NIST labora-
tories. One of these is the collaborative research project
conducted between NIST and other national measurement in-
stitutes (NhLIs) in North, Central, and South America. Its goal
is to significantly reduce the amount of time it takes to per-
form a complete round robin of international comparisons of
measurement standards. NIST is finding ways to reduce this
time by leveraging Internet technology.
This pilot project, involving 12 participating member
countries of the Inter-American System of Metrology (SIM)
and dubbed "SIMnet," has just been demonstrated at NIST.
The SIMnet system, shown in Fig. 5, effectively provides an
Internet-based environment for real-time comparisons of elec-
trical calibration measurements among 12 participating SIM
countries. NIST's focus in the SIMnet effort is to use the
Internet to provide real-time audio, video, data, and applica-
tion-sharing capabilities. This system allows the metrologists
of the 12 NMIs collaborating in real time to perform compari-
sons of electrical measurements, calibration procedures, and
measurement data of digital multimeters (DMMs). Through
the use of SIMnet, we expect to shorten a round robin of inter-
national comparisons of electrical measurement standards for
the DMMs from three years to six months.
In another research study, NIST scientists are using DMC
research results to incorporate telepresence technology into
calibrating gas flowmeters. In order to expand their flowmeter
calibration capacity without spending millions of dollars to
build a new facility, flow measurement scientists at NIST have
set up a project and collaboration agreement with the Colo-
rado Engineering Experiment Station Inc. (CEESI) to use its
Fig. 5. SlMnet provides an Internet-based environment for real-time
comparisons of electrical calibration measurements among 12 participating SIM
countries.
26 IEEE Instrumentation & Measurement Magazine J une 1999
via simple plug-and-play sensor and actuator components. In
addition, using standardized components and interfaces will
certainly reduce interoperability problems.
[l] N. R. J ohnson, Building plug-and-play networked smart
transducers, Sensors Magazine, Oct. 1997, pp. 40-61.
[Z] K. Leeand R. Schneeman, A Standardized Approach for
Transducer Interfacing: Implementing IEEE-P1451 Smart
Transducer Interface Draft Standards, Proc. of Sensors Conference,
Philadelphia, PA, Oct. 22-24,1996, pp. 87-100.
[3] R. Schneeman, and K. Lee, Multi-Network Access to IEEE P1451
Smart Sensor Information Using World Wide Web Technology,
Proc. Sensors Conference, Boston, MA, May 13-15,1997, pp. 15-34.
Automation Research Corporation. Oct. 1997, pp. 1-31.
the fieldbus, ISA Tech/97, Anaheim, CA., Oct. 7-9,1997.
networks, InTech Magazine, Apr. 1998, p. 18.
Actuators Network Capable Application Processor (NCAP) Information
Model, IEEE P1451.1 D1.83, Dec. 16,1996.
Actuators-Transducer to Microprocessor Communication Protocols
and Transducer Electronic Data Sheet (TEDS) Formats, IEEE Std
1451.2-1997, Sept. 26,1997.
[9] P.I. Espina, Tele-metrology: remote flowmeter calibration, Flow
Control, vol. 5, no. 1, J an. 1999, pp. 16-21.
[4] Automation Strategies: Ethernet-based Control Network Strategies,
[5] J .C. Eidson, and W. Cole, Closed-loop control using Ethernet as
[6] B. Baranski, and J . Strothman, Ethernet picked for FieldBus H2
[7] Draft Standard for a Smart Transducer Interfacefor Sensors and
[8] Standardfor a Smart Transducer lntevface for Sensors and
E
DisclaimerXertain commercial products, hardware,
and software are identified in this article in order to de-
scribe the system. Such identification does not imply
recommendation or endorsement by the National In-
stitute of Standards and Technology, nor does it imply
that the products identified are necessarily the best or
the only one available for the purpose.
Fig. 6. A large flowmeter calibration facility at GEES1
large-flow calibration facility. This project will use the 1451
smart transducer interface technology coupled with the ap-
plet-based Internet techniques developed at NIST to remotely
monitor and control the calibration process of gas flowmeters
at CEESI, with flow rate up to 3.95 standard liters per minute
(slm) [9]. NET scientists will use real-time video, audio, and
computer means through the Internet to remotely check the
health of the sensors and equipment before and after the cali-
bration, coordinate the calibration procedures with the remote
staff, control the collection of calibration data, monitor the
measurement results, and coordinate sending the measure-
ment data back to NIST for computation and validation. A
typical, large-scale, outdoor pipeline setting for flowmeter
calibration at CEESI is shown in Fig. 6. This project is currently
in the design phase, with a small-scale model already demon-
strated in a recent workshop held at NIST.
U m ma ry
In summary, the NIST DMC demonstration project that has
been described here spans all levels of the DMC system. This
multilevel data communication capability is a viable one as a
result of standardized networks, protocols, and transducer in-
terfaces. Common Intemet-based software technology was
used to provide the ease of data migration between the various
communication pathways. Standard and de facto software lan-
guages such as C and Java were used with off-the-shelf devel-
opment tools to implement the embedded network node
applications and the web-based application, respectively.
Intemet-based TCP/IP protocols and Ethernet technology
were used to design the networking infrastructure.
The distributed measurement and control concept with
networking sensors and actuators should work quite well
with advanced system design, particularly where sen-
sor-based monitoring and control applications are important.
Taking this approach in systemdesign can reduce the total
life-cycle cost of the systemby use of modular sensor system
design and commercially available smart transducers with
standardized interfaces, and easy maintenance and upgrade
Kang Lee is Leader of the Sensor Integration Group at the National
Institute of Standards and Technology (NIST). He is also is the
Chairman of TC9, I EEE Instrumentation and Measurement Soci-
etys Technical Committee on Sensor Technology. Currently he is
coordinating an effort in industry to establish a set of I EEE stan-
dards, A Smart Transducer Interface for Sensors and Actuators, to
ease the connectivity of sensors to systems and networks. He received
his B.S.E.E.from the Johns Hopkins University and M.S.E.E.from
the University of Maryland. Rick Schneeman is a Computer Scien-
tist in the Manufacturing Engineering Laboratory at NIST. He is
also a member of the Automated Production Technology Division.
He is a inember of one of TC9s four working groups, which has re-
cently developed a draft specifcation f or a smart transducer infor-
mation model, I EEE P1451.1. He received his B.S. in Computer
Science and Information Systems Management from the University
of Maryland and M.S. from the Johns Hopkins University.
J une 1999 IEEE Instrumentation & Measurement Magazine 27