Você está na página 1de 60



Also in this issue:

> Technology Policy and the Trump Administration
> The First Female Engineer at JPL

FEBRUARY 2018 www.computer.org

SkillChoice™ Complete
Now with expanded libraries and an upgraded platform! Valued at

6,000+ Practice
videos Exams 15,000+ Books24x7 titles

OVER 20x as many resources as before

One membership.
Unlimited knowledge.
Did you know IEEE Computer Society membership comes with access to a
high-quality, interactive suite of professional development resources, available 24/7?

Powered by Skillsoft, the SkillChoice™ Complete library contains more than $3,000
worth of industry-leading online courses, books, videos, mentoring tools and exam
prep. Best of all, you get it for the one low price of your Preferred Plus, Training &
Development, or Student membership package. There’s something for everyone,
from beginners to advanced IT professionals to business leaders and managers.

The IT industry is constantly evolving. Don’t be left behind. Join the IEEE Computer
Society today, and gain access to the tools you need to stay on top of the latest
trends and standards.

Learn more at www.computer.org/join.


IEEE COMPUTER SOCIETY computer.org • +1 714 821 8380

Editor Managers, Editorial Content
Meghan O’Dell Brian Brannon, Carrie Clark

Contributing Staff Publisher

Christine Anthony, Lori Cameron, Lee Garber, Cathy Martin, Robin Baldwin
Chris Nelson, Dennis Taylor, Rebecca Torres, Bonnie Wylie
Director, Products and Services
Production & Design Evan Butterfield
Carmen Flores-Garvey
Senior Advertising Coordinator
Debbie Sims

Circulation: ComputingEdge (ISSN 2469-7087) is published monthly by the IEEE Computer Society. IEEE Headquarters, Three Park Avenue, 17th Floor, New
York, NY 10016-5997; IEEE Computer Society Publications Office, 10662 Los Vaqueros Circle, Los Alamitos, CA 90720; voice +1 714 821 8380; fax +1 714 821 4010;
IEEE Computer Society Headquarters, 2001 L Street NW, Suite 700, Washington, DC 20036.
Postmaster: Send address changes to ComputingEdge-IEEE Membership Processing Dept., 445 Hoes Lane, Piscataway, NJ 08855. Periodicals Postage Paid at
New York, New York, and at additional mailing offices. Printed in USA.
Editorial: Unless otherwise stated, bylined articles, as well as product and service descriptions, reflect the author’s or firm’s opinion. Inclusion in ComputingEdge
does not necessarily constitute endorsement by the IEEE or the Computer Society. All submissions are subject to editing for style, clarity, and space.
Reuse Rights and Reprint Permissions: Educational or personal use of this material is permitted without fee, provided such use: 1) is not made for profit;
2) includes this notice and a full citation to the original work on the first page of the copy; and 3) does not imply IEEE endorsement of any third-party products
or services. Authors and their companies are permitted to post the accepted version of IEEE-copyrighted material on their own Web servers without permission,
provided that the IEEE copyright notice and a full citation to the original work appear on the first screen of the posted copy. An accepted manuscript is a version
which has been revised by the author to incorporate review suggestions, but not the published version with copy-editing, proofreading, and formatting added by
IEEE. For more information, please go to: http://www.ieee.org/publications_standards/publications/rights/paperversionpolicy.html. Permission to reprint/republish
this material for commercial, advertising, or promotional purposes or for creating new collective works for resale or redistribution must be obtained from IEEE by
writing to the IEEE Intellectual Property Rights Office, 445 Hoes Lane, Piscataway, NJ 08854-4141 or pubs-permissions@ieee.org. Copyright © 2018 IEEE.
All rights reserved.
Abstracting and Library Use: Abstracting is permitted with credit to the source. Libraries are permitted to photocopy for private use of patrons, provided the per-
copy fee indicated in the code at the bottom of the first page is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923.
Unsubscribe: If you no longer wish to receive this ComputingEdge mailing, please email IEEE Computer Society Customer Service at help@computer.org and
type “unsubscribe ComputingEdge” in your subject line.
IEEE prohibits discrimination, harassment, and bullying. For more information, visit www.ieee.org/web/aboutus/whatis/policies/p9-26.html.

IEEE Computer Society Magazine Editors in Chief

Computer IEEE Micro IEEE Intelligent Systems
Sumi Helal, Lancaster University Lieven Eeckhout, Ghent V.S. Subrahmanian, University
University of Maryland
IEEE Software IEEE MultiMedia
Diomidis Spinellis, Athens IEEE Computer Graphics Shu-Ching Chen, Florida
University of Economics and and Applications International University
Business Torsten Möller, Universität Wien

IEEE Annals of the History

IEEE Internet Computing IEEE Pervasive Computing of Computing
M. Brian Blake, University of Marc Langheinrich, University Nathan Ensmenger, Indiana
Miami of Vienna University Bloomington

IT Professional Computing in Science IEEE Cloud Computing

Irena Bojanova, NIST & Engineering Mazin Yousif, T-Systems
Jim X. Chen, George Mason International
IEEE Security & Privacy University
David M. Nicol, University of
Illinois at Urbana-Champaign

www.computer.org/computingedge 1


Standardizing Ethical
How Much to
Design for Artificial Trust Artificial Human Intellect
Intelligence and Intelligence? and Amplifying
Autonomous Systems Perception and
8 Editor’s Note: Artificial Intelligence
10 Standardizing Ethical Design for Artificial
Intelligence and Autonomous Systems

14 A Layered Model for AI Governance


19 Deep Learning Triggers a New Era in Industrial


25 How Much to Trust Artificial Intelligence?


30 Augmenting Human Intellect and Amplifying

Perception and Cognition

36 Katie Malone on Machine Learning


41 The Problem with AI


46 Technology Policy and the Trump Administration


48 Cloud-Native Applications and Cloud Migration:

The Good, the Bad, and the Points Between

51 Dana Ulery: Pioneer of Statistical Computing and

Architect of Large, Complex Systems

56 Computing in World War I



4 Magazine Roundup

The Problem
with AI

Subscribe to ComputingEdge for free at


by Lori Cameron

policies, regulations, and fund-

ing priorities to address emerg-
ing technologies. Although
many developers and technolo-
gists bemoan the complexity,
pace, and ineptitude of govern-
ment, others familiarize them-
selves with the challenges and
seize opportunities in the policy
arena. In this article from the
December 2017 issue of Com-
puter, researchers survey current
federal policies and activities
impacting technology develop-
ers, with special emphasis on
privacy, cybersecurity, safety
regulation, energy and environ-
ment, and ethical issues.

T he IEEE Computer
Society’s lineup of 13
peer-reviewed tech-
nical magazines covers cut-
ting-edge topics ranging from

Current US Federal Policy

Framework for Self-Driving
Vehicles: Opportunities and
Computing in Science &

Product Innovation through

Computational Prototypes
software design and computer Challenges and Supercomputing
graphics to Internet comput- Self-driving systems raise Converting from physical pro-
ing and security, from scien- many economic, social, and totype–based product design
tific applications and machine security issues that draw fed- to computational (virtual) pro-
intelligence to cloud migration eral attention. The landscape totype–based product design
and microchip design. Here are is constantly shifting as US requires leading-edge computa-
highlights from recent issues. government agencies adapt tional engineering codes; joint

4 February 2018 Published by the IEEE Computer Society 2469-7087/18/$33.00 © 2018 IEEE
R&D efforts; dedicated research- which can be viewed as ancestors of the Apulia region in southern
ers; and meticulous verification, of the modern mechanical counter. Italy, the authors proposed an
validation, and uncertainty quan- alternative approach to traditional
tification. For the most difficult IEEE Cloud Computing tourism marketing and advertis-
engineering design problems, ing efforts: an interactive, innova-
most of these are necessary but Realizing Software Reliability tive, and attractive VR experience
not sufficient. In extreme cases, an in the Face of Infrastructure called the Multisensory Apulia
impending crisis might force con- Instability Touristic Experience (MATE).
version even in the face of strong Cloud computing has brought
resistance. This article from the with it the utilization of off-the- IEEE Intelligent Systems
November/December 2017 issue of shelf commodity hardware that
Computing in Science & Engineer- has higher failure rates than the Design and Prototyping a
ing illustrates the Goodyear Tire systems used in enterprises for Smart Deep Brain Stimulator:
& Rubber Company’s successful, the past several decades. Coupled An Autonomous Neuro-
crisis-driven transition to virtual with increasingly complex, highly Sensing and Stimulating
prototype–based product design. distributed, constantly changing Electrode System
datacenter environments that can In this article from the September/
IEEE Annals of the History no longer be treated as determin- October 2017 issue of IEEE Intelli-
of Computing istic systems, this forces us to gent Systems, researchers present
change the way we view the sta- the design and prototyping of a
Carries Stripped to the Bone: bility of that infrastructure. This smart deep brain stimulator (SDBS)
Episodes in the History of article from the September/Octo- that consists of brain-implantable
Coaxial Modular Digital ber 2017 issue of IEEE Cloud Com- smart electrodes and a wireless-
Counters puting studies how Netflix has fully connected external controller. SDBS
Although much has been writ- embraced this mindset change electrodes operate as completely
ten on the history of calculating and was able to avoid any signifi- autonomous electronic implants
machines, very little attention cant impact during a major outage that are capable of sensing and
has been paid to the evolution of experienced by their cloud infra- recording neural activities in real
mechanical counters and their structure provider, Amazon Web time, performing local processing,
components. Mechanical coun- Services (AWS). and generating arbitrary wave-
ters were ubiquitous and could forms for neuro-stimulation. A
be found in many cars, where Experiencing the Sights, bidirectional, secure, fully passive
they served as odometer displays, Smells, Sounds, and Climate wireless communication backbone
or in cash registers and various of Southern Italy in VR was designed and integrated into
calculating machines. The most In this article from the November/ this smart electrode to maintain
common construction for such December 2017 issue of IEEE Com- contact between the electrodes
counters is made of rotating disks, puter Graphics and Applications, and the controller.
which are similar and located on researchers explore what it takes to
the same axis. Although they look make interactive computer graph- IEEE Internet Computing
simple in appearance and perhaps ics and virtual reality (VR) attrac-
standardized, they have a history tive as a promotional vehicle, from PACMAN: Personal Agent for
of their own. In this article from the points of view of the tourism Access Control in Social Media
the July–September issue of IEEE agencies and the tourists them- Given social media users’ plethora
Annals of the History of Computing, selves. Specifically, in response to of interactions, appropriately con-
the author analyzes three of the a call from local authorities seek- trolling who can access what infor-
earliest known models of counters, ing to increase the tourism appeal mation becomes a challenging task

www.computer.org/computingedge 5

for users. Selecting the appropriate neural network models in near users. The authors of this article
audience, even from within their real time with software optimiza- from the October–December 2017
own friend network, can be fraught tions such as downsampling and issue of IEEE Pervasive Comput-
with difficulties. In this article from compression, while consuming ing introduce the use of a deform-
the November/December 2017 less power than state-of-the art able device prototype, Typhlex,
issue of IEEE Internet Computing, embedded processors such as the with strategically placed grooves
researchers present PACMAN as Jetson TX1. to elicit bend gestures. They con-
a potential solution. It’s a personal ducted two exploratory studies
assistant agent that recommends IEEE MultiMedia with sighted participants (with
personalized access control deci- the prototype hidden from view)
sions based on the social context Crowdsensing Multimedia and blind participants, focus-
of any information disclosure by Data: Security and Privacy ing on comparing the usability of
incorporating communities gener- Issues bend gestures to touch as primary
ated from the user’s network struc- Smartphones are equipped with forms of input. Their findings sug-
ture and utilizing information in various sensors and have high- gest that while easily learnable
the user’s profile. PACMAN pro- performance wireless communica- and enjoyed by both groups, the
vides accurate recommendations tion capabilities. Through the ubiq- prototype had yet to improve blind
while minimizing intrusiveness. uitous presence of powerful mobile users’ performances when com-
devices, crowdsensing lets ordinary pared to the commonly used touch
IEEE Micro people collectively gather and share input paradigm. They present les-
real-time multimedia data. Multime- sons learned from their design
Flying IoT: Toward Low- dia crowdsensing has made large- process and studies, and discuss
Power Vision in the Sky scale participatory sensing viable the promise of deformable input
Many Internet of Things (IoT) in a speedy and cost-efficient man- devices in the area of accessibility
devices require some level of ner, but it also introduces security for blind users.
machine learning or cognitive and privacy concerns. For example,
capability to be truly effective, but participants’ personally identifiable IEEE Security & Privacy
the high computational complex- information can be exposed while
ity of cognitive algorithms makes sharing individually owned sen- Botnet Fingerprinting:
them unsuitable for low-power IoT sor data. In this article from the Anomaly Detection in SMTP
processors. In this article from the October–December 2017 issue of Conversations
November/December 2017 issue IEEE MultiMedia, the authors iden- In this article from the Novem-
of IEEE Micro, the authors study tify security and privacy issues ber/December 2017 issue of IEEE
a cognitive drone application in a in multimedia crowdsensing and Security & Privacy, the authors
design space called Flying IoT to describe existing solutions that are present the results obtained dur-
better understand the design chal- designed to protect both data pro- ing their research on detecting
lenges of cognitive IoT devices. ducers and consumers. unsolicited emails sent by botnets.
To improve their processor’s per- The distinction from most existing
formance while maintaining its IEEE Pervasive Computing solutions is that this approach is
low-power consumption and small based on the analysis of network
form factor, the authors propose a Typhlex: Exploring traffic, specifically the sequence
sensor-cloud architecture in which Deformable Input for Blind and syntax of SMTP commands
data collection is done at the edge Users Controlling a Mobile observed during email deliv-
and data processing is offloaded Screen Reader ery. The authors present several
to the cloud. This architecture Current smartphone technology improvements for detecting unso-
can process complex convolution presents many challenges for blind licited email sources from different

6 ComputingEdge February 2018

botnets (fingerprinting) that can a dynamic risk management and in streamlining major knowl-
be used during network forensic mitigation approach based on edge management processes. The
investigation. probabilistic threat estimation. A author develops a property graph
smart pacemaker case study illus- data model to facilitate the process
IEEE Software trates the approach. model of knowledge management.
In addition, the model is imple-
Probabilistic Threat Detection IT Professional mented through the Neo4j graph
for Risk Management in Cyber- database system. This research
Physical Medical Systems Graph Databases for provides some guidance for prac-
Medical devices are complex Knowledge Management titioners in seeking alternative
cyber-physical systems incorpo- Emerging technologies let com- approaches to traditional methods
rating emergent hardware and panies manage their knowledge of knowledge management.
software components. However, assets with more innovative and
this complexity leads to a wide effective methods. Due to the com- Computing Now
attack surface, posing security plex nature of knowledge manage-
risks and vulnerabilities. Mitiga- ment processes, it is cumbersome The Computing Now website
tion and management of such to design, develop, and implement (computingnow.computer.org) fea-
risks during premarket design a system based on relational data- tures up-to-the-minute computing
and postmarket deployment are bases. This article, which appears news and blogs, along with arti-
required. In this article from the in the November/December 2017 cles ranging from peer-reviewed
January/February 2018 issue of issue of IT Pro, proposes a spe- research to opinion pieces by
IEEE Software, the authors present cific graph database application industry leaders.


Advertising Personnel Southwest, California:

Mike Hughes
Debbie Sims: Advertising Coordinator Email: mikehughes@computer.org
Email: dsims@computer.org Phone: +1 805 529 6790
Phone: +1 714 816 2138 | Fax: +1 714 821 4010

Advertising Sales Representative (Classifieds & Jobs Board)

Advertising Sales Representatives (display)
Heather Buonadies
Central, Northwest, Southeast, Far East: Email: h.buonadies@computer.org
Eric Kincaid Phone: +1 201 887 1703
Email: e.kincaid@computer.org
Phone: +1 214 673 3742
Fax: +1 888 886 8599 Advertising Sales Representative (Jobs Board)

Northeast, Midwest, Europe, Middle East: Marie Thompson

David Schissler Email: marie@4caradio.org
Email: d.schissler@computer.org Phone: 714-813-5094
Phone: +1 508 394 4026
Fax: +1 508 394 1707

www.computer.org/computingedge 7

Artificial Intelligence

F rom Siri and Alexa to self-driving cars

and robots fulfilling Amazon orders, arti-
ficial intelligence already surrounds us.
As AI advances and permeates more areas of life,
there are obvious benefits and drawbacks—super-
Cognition,” the author examines various technol-
ogies designed to augment human intellect and
amplify human perception and cognition, consider-
ing how novel technologies can create a new rela-
tionship between digital technologies and humans.
intelligence could help us eradicate war, poverty, In IEEE Software’s “Katie Malone on Machine
and disease, but it could also intentionally or unin- Learning,” Edaena Salinas interviews Katie
tentionally cause great harm. The February 2018 Malone, a data scientist in the R&D department at
issue of ComputingEdge features cutting-edge Civis Analytics, which specializes in data science
research and thought pieces on AI and more. software and consulting. Katie and Edaena discuss
In Computer’s “Standardizing Ethical Design the major types of machine-learning algorithms—
for Artificial Intelligence,” the authors recognize the backbone of AI—and some examples, includ-
that AI’s consequences for social order aren’t well ing supervised and unsupervised classification.
understood, and look at how standards can guide Finally, in IT Professional’s “The Problem with
the way technology impacts society. AI,” the author looks at how AI and machine-
Researchers at Harvard University propose a learning technologies can help or hinder organiza-
conceptual framework for thinking about gover- tions in curating vast amounts of data.
nance for AI in IEEE Internet Computing’s “A Lay- This ComputingEdge issue also includes arti-
ered Model for AI Governance,” noting that there cles on topics other than AI:
is a large information gap between AI developers
and consumers and policymakers. • The author of IEEE Micro’s “Technology Policy
In IEEE MultiMedia’s “Deep Learning Trig- and the Trump Administration” considers how
gers a New Era in Industrial Robotics,” the author Trump’s (likely) technology policies will affect
examines deep learning applications in robot- the value of US firms in IT markets.
ics and introduces promising research that will • IEEE Cloud Computing’s “Cloud-Native Applica-
likely impact how industrial robot systems are tions and Cloud Migration: The Good, the Bad,
designed in the near future. and the Points Between” looks at the advan-
The author of IT Professional’s “How Much to tages and costs of cloud-native features in IT.
Trust Artificial Intelligence?” notes that AI is typi- • IEEE Annals of the History of Computing’s
cally software dominant, and that software is prone “Dana Ulery: Pioneer of Statistical Computing
to vulnerabilities. He recommends using caution and Architect of Large, Complex Systems” pro-
until some reliable methodology is adopted for files JPL’s first female engineer.
the assessment of assured trust within AI. • CiSE’s “Computing in World War I” looks back
In IEEE Pervasive Computing’s “Augmenting at the Royal Navy’s use of computers in their
Human Intellect and Amplifying Perception and battleships.

8 February 2018 Published by the IEEE Computer Society 2469-7087/18/$33.00 © 2018 IEEE
Be at the Center of It All
IEEE Computer Society membership puts you at the
heart of the technology profession—and helps you grow with it.
Here are 10 reasons why you need to belong.

A robust jobs board, Training that

plus videos, articles sharpens your edge
and presentations to in Cisco, IT security,
help you land that next MS Enterprise, Oracle
opportunity. and more.

Scholarships awarded
to computer science Certifications
and engineering student and exam preparation
members each year. that set you apart.

Opportunities to Industry intelligence,

get involved through including Computer, myCS,
speaking, publishing and Computing Now, and
volunteering opportunities. myComputer.

Deep discounts Over 200 annual

on magazines, journals, conferences and technical
conferences, symposia events held worldwide.
and workshops.

Access to 300-plus chapters

hundreds of and more than 30
books, 13 technical technical committees
magazines and keep you
20 research journals. connected.

IEEE Computer Society—keeping you ahead

of the game. Get involved today.

Ethical Design for
Artificial Intelligence
and Autonomous Mind, or alphaGo), AI has been a
standard part of the industrial rep-

ertoire since at least the 1980s, with
expert systems checking circuit
boards and credit card transactions.
Machine learning (ML) strat-
Joanna Bryson, University of Bath egies for generating AI have also
long been used, such as genetic al-
Alan Winfield, University of the West of England
gorithms for fi nding solutions to
intractable computational problems
AI is here now, available to anyone with access like scheduling, and neural net-
works not only to model and under-
to digital technology and the Internet. But stand human learning but also for
basic industrial control, monitor-
its consequences for our social order aren’t ing, and classification. In the 1990s,
probabilistic and Bayesian methods
well understood. How can we guide the way revolutionized ML and opened the
technology impacts society? door to one of the most pervasive AI
abilities now available: searching

through massive troves of data. In-
or decades—even prior to its inception—AI has novations in AI and ML algorithms have extended our ca-
aroused both fear and excitement as humanity pacity to fi nd information in texts, allowing us to search
has contemplated creating machines like our- photographs as well as both recorded and live video and
selves. Unfortunately, the misconception that audio. We can translate, transcribe, read lips, read emo-
“intelligent” artifacts should necessarily be human-like tions (including lying), forge signatures and other hand-
has largely blinded society to the fact that we have been writing, and forge video.
achieving AI for some time. Although AI that surpasses Yet, the downside of these benefits is ever present.
human ability grabs headlines (think of Watson, Deep As we write this, allegations are circulating that the

10 February 2018 Published by the IEEE Computer Society  2469-7087/18/$33.00 © 2018 IEEE
116 COM PUTE R PUBLISHED BY THE IEEE COMPUTER SOCIET Y 0 0 1 8 - 9 1 6 2 / 1 7/ $ 3 3 .0 0 © 2 0 1 7 I E E E
Davenport Consulting; cwalrad@daven.org

outcomes of the recent US presiden- when its orientation is changing or ambition. Human memory has been
tial election and UK referendum on when it is falling. outstripped by books for centuries—
EU membership were both influenced Autonomy is technically the capac- mere intelligence is no more of a direct
by the use of AI to detect and target ity to act as an individual. For social threat than mere strength.
“swing voters” via public social media. animals like humans, autonomy is
To address these and other concerns, normally situated somewhere along a Will AI undermine
the IEEE Computer Society Standards scale. For example, it is fully expected societal stability?
Activities Board is creating standards that family, workplace, government, For centuries, people have had signifi-
for responsible designers who will and other organizations might regu- cant concerns about the displacement
shape our brave new world and ensure larly have some impact on our actions. of workers by technology. There is no
AI’s benefit to humanity. Similarly, a technical system that can question that new technologies dis-
sense the world and select an action rupt communities, families, and lives,
DEFINING AI specific to its present context is called but historically, the majority of this
Although the following defi nitions “autonomous” even though its actions disruption has been for the better. In
are not universally used, they’re are ultimately determined by the de- general, lifespans are longer and infant
well-established.1 Intelligence is the signers that constructed its intelli- mortality is lower than ever before, and
capacity to do the right thing at the gence and its operators. these indicators are well associated
right time, in a context where doing with political stability. Nevertheless,
nothing (or making no change in be- CONCERNS ABOUT we are currently seeing a disruption
havior) would be worse. Intelligence DOMESTIC AND that seems to be undermining political
then requires COMMERCIAL AI stability. This disturbance is termed
AI is core to some of the most success- political polarization, which seems to
› the capacity to perceive contexts ful companies in history in terms of co-occur with inequality, although
for action, market capitalization and, along with causality between these is unclear.2
› the capacity to act, and information and communications Polarization has happened before, for
› the capacity to associate con- technology (ICT) more generally, has example, in the early 20th century,
texts to actions. revolutionized the ease with which reaching its climax in World War I.
people from all over the world can New technologies could play a role in
By this defi nition, plants are intelli- create, access, and share knowledge. increasing inequality—and therefore
gent. They can perceive and respond to However, possible pitfalls of AI could polarization—by eliminating costs
the direction of light, for example. The have quite serious consequences. such as distance that formerly sup-
more conventional understanding of Here we briefly review some common ported economic diversity. This time,
“intelligent” includes being cognitive, concerns to see which are both realis- AI and ICT might be the technologies
that is, being able to learn new con- tic and specific to AI. changing the economic landscape.
texts and actions, and the associations
between them. Will AI outcompete us? Will AI harm privacy, personal
AI, by convention, describes (typ- Some of the most sensational fears are liberty, and autonomy?
ically digital) artifacts that demon- that, as AI increases to the point that What really makes AI special is its re-
strate any of these capacities. So, for it surpasses human abilities, it might lationship to information, especially
example, machine vision, speech rec- take control over our resources and personal information. Previous peri-
ognition, pattern recognition, and outcompete our species, leading to ods of domestic spying have been asso-
static production systems are all ex- human extinction. AI is already super- ciated with everything from prejudice
amples of AI, with algorithms that can human in many domains. With ma- in opportunities to pogroms. How-
be found in standard AI textbooks. chines, we can already do arithmetic ever, AI and ICT can greatly facilitate
Robots are artifacts that sense and better, play chess and Go better, tran- such knowledge gathering. We are
act in the physical world in real time. scribe speech better, read lips better, now able to keep and access long-term
By this defi nition, a smartphone is a remember more things for longer, and records on anyone who produces stor-
(domestic) robot. It has not only mi- indeed be faster and stronger than we able data—for example, anyone with
crophones but also a variety of pro- are unaided. However, these capaci- bills, contracts, or a credit history, not
prioceptive sensors that let it know ties have in no sense led to machine to mention public writing and social

www.computer.org/computingedge 11
M AY 2 0 1 7 117

media use. With ML, this data lets us autonomous and intelligent systems.”4 .ieee.org/develop/project/7001
make predictions concerning individ- The first output from the initiative is a .html), which we discuss below;
uals’ behavior and preferences, which discussion document called Ethically › P7002—Data Privacy Process
in turn opens the possibilities of con- Aligned Design (EAD), version 1, pub- (standards.ieee.org/develop
trol or persecution. lished in December 2016.4 The work of /project/7002.html), which aims
eight committees, it covers to create one overall method-
CAN STANDARDS PROMOTE ological approach that specifies
ETHICS IN AI? › general principles, practices to manage privacy
Standards are consensus-based agreed- › how to embed values into auton- issues; and
upon ways of doing things, setting out omous intelligent systems, › P7003—Algorithmic Bias Consid-
how things should be done. If a system › methods to guide ethical design erations (standards.ieee.org
or process can be shown to do things and design, /develop/project/7003.html),
as prescribed, it is said to be compliant › safety and beneficence of arti- which aims to specify method-
with the standard. Such compliance ficial general intelligence and ologies to ensure that negative
provides confidence in a system’s effi- artificial superintelligence, bias in algorithms is addressed
cacy in areas important to users, such › personal data and individual and eliminated.
as safety, security, and reliability. access control,
P7001 is an effort in which both au-
IEEE’s Initiative for Ethical Considerations thors are involved. It is based on the
radical proposition that it should al-
in Artificial Intelligence Systems has
ways be possible to find out why an AS
as its mission to “ensure every technologist made a particular decision.
is educated, trained, and empowered to prioritize Transparency is not one thing.
ethical considerations.” Clearly, elderly persons don’t require
the same level of understanding of their
care robot as the engineer who repairs
Few standards explicitly address › how to reframe autonomous it. Nor would patients expect the same
ethics in robotics and AI. One that does weapons systems, appreciation of the reasons a medical-
is British Standard (BS) 8611:2016, Ro- › economics and humanitarian diagnosis AI recommends a particular
bots and Robotic Devices: Guide to the Eth- issues, and course of treatment as their doctor. The
ical Design and Application of Robots and › law. P7001 working group has identified five
Robotic Systems.3 Published in April categories of stakeholder—users, safety
2016, it provides designers with a tool EAD articulates a set of about 60 certification agencies, accident investi-
to assess ethical risk. At the heart of BS draft issues and recommendations. gators, lawyers or expert witnesses, and
8611:2016 is a set of 20 distinct ethical Each committee was asked to identify wider society—and proposes that ASs
hazards and risks, grouped under four issues that could be addressed through a must be transparent to each in different
categories: societal, application, com- new standard. Presently, four standards ways and for different reasons.
mercial/financial, and environmental. working groups are drafting candidate
Advice on measures to mitigate the standards to address an ethical concern › For users, transparency is im-
impact of each risk is given, along with articulated by one or more of the eight portant because it builds trust in
suggestions on how such measures committees outlined in the EAD docu- the system by providing a simple
might be verified or validated. ment. The candidate standards are way for users to understand what
IEEE’s Initiative for Ethical Con- the system is doing and why.
siderations in Artificial Intelligence › P7000—Model Process for › For AS safety certification,
and Autonomous Systems, a program Addressing Ethical Concerns transparency is important
designed to bring together “multiple during System Design (standards because it exposes the system’s
voices in the AI and Autonomous Sys- .ieee.org/develop/project processes for independent
tems (AS) communities,” has as its /7000.html), which aims to certification against safety
mission to “ensure every technologist establish a value-based system standards.
is educated, trained, and empowered design methodology; › If accidents occur, an AS needs
to prioritize ethical considerations › P7001—Transparency of Autono- to be transparent to investiga-
in the design and development of mous Systems (standards tors; the internal process that

12 ComputingEdge February 2018

This article originally appeared in
Computer, vol. 50, no. 5, 2017.

led to the accident must be he changes artificial intelli-
traceable. gence and autonomous systems
› Following an accident, lawyers are bringing to the world are
or other expert witnesses who real, and already in progress. Although
might be called on to give evi- we cannot say with certainty that the
dence require transparency to situation is in hand, we as members of
inform their evidence. the global initiative are optimistic that
› Disruptive technologies, such as the right steps are being taken and that
driverless cars, require a certain IEEE will be key to ensuring that AI and
level of transparency for wider ASs benefit all of humanity.

society to gain the public’s con-
fidence in the technology and to ACKNOWLEDGMENTS
ensure that trust is deserved. We thank John C. Havens for introducing
us to the IEEE initiative and forwarding
Of course, the way in which trans- Dr. Walrad’s request for an article to us.
parency is provided is likely to be very

different for each group. If we take a REFERENCES
care robot as an example, transpar- 1. P.H. Winston, Artificial Intelligence,
ency means users can understand Addison-Wesley, 1984.
what the robot might do in different 2. N.M. McCarty, K.T. Poole, and H.
circumstances. If the robot does any- Rosenthal, Polarized America: The
thing unexpected, they should be able Dance of Ideology and Unequal Riches,
to ask it “Why did you just do that?” MIT Press, 2006.
and receive an intelligible reply. 3. Robots and Robotic Devices: Guide to
Safety certification agencies will the Ethical Design and Application
need access to technical details of how of Robots and Robotic Systems, BS
the AS works, together with verified 8611:2016, British Standards Inst.,
test results. Accident investigators 2016; shop.bsigroup.com/Product
will need access to data logs of exactly Detail?pid=000000000030320089.
what happened prior to and during an 4. Ethically Aligned Design: A Vision for
accident, most likely provided by some- Prioritizing Human Wellbeing with
thing akin to an aircraft flight data re- Artificial Intelligence and Autonomous
corder—and it should be illegal to oper- Systems, version 1, IEEE Standards
ate an AS without such a system. Wider Assoc., 2016; standards.ieee.org
society would need accessible docu- /develop/indconn/ec/ead_v1.pdf.
mentary-type science communication
to explain an AS (such as a driverless
car autopilot) and how it works. Find out more
In P7001, we aim to develop a stan- JOANNA BRYSON is an associate and get involved:
dard that sets out measurable, test- professor in the Department of
able levels of transparency in each of Computer Science at the University of cybersecurity.ieee.org
these categories (and perhaps new cat- Bath and an affiliate of the Princeton
egories yet to be determined) so that Center of Technology Policy. Contact
we can assess an AS objectively and her at jjb@alum.mit.edu.
determine compliance. It is our aim
that P7001 will also articulate trans- ALAN WINFIELD is a professor of
parency levels in a range that defines robot ethics at the Bristol Robotics
minimum levels up to the highest Laboratory in the University of
achievable standards of acceptance. the West of England and a visit-
The standard will provide AS design- ing professor in the Department
ers with a toolkit for self-assessing of Electronic Engineering at the
transparency as well as recommenda- University of York. Contact him at
tions for how to address shortcomings alan.winfield@uwe.ac.uk.
or transparency hazards.

www.computer.org/computingedge 13
M AY 2 0 1 7 119
Internet Governance
Editor: Virgilio A.F. Almeida • virgilio@dcc.ufmg.br

A Layered Model for

AI Governance
Urs Gasser and Virgilio A.F. Almeida • Harvard University

AI-based systems are “black boxes,” resulting in massive information

asymmetries between the developers of such systems and consumers and
policymakers. In order to bridge this information gap, this article proposes a
conceptual framework for thinking about governance for AI.

any sectors of society rapidly adopt digi- technology.5 Yet the process of designing a gov-
tal technologies and big data, resulting ernance ecosystem for AI, autonomous systems,
in the quiet and often seamless integra- and algorithms is complex for several reasons.
tion of AI, autonomous systems, and algorith- As researchers at the University of Oxford
mic decision-making into billions of human point out,3 separate regulation solutions for
lives.1,2 AI and algorithmic systems already decision-making algorithms, AI, and robot-
guide a vast array of decisions in both private ics could misinterpret legal and ethical chal-
and public sectors. For example, private global lenges as unrelated, which is no longer accurate
platforms, such as Google and Facebook, use in today’s systems. Algorithms, hardware,
AI-based filtering algorithms to control access software, and data are always part of AI and
to information. AI algorithms that control self- autonomous systems. To regulate ahead of time
driving cars must decide on how to weigh the is difficult for any kind of industry. Although
safety of passengers and pedestrians.3 Vari- AI technologies are evolving rapidly, they are
ous applications, including security and safety still in the development stages. A global AI
decision-making systems, rely heavily on AI- governance system must be flexible enough to
based face recognition algorithms. And a recent accommodate cultural differences and bridge
study from Stanford University describes an gaps across different national legal systems.
AI algorithm that can deduce the sexuality of While there are many approaches we can take
people on a dating site with up to 91 percent to design a governance structure for AI, one
accuracy.4 Voicing alarm at the capabilities option is to take inspiration from the develop-
of AI evidenced within this study, and as AI ment and evolution of governance structures
technologies move toward broader adoption, that act on the Internet environment. Thus, here
some voices in society have expressed con- we discuss different issues associated with gov-
cern about the unintended consequences and ernance of AI systems, and introduce a concep-
potential downsides of widespread use of these tual framework for thinking about governance
technologies. for AI, autonomous systems, and algorithmic
To ensure transparency, accountability, and decision-making processes.
explainability for the AI ecosystem, our gov-
ernments, civil society, the private sector, and The Nature of AI
academia must be at the table to discuss gover- Although AI-based applications are increas-
nance mechanisms that minimize the risks and ingly adopted in hospitals, courtrooms, schools,
possible downsides of AI and autonomous sys- at home, and on the road to support (and in some
tems while harnessing the full potential of this instances, even guide) human decision-making,

58 Published by the IEEE Computer Society 1089-7801/17/$33.00 © 2017 IEEE IEEE INTERNET COMPUTING
14 February 2018 Published by the IEEE Computer Society  2469-7087/18/$33.00 © 2018 IEEE
A Layered Model for AI Governance

currently there is no universally similar to previous conversations on public finances if robots don’t
accepted definition of AI, a term about digital technologies’ impact on pay taxes?
coined in the mid-1950s by US re- society, the challenges related to AI,
searchers.6,7 One reason for the lack autonomous systems, and algorithms Such lists of substantive issues,
of a definition is that AI, from a are often presented and discussed in to which several others could be
technical perspective, is not a single the form of lists of substantive issues added (for instance, intellectual
technology, but rather a set of tech- (including policy, legal, governance, property or liability), can be sup-
niques and subdisciplines ranging and ethical considerations) that must plemented by cross-cutting themes
from areas such as speech recogni- be addressed.6 surrounding transparency, account-
tion and computer vision to attention A recent roadmap on AI policy ability, and explainability; inclusion
and memory, to name just a few.6 by one leading expert, for instance, and fairness; global governance; and
From a phenomenological per- identifies the following clusters of more that span across the different
spective, however, the term AI is of- core issues and questions where AI application areas of AI-based sys-
ten used as an umbrella term to refer applications either lead to new chal- tems (see https://cyber.harvard.edu
to a certain degree of autonomy ex- lenges or amplify pre-existing policy /research/ai/usecases).
hibited in advanced health diagnos- concerns and pressure points10:
tic systems, next-generation digital Models for AI Governance
tutors, self-driving cars, and other • Justice and equality. To what ex- When considering future governance
AI-based applications. Often, such tent can AI systems be designed models for AI that address the afore-
applications in turn impact human and operated to reflect human mentioned issues, it might be help-
behavior and evolve dynamically in values such as fairness, ac- ful and necessary to move beyond
ways that are at times unforeseen countability, and transparency such lists and consider some of the
by the systems’ designers. In this and avoid (new) inequalities and larger structural challenges associ-
context, the differentiation between biases? ated with the “regulation” (broadly
weak (or narrow) and strong (or gen- • Use of force. As AI-based systems defined) of AI-based technologies.
eral) AI is often used and helpful are now involved in making deci- In the following, we highlight three
when discussing the nature of AI. sions about the use of force — for such challenges that translate into
Weak AI describes the current gener- instance, in the case of autono- design requirements for a future
ation of applications that are focused mous weapons — how much human governance model of AI.
on a relatively narrow task such as control is necessary or required?
playing a game, recognizing a voice, Who bears responsibility for the Information asymmetries. While AI
or detecting certain patterns on a AI-based outputs? has the potential to shape the lives of
CT-scan. Strong AI, in contrast, • Safety and certification. Particu- billions of people, only a few experts
refers to machines with genuine larly where AI-based systems really understand the underlying
intelligence and self-awareness in have a physical manifestation, techniques. AI-based systems are of-
the sense that the machine has the how do we define and validate ten inscrutable, sometimes resulting
ability to apply intelligence to any safety thresholds — for instance, in massive information asymmetries
problem.8 At present, the technical through standard-setting and between the developers of such sys-
possibility and (potential) societal certification? tems and other stakeholders, includ-
impact of strong AI is discussed • Privacy. As AI-systems are en- ing consumers and policymakers.
controversially, while the current abled and powered by data, what An effective governance system for
adaptation of weak AI already leads are the privacy implications and AI needs to incorporate mechanisms
to a series of real governance is- new privacy threats of next- aimed at improving our collective
sues that deserve attention in the generation technologies — for understanding of the AI phenom-
present. instance, in terms of government enon in its different manifestations
surveillance or corporate influ- and contexts of application.
AI Governance Challenges ence over customers?
Following a typical pattern when • Displacement of labor and taxa- Finding normative consensus. The
new technologies become more tion. To what extent will AI- current policy and governance de-
widely available, policymakers based machines replace jobs bate is largely focused on risks and
and other stakeholders are focus- previously performed by humans, challenges associated with AI. But
ing largely on the risks and harms or at least transform what labor AI also offers tremendous potential
of AI-based technologies.9 Again, means? What are the effects of AI benefits to society, as the discussions

www.computer.org/computingedge 15
Internet Governance

The Layered Model

Society Modularity is one of the main mech-
anisms for managing complex sys-
• • Norms
Social & legal tems. Modularity aims to reduce
• • Regulation Long-term
• • Legislation the number of interdependencies
• • Criteria
that must be analyzed by identify-
Ethical layer Mid-term ing which tasks are highly interde-
• • Principles
pendent and which ones are not.15
• • Data governance
Technical layer Layering represents a particular
• • Algorithm accountability Near-term
Algorithms & data
• • Standards form of modularity, in which differ-
ent parts of the overall system are
arranged into parallel hierarchies. A
AI systems
frequently cited example of layering
is the Open System Interconnection
Figure 1. A layered model for AI governance. The interacting layers (which sit (OSI) Reference model used during
between society and AI applications) are social and legal; ethical; and technical the late 1970s.15 Another example of a
foundations that support the ethical and social layers. layered model was proposed by David
Clark16 to represent the nature of
cyberspace using a model with four
about the use of AI in the context requirements for a future gover- layers, that are: first, the people who
of Sustainable Development Goals nance model of AI point away from participate in the cyber-experience;
illustrate (see www.itu.int/en/ITU-T simple state-centric, command-and- second, the information that is stored,
/A I/Pages/201706-defau lt.aspx). control regulatory schemes toward transmitted, and transformed in
A governance model must open more complex approaches to gover- cyberspace; third, the logical build-
up spaces for cost-benefit analyses nance emergent in fields as diverse ing blocks that make up the services,
and normative consensus build- as the Internet, nanotechnology gov- and fourth, the physical foundations
ing among different stakeholders, ernance, or gene driver governance. that support the logical elements. The
particularly where tradeoffs are While the exact contours of a future scale, heterogeneity, complexity, and
involved in the design of AI sys- AI governance model are still in flux, degree of technological autonomy
tems. A future governance model advanced governance models such of AI systems require new thinking
also needs to deal with norma- as active matrix theory, polycentric about policy, law, and regulation. We
tive differences among contexts governance, hybrid regulation, and attempt to capture the complex na-
and geographies, and provide for mesh regulation can provide both ture of AI governance by using an
interoperability among different inspiration and conceptual guidance analytical model with three layers.
frameworks and approaches.11 on how such a future governance From the top down, the interacting
regime might be designed.13 In the layers are as follows:
Government mismatches. Even where next section, we highlight one fea-
we have a shared understanding of ture that is common across many of • social and legal;
AI technologies, the underlying tech- these models: the idea of modularity • ethical; and
niques, and societal consensus about embodied in the form of layered gov- • technical foundations that sup-
what is or isn’t desirable, the design ernance, which also combines dif- port the ethical and social layers.
of effective, efficient, and legiti- ferent instruments to grapple with
mate means (strategies, approaches, and address the aforementioned sub- Figure 1 shows a representation
tools, and so forth) to resolve the stantive issues, making it a shared of the layered governance model. It
aforementioned substantive issues is responsibility among all relevant will sit between society and AI ap-
challenging, given the conditions of actors involved. It is important to plications. The instruments mapped
uncertainty and complexity in the AI note that any such emerging model onto the layers can be developed at
ecosystem. But larger undercurrents must be situated in and interact with different times. In the near term,
also put limits on traditional ap- existing institutional frameworks of governance proposals could concen-
proaches to law- and policymaking applicable laws and policies, particu- trate on developing standards and
in the digital age.12 larly human rights, as the develop- principles for AI algorithms. For the
Taken together, these structural ment and deployment of AI does not mid- and long-term, nation-states
challenges and associated design take place in a vacuum.14 can work on specific legislation to

60 www.computer.org/internet/ IEEE INTERNET COMPUTING

16 ComputingEdge February 2018
A Layered Model for AI Governance

regulate mature AI applications. The an insurance company and charges a data protection for all individuals
model can be a helpful heuristic that certain group of people higher premi- within the European Union (www
illustrates how principles, policies, ums, based on variables such as gen- .eugdpr.org). It offers a (limited) “right
norms, and laws in response to AI- der or age, such a decision-making to explanation” that will oblige com-
based challenges and opportunities application would be violating the panies to explain the purpose of an
can be combined and work together, ethical principle of equal or fair algorithm and the kind of data it uses
within and across layers. treatment. when making automated decisions.21
Absent an AI-specific international
The Technical Layer The Social and Legal Layer legal framework, a global oversight
The technical layer is the foundation The social and legal layer could ad- body, which can take the form of a
of the AI governance ecosystem — dress the process of creating institu- multistakeholder committee, could be
the algorithms and data out of which tions and allocating responsibilities the curator of global principles and
it is built. AI systems and autono- for regulating AI and autonomous emerging norms for AI systems.
mous systems rely on data and al- systems. For example, Matthew
gorithms, regardless of whether Scherer20 describes a policymaking References
they are physical systems (such as body that would have the power to 1. E. Horvitz, “AI, People, and Society,” Sci-
self-driving cars and commercial define AI, create exceptions allow- ence, vol 357, no. 6346, 2017, p. 7.
robots) or software systems (such as ing for AI research to be conducted 2. National Science and Technology Council
criminal justice or medical diagnos- in certain environments without the Committee on Technology, Preparing for
tic systems, or intelligent personal researchers being subjected to strict the Future of Artificial Intelligence, tech.
assistants).17 A set of principles for liability, and establish an AI certifi- report, Executive Office of the President,
accountable algorithms and an asso- cation process. One starting point for 2016.
ciated suggested social impact state- specific norms aimed at regulating AI 3. S. Wachter, B. Mittelstadt, and L. Floridi,
ment were developed as part of a can be the principles and criteria that “Transparent, Explainable, and Account-
Dagstuhl Seminar on “Data, Respon- emerge from the ethical and techni- able AI for Robotics,” Science Robotics,
sibly.”18 The proposed principles for cal layers, in addition to pre-existing vol. 2, no. 6, 2017; doi:10.1126/scirobotics
accountable algorithms with social and more general national and inter- .aan6080.
impact are as follows: responsibility, national legal frameworks, includ- 4. S. Levin, “New AI Can Guess Whether
explainability, accuracy, auditabil- ing human rights. The layered model You’re Gay or Straight from a Photo-
ity, and fairness. The collection, use, provides a framework for thinking graph,” The Guardian, 8 Sept. 2017;
and management of data by AI algo- about AI governance, aiming at the www.theguardian.com/technology/2017
rithms, known as data governance, definition of appropriate behavior for /sep/07/new-artificial-intelligence-can
should follow principles that promote AI and autonomous systems. -te l l-whet her-you r e-gay-or-st r a ig ht
fairness and safeguard against race, -from-a-photograph.
color, national origin, religion, sex, 5. I. Rahwan, “Society-in-the-Loop: Pro-
gender, sexual orientation, disabil-
ity, or family status discrimination.19 I mplementing governance struc-
tures for AI and algorithmic
decision-making systems can occur
gramming the Algorithmic Social Con-
tract,” Ethics and Information Technology,
2017; doi:10.1007/s10676-017-9430-8.
The Ethical Layer at multiple layers and involve blended 6. P. Stone et al., “Artificial Intelligence and
On top of the technical layer, we approaches. Here, we describe some Life in 2030: One Hundred Year Study
could articulate high-level ethi- of these layers, taking into consider- on Artificial Intelligence,” Report of the
cal concerns that apply to different ation that some of them would only be 2015 Study Panel, tech report, Sept. 2016;
types of AI applications and systems. considered if the risks that certain AI https://ai100.stanford.edu/sites/default
One important source for the devel- applications present are substantial /files/ai_100_report_0831fnl.pdf.
opment of such ethical principles are and concrete. Governance processes 7. J. McCarthy et al., “A Proposal for the
human rights principles. Another ex- can range from market-oriented so- Dartmouth Summer Research Project on
ample of the emergence of AI ethics lutions to government-based struc- Artificial Intelligence,” 31 Aug. 1955.
norms is the IEEE general principles tures and can be applied nationally 8. E. Kumar, Artificial Intelligence, I.K.
for AI and autonomous systems.17 or internationally. On the regional International, 2008.
Actions driven by algorithms can be level, a rich example is the General 9. R. Brownsword and K. Young, eds., Regu-
assessed according to ethical criteria Data Protection Regulation (GDPR), lating Technologies: Legal Futures, Regula-
and principles. For instance, when an a wide-ranging and complex regula- tory Frames, and Technological Fixes, Hart,
AI application analyzes the data of tion intended to strengthen and unify 2008.

www.computer.org/computingedge 17
Internet Governance

10. R. Calo, “Artificial Intelligence Policy: A and Autonomous Systems, Ethically AI initiative, and serves as a professor of
Roadmap,” Social Science Research Net- Aligned Design: A Vision for Prioritizing practice at Harvard Law School. His re-
work (SSRN), 8 Aug. 2017; https://ssrn Human Wellbeing with Artificial Intelli- search and teaching focus on the interplay
.com/abstract3015350. gence and Autonomous Systems (AI/AS), between law and technology. Gasser is a
11. J. Palfrey and U. Gasser, Interop: The IEEE, 2017; http://standards.ieee.org graduate of the University of St. Gallen
Promise and Perils of Highly Intercon- /develop/indconn/ec/ead_v1.pdf. and Harvard Law School. Contact him at
nected Systems, Basic Books, 2012. 18. S. Abiteboul et al., “Data, Responsibly ugasser@cyber.harvard.edu.
12. C. Scott, “Regulation in the Age of Gov- (Dagstuhl Seminar 16291),” Dagstuhl Re-
ernance: The Rise of the Post-Regulatory ports, vol. 6, no. 7, 2016, pp 42–71. Virgilio A.F. Almeida is a faculty associate
State,” The Politics of Regulation: Institu- 19. National Science and Technology Coun- at the Berkman Klein Center for Inter-
tions and Regulatory Reforms for the Age of cil Committee on Technology, Big Data: net and Society at Harvard University,
Governance, J. Jordana and D. Levi-Faur, A Report on Algorithmic Systems, Op- and a professor in the Computer Science
eds., Edward Elgar, 2004, pp. 145–174. portunity, and Civil Rights, tech. report, Department at the Federal University
13. R.H. Weber, Realizing a New Global Executive Office of the President, 2016. of Minas Gerais (UFMG), Brazil. His re-
Cyberspace Framework: Normative Foun- 20. M. Scherer, “Regulating Artificial Intel- search interests include cyber policies,
dations and Guiding Principles, Schul- ligence Systems: Risks, Challenges, Com- large-scale distributed systems, the In-
thess 2014. petencies, and Strategies,” Harvard J. Law ternet, and social computing. Almeida
14. U. Gasser, “AI and the Law: Setting the & Technology, vol. 29, no. 2, 2016; http:// has a PhD in computer science from
Stage,” Medium, 26 June 2017; https:/ dx.doi.org/10.2139/ssrn.2609777. Vanderbilt University. Contact him at
/medium.com/berkman-klein-center/ai-and 21. S. Wachter, B. Mittelstadt, and L. Floridi, virgilio@dcc.ufmg.br or valmeida@
-the-law-setting-the-stage-48516fda1b11. “Why a Right to Explanation of Auto- cyber.harvard.edu.
15. C.S. Yoo, “Protocol Layering and Internet mated Decision-Making Does Not Exist in
Policy,” Faculty Scholarship Paper 454, the General Data Protection Regulation,”
Univ. of Pennsylvania, 2013; scholarship Int’l Data Privacy Law, 2017; https://ssrn
.law.upenn.edu/faculty_scholarship/454. .com/abstract2903469.
16. D. Clark, “Characterizing Cyberspace:
Read yourappeared
This article originally subscriptions
Past, Present, and Future,” MIT CSAIL, Urs Gasser is the executive director of the
through the
IEEE Internet Computing, myCS
v. 1.2, 12 Mar. 2010. Berkman Klein Center for Internet & So-
17. The IEEE Global Initiative for Ethical ciety at Harvard University, where he no. 6, 2017. cations portal at http://
Considerations in Artificial Intelligence co-leads the Ethics and Governance of

Subscribe today for the latest in computational science and engineering research, news and analysis,
CSE in education, and emerging technologies in the hard sciences.


62 www.computer.org/internet/ IEEE INTERNET COMPUTING

18 ComputingEdge February 2018

Deep Learning Triggers a New

Era in Industrial Robotics

Ryo Miyajima ne reason deep learning has attracted area. Either way, experienced human operators
Preferred Networks the attention of so many researchers must carefully tune parameters to achieve reli-
and engineers, even outside of the AI able results.
community, is because it can capture abstract Researchers and engineers at Preferred
features and recognize patterns in ways many Networks and FANUC have teamed up to
once thought impossible for computers. The demonstrate that deep learning could offer
breakthrough was exemplified by the emer- an alternative solution for this task. We use
gence of AlphaGo.1 Prior to the successes a depth camera image around a given suc-
of AlphaGo, experts had thought that the tion hand position as input, together with the
abstract strategical thoughts and theories that output representing whether that suction was
a human Go player develops over time through successful. Through trial and error, we collect
training and experience were not replicable by this input and output pair with the actual
a computer—that is, without some technolog- robot, initially starting with a random policy
ical breakthrough. AlphaGo proved that deep (see Figure 2). By training the deep neural
learning is indeed that breakthrough. network with thousands of these inputs and
The pattern recognition capabilities of outputs, we have achieved 90 percent accuracy,
deep learning have pushed the limits in various which is comparable to a conventional system
fields—and industrial robotics is no exception. whose parameters are tuned by experienced
Deep learning will arguably not solve all of the operators.2 Furthermore, our deep learning
problems we encounter in industrial robotics, approach doesn’t require us to predefine the
but it will improve the perception capabilities object’s appearance or shape.
of robotics systems, given its power to rec-
ognize complex real-world patterns robustly. Amazon Picking Challenge
Here, I examine some deep learning applica- Another, more complex application that might
tions in robotics. benefit from deep learning involves picking
items of various shapes and forms stored on
Automated Bin Picking shelves. The Amazon Picking Challenge is
Editor: Tao Mei, Microsoft Research Asia

Let’s first consider the application of auto- a competition held by Amazon that aims to
matically picking steel cylinders with a suction “strengthen the ties between the industrial
hand, as shown in Figure 1a. Given a depth and academic robotic communities to pro-
camera image as input, the robot system is mote shared and open solutions to some of
expected to figure out a point within the bin the big problems in unstructured automation”
at which the suction hand will most reliably (www.robocup2016.org/en/events/amazon
suck a cylinder. Conventional systems usually -picking-challenge).
tackle this task by matching the image with pre- For the 2016 Challenge, competitors were
defined photographs or CAD data (see Figure instructed to pick items off a shelf and place
1b). If the object’s appearance or shape cannot them back again. As easy as this task is for
be predefined, conventional systems will often humans, the industry has yet to see automated
look for a planar surface larger than a certain robots replace all human pickers. Of the 15

Published by the©IEEE
2469-7087/18/$33.00 Computer
2018 IEEE Society Published by the IEEE Computer Society 1070-986X/17/$33.00 ©2017 IEEE
February 2018 91

it over algorithms that rely on hand-crafted

features. The Faster Region-based Convo-
lutional Neural Network (Faster-RCNN)
system was one of the most popular deep
neural-network-based object detection sys-
tems when the team started working on their
system. It had shown state-of-the-art accuracy
(a) (b)
when it was released on datasets such as Pas-
cal’s visual object challenge (VOC) 2007 and
FIGURE 1. A bin picking application of steel cylinders: (a) Given a depth 2012 and the Microsoft Common Objects in
camera image as input, the robot system is expected to figure out a Context (MS COCO) dataset.4 It also had the
point within the bin at which the suction hand will most reliably suck a advantage of running at near real-time frame
cylinder. (Photo courtesy of Eiichi Matsumoto; used with permission.) (b) rates, which was essential for developing a fast
Conventional systems usually tackle this task by matching the image robotic system. The underlying idea behind
with predefined photographs or CAD data. (Photo courtesy of FANUC Faster-RCNN is to use a fully convolutional
Corporation; used with permission.) network that generates high-quality region
proposals (that is, bounding boxes that enclose
an object and their “objectness score”).
teams that advanced to the finals, two-thirds
used deep learning3 in their system, including Object Pose Estimation
the top scoring teams. As an example of how The next step in the pipeline is object pose esti-
deep learning is integrated into these compet- mation. According to Morariu, the Team Delft
itive systems, I look at the work of last year’s system estimated the 6D pose of the detected
winner, Team Delft—a joint team from the object and matched a premade CAD model
TU Delft Robotics Institute and the company of the object against the real-time point cloud
Delft Robotics. retrieved from the camera. Using the bounding
I spoke with former Team Delft mem- box obtained from Faster R-CNN, only the
ber Wilson Ko to learn more about how they relevant part of the captured point cloud was
used deep learning in their object detection being matched. For deformable items, how-
component to classify objects in a camera ever, a premade CAD model is meaningless
image and output the bounding box for each because the objects change in shape. Instead,
object (Figure 3). Although Team Delft’s object pose estimation was skipped and the
success cannot be solely attributed to the use system generated grasps directly on the filtered
of deep learning, Ko asserted that properly point cloud.
integrating deep learning into conventional For the rigid items, Morariu explained
systems will bring us closer to fully autono- that the team used the Super 4-Points Con-
mous, robust picking systems. In the follow- gruent Sets (Super 4PCS) algorithm5 to do the
ing, each module in Team Delft’s pipeline matching. The system can use the transforma-
(the process from recognizing the object to tion that is found at the end of this process to
sending a motor command to the robot) is estimate the object’s pose with regard to the
explained in detail, based on information robot. Finally, the system refines the object
provided by Ko and his former teammate pose using the Iterative Closest Point (ICP)
Mihai Morariu. algorithm.

Object Detection Grasp Generation

Because deep learning has shown impres- According to Ko, after the system determined
sive performance during the last few years the object’s pose, it used this information
in addressing object detection problems, to generate the grasp poses, and the grasps
Morariu said that the team decided to use were predefined for the CAD model in the

20 IEEE MultiMedia
ComputingEdge www.computer.org/multimedia
February 2018
.. .. success/
.. .. failure
Global depth
camera image ....

Deep neural network

Success probability
Train and update
neural network
(every few Best place to
hundred iterations) suck based
on prediction



Local depth camera image Record

with success/failure labels success/
Run real-world robot

FIGURE 2. Overview of our novel bin picking system. It uses a depth camera image around a given suction hand position
as input, together with the output representing whether that suction was successful. Through trial and error, we collect this
input and output pair with the actual robot, initially starting with a random policy. We then train the deep neural network with
thousands of these inputs and outputs.

environment. The team used shape primitives The robot used online motions to move
to describe the object geometry and predefine inside the shelf, and such motions are vari-
the grasps. Because an estimate of the object able because they depend on the target’s loca-
pose was available, the predefined grasp poses tion and orientation. Online motions were
could be transformed with the object pose, split into different parts: approach, contact,
providing the robot with poses to move into lift, and retreat. Again, MoveIt was employed
so it could pick up the item. The system then here for collision checking, with Trac-IK and
scored the grasps and eliminated some based RRT-Connect plugins for inverse kinematics
on reachability and robustness. For deformable and path planning.
items, the system used surface normals of the To execute the motions, the team used
segmented point cloud to generate grasp poses. the MotoROS driver from Robot Operating
System (ROS)-Industrial and enhanced it to
Motion Planning fit their needs.
Ko explained that the team distinguished
between two types of trajectories: offline and Inspiring Research
online motions. Offline motions were used for Here, I introduce some promising research that
motions outside of the shelf, which could be will likely impact how industrial robot systems
pre-generated using RRT-Connect of MoveIt are designed in the near future.
(http://moveit.ros.org), based on Rapidly
exploring Random Trees, RRTs, and still be End-to-End Training
collision free, because it was assumed that the Previously the “best practice” for designing
environment outside the shelf would remain robotics systems was to connect modular sys-
static. tem components as a pipeline. For example, if

www.computer.org/computingedge October–December 2017 93


FIGURE 3. Team Delft used deep learning in their object detection component to classify the objects in the camera image
and output the bounding box for each object: detection for the (a) picking task and (b) stowing task. (Photos courtesy of
Carlos Hernandez Corbato; used with permission.)

you were to design a robot that serves coffee, The terminology “end-to-end” refers to
you might build system components, such as methods that don’t require intermediate com-
a state estimator that realizes which phase of ponents in this sense. Naturally enough, the
pouring the coffee the robot is in, a planner to transition from “pipelines” to “end-to-end” is
designate the next action the robot should take, happening in the robotics field as well. One
a controller to actuate the motors in the robot example is the work from the University of
so that the desired action is achieved, and so on. California, Berkeley, in which a robotic arm
A similar approach might be applied to was trained end-to-end to perform tasks like
program a computer to play video games. inserting toy blocks into boxes.7
Let’s say you want a computer to play the Admittedly, with the current technol-
game of Pong. The program receives only the ogy, end-to-end trained systems will not be
video image of the game, so you could build a as precise and accurate as conventional sys-
module to figure out the abstract state of the tems that are tweaked and tuned for a very
player (for example, where is the ball and pad- specific task, such as controlling the position
dle?). You could build another module to plan of a robotic arm. However, being able to teach
what the player should do (in which direc- abstract tasks (opening the cap of a bottle, for
tion should the ball be hit back, and where example) to robots end-to-end was not some-
should the paddle be to achieve that?), and thing experts thought was practical until a few
yet another module to decide the actual com- years ago. With new technology, we might see
mands to input into the game (which button robotic system designs radically reshaped in
should the player press, if any?). Finally, you’d the near future.
connect all these modules.
A group from DeepMind showed that Toward Robust Grasping
you don’t need to go through the trouble of As exciting and astonishing as the state-of-the-
building all of these modules—by combin- art achievements of deep learning are, many
ing deep learning and some techniques from will point out the caveats of the technology. For
a field called reinforcement learning, you can example, deep learning requires a large dataset
directly train a computer program to output the for training, which is why many of those work-
game commands given visual images as input.6 ing on automated grasping have welcomed the
DeepMind’s trained computer program has release of Dex-Net 2.0 from the University
outperformed humans in some games. of California, Berkeley. Dex-Net 2.0 contains

22 IEEE MultiMedia
ComputingEdge www.computer.org/multimedia
February 2018
millions of datapoints to train a deep learning its repetitive work will likely be recovered in
network8 that tells you the quality of a grasp the foreseeable future. However, there are two
with a parallel jaw gripper. issues that need to be addressed. First, there
Collecting large amounts of data for deep is a growing demand for automation in mass
learning reminds me of the work at Goo- customization solutions (as opposed to mass
gle Research, where they ran as many as 14 production). Second, even in some mass pro-
robots simultaneously over the course of two duction factories that will benefit from auto-
months to collect 800,000 grasp attempts.9 mation, we still see human workers perform-
In contrast, Dex-Net does not rely on time ing repetitive tasks that are technologically
and a vast number of robots to collect data. challenging (requiring dexterity, for example)
Rather, it exploits physics-based models so or not worth the investment to automate. In
that grasping attempts can be synthesized either case, deep learning is a promising tech-
instead of experimenting with the grasp in nology for cultivating undeveloped areas and
the real world. providing us with more robust, adaptive, and
For example, let’s say you want to pick reliable systems.
up a cube with two fingers. You immediately
choose to place your fingers on the two fac- References
ing sides and not on any other combination of 1. D. Silver et al., “Mastering the Game of
sides. Your intuition about your grasp quality Go with Deep Neural Networks and Tree
is in line with what grasp analytics predict. Search,” Nature, 28 Jan. 2016; doi:10.1038
Therefore, instead of spending time and using /nature16961.
numerous robots to predict the outcome of 2. E. Matsumoto, “Learning from 0 in
every possible grasp, it makes sense to use our a Bulk Loading Robot with Deep
knowledge about grasp quality and inject it Learning,” Preferred Networks Research
into the training dataset to train a deep neural Blog, 2015; https://research.preferred
network. .jp/2015/12/robot_binpick_deep_learning
(in Japanese).
3. “Detail Report on Amazon Picking Chal-

O ne impressive trait of these methods that

use deep learning is their ability to adapt
to data they didn’t see in the training phase—a
lenge 2016 (First Part),” Nikkei Robotics [in
Japanese], Sept. 2016; http://ec.nikkeibp
capability is referred to as “generalization.” 4. S. Ren et al., “Faster R-CNN: Towards
Even without deep learning, you could pro- Real-Time Object Detection with Region
gram a robot arm to grasp a specific object. Proposal Networks,” Proc. Advances in Neu-
However, that robot will usually have difficul- ral Information Processing Systems (NIPS),
ties adapting to other objects of various shape, 2015, https://arxiv.org/abs/1506.01497.
friction, or appearance without reprogram- 5. N. Mellado, D. Aiger, and N.J. Mitra,
ming. Deep learning relaxes that restriction, “Super 4PCS Fast Global Pointcloud
and it is exactly that capability that makes it a Registration via Smart Indexing,” Com-
promising method with which to tackle many puter Graphics Forum, vol. 33, no. 5, 2014,
of the unsolved problems and challenges in pp. 205–215.
industrial robotics. 6. V. Mnih et al., “Human-Level Con-
Conventional industrial robotics was trol through Deep Reinforcement
all about controlling and reducing the vari- Learning,” Nature, vol. 518, Feb. 2015,
ance of the environment so that unintelligent pp. 529–533.
robots could do their repetitive work. Perhaps 7. S. Levine et al., “End-to-End Training
this practice is acceptable if the system is for of Deep Visuomotor Policies,” J. Machine
mass production and the initial investment in Learning Research, vol. 17, no. 1, 2016,
programming/teaching the robot to perform pp. 1334–1373.

www.computer.org/computingedge October–December 2017 95


8. J. Mahler et al., “Dex-Net 2.0: Deep Ryo Miyajima is an engineer at Preferred

Learning to Plan Robust Grasps with Networks. Contact him at ryo@preferred.jp.
Synthetic Point Clouds and Analytic
Grasp Metrics,” Proc. Robotics Science and
Systems, 2017; www.roboticsconference This article originally appeared in
.org/program/papers/19. IEEE MultiMedia, vol. 24, no. 4, 2017.
9. S. Levine et al., “Learning Hand-Eye
Coordination for Robotic Grasping with
Read your subscriptions through
Deep Learning and Large-Scale Data the myCS publications portal at
Collection,” Int’l J. Robotics Research, Mar. http://mycs.computer.org
2016; https://arxiv.org/abs/1603.02199.

This series of in-depth interviews with

prominent security experts features Gary
McGraw as anchor. IEEE Security & Privacy
magazine publishes excerpts of the 20-minute
conversations in article format each issue.

*Also available at iTunes

24 IEEE MultiMedia
ComputingEdge www.computer.org/multimedia
February 2018
EDITOR: Irena Bojanova, NIST, irena.bojanova@computer.org

How Much to
Trust Artificial

George Hurlburt, STEMCorp

here has been a great regimes. This results in vehicles but the driver failed to appreciate
deal of recent buzz about that obey the rules of the road, that his autopilot still required his
the rather dated notion except when they do not. Such full, undivided attention. In this
of artificial intelligence was the case when a motor vehi- rare case, misplaced trust in an
(AI). AI surrounds us, involv- cle in autonomous mode broad- AI-based system turned deadly.
ing numerous applications rang- sided a turning truck in Florida,
ing from Google search, to Uber killing its “driver.” The accident Establishing a Bar for Trust
or Lyft ride-summoning, to air- was ultimately attributed to driver AI advancement is indeed impres-
line pricing, to Alexa or Siri. To error, as the autonomous con- sive. DARPA, sponsor of early
some, AI is a form of salvation, trols were deemed to be perform- successful autonomous vehi-
ultimately improving quality of ing within their design envelope. cle competitions, completed the
life while infusing innovation The avoidance system design at Cyber Grand Challenge (CGC)
across myriad established indus- the time required that the radar competition in late 2016. The
tries. Others, however, sound dire and visual systems agree before CGC established that machines,
warnings that we will all soon evasive action would be engaged. acting alone, could play an estab-
be totally subjugated to superior Evidence suggests, however, that lished live hacker’s game known as
machine intelligence. AI is typi- the visual system encountered Capture the Flag. Here, a “flag” is
cally, but no longer always, soft- glare from the white truck turn- hidden in code, and the hacker’s
ware dominant, and software is ing against bright sunlight. This job is to exploit vulnerabilities to
prone to vulnerabilities. Given system neither perceived nor reach and compromise an oppo-
this, how do we know that the AI responded to the looming haz- nent’s flag. The CGC offered a $2
itself is sufficiently reliable to do ard. At impact, however, other million prize to the winning team
its job, or—put more succinctly— evidence implicated the “driver,” that most successfully competed
how much should we trust the who was watching a Harry Potter in the game. The final CGC round
outcomes generated by AI? movie. The driver, evidently over- pitted seven machines against one
confident of the autopilot, did another on a common closed net-
Risks of Misplaced Trust not actively monitor its behavior work without any human inter-
Consider the case of self-driv- and failed to override it, despite vention. The machines had to
ing cars. Elements of AI come an estimated seven-second vis- identify vulnerabilities in an oppo-
into play in growing num- ible risk of collision.1 The design nent’s system, fix them on their
bers of self-driving car autopilot assurance level was established, own system, and exploit them in

2469-7087/18/$33.00 © 2018 IEEE Published by the IEEE Computer Society February 2018 25
1520-9202/17/$33.00 © 2017 IEEE Published by the IEEE Computer Society computer.org/ITPro 7

opponents’ systems to capture the

flag. Team Mayhem from Carne-
gie Mellon University was declared
the winner.2
John Launchbury, director of
DARPA’s Information Innovation
Office, characterizes the type of
AI associated with the CGC as
handcrafted knowledge. Emerging
from early expert systems, this
technology remains vital to the
advancement of modern AI. In
handcrafted knowledge, systems
reason against elaborate, manu-
ally defined rule sets. This type of
AI has strength in reasoning but
is limited in forms of perception.
However, it possesses no ability to
learn or perform abstraction.3
While building confidence that Figure 1. Some prevalent AI machine learning algorithms.
future reasoning AI can indeed
rapidly diagnose and repair soft-
ware vulnerabilities, it is impor-
tant to note that the CGC was capable of harnessing the same AI systems could go awry in unex-
intentionally limited in scope. vulnerabilities to assume control pected ways, effectively defining
The open source operating sys- over others’ networks, including the level of trust in AI based tools
tem extension was simplified for the growing and potentially vul- becomes a high hurdle.6
purposes of the competition,4 and nerable Internet of Things (IoT). At its core, AI is a high-order
known malware instances were This concern prompted the Elec- construct. In practice, numerous
implanted as watered-down ver- tronic Frontier Foundation to loosely federated practices and
sions of their real-life counter- call for a “moral code” among AI algorithms appear to compose
parts.5 This intentionally eased developers to limit reasoning sys- most AI instances—often cross-
the development burden, permit- tems to perform in a trustworthy ing many topical domains. Indeed,
ted a uniform basis for competi- fashion.4 AI extends well beyond com-
tive evaluation, and reduced the puter science to include domains
risk of releasing competitors’ soft- Machine Learning Ups such as neuroscience, linguistics,
ware into the larger networked the Trust Ante mathematics, statistics, physics,
world without requiring signifi- Launchbury ascribes the term sta- psychology, physiology, network
cant modification. tistical learning to what he deems science, ethics, and many others.
The use of “dirty tricks” to the second wave of AI. Here, Figure 1 depicts a less than fully
defeat an opponent in the game perception and learning are inclusive list of algorithms that
adds yet another, darker dimen- strong, but the technology lacks underlie second-wave AI phe-
sion. Although the ability to re- any ability to perform reasoning nomena, often collectively known
engineer code to rapidly isolate and abstraction. While statisti- as machine learning.
and fix vulnerabilities is good, it is cally impressive, machine learn- This myriad of potential under-
quite another thing to turn these ing periodically produces indi- lying algorithms and methods
vulnerabilities into opportunities vidually unreliable results, often available to achieve some state of
that efficiently exploit other code. manifesting as bizarre outliers. machine learning raises some sig-
Some fear that if such a capabil- Machine learning can also be nificant trust issues, especially for
ity were to be unleashed and grow skewed over time by tainted train- those involved in software testing
out of control, it could become a ing data.3 Given that not all AI as an established means to assure
form of “supercode”—both exempt learning yields predictable out- trust. When the AI becomes asso-
from common vulnerabilities and comes, leading to the reality that ciated with mission criticality, as

26 ComputingEdge February 2018

8 IT Pro July/August 2017
is increasingly the case, the tester One high-level AI test assesses under controlled conditions, sig-
must establish the basis for multi- the ability to correctly recognize nificant differences result between
ple factors, such as programmatic and classify an image. In some the use of single or multiple well-
consistency, repeatability, penetra- instances, this test has surpassed validated datasets used to train and
bility, applied path tracing, or iden- human capability to make such test classifiers. Thus, even con-
tifiable systemic failure modes. assessments. For example, the trolled testing for classifiers can
The nontrivial question of Labeled Faces in the Wild (LFW) become highly complicated and
what is the most appropriate dataset supports facial recogni- must be approached carefully.8
AI algorithm goes as far back as tion with some 13,000 images Other trust-related factors
1976.3 The everyday AI practi- to train and calibrate facial rec- extend well beyond code. Because
tioner faces perplexing issues ognition machine learning tools coding is simultaneously a cre-
regarding which is the right algo- using either neural nets or deep ative act and somewhat of a syn-
rithm to use to suit the desired learning. The new automated AI tactic science, it is subject to some
AI design. Given an intended image recognition tools can sta- degree of interpretation. It is fea-
outcome, which algorithm is tistically outperform human facial sible that a coder can inject either
intentional or unintentional cul-
tural or personal bias into the
The everyday AI practitioner faces perplexing issues resulting AI code. Consider the
case of the coder who creates a
regarding which is the right algorithm to use to suit highly accurate facial recognition
the desired AI design. routine but neglects to consider
skin pigmentation as a deciding
factor among the recognition cri-
the most accurate? Which is recognition capability using this teria. This action could skew the
the most efficient? Which is the dataset.7 The task at hand, how- results away from features oth-
most straightforward to implement ever, is fundamentally perceptual erwise reinforced by skin color.
in the anticipated environment? in nature. These tasks functionally Conversely, the rates of recid-
Which one holds the greatest discriminate through mathemat- ivism among criminals skews
potential for the least corruption ically correlated geometric pat- some AI-based prison release
over time? Which ones are the terns but stop short of any form of decisions along racial lines. This
most familiar and thus the most higher-order cognitive reasoning. means that some incarcerated
likely to be engaged? Is the design Moreover, while it compares selec- individuals stand a better statisti-
based on some form of central- tive recognition accuracy against cal chance of gaining early release
ity, distributed agents, or even human ability, other mission-crit- than others—regardless of pre-
swarming software agency? How ical aspects of the underlying code vailing circumstances.9 Semantic
is this all to be tested? base remain unchecked under this inconsistency can further jeop-
These questions suggest that test. ardize the neutrality of AI code,
necessary design tradeoffs exist especially if natural language
between a wide range of alterna- Beyond the Code processing or idiomatic speech
tive AI-related algorithms and Testing machine learning becomes recognition are involved.
techniques. The fact that such further complicated as extensive Some suggest that all IT careers
alternative approaches to AI exist datasets are required to “train” the are now cybersecurity careers.10
at all suggests that most AI archi- AI in a learning environment. Not This too has a huge implica-
tectures are far from consistent only should the AI code be shown tion for the field of AI develop-
or cohesive. Worse, a high degree to be flawless, but the data used in ment and its implementation. The
of contextually-based customiza- the training should theoretically question of “who knows what the
tion is required for both reason- bear the highest pedigree. In the machine knew and when it knew
ing and learning systems. This, real world, however, datasets often it” becomes significant from a
of course, extends to AI testing, tend to be unbalanced, sparse, cybersecurity standpoint. What a
because each algorithm and its inconsistent, and often inaccurate, machine learns is often not readily
custom implementation brings if not totally corrupt. Figure 2 sug- observable, but rather lies deeply
its own unique deep testing chal- gests that information often results encoded. This not only affects
lenges, even at the unit level. from resolving ambiguity. Even newly internalized data, but—in

www.computer.org/computingedge 27
computer.org/ITPro 9

the IoT—these data can trip deci-

sion triggers to enliven actua-
tors that translate the “learning”
into some sort of action. Lack-
ing concrete stimulus identity and
pedigree, the overall AI-sparked
IoT stimulus-response mecha-
nism becomes equally uncertain.
Nonetheless, the resulting actions
in mission-critical systems require
rigorous validation.

The Third Wave

Launchbury foresees the need
for a yet-to-be-perfected third
wave of AI, which he names con-
textual adaptation. This technol-
ogy, requiring much more work,
brings together strengths in per-
ception, learning, and reason-
ing and supports a significantly Figure 2. Information provenance can often be unclear.
heightened level of cross-domain
The 2017 Ontology Sum-
mit, aptly entitled “AI, Learning, viewed in context. AI, largely emerging from applied network
Reasoning, and Ontologies,” subsymbolic today, will need to science, offer a better means of
concluded in May 2017. Rein- deal with applied semantics in a assessing dynamic AI behav-
forcing Launchbury’s observa- far more formal sense to achieve ior that emerges over time. This
tion, the draft summit commu- third-wave status. Under such cir- becomes increasingly true as
nique concluded that, to date, cumstances, AI becomes nonlin- the temporal metrics associ-
most AI approaches, including ear, in which cause and effect are ated with graph theory become
machine learning tools, oper- increasingly decoupled via multi- better understood as a means
ate at a subsymbolic level using ple execution threads. This leads of describing dynamic behaviors
computational techniques that do to the establishment of complex that fail to follow linear paths to
not approximate human thought. adaptive systems (CAS), which tend achieve some desired effect.12
Although great progress has been to adhere to and be influenced by
achieved in many forms of AI, the nonlinear network behavior.

full treatment of knowledge rep- In a CAS, new behaviors ntil some reliable method-
resentation at the symbolic level emerge based on environmen- ology is adopted for the as-
awaits maturity (bit.ly/2qMN0it). tal circumstance over time. Here, sessment of assured trust
Correspondingly, the utility of there can be multiple self-orga- within AI, the watchword must
ontology as a formal semantic nizing paths leading to success be caution. Any tendency to put
organizing tool offers only limited or failure, all triggered by highly blind faith in what in effect remains
advantages to AI and its ultimate diversified nodes and arcs that largely untrusted technology can
test environment. can come, grow, shrink, and go lead to misleading and sometimes
The semantic network involves over time. Such networks defy dangerous conclusions.
graph representations of knowl- traditional recursive unit testing
edge in the form of nodes and when composed using embed- References
arcs. It provides a way to under- ded software, which is interre- 1. N.E. Boudette, “Tesla’s Self-Driving
stand and visualize relationships lated to data. This is because in a System Cleared in Deadly Crash,”
between symbols, often repre- CAS, the whole often becomes far New York Times, 19 Jan. 2017.
sented by active words, which more than merely the sum of the 2. D. Coldewey, “Carnegie Mellon’s
convey varying meanings when parts.11 Rather, new approaches, Mayhem AI Takes Home $2 Million

28 ComputingEdge February 2018

10 IT Pro July/August 2017
from DARPA’s Cyber Grand Chal- 7. A. Jacob, “Forget the Turing Test— George Hurlburt is chief scientist at
lenge,” TechCrunch, 5 Aug. 2016; There Are Better Ways of Judging STEMCorp, a nonprofit that works to
tcrn.ch/2aM3iS7. AI,” New Scientist, 21 Sept. 2015; bit. further economic development via adop-
3. J. Launchbury, “A DARPA Perspec- ly/1MoMUnF. tion of network science and to advance
tive on Artificial Intelligence,” DAR- 8. J. Demsar, “Statistical Compari- autonomous technologies as useful tools
PAtv, 15 Feb. 2017; www.youtube sons of Classifiers over Multiple for human use. He is engaged in dynamic
.com/watch?v5-O01G3tSYpU. Data Sets,” J. Machine Learning Re- graph-based Internet of Things architec-
4. N. Cardozo, P. Eckersley, and J. Gil- search, vol. 7, 2006, pp. 1–30. ture. Hurlburt is on the editorial board of
lula, “Does DARPA’s Cyber Grand 9. H. Reese, “Bias in Machine IT Professional and is a member of the
Challenge Need a Safety Protocol?” Learning, and How to Stop It,” board of governors of the Southern Mary-
Electronic Frontier Foundation, TechRepublic, 18 Nov. 2016; tek.io land Higher Education Center. Contact
4 Aug. 2016; bit.ly/2aPxRXc. /2gcqFrI. him at ghurlburt@change-index.com.
5. A. Nordrum, “Autonomous Security 10. C. Mims, “All IT Jobs Are Cyberse-
Bots Seek and Destroy Software curity Jobs Now,” Wall Street J., 17
Bugs in DARPA Cyber Grand Chal- May 2017; on.wsj.com/2qH5VP2.
lenge,” IEEE Spectrum, Aug. 2016; 11. P. Erdi, Complexity Explained, Springer-
bit.ly/2arLOcR. Verlag, 2008. Read your subscriptions
6. S. Jontz, “Cyber Network, Heal 12. N. Masuda and R. Lambiotte, A through the myCS
This article originally appeared in
publications portal at
Thyself,” Signal, 1 Apr. 2017; bit Guide to Temporal Networks, World IT Professional, vol. 19, no. 4, 2017.
.ly/2o0ZCVe. Scientific Publishing, 2016.

Executive Director: Angela R. Burgess; Director, Governance & Associate
Executive Director: Anne Marie Kelly; Director, Finance & Accounting:
PURPOSE: The IEEE Computer Society is the world’s largest association Sunny Hwang; Director, Information Technology & Services: Sumit
of computing professionals and is the leading provider of technical STANDARDS AWARD
Director, Membership NOMINATIONS
Development: Eric Berkowitz; Director,
information in the field. Products & Services: Evan M. Butterfield
MEMBERSHIP: Members receive the monthly magazine Computer,IEEE COMPUTER SOCIETY HANS K ARLSSON
discounts, and opportunities to serve (all activities are led by volunteer COMPUTERSTANDARDS SOCIETY OFFICES
members). Membership is open to all IEEE members, affiliate society Washington, D.C.: 2001 L St., Ste. 700, Washington, D.C. 20036-4928 Phone:
members, and others interested in the computer field. +1 202 371 0101 • Fax: +1 202 728 9614 • Email: hq.ofc@computer.org Los
OMBUDSMAN: Email ombudsman@computer.org. AAlamitos: and $2,000
plaque 10662 honorarium
Los Vaqueros is presented
Circle, Los in 90720
Alamitos, CA recognition of
COMPUTER SOCIETY WEBSITE: www.computer.org Phone: +1 714
outstanding 821
skills 8380
and • Email: to
dedication help@computer.org
diplomacy, team facilitation, and
joint achievement
MEMBERSHIP in the development
& PUBLICATION or promotion of standards in the
Next Board Meeting: 1-2 February 2018, Anaheim, CA, USA
Phone: +1 industry
800 272where
6657 •individual aspirations,
Fax: +1 714 821 4641 corporate competition,
• Email: help@computer.org
and Watanabe
organizational Building,
rivalry could 1-4-2 Minami-Aoyama,
otherwise be counter Minato-ku, Tokyo
to the benefit
107-0062, Japan • Phone: +81 3 3408 3118 • Fax: +81 3 3408 3553 • Email:
President: Hironori Kasahara; President-Elect: Cecilia Metra; Past President: of society.
Jean-Luc Gaudiot; First VP, Publications: Greg T. Byrd; Second VP, Secretary tokyo.ofc@computer.org
Dennis J. Frailey; VP, Member & Geographic Activities: Forrest Shull; VP,
Professional & Educational Activities: Andy T. Chen; VP, Standards

Activities: Jon Rosdahl; VP, Technical & Conference Activities: Hausi A. President & CEO: James Jefferies; President-Elect: Jose M.F. Moura; Past
Müller; 2017-2018 IEEE Division VIII Director: Dejan S. Milojičić; 2018-2019 President: Karen Bartleson; Secretary: William Walsh; Treasurer: Joseph V.
Little; Director & President, IEEE-USA: Sandra "Candy" Robinson;
IEEE Division V Director: John W. Walz; 2018 IEEE Director-Elect Division3 endorsements.
• Requires Director & President, Standards Association: Forrest D. Wright; Director
VIII Director-Elect: Elizabeth L. Burd
• Self-nominations & are
VP, not accepted. Activities: Witold M. Kinsner; Director & VP,
BOARD OF GOVERNORS Membership & Geographic Activities: Martin Bastiaans; Director & VP,
• Do not need IEEE or IEEE Computer Society membership to apply.
Term Expiring 2018: Ann DeMarle, Sven Dietrich, Fred Douglis, Vladimir Publication Services and Products: Samir El-Ghazaly; Director & VP,
Getov, Bruce M. McMillin, Kunio Uchiyama, Stefano Zanero Technical Activities: Susan "Kathy" Land; Director & Delegate Division V:
Term Expiring 2019: Saurabh Bagchi, Leila De Floriani, David S. Ebert, Jill I. John W. Walz; Director & Delegate Division VIII: Dejan S. Milojičić
Submit your nomination electronically: awards.computer.org | Questions: awards@computer.org
Gostin, William Gropp, Sumi Helal, Avi Mendelson
Term Expiring 2020: Andy Chen, John Johnson, Sy-Ken Kuo, David Lomet,
Dimitrios Serpanos, Forrest Shull, Hayato Yamana
revised 13 December 2017

www.computer.org/computingedge 29
computer.org/ITPro 11
Human Augmentation
Editor: Albrecht Schmidt n University of Stuttgart n albrecht@computer.org

Augmenting Human Intellect

and Amplifying Perception
and Cognition
Albrecht Schmidt, University of Stuttgart

I n this first installment of IEEE Perva-

sive Computing’s new Human Aug-
mentation department, I look at vari-
human abilities and flexibility. From the
18th through the 20th century, power
machines—such as steam engines,
ples of tools that have increased our
physical capabilities, making humans
stronger, faster, and more precise. This
ous technologies designed to augment combustions engines, electric motors, has inevitably changed what we value
the human intellect and amplify human and hydraulic lifts—revolutionized our in individuals and how we structure
perception and cognition. Linking back world, from the workplace to family our society. Tool use and tool making
to early work in interactive comput- life. Power (that is, muscle power) was are fundamental to a species and linked
ing, I consider how novel technologies no longer an area where humans were to advances in evolution.
can create a new relationship between superior to machines. Nevertheless, Tools for the mind and for augment-
digital technologies and humans. Forth- these advances in physical technologies ing the human intellect have been a cen-
coming articles will provide examples of led to the world in which we now live. tral goal since the early days of comput-
how digital systems can amplify human In the 21st century, computing, net- ing (see the “Mandatory Reading: Past
performance—in particular, human working, and digital media, combined Visions of the Future” sidebar). Search-
cognition and perception. with sensing and actuation, are the new ing through vast amounts of informa-
ingredients for fundamental change. tion has become an essential tool for
A REvolutIon In thE We’re likely at the dawn of another tech- many professions (just try to write soft-
MAkIng nical revolution that will question all we ware without Internet access). Extend-
It’s apparent that artificial intelligence know about work, economics, social ing our memory and externalizing
(AI) is challenging humans in many environments, family, and even ethics. information is becoming commonplace,
established domains. In games such as I suggest reading up on the first indus- as media capture and access become
chess and GO, algorithms have outper- trial revolution and its wide impact, more simplified. Tools for ubiquitous
formed humans,1 and autonomously looking in particular at the opportuni- communication are providing value to
driven cars have started to exhibit stun- ties it created beyond the workplace. An people at home and in the workplace.
ning performance.2 In both the popular interesting starting point is Energy and There are many positive effects, but
press3 and articles from esteemed col- the English Industrial Revolution, by humans must learn how these tools fit
leagues,4 we see warnings of AI pos- Edward Anthony Wrigley (Cambridge into our lives, and it’s apparent that new
ing a risk to humanity. In fact, there’s University Press, 2010). technologies affect how we think, some-
a website where you can plug in your times literally changing our brains.6
profession and job title to find whether thE BRIght SIdE of thE Evolution, however, is slow, even if the
you’re at risk of being replaced by com- futuRE tools are quickly changing the condi-
puting technologies. 5 Some experts Human history is full of technologi- tions and requirements around us.
have painted a dark future in which cal advances that have changed how
humans are sidelined—or even made we work and live. A major talent of AugMEntIng thE huMAn
extinct—by machines. But I disagree humans is our ability to develop tools IntEllECt
with these bleak outlooks! and devices that help us adapt to differ- Amplifying human abilities follows
To me, this feels like history repeating ent environments. The many machines Joseph Lickider’s idea of a “Man-Com-
itself, and people are underestimating invented in the last 200 years are exam- puter Symbiosis”7 and extends research

6 30 VA SI V E computing
PER February 2018 Published by the IEEE Computer Society
Published by the IEEE CS n 2469-7087/18/$33.00
© 2018
2017 IEEE
mAndAtory rEAdIng: PASt VISIonS of thE fUtUrE

Ideas of using information technologies to augment human www.1962paper.org/web.html).3 the idea was that computing
cognitive and perceptual abilities have been stated by visionar- technologies could increase human capability in dealing with
ies of the last century. It’s exciting to go back to these powerful complex problems by offering faster comprehension. over-
past visions of the future and read them with current technolo- all, he foresaw such systems solving problems that otherwise
gies in mind. couldn’t be tackled by humans.
In 1945, Vannevar Bush outlined, in “the memex—As We mark Weiser and his colleagues at XEroX PArc explored in
may think,” a vision for making (scientific) knowledge widely the 1980s and 1990s how the ubiquity of digital computing,
available and for allowing for sharing and collaboration.1 his networking, and storage technologies would change our lives.4
vision was bold! now, 70 years later, advances in technologies— technologies for networked mobile devices and large-scale
especially in networking and capturing information—have interactive displays were experimentally explored from a wide
created a world where information is readily available and where range of angles, going beyond the purely technological ques-
everyone can contribute with very little skill. tions, and looking at the impact on business and society.
joseph Licklider foresaw a close relationship between comput-
ers and humans. In “man-computer Symbiosis,” he envisioned REfEREnCES
computers and humans working together with a high degree of
1. V. Bush, “the memex—As We may think,” The Atlantic, july 1945; www.
flexibility, allowing for joint decision making and collaboration
in solving complex (cognitive) tasks.2 the way in which we now
2. j.c.r. Licklider, “man-computer Symbiosis,” IRE Trans. Human Factors
use computers as tools, and the seamlessness in human-com- in Electronics, mar. 1960, pp. 4-11; http://groups.csail.mit.edu/medg/
puter interaction, underline that this has become the dominant people/psz/Licklider.html.
way of working today. 3. d.c. Engelbart and W.K. English, “A research center for Augmenting
douglas Engelbart saw how interactive computing can human Intellect,” Proc. Am. Federation of Information Processing Societies
augment the human intellect. he envisioned and explored ex- (AfIPS), 1968; http://dl.acm.org/citation.cfm?id=1476589.1476645.
perimentally how interactive applications can support humans. 4. m. Weiser, “the computer for the 21st century,” Scientific American,
he coined the term of “augmenting human intellect” (see vol. 265, no. 3, 1991, pp. 94–104.

by Douglas Engelbart on “Augmenting Many of these technologies, rang- compared to our raw capabilities. Imag-
[the] Human Intellect.”8 Lickider and ing from early work on development ing sensors have a higher spatial and
Engelbart understood early on how tools to remote collaboration, mobile temporal resolution than the human
interactive computing technology could computing devices, and (more recently) eye, and the amount of information we
be used to augment the human intellect. machine learning, have undoubtedly can store externally is much higher than
The idea of ubiquitous computing, as augmented the human intellect for the what we can remember. Furthermore,
described by Mark Weiser,9 moved the masses. However, looking at technolo- algorithms are faster at picking out
vision forward to interactive comput- gies currently in the making reveals that details from a vast amount of images
ing technologies pervasively integrated this is only the beginning. If we can than human observers.
with everyday environments. amplify ourselves, we will stay ahead The goal behind amplifying human
The idea of externalizing cognitive of machines. Once muscle power was perception and cognition is to cre-
efforts and the notion of distributed no longer a central need, our intellec- ate systems where we closely couple
cognition is based on the fact that user tual abilities bloomed. If machines take humans with technologies to provide
actions are situated in place and time over basic intellectual tasks, might we us with super-human abilities. A key to
and are often a reaction to the imme- have the time to achieve new abilities such technologies is that they act like a
diate environment.10 One important in social, emotional, or moral domains? natural extension of our own abilities,
facet of distributed cognition is that with no added effort or increased cogni-
cognitive capabilities are a combina- AMPlIfyIng PERCEPtIon And tive load required. Amplified perception
tion of a person’s cognitive abilities CognItIon is based on combining human senses
and their environment. Knowledge and The ability to see and to hear helps us with technical sensors. I envision new
skill are not only in our minds but also make sense of our surroundings. Our amplified senses that are seamless to
in the environment.11 Another impor- perception and cognition determine use, and I’m confident they can be built
tant example is the intelligent use of how we experience the world, and with current technologies. Amplifying
space for problem solving.12 By creat- human senses are incredibly powerful. cognition includes a wide range of digi-
ing dynamic environments that can However, with current and upcoming tal technologies. Examples are memory
change technically and adapt to the technologies, we have reached a point extensions and visualizations that help
tasks at hand, we inevitably alter our where sensors, databases, and algo- us take in information and enhance our
cognitive abilities. rithms can outperform human abilities short-term memory.

jAnUAry–mArch 2017 31
PER VA SI V E computing 7
Human augmentation

Human augmentation

perspectives (a first-person versus third-

person view).

Amplified Cognition and Creativity

You can amplify cognition and creativ-
ity in many ways. Examples include

• amplifying a user’s personal memory

through contextualized capture and
repeated presentation1 (for example,
with a wearable camera, a user can
capture pictures throughout the day
and use them for memory augmenta-
tion—see http://recall-fet.eu);
• enhancing information intake of vari-
Figure 1. Prototype of a video see-through system that combines an Oculus with an ous media (for example, for speed read-
RGB, depth, and thermal image and allows implicit control by different physiological ing or for nonlinear viewing of videos);
sensors. (Source: Yomna Abdelrahman and Pascal Knierim, Project FeuerWeRR, • offering a parallel presentation of mas-
University of Stuttgart; used with permission.) sive amounts of information (such as
presenting large documents on large
and high-resolution screens); and
Basic technologies that implement A vision for such a system would • presenting related solutions to
these amplifiers for cognition and be glasses (or contact lens, or even an amplify creativity (such as presenting
perception are already available. The implant) that let you seamlessly oper- hundreds of images of existing solu-
missing, but central, piece is the seam- ate across a wide visual spectrum in tions in a room).
less integration with our existing per- which you can manipulate the focus,
ceptual and cognitive abilities and speed, and spectrum. Imagine you’re Consider the scenario shown in
effective mechanisms for explicit and walking along a path in the forest, and Figure 2. The idea here is to build on
implicit control. Physiological sensing you see a squirrel in the distance. Once human perceptual capabilities and sup-
could be the missing link, and I expect you look at it, you concentrate and can port them with technologies. Instead
that using eye-gaze, brain signals, and zoom in and see how it’s nibbling on a of searching within categories, (which
muscle activity will be key to creating a nut. When it jumps from one tree to the requires significant knowledge), we nar-
seamless user experience for amplified next, you can, by holding your breath, row down the search space to a set that
perception and cognition. slow down what you see and appreciate humans can easily perceive and match.
how it lands on the branch. Once you In this example, let’s say we have 500
Amplified Perception start walking again, you return to your plants in the book. By narrowing it
If we look at how to amplify perception, normal view. down to blue plants that are approxi-
we see two major directions: We’re not there yet, but at the Human mately 30 cm high, we could probably
Computer Interaction Lab at the Uni- reduce the set to 50 plants. We assume
• enhancing and amplifying existing versity of Stuttgart, we have experi- that a person could quickly pick the
senses (such as vision, hearing, and mented with different technologies right plant from a set of just 50, shown
touch), and for amplifying perception. Currently, on a high-resolution wall-sized display.
• extending perceptual abilities to technologies are still bulky—but the By not fully automating the task, and by
domains where humans have no per- vision is that the technologies might be, adding a human perceptual step, we can
ception but technical sensors exist in 20 years’ time, embedded into your increase the person’s knowledge as she
(such as sensors indicating magnetic glasses or contact lenses. Figure 1 shows sees what’s around the actual match.
north or solar radiation). a prototype of a video see-through sys-
tem that lets you have visual perception Quantifying Cognitive and
I expect that systems for amplifying per- beyond the human visual spectrum. You Perceptual Amplification
ception could be seamlessly integrated can move between a normal color video Even though there are now many tools
with current perception such that, in and a thermal video, and you can add that augment and amplify our abili-
the long term, a person wouldn’t even a depth view. With a similar setup, we ties, solid metrics don’t exist, and we,
realize the amplification. have explored how to provide different as a community, have made little effort

8 32 VA SI V E computing
PER ComputingEdge February 2018
Human augmentation

to quantify such augmentations and

amplifications beyond single cases. For
tools in the physical world, the effect
is typically easy to quantify. A human
without any technical support can
travel about 5 km an hour; with a bike,
that goes up to 20 km per hour, and
with a car, up to 100 km. You can also
clearly quantify the difference in using
an electric drill versus a mechanical
drill, noting the number of holes drilled
and the increase in precision.
In the digital world, there’s little scien-
tific work on quantification. How much
quicker can you develop software with a
50 mbit/s Internet connection versus a 1
mbit/s connection? What is the effect on
the productivity of a software developer
if she has access to the website stackover-
flow.com compared to using a printed
reference manual for the program-
ming language? Is having an interactive
60-inch screen in the meeting room
more effective than everyone having a
tablet computer? How much quicker are Figure 2. A sample scenario—naming a plant. this examples illustrates comparing
you at solving problems when you can the usage of a traditional book based on categories with a large screen presentation.
access YouTube versus using the manual All potential candidates are shown at once, but filters (based on color, size, and so on)
provided with the product? can be applied. Spotting the match relies on human perceptual abilities. (Artwork by
To scientifically validate the amplifi- Katrin Wolf; used with permission.)
cation of human capabilities, it’s essen-
tial to understand how to quantify the
amplification. Can we create metrics us smarter than the technology, or do we So what’s new? The difference now
that let us state that using a certain dig- assume that technologies will take over? is that the upcoming technologies for
ital tool will increase your externally Many are skeptical of technolo- cognition and perception are moving
perceived IQ by 5 points? Or that using gies that amplify our abilities. There’s much closer to our bodies. Natural and
another tool will allow you to perform a natural skepticism about creating implicit control will thus make us feel as
(with the tool) as well as someone (with- super humans through technologies, though such technologies are a part of
out a tool) whose average school grades but looking back at the machines that us—if we get it right.
are 0.5 points better than yours? replaced human muscle power shows
Validating amplifications obviously us that super human technologies aren’t ACKnOWledGement
won’t be easy, and the methods will be new. Humans move at great speeds (in this work has received funding from the European
disputed. Furthermore, validation will cars) and lift amazing weights (with research council (Erc), under the EU’s horizon
most certainly require complex, large- cranes). Technologies have and always 2020 research and innovation program (grant
scale experiments, but such experi- will change our abilities. The introduc- agreement no. 683008) and from the german fed-
ments are essential to making advances tion of written texts, book printing, and eral ministry of Education and research (BmBf) in
not just visible but also measurable. photography are just some examples project feuerWerr.
that made us super human with regard
towARd SuPER huMAn to our cognitive abilities—compared
tEChnologIES? to people who don’t have these tech-
It’s apparent that many of these ideas nologies. We also have perceptual aids 1. C. Moyer, “How Google’s AlphaGo
will also question what we really want (such as microscopes or thermal cam- Beat a Go World Champion,” The
Atlantic, 28 Mar. 2016; www.theatlan-
humans to be. Do we create (or at least eras) as well as cognitive aids (such as tic.com/technology/archive/2016/03/
attempt to create) technologies that make calculators). the-invisible-opponent/475611.

jAnUAry–mArch 2017 33
PER VA SI V E computing 9


2. J. Vincent, “World’s First Self-Driving 6. N. Carr, The Shallows: How the Fou ndation for Hu man- Computer
Taxi Trial Begins in Singapore,” The Internet Is Changing the Way We Interaction Research,” ACM Trans.
Verge, 25 Aug 2016; www.theverge. Think, Read and Remember, Atlantic Computer-Human Interaction (TOCHI),
com/2016/8/25/12637822/self-driving- Books, 2010. vol. 7, no. 2, 2000, pp. 174–196.
7. J.C .R. Licklider, “Man- Computer 12. D. Kirsh, “The Intelligent Use of Space,”
3. M. Sainato, “Stephen Hawking, Elon Symbiosis,” IRE Trans. Human Factors Artificial Intelligence, vol. 73, no. 1, 1995,
Musk, and Bill Gates Warn about in Electronics, Mar. 1960, pp. 4 –11; pp. 31–68.
Artificial Intelligence,” The Observer, http://groups.csail.mit.edu/medg/people/
19 Aug. 2015; http://observer. psz/Licklider.html.
musk-and-bill-gates-warn-about-artifi- 8. D.C. Engelbart and W.K. English,
cial-intelligence. “A Research Center for Augmenting
Human Intellect,” Proc. Am. Federation
4. S. Hawking et al., “Transcendence of Information Processing Societies Albrecht Schmidt is a professor at the Univer-
Looks at the Implications of Artificial (AFIPS), 1968; http://dl.acm.org/citation. sity Stuttgart, germany. contact him at albrecht.
Intelligence—But Are We Taking AI cfm?id=1476589.1476645.
Seriously Enough?” The Independent, schmidt@vis.uni-stuttgart.de
1 May 2014; www.independent.co.uk/ 9. M. Weiser, “The Computer for the 21st
news/science/stephen-hawking-tran- Century,” Scientifi c American, vol. 265,
scendence-looks-at-the-implications- no. 3, 1991, pp. 94–104.
taking-9313474.html. 10. L .A. Suchman, Plans and Situated
Actions: The Problem of Human-
5. Q. Bui, “Will Your Job Be Done Machine Communication, Cambridge This article originally
read yourappeared in
by a Machine?” NPR, 21 May Univ. Press, 1987. IEEE Pervasive through the mycSvol.
2015; www.npr.org/sections/ publications portal at
money/2015/05/21/408234543/will- 11. J. Hollan, E. Hutchins, and D. Kirsh, 16,http://mycs.computer.org.
no. 1, 2017.
your-job-be-done-by-a-machine. “Distributed Cognition: Toward a New

Paper Deadline Extended to February 7

Keeping Call for Papers

IEEE SERVICES 2018 What’s Trending?

YOU at the July 2-7, 2018 | San Francisco, CA

The information you need
and only the information you need.
Industry intelligence delivered on your

terms, when and how you want it.
Papers are being accepted until 7 February for the IEEE IMPORTANT DATES
• myCS—delivers your publications
2018 World Congress on Services (IEEE SERVICES 2018), your way

of Technology
the one and only services conference sponsored by IEEE REGULAR PAPERS:• myComputer—customizable mobile
app delivering targeted information
and IEEE Computer Society’s Technical Committee on Full Paper Submission Due Date: February 7, 2018
specific to your specialty
Services Computing (TC-SVC). Decision Notification (Electronic):
• Computing Marchaward
Now—this 22, 2018
Camera-Ready Copy Due Date: April 6, 2018
winning website features industry
IEEE SERVICES 2018 will cover all aspects of innova-
myComputer, myCS,
tive services computing and applications, current and WORKSHOP AND
news and developments.
Learn something new.
Computing Now
emerging. IEEE’s 14th annual conference will involve vari-
ous systems and networking aspects.
Full Paper Submission
Check Due Date:
out these February
resources 14, 2018
Decision Notification (Electronic): March 22, 2018
Camera-Ready Copy Due Date: April 6, 2018
IEEE SERVICES 2018 is the exclusive services
conference that publishes its proceedings in the For more information, go to:
Stay relevant with the IEEE Computer Society
IEEE Xplore Digital Library. http://conferences.computer.org/services/2018/cfp/
More at www.computer.org

34 VA SI V E computing
10 PER ComputingEdge February 2018
Take the
CS Library
you go!
IEEE Computer Society magazines and Transactions
are available to subscribers in the portable ePub format.

Just download the articles from the IEEE Computer

Society Digital Library, and you can read them on any device
that supports ePub, including:

• Adobe Digital Editions (PC, MAC)

• iBooks (iPad, iPhone, iPod touch)
• Nook (Nook, PC, MAC, Android, iPad, iPhone, iPod, other devices)
• EPUBReader (FireFox Add-on)
• Stanza (iPad, iPhone, iPod touch)
• ibis Reader (Online)
• Sony Reader Library (Sony Reader devices, PC, Mac)
• Aldiko (Android)
• Bluefire Reader (iPad, iPhone, iPod touch)
• Calibre (PC, MAC, Linux)
(Can convert EPUB to MOBI format for Kindle)

SOFTWARE Editor: Robert Blumen
ENGINEERING rober t@rober tblumen.com

Katie Malone
on Machine Learning
Edaena Salinas

MACHINE LEARNING WAS featured fl ix recommendations, and most recently

in episode 193 of Software Engineering in driverless cars. In the coming years,
Radio with Grant Ingersoll in 2013. But we’ll see it used in more fields. So, what
because this area has changed consider- is machine learning?
ably in the past four years, it made sense
to revisit it with a fresh outlook. In epi- Katie Malone: My background is in sci-
sode 286, Edaena Salinas talks with Ka- ence. I believe that there’s truth in the
tie Malone, a data scientist in the R&D world and that science is one of the ways
department at Civis Analytics, which we get to that truth. It’s really hard to
specializes in data science software and measure truth directly. Instead, we col-
consulting. Katie earned a PhD in phys- lect data on the world. If we analyze
ics from Stanford University; during her that data, sometimes we can pull out
studies she searched for new particles the truth. A true thing about the world
at CERN. She teaches Udacity’s Intro might be, “I’m interested in watching
to Machine Learning course and hosts this movie.” Or it might be, “There’s a
Linear Digressions, a podcast about ma- good way to translate this sentence from
chine learning (lineardigressions.com). English to French.” Machine learning is,
Here, Katie and Edaena discuss the in my view, a suite of tools that allows
major types of machine-learning algo- you to analyze data to figure out what’s
rithms and some examples, including going on in the world, and how that’s
supervised and unsupervised classifica- expressed in the data.
tion. Portions of the interview not in- Usually it involves heavy computa-
cluded here for reasons of space include tional lifting. The “machine” compo-
topics such as cleaning the raw data, nent implies computers. Then there’s
training data versus test data, randomiza- usually a heavy dose of statistics, and
tion, evaluation metrics, and Katie’s take often additional scientific fields. If you’re
on popular programming languages. To studying human behavior, you should be
hear the full interview, visit se-radio.net or aware of [other fields that study humans]
access our archives via RSS at feeds.feed like behavioral psychology and econom-
burner.com/se-radio. —Robert Blumen ics. Those areas give you context about
the thing you’re interested in.
Edaena Salinas: Machine learning is
widely used—in search engines, speech Edaena: How does machine learning re-
recognition, language translation, Net- late to AI?

92 I E E E S O FFebruary
T W A R E2018
| P U B L I S H E D B Y T H E I E E E CPublished
C I E Computer
TY Society  2469-7087/18/$33.00
0740-7459 / 1 7 / $ 3 3 . 0 0 © 2 0 1 7©I E2018

Katie: I once heard that ma- ular words like “Nigerian prince,” up in a particular email, you’ll get
chine learning focuses more on from the cases where you have said a 1 in that spot in the matrix; if it
understanding— measuring or mak- “this is spam,” the model can learn doesn’t show up, you’ll get a 0. Then
ing predictions—while AI is think- those patterns and apply them to you can put that matrix into a stan-
ing one step further. Once we under- new cases. Hopefully at some point dard machine-learning algorithm,
stand what’s going on, how can we you don’t have to manually label and it will find the structure in the
make better decisions? How can we emails because the model will have matrix that allows it to understand
change the way we do things to take figured out what spam looks like. which words are most closely associ-
advantage of those insights? AI adds ated with the emails you’ve classified
a layer of decision making on top of Edaena: By that time, is the system able as spam.
machine learning. to figure out those common words? Another important aspect of ma-
chine learning is thinking about dif-
Edaena: Let’s walk through a simple Katie: Spam is an interesting case, ferent representations of your data.
example of machine learning: spam as presumably spammers are getting What I just described is the simplest
detection in email. Once I indicate more sophisticated. The spam filters way you might represent data in an
that an email is spam, I’m telling the that worked five years ago probably email, but there are other algorithms
system something. What happens wouldn’t work that well right now. that can be more compressed with
under the hood? That’s another important aspect of respect to how the words or sen-
machine learning: it’s pretty rare to tences showing up in an email are
Katie: Email is an example of super- have a problem that you solve once represented. How you represent your
vised classification. Let me break and for all. Usually you want to re- data has an intimate connection
this into two parts. Supervision oc- visit it periodically to see if the solu- with the type of algorithm you’re go-
curs when you have the correct an- tions you came up with last year or ing to use to do the supervised classi-
swer for some of the cases. In this last month still apply. fication. The way the data is format-
example, you provide the answer Spam is a good example of that. ted can make it very easy for us to
when you manually label the email I don’t know if people talk about find the truth we’re seeking, or it can
as spam. If you don’t tell the model Nigerian princes anymore, because make it very hard. It’s worth think-
that it’s spam, then the model as- that’s such a cliché at this point, ing about carefully.
sumes it’s a legitimate email. Clas- but the formula of “We’re going to
sification is sorting things into two pretend there’s money sitting in an Edaena: What’s one way in which
buckets: spam and not spam. account and if you send a small de- this data has been compressed in
Machine learning is making pre- posit we’ll release it to you” remains other data structures?
dictions based on the attributes of popular, although the exact details
an email. We learn what spam email change. In that scenario, you have Katie: One case is Netflix movie rec-
looks like, and then we extrapolate to keep retraining your algorithm to ommendations. Imagine that each
those patterns onto new emails to continue to make good decisions. person who watches movies is a row
predict whether they’re spam or not. and each possible movie they could
The model is probably going to Edaena: How is this information watch is a column. Most people will
look at the words in the email, and represented? Is there a specific for- only watch one percent of all the
potentially the sender’s domain. mat for the model? movies out there. And most movies
Spam emails have very particular pat- are not going to be watched by even
terns. The words in spam tend to be Katie: The simplest thing you can do a significant fraction of all users.
distinctive—usually they’re trying to is to treat each word as its own fea- You have a big sparse matrix.
sell you something with lots of super- ture. Many machine-learning algo- In a case like this, you can use ma-
lative adjectives. Or maybe they’re rithms assume there’s a big matrix trix factorization. Instead of having
trying to get you to send money to of attributes. Imagine a matrix as a this big sparse matrix, imagine that
somebody in a foreign country. Very big data table, and each row of the there are two factors—two different
often there are grammatical mistakes. table is an email and each column is types of attributes—that we’re trying
Based on the presence of partic- a word. If a particular word shows to understand. [In this case, users and

www.computer.org/computingedge J U LY / A U G U S T 2 0 1 7 | I E E E S O F T WA R E 37

sion. A lot of the same algorithms

SOFTWARE ENGINEERING RADIO can be used for classification and re-
gression, depending on the fi nal type
of output you want.
Visit www.se-radio.net to listen to these and other insightful hour-long podcasts.
Edaena: Can you explain regression
RECENT EPISODES a bit more? For example, what is the
• 283—Host Felienne talks with Alexander Tarlinder about developer testing objective of linear regression?
and his book on the topic.
• 289—James Turnbull returns to the show to tell host Robert Blumen how Katie: Linear regression has a con-
to automate infrastructure builds using declarative programming with the tinuous output. Classification is try-
Terraform tool. ing to figure out if something is A or
• 290—Docker’s Diogo Mónica talks with host Kim Carter about Docker B, spam or not spam. You wouldn’t
security aspects. say that spam has an inherently
higher value than not spam or vice
UPCOMING EPISODES versa. There isn’t a natural ordering
• 292—Philipp Krenn talks to host Jeff Meyerson about the search server of those two things.
Elasticsearch. We’ll use income as an example.
• 293—Yakov Fain and new host Matthew Farwell discuss the popular Java- If you’re trying to predict some-
Script framework Angular. body’s income from other attributes
• 294—New host Alex Newman talks asynchronous I/O with Rust language that you have, then obviously there’s
expert Carl Lerche. a natural ordering. There’s a natural
ordering to values like $10,000 and
Linear regression tries to use
types of movies.] Say there are buck- it. Should they give it valuable real known attributes about a person
ets, or segments, of users, and users estate on the front page to advertise to predict another attribute, like
in each bucket watch certain types of it? And to which people? But they income. Do I see a relationship be-
movies or certain mixtures of movies. have no data on this movie yet. They tween a person’s attributes and their
Then we have types of movies: action don’t know who has watched it or income? One example is age. The
movies or foreign documentaries. who liked it before. This is a tricky older someone is, up to a certain
Whether a particular user likes a par- place to be in terms of machine point, the more money they tend to
ticular movie is a combination of the learning, because machine learning make. You might observe what kind
type of user and type of movie. Rep- is usually about pattern recognition, of car this person drives. I know
resenting the same data differently and there’s no pattern yet. there are patterns in income versus
can make it easier and more direct to But if you have some other contex- the type of transportation you use:
figure out whether a user is going to tual information, like “this is an ac- richer people have nicer cars.
like a movie. tion movie,” then you have a better None of these patterns is going to
place to start from. You have some hold absolutely for every single case,
Edaena: How does machine learn- idea of the people who like action but statistically it’ll usually hold.
ing handle cases where there’s no movies. This is effective at the begin- And from that you can make predic-
prior data? ning, and then you can refine those es- tions. The quality of those predic-
timates as you collect more data. tions will depend on how good your
Katie: In recommendation engines, data is, and to a lesser extent, how
there’s the classic problem of “cold Edaena: Can supervised learning be good your algorithm is. But you’re
starts.” This is when a new movie applied to values that are continuous getting a little closer than if you were
will be added to Netfl ix next month, instead of discrete? to make a shot-in-the-dark guess.
and they need to figure out if a lot
of people are going to want to watch Katie: That’s usually called regres- Edaena: If we plot the income on one

94 I E E E S O F ComputingEdge
T W A R E | W W W. C O M P U T E R . O R G / S O F T W A R E | @ I E E E S O F T WA R E February 2018

axis and the car’s price on another, this line. For classification you can pervised learning. In my experience,
you could predict from income how use accuracy, which is how many it’s really hard. It’s a different way of
much you spend on a car, right? you got right divided by the total. thinking about your data, because
One of the big challenges of ma- when you have the correct answer,
Katie: That would give you some chine learning is figuring out if those you want to get as close as possible
idea. You could imagine fitting a line metrics are really measuring the thing to it. With unsupervised machine
to the distribution you see. The slope you care about, because usually they learning, it can be very tricky to try
of that line will give an estimate of aren’t. You have to be a little smarter to understand what a good answer
someone’s income once you know to figure out exactly what you care looks like.
how expensive their car is. For every about, and modify the metrics to re-
data point in your dataset, the line flect that. You usually have good Edaena: What do some of those
gives a predicted income. options, but the artistry is knowing questions look like?
when to leave those options behind.
Edaena: How do we measure how Katie: The biggest one in my expe-
good the line is compared to the data? Edaena: How is unsupervised ma- rience has been clustering. Cluster-
chine learning different from ing is the idea that you have blobs
Katie: For a lot of machine-learning supervised? or coherent clumps in the data. You
algorithms, there are standard met- want to find them. This is hard
rics. Somebody drives an expensive Katie: The canonical answer is that because with a lot of real-world
car, so I think they make $100,000 in supervised machine learning, you datasets, you don’t know if there
a year. If you have the actual in- have correct answers that came with are clusters in the dataset to begin
come in your dataset, then the data- your dataset. With unsupervised with. If you don’t find any, it’s re-
set shows that the person makes machine learning, you don’t have ally hard to know if it’s because
$110,000 a year. The $10,000 dif- that luxury. You just have data. The you’re doing a bad job or because it
ference is my error. Sometimes you types of questions you can ask of doesn’t exist. This gets back to the
square it for other reasons that we that data are different and very often idea of truth—what are we trying
don’t have to get into. You sum that constrained by the fact that there’s to understand here?
over your whole dataset and divide no correct answer. Principal component analy-
by the number of points. This is a There are different types of ques- sis is another unsupervised tech-
metric for the “goodness of fit” of tions you try to answer with unsu- nique. You’re trying to find ways
july • august 2016

IEEE Software (ISSN 0740-7459) is published bimonthly by the and their companies are permitted to post the accepted version of
IEEE Computer Society. IEEE headquarters: Three Park Ave., IEEE-copyrighted material on their own webservers without permis-
march • april 2016

may • june 2016


17th Floor, New York, NY 10016-5997. IEEE Computer Soci- sion, provided that the IEEE copyright notice and a full citation to
nov ember • december 2015

jUly • aUGUsT 2016

january • february 2016


ety Publications Office: 10662 Los Vaqueros Cir., Los Alamitos, the original work appear on the first screen of the posted copy. An
MaRCh • aPRIl 2016

May • jUNE 2016

CA 90720; +1 714 821 8380; fax +1 714 821 4010. IEEE Com- accepted manuscript is a version which has been revised by the au-

jaNUaRy • fEbRUaRy 2016

puter Society headquarters: 2001 L St., Ste. 700, Washington, DC thor to incorporate review suggestions, but not the published version
20036. Subscribe to IEEE Software by visiting www.computer with copyediting, proofreading, and formatting added by IEEE. For



.org/software. more information, please go to: http://www.ieee.org/publications

_standards/publications/rights/paperversionpolicy.html. Permission
to reprint/republish this material for commercial, advertising, or
VOl. 20, NO. 1
vOl. 19, NO. 6

Postmaster: Send undelivered copies and address changes to IEEE

VOl. 20, NO. 3
VOl. 20, NO. 2

VOl. 20, NO. 4

Software, Membership Processing Dept., IEEE Service Center, 445 promotional purposes or for creating new collective works for resale

Hoes Lane, Piscataway, NJ 08854-4141. Periodicals Postage Paid at or redistribution must be obtained from IEEE by writing to the IEEE


New York, NY, and at additional mailing offices. Canadian GST Intellectual Property Rights Office, 445 Hoes Lane, Piscataway, NJ IC-20-01-c1 Cover-1 December 7, 2015 1:45 PM

#125634188. Canada Post Publications Mail Agreement Number

IC-19-06-c1 Cover-1 October 9, 2015 3:26 PM

08854-4141 or pubs-permissions@ieee.org. Copyright © 2017 IEEE.

40013885. Return undeliverable Canadian addresses to PO Box IC-20-02-c1 Cover-1 February 11, 2016 10:30 PM
All rights reserved. IC-20-03-c1 Cover-1 April 13, 2016 8:45 PM

122, Niagara Falls, ON L2E 6S8, Canada. Printed in the USA.

Want to know more about the Internet?
Abstracting and Library Use: Abstracting is permitted with credit to
Reuse Rights and Reprint Permissions: Educational or personal use the source. Libraries are permitted to photocopy for private use of
of this material is permitted without magazine
providedcovers all 1)aspects
such use: of Internet
is patrons, providedcomputing,
the per-copy fee indicated in the code at the bot-
not made for profit; 2) includesfrom programming
this notice and standards
and a full citation to the tomto security and networking.
of the first page is paid through the Copyright Clearance Center,
original work on the first page of the copy; and 3) does not imply 222 Rosewood Drive, Danvers, MA 01923.
IEEE endorsement of any third-party products or services. Authors
www.computer.org/computingedge J U LY / A U G U S T 2 0 1 7 | I E E E S O F T WA R E 95 39

This article originally appeared in

IEEE Software, vol. 34, no. 4, 2017.

Looking for the of compressing the data down to

the aspects that make it the most

BEST Tech Job variable. You’re trying to find di-

rections in your dataset in a lower-

for You? dimensional space that maximize

the variance in your data.
The point is that this is some-
thing you can do without knowing
Come to the Computer Society Jobs
any “correct” answers for your data,
Board to meet the best employers which can be pretty useful when you
in the industry—Apple, Google, Intel, don’t have labels to rely on.
NSA, Cisco, US Army Research,
Oracle, Juniper... Edaena: Is unsupervised machine
learning widely used? Are there any
Take advantage of the special systems that we interact with that
resources for job seekers—job might be using it?
alerts, career advice, webinars,
templates, and resumes viewed by Katie: Segmentation within mar-
top employers. keting is one example. If you think
your users might fall into a few
main buckets, you want to seg-
ment them. For example, buyers of
www.computer.org/jobs computers might be power users,
programmers and data scientists,
people who watch movies and use
Facebook, and people who use it as
a work machine but don’t program
on it. Maybe there’s a dataset that
Are Enemy Hackers Slipping would allow you to pick out these
through Your Team’s Defenses? distinct groups.

Protect Your Organization from Hackers Edaena: Unsupervised might also be

by Thinking Like Them more about discoverability, because
you don’t necessarily know what
Take Our E-Learning Courses you’re looking for.
in the Art of Hacking
Katie: That’s fair. Supervised meth-
ods are used when you know what
You and your staff can take these courses where you are you’re trying to answer. Unsuper-
and at your own pace, getting hands-on, real-world training
that you can put to work immediately.
vised methods are for when you
don’t have that labeled data available
www.computer.org/artofhacking or when you don’t exactly know how
to slice and dice the data yet.

EDAENA SALINAS is a software engineer in

Microsoft Research’s Knowledge Technologies
Group. She also hosts The Women in Tech Show
(thewomenintechshow.com). Contact her at

96 I E E E S O F ComputingEdge
T W A R E | W W W. C O M P U T E R . O R G / S O F T W A R E | @ I E E E S O F T WA R E February 2018
EDITOR: Seth Earley, Earley Information Science, seth@earley.com

The Problem
With AI

Seth Earley, Earley Information Science

cience is actually pret- 360-degree understanding of trends impact sales. Messy data
ty messy. When I was customers and their needs. can be missing values or can be in
a chemistry under- Artificial intelligence (AI), formats that are difficult to ingest
graduate, I loved the though broadly applied these and process. The data can be very
theory behind biochemistry— days to mean almost any algo- good, but requires work to get it
the endless complexity allowed rithm, is primarily dependent on into a format for processing.
by simple rules; how massive, some form of machine learning. A recent article in Sloan Man-
complex cellular machines could Machine learning in turn is fre- agement Review stated that
arise from a few building blocks. quently fueled by what is called
In lab, however, I struggled to big data (high-velocity, high- Organizations can now load all of
make the simplest reactions volume, highly variable data the data and let the data itself point
work. Starting with pure crystal- sources) but can also be fueled by the direction and tell the story. Un-
line compounds and expensive traditional data sources. necessary or redundant data can be
laboratory equipment, when the culled … [This process is] often re-
result was also expected to be Variable Does Not ferred to … as ‘load and go.’1
crystalline, I ended up with piles Mean Poor Quality
of brown goo—with my instructor There is a common misconcep- While conceptually accurate, there
concluding, “Well, it could be in tion that “variable” data can mean is much left open to misinterpre-
there” in reference to the experi- “messy” data and that “messy” tation. “All the data” needs to be
ment’s objective. data can mean “poor-quality” defined. Does it mean all product
Data science is also very messy. data. Simply put, variable does data, social media data, accounting
Frequently the starting point not mean messy, and messy does data, transactional data, knowledge
is the data equivalent of brown not mean poor quality. Variable base data? Clearly “all” is an over-
goo—messy, poor quality, incon- data is data that has different for- generalization. And this approach
sistent data—with the expecta- mats and structures. To use it, we has its drawbacks. Sandy Pentland,
tion that pure crystalline results need to understand how the dif- MIT professor, remarked at the re-
will be the output of the next ferent types of data can be used as cent MIT CIO Symposium that
best action, personalized mar- signals to achieve a result. Twitter “Putting all of your data in a data
keting campaigns, highly effec- data is very different than transac- lake makes it convenient for hack-
tive custom email campaigns, or a tional data. The two together can ers to go to one place to steal all of
cross-department, cross-functional, provide insights about how social your data.”

2469-7087/18/$33.00 © 2018 IEEE Published by the IEEE Computer Society February 2018 41
1520-9202/17/$33.00 © 2017 IEEE Published by the IEEE Computer Society computer.org/ITPro 63

No matter what the scope is, we A Range of AI Applications to augment their knowledge and
have to select data that is appro- AI applications exist along a spec- experience when developing treat-
priate to the domain of the prob- trum. At one end lies embedded ment regimens. Another type is
lem space. The data needs to be AI, which is transparent to the creation of an intelligent virtual
in a consistent format. It can- user, but makes applications work assistant (IVA) that retrieves an-
not contain incorrect values. If better and easier for them. Spell- swers to procedural questions
the data is incorrect or missing, ing correction is an example that rather than lists of documents.
then the algorithm cannot func- people take for granted. Machine IVA functionality requires vari-
tion correctly unless we are mak- translation is another. Search en- ous mechanisms that are powered
ing accommodations for those gines use machine learning, and by machine learning. The first is
issues. “It’s an absolute myth that AI, and, of course, speech recog- speech recognition, which trans-
you can send an algorithm over nition, which has made enormous lates spoken language into text.
raw data and have insights pop progress in recent years. The next is a mechanism for de-
up,” according to Jeffrey Heer, a At the other end of the spec- riving intent from the user query
professor of computer science at trum are the applications that or utterance. Intent can be based
the University of Washington, as require deep data science and al- on training sets and examples
quoted in the New York Times.2 gorithm development expertise. of phrase variations, or it can be
Technology writer Rick Delga- The people who develop these ap- from parsing language to derive
do notes that “many data scientists plications are technical experts meaning.
jokingly refer to themselves as data with deep mathematical and data
janitors, with a lot of time spent science knowledge. They devise The Role
getting rid of the bad data so that and tune the algorithms that pro- of Machine Learning
they can finally get around to uti- vide advanced functionality. Each of these approaches lever-
lizing the good data. After all, bad Along the continuum are the ages machine learning. Some dia-
data can alter results, leading to in- platforms and development en- log management approaches can
correct and inaccurate insights.”3 vironments that make use of the use mechanisms akin to language
In a recent conversation I had tools (many of which are open translation. Given enough ques-
with Laks Srinivasan, chief oper- source). These applications re- tions and enough answers, a ma-
ating officer (COO) of Opera Solu- quire various levels of configu- chine learning algorithm can
tions, he asserted that “80 percent ration and integration to provide “translate” questions into the cor-
of the work the data scientists are capabilities. rect responses. When the intent is
doing is data cleaning, linking, derived via natural language un-
and organizing, which is an infor- Types of derstanding or training set classifi-
mation architecture (IA) task, not Cognitive Computing cation, a response can be retrieved
a data scientist function.” For example, consider a type of from a corpus of content via a
Opera, founded in 2004, was “cognitive computing” applica- ranking algorithm that uses signals
one of the firms that tied for first tion. Cognitive computing is a generated through determining the
prize in a Netflix contest that class of application that helps hu- intent of the user as well as addi-
was offering US$1 million to the mans interface with computers in tional metadata that can inform
company that could beat its rec- a more streamlined, natural way. the user’s context—anything from
ommendation engine by 10 per- Such applications are also capa- purchased products, to configured
cent or more. (The three-year ble of processing information in a applications, to demographic or so-
contest, which ended in August less traditionally structured man- cial media data.
2009, awarded the prize to a team ner to provide a range of answers, Inference can use relation-
from AT&T Labs, which sub- with probabilities based on the ships mapped in an ontology—
mitted its response just minutes user’s context and details about for example, products associated
before Opera.) Opera is an exam- the data sources. with a particular solution or steps
ple of a company that developed One type of cognitive comput- to troubleshoot a specific de-
a platform to help data scien- ing application is the processing of vice configuration. Some of this
tists in many aspects of analysis, large amounts of patient observa- knowledge is inferred from the
feature engineering, modeling, tional data and providing a “sec- data and some is intentionally
data preparation, and algorithm ond opinion” about a diagnosis. structured—the knowledge engi-
operationalization. Physicians are using this approach neering approach to AI.

42 ComputingEdge February 2018

64 IT Pro July/August 2017
Contextualizing Endless requires human judgment and in- typing a question into the com-
Knowledge Sources puts to fine-tune results. Those re- pany’s support chat bot. The bot
Organizations have enormous sults might be the componentized interprets the natural language
repositories of knowledge in the information containing specific question as an intent, and the on-
form of processes, procedures, answers to questions rather than tology allows retrieval of the cor-
manufacturing techniques, re- large amounts of text. When the rect responses from a knowledge
search methodologies, embed- content is fine-tuned and compo- repository. The ontology manages
ded designs, programming code, nentized, the specific answers can intents and responses as well as
configured applications, technical be more readily retrieved. A user terminology and phrase varia-
documentation, knowledge bases looking for an answer does not tions for algorithm training.
of various kinds, engineering want a list of documents, but the The advantage of a natural lan-
libraries, expert systems, tradi- answer to the question. Bots and guage question over a search is that
tional libraries, technical publica- intelligent virtual assistants are it becomes easier to derive the us-
tions, scientific, engineering, and designed to respond with an an- er’s intent when they ask a fully
trade journals—the list of explic- swer or a short list of suggestions formed question rather than typing
it knowledge sources is endless. presented in the correct context a few ambiguous keywords. A bot
Historically, humans have always (the user’s query or intent). Auto- can also be programmed to further
limited the scope of the informa- tagging and autoclassification disambiguate intent by request-
tion that they consume—for ex- machine learning algorithms can ing more detail from the user. This
ample, by picking up a book on a apply the correct metadata to con- type of natural interface can also be
topic, searching for a specific area tent to allow for those contextual- used to access corporate informa-
in a library, pursuing a special- ized results. tion sources—running a financial
ized library, or seeking out a par- analysis or retrieving underwriting
ticular journal. Even in our digital The Role of Ontologies procedures, for example.
age, engineers will go to engineer- Ontologies are the containers of
ing sites for nuanced, specialized metadata—the knowledge scaf- Maturing Algorithms Still
information. Scientists will go to folds or structures that can be Necessitate Data Clean-Up
scientific sites, and so on. abstracted from systems of knowl- While machine algorithms play
Information from highly di- edge and applied to other bodies an important role in both the
verse sources cannot be processed of information for organization preparation of data and interpre-
as raw data inputs for any pur- and contextualization. The on- tation of user intent, these types
pose without restriction. It needs tology can capture the relation- of applications require a signifi-
to be parsed, curated, packaged, ships between knowledge elements cant amount of knowledge engi-
contextualized, and componen- and ways of organizing those neering to be successful.
tized for consumption by users elements—for example, the list of As machine learning algorithms
or ingested by systems for ap- user intents with corresponding mature, the heavy lifting will be-
plication to a limited number of actions. A taxonomy of products come more invisible and behind
scenarios. As powerful as it was, can be related to a taxonomy of so- the scenes, and data or content
the Jeopardy-playing Watson pro- lutions composed of those prod- preparation as well as application
gram required specific informa- ucts. Or a list of problem types can tuning and configuration will con-
tion sources to function correctly. be associated with corresponding stitute the bulk of the work and re-
troubleshooting approaches. quire the greatest effort. With data
Can Curation Tools such as virtual assis- scientists increasingly in short sup-
Be Automated? tants become channels for knowl- ply, business users will need to
Machines can help when given the edge structured with an ontology, perform more analysis so that a
correct scaffolding and represen- along with rules and contexts that backlog does not develop behind
tative training sets. Data and con- apply to specific problem sets. scarce data science resources. Data
tent sources can be processed by Take, for example, the task of ser- preparation is a major challenge,
machine algorithms, overlaying vicing a customer who is trying and operationalizing capabilities
the structure and identifying pat- to set up and operate a new fit- is an even bigger one. This is be-
terns in the information to assist in ness tracker. Instead of search- cause knowledge of deep analy-
componentization and contextual- ing on the website or calling the sis approaches is becoming lost
ization. The process is iterative and help desk, the customer might try in translation from the laboratory

www.computer.org/computingedge 43
computer.org/ITPro 65

environment to the operational client and the same department, the data to a point, independent
environment. Given that detailed multiple teams start with the same of its future use, and then apply
machine learning approaches are raw data, which can be inefficient. AI and machine learning in con-
less accessible to business peo- The second factor is that so much verting big data to small data. By
ple, there is increasingly a gulf be- of the work data scientists are do- putting a semantic layer around
tween the business world and the ing is data cleaning, linking, and analytic models and tools, all the
IT world. However, two trends are organizing, which, as Srinivasan users can find them once the se-
in play. Sophisticated tools are be- mentioned, is an IA task, not a mantic layer is operationalized.
coming more commoditized, while data scientist function. According to Srinivasan, “By
more advanced capabilities are be- The third factor is that even making data independent of use
ing made available to business peo- after the data is cleaned up and cases and operationalizing it, and
ple through platform approaches. models are developed that accu- then making it machine-learning-
The key component of data prep- rately predict (for example) who is driven and AI-driven, the signals
aration, data operationalization, likely to buy a certain product, it learn about the data. The sys-
and translation between business takes a lot of time to go from the tem becomes a learning system,
challenges and analytical tools is data science sandbox to actually not a static, one-time data model-
the semantic layer—the glossaries, operationalizing the analytics that ing system. It becomes a continu-
thesaurus structures, metadata create a business impact. ous feedback, loop-based, living,
standards, data architectures, and This disconnect occurs because breathing kind of a central ner-
quality mechanisms. the development environment and vous system, in the enterprise.”
As the tools get more mature, the production environment are In other words, the seman-
organizations will get value from very different. As Srinivasan ex- tic layer acts as a way to translate
them only if they take control of plains, “The data scientists might business problems into the inputs
the things that will not be com- build a model using SAS in the needed to query a big dataset. The
moditized by the marketplace— sandbox and using certain data- technical predictive algorithms
their data, content, processes, and sets, but the IT department needs operate under the covers, and this
semantic translation layers. For ex- to re-code the variables and mod- complexity is hidden from the
ample, organizations will not get a els in Java or optimize R code to user. The algorithms simply have
competitive advantage by building scale in Hadoop when the appli- to point to the big data sources
speech recognition. That problem cation goes into production. At (that are correctly cleansed and
has been solved (for the most part— this point, the data is also very dif- prepared, of course) and then pro-
it is still improving, but building ferent because it goes beyond the vide their parameters as inputs to
the algorithms from scratch would test datasets, so the data scientists predict their outcomes, run simu-
not have business value). They will, have to retest it against the model. lations, segment audiences, cus-
however, gain a competitive advan- Finally, even when the projects are tomize campaigns, and so on.
tage from servicing their customers in production, all these insights
uniquely with a speech recognition and know-how [are] fragmented Developing an
agent that accesses the knowledge into documents or code or peo- Orchestration Layer
they have about their customers ple’s heads. As staff turns over, In the case of Opera, the com-
and serves up the products and knowledge is lost.” pany went on to build a platform
content they need. from the ground up to create and
Re-Imagining the manage the signal layer, and ran
Rethinking High-Power Analytics Lifecycle mission-critical applications on
Analytics When Opera began consider- it. The platform, called Signal
As demand is exploding for big ing how to address these issues, Hub, processes data from about
data analytics, data scientists are it came up with the approach of 500 million consumers for global
increasingly in short supply. When fundamentally re-imagining the blue-chip clients across indus-
a company is building predictive analytic development lifecycle by tries. This approach allowed Op-
models or machine learning mod- developing a “semantic layer” be- era to essentially outsource the
els, a few factors stand out. tween the data layer (raw data) data science work, operate on
Every journey starts out with and the use case, application, and its platform, and sell solutions
raw data, so if a company is do- UI layer. The thought was that to business buyers. When Opera
ing multiple projects for the same the company could preprocess developed and then productized

44 ComputingEdge February 2018

66 IT Pro July/August 2017
This article originally appeared in
IT Professional, vol. 19, no. 4, 2017.

the platform as an orchestration source to orchestrate more and ost of the work that data
layer in 2013, many organizations more process actions, that, in our scientists do is “data jani-
did not have the IT or data science case, prevents service disruptions.” torial” work, as opposed
resources to fully exploit the pow- This approach is beginning to be to science work, and there is a gulf
er of advanced tools. The market exploited in ways that allow for between prototype and sandbox,
has matured since then, and that interoperability between platforms and innovation and production. In
strategy—to productize as an end- that are exposing functionality addition, having pockets of knowl-
to-end AI and machine learning through a services layer. Those edge and expertise throughout
enterprise platform by harden- “normalized knowledge bases” are the enterprise, which may be gone
ing with security, scalability, and powering chat bots that are driving when an employee leaves, poses a
governance capabilities—provides the next-generation digital worker. problem when the knowledge is
valuable lessons for organizations not institutionalized or captured
building data-driven solutions. Leveraging Platforms in a system. Organizations are best
Thinking about data as a service and Orchestration Layers off if they focus on understanding
and the platform as an orchestra- Many organizations are attempt- their own data, focus on the busi-
tion layer between business prob- ing to build their own platforms ness problems they are trying to
lems and technology solutions can and believe this is required to cre- solve, and build the semantic lay-
help organizations achieve dra- ate a competitive advantage from ers that can allow for data portabil-
matic improvement in data sci- machine learning and AI capa- ity across various platforms. This
entists’ productivity, and in the bilities. The key decision point is lets them take advantage of best-
productivity of business analysts whether the platform is the differ- of-breed solutions and not become
and business intelligence work- entiator or whether it is the data locked into a particular vendor
ers. “The maturing of technolo- and orchestration layer that will that does not abstract the business
gies and emergence of platforms be the differentiator. “I frequently problem, analytic, data, and plat-
is democratizing insights derived hear CIOs say they have a plat- form layers required to operation-
through machine learning and ca- form or that they are building alize the fast-evolving advanced
pabilities provided by AI in a way machine learning. The problem is machine learning analytic and AI
that we say makes ordinary people that it is easy to go through $100 technologies.
extraordinary,” says Srinivasan. million or more, and a lot of pain
“If all the insights and expertise and suffering. I say, ‘Do not try References
are buried in a small team within a this at home’ in my presentations 1. R. Bean, “How Big Data Is Empow-
company, it doesn’t really leverage and hope they take it to heart,” ering AI and Machine Learning at
the value of AI tools to be used by cautions Srinivasan. Scale,” MIT Sloan Management Rev.,
an average call center rep.” A core premise for success with 8 May 2017; bit.ly/2psZyMm.
The concepts of data as a ser- advanced analytics is that orga- 2. S. Lohr, “For Big-Data Scientists,
vice and platforms as an orches- nizations need to build metadata ‘Janitor Work’ Is Key Hurdle to
tration layer have far-reaching structures and ontologies to de- Insights,” New York Times, 17 Aug.
implications for the future of AI- fine relationships among data ele- 2014; nyti.ms/2kl1V3Y.
driven enterprises. Not only can ments relevant to their companies. 3. R. Delgado, “Why Your Data Sci-
data be more fully exploited by Srinivasan continues: “That is entist Isn’t Being More Inventive,”
this paradigm, but so can knowl- the investment that organizations Dataconomy, 15 Mar. 2016; bit.ly/
edge and content—the raw materi- should be making rather than 2rxoy73.
al on which cognitive applications building their own platforms.
are being developed. According to They should be building their Seth Earley is CEO of Earley Infor-
Henry Truong, CTO of TeleTech, own representation of the core of mation Science (www.earley.com). He’s
a $1.4 billion call center services the business, the soul of the busi- an expert in knowledge processes, en-
firm, “Organizations can normal- ness, which is the ontology that terprise data architecture, and customer
ize knowledge in the same way can embody all that knowledge of experience management strategies. His
that they normalize data—through processes and customers. Insights interests include customer experience an-
componentizing knowledge into can then be fed back into the on- alytics, knowledge management, struc-
the building blocks that provide tology, so it becomes that living, tured and unstructured data systems and
solutions to problems. The knowl- breathing thing. It is a semantic strategy, and machine learning. Contact
edge ontology becomes the data layer that evolves around that.” him at seth@earley.com.

www.computer.org/computingedge 45
computer.org/ITPro 67
Micro Economics

Technology Policy and the Trump


Harvard Business School

...... Technology policy has been a What can changes in trade policy do to Beyond that, one additional trade
low priority for most voters in presidential IT firms? At a general level, every large US question is worth watching. Several US
elections in the post-war era. The most tech firm is integrated into the world firms, including Cisco, IBM, Google,
recent contest was no exception. Argu- framework for trade, so tearing up the sys- Apple, and Facebook, have a large foreign
ments about technology policy never tem could cause considerable damage. presence and would like to repatriate their
made it into campaign commercials, to Generally speaking, every large US foreign earnings as US dollars without
say the least, nor even a minute of the firm sources inputs from outside the US paying taxes. The Obama administration
televised presidential debates. and sells final products outside the US. refused to initiate such a tax holiday, and
So it goes. Every major software firm uses pro- Trump might think differently. That would
Many denizens of the high-tech world grammers in the US and outsources move stock prices if implemented.
did not expect Donald Trump to win, some amount of work to programmers That adds up to a big unknown for firm
woke up to his triumph, and suddenly outside the US, and, again, sells their values. The value of virtually every US-
wondered what impact his new policies products to buyers outside the US. That based IT firm depends on the outcomes
might have on their business. Needless goes for Apple, Cisco, IBM, Microsoft, of these policy debates. That supports an
to say to anybody who paid attention, his HP, Oracle, Google, Facebook, and on approach for investors: expect volatility
campaign was not much help answering and on. These firms will lose market share across the entire sector and adopt invest-
those queries, since he was not very and profits if the costs of inputs and labor ment strategies to hedge against it.
specific about his technology policy. increase, or if the number of potential
This column considers two nonparti- buyers declines. Immigration
san questions. How will his (likely) tech- Since Trump’s populist antitrade During the campaign, Trump focused
nology policies affect the value of US tirades conflict with his generally pro- hostility toward immigration. Attention
firms in information technology markets? business attitudes, we should expect was directed at unskilled immigrants
Details about policy should become quite a fight inside the administration from south of the US border, as well as
known in the first half of 2017, and they over the practical details of trade policies. those from nations with Muslim major-
suggest a second question: what details Whether input and labor costs will ities. Quite frankly, if immigration from
should an investor care about, as the new increase, and how much, and whether Mexico slows, then industries other than
administration hashes out the details? market buyers will decline, and how high technology—such as agriculture,
much, cannot be determined until those service work, and construction—will be
Trade details get set. most affected. It is hard to see any direct
Let’s start with trade, which was a visible As for Trump’s dissatisfaction with effect for IT firms arising from that type
aspect of Trump’s campaign. He Chinese mercantilism, most experts pre- of policy.
expressed dissatisfaction with the posi- dict that his confrontational policy will go What if the US (deliberately) started
tion of the US in the world trade system. nowhere. I find these experts so persua- processing visas from Muslim-majority
He focused on the exit of jobs in footloose sive that I’ll take a bet—$100 to your countries with more laborious delays? It
manufacturing industries, the North Amer- favorite charity or mine. You win if a major would affect the visits of foreign execu-
ican Free Trade Agreement (NAFTA), and US IT firm improves its mainland Chinese tives from countries such as Dubai, Saudi
China’s mercantilist actions. market share in the next four years. Arabia, and Malaysia. That might affect

46 78 February 2018 Published

Published by the by the IEEE
IEEE Computer Computer Society 
Society 2469-7087/18/$33.00
© 2018
This article originally appeared in
IEEE Micro, vol. 37, no. 1, 2017.

boutique tourist and consulting busi- Energy, which also support R&D that Trump expressed dismay about the pro-
nesses, but that touches only the edge works its way into commercial IT products. posed merger of AT&T and Time Warner,
of US IT firms. That adds up to a straightforward but most of his circle of advisors are hostile
Investors should focus on policies for forecast: if the budget for R&D at these to blocking mergers. So, the outcome to
high-skill immigrants. Every major high- agencies declines, that is bad for the that merger proposal will show a lot.
tech firm employs high-skill immigrants, whole sector. Drastic cuts slow down Another test case could be a proposed
and this group comprises founders for innovation, whereas growth speeds it up merger between T-Mobile and Sprint,
many venture firms. Many high-skill across the entire portfolio. which the Obama administration’s
immigrants have master’s degrees or appointees talked down before it was for-
PhDs from US universities. The present Commercial Policy mally proposed. Will management
system works reasonably well for those The administration can have immediate attempt to resurrect it? We’ll have to
with degrees. impact through staff and personnel watch and wait.
Drastic changes—such as slowing appointments to agencies that make Also watch the administration policy
the granting of visas and green cards to commercial policy for high tech. For in privacy, which the FTC took a lead on
those with degrees—will slow down example, most observers expect the in the last few years. The issues are var-
innovation in the commercial IT sector. administration to appoint directors to the ied, subtle, and difficult. The policy out-
That holds for virtually all US IT firms, so Securities and Exchange Commission comes make an enormous difference to
any slowdown hurts investments across who oppose government actions. Expect product design and the operations of
the sector. government regulators not to stop Wall many firms, especially those in online
One aspect of high-skill immigration is Street banks from returning to the kind advertising and healthcare. Again, invest-
difficult to forecast—namely, changes to of self-serving (and sometimes unethical) ors will have to wait and see.
the system for H-1B visas. The H-1B sys- actions observed in the 1990s during the Similarly, Apple faced a quandary pro-
tem is already rather constrained, and IPO boom. Frankly, I think this will be tecting privacy when it negotiated with
nobody expects that the Trump administra- bad for the US startup economy. the FBI about breaking into an old iPhone.
tion will try to reform it. More to the point, Investors should also expect the Fed- The FBI learned of another way into the
any tighter limits would hit a few firms eral Communications Commission (FCC) phone, so the broad issues never got
hard. Will that happen? It is hard to say right to appoint decision makers who will not resolved. The Obama administration sided
now. Investors have to watch and wait. intervene in Internet markets. Net neu- with law enforcement, and Trump did, too
trality will not be enforced, and many (even calling for a boycott of Apple).
Research and Development other recent initiatives will be reversed, Expect more volatility from these issues.
A new administration can also change such as those aimed at opening up the It is a wild card for values at many firms.
policy for R&D. Although it’s less visible set-top box for cable television.
to the average voter, the US government
is the single largest funder of basic sci-
ence and also a large funder of experi-
That will raise the value of big cable
firms, such as Comcast, and other car-
riers, such as Verizon, because it will
O verall, Trump’s policies look like
a mixed bag for the value of
many US IT firms, and contain many
ments in applied science. give them the upper hand in negotiations dangers. This much I can forecast:
This matters to US IT firms, who have on a range of issues, such as zero rating, most savvy tech firms woke up the
benefited from this funding in the past. interconnection fees for moving data into day after the election and added staff
For example, new network engineering, ISP networks, and collocation fees for to their Washington lobby organiza-
search engines, AI, and robotics can trace content delivery networks. Again, tions. The cynic in me also expects the
their invention to federal funding. In addi- frankly, I think this will be bad for the Trump administration to try a quid pro
tion, many computer scientists got their value of content firms with big data appli- quo, such as offering a tax holiday as a
first experiences on projects funded by cations, such as Netflix, YouTube, Face- bribe to gain silence on other issues.
this federal money, which effectively sub- book, and venture capitalists backing That suggests a somewhat partisan
sidized US technology workforce training. new streaming entrants. forecast. I do not expect that most CEOs
There is no reason to expect the emer- Antitrust is a more ambiguous area. want their issues to be invisible in the
gence of additional technologies to be any The Obama administration let 99 percent next election. I also expect their stance
less sensitive to federal funding, so of mergers go through, and that will con- next time to depend on their experience
changes to the funding level at DARPA tinue. The only open questions concern in the next few years. MICRO
and the National Science Foundation are big mergers.
key budgets to watch. The same goes for Watch the early test cases for clues Shane Greenstein is a professor at the
R&D funding at the National Institutes of about the general approach of the new Harvard Business School. Contact him
Health, NASA, and the Department of administration. On the campaign trail, at sgreenstein@hbs.edu.

www.computer.org/computingedge JANUARY/FEBRUARY 2017 79 47


Cloud-Native Applications
and Cloud Migration
The Good, the Bad, and the Points Between

loud-native features in your information technology (it) systems

come with many advantages, but they also come with a cost. The
costs vary greatly, based upon your applications and data. Sometimes
being cloud native doesn’t make economic sense, and sometimes
it does.

Keep your eye on that ball as you migrate to the changing a small percentage of the code to lever-
cloud. age some cloud-native features. Complete refactor-
A global 2000 company or a government agency ing means rewriting most of the applications so that
typically has more than 5,000 applications on legacy they become cloud-native applications.
platforms. These applications are placed into cate- You need to pick a path from one of the 7 R’s for
gories: ones that should move to the cloud and ones each application, based upon its immediate, as well
that should not. If they move to the cloud, then do as long-term, return on investment. In some cases,
they need to be altered to leverage cloud-native fea- you need to break applications apart into reusable
tures (refactoring), or they could be moved with few components, and in other cases, applications can be
or no changes (lift and shift)? replaced with a software as a service (SaaS) applica-
As you can see in Figure 1, an application can tion analog. Some applications need to be removed,
take many paths on its migration to the cloud. and many applications should stay put.
They’re called the 7 R’s of migration. This includes The toughest part of application migration to the
lift and shift (rehosting), or moving the applications cloud is figuring out the correct path. The retire path
with few or no changes. Partial refactoring means is usually obvious from the start. If applications are
movable to the cloud, how should they be configured,
changed, or not? Enterprises still struggle with these
issues, but best practices are starting to emerge.

Why Go Native
The pros of going to cloud-native features include
EDITOR the following:
david@davidlinthicum.com • Performance. You’ll typically be able to access
the native features of public cloud services to

12 I E E E C L O U D CFebruary
O M P U T I2018
N G P U B L I S H E D B Y T H E I E E E C O M Published
P U T E R S Oby
C Ithe
E T YIEEE Computer Society  2 3 2 52469-7087/18/$33.00
- 6 0 9 5 / 1 7/ $ 3 3 . 0 0 ©©2 2018
0 1 7 I IEEE
provide better performance than is • Replace application with SaaS service
possible with nonnative features. Replace • Build cloud native application with similar/improved
characteristics and features
For example, you can deal with
an input/output (I/O) system that
• Develop and productize common business and technical services
works with autoscaling and load- Reuse
• Consolidate similar applications and services services
balancing features.
• Efficiency. Cloud-native applica- • Targeted changes to the application to address issues; leverage
tions’ use of cloud-native features Refactor the cloud
• Revise applications to leverage common services
and application programming inter-
faces (APIs) should provide more
• Move to more cloud-aligned technology and platform services
efficient use of underlying resources. Replatform
• Integration with cloud operations and monitoring
That translates to better perfor-
mance and/or lower operating costs. • Lift and shift, minimizing changes, virtualization replatforming
• Cost. Applications that are more • Physical to Virtual to Cloud (P2V2C), Virtual to Cloud (V2C)
efficient typically cost less to run.
Cloud providers send you a monthly -
Retain • Leave application as-is; do not move to cloud
bill based upon the amount of
resources consumed, so if you can
do more with less, you save on dol- Retire • Application end of life
lars spent.
• Scalability. Because you write the FIGURE 1. The 7 R’s of cloud migration. Source: Cloud Technology Partners
applications to the native cloud
interfaces, you have direct access to A General Approach to Becoming for the resources you use, so applica-
the autoscaling and load-balancing Cloud Native tions that work more efficiently with
features of the cloud platform. To take proper advantage of a cloud those resources run faster and generate
platform, including infrastructure as a smaller cloud service bills at the end of
The price you pay for these advan- service and platform as a service (PaaS), the month. Several cloud-cost gover-
tages is portability. Applications that you have to design the applications so nance tools can monitor these costs for
are localized for specific cloud plat- that they’re decoupled from any spe- you. I highly recommend using one or
forms are not easily ported to other cific physical resource. For example, if more of these tools if you plan to spend
cloud platforms. Doing so takes a great you access I/O directly from a platform the time and money to move and refac-
deal of rewriting or refactoring of the as Linux, you need to access the cloud’s tor applications to be cloud native.
code. For all practical purposes, you are abstraction layer, or its native APIs.
locked into that cloud platform. Clouds can provide an abstrac- The Downsides of Cloud Native
If applications run on the tar- tion or virtualization layer between the There are trade-offs to cloud-native
get cloud platform for years, you’re application and the underlying physical application development and design. Be
bound to get your investment back in (or virtual) resources, whether they’re aware of the trade-offs before you bind
code changes and testing. You have to designed for cloud or not. But that’s not your application to a specific cloud. You
look at the advantages of going cloud good enough. If you’re truly going cloud should do this with the heart of a chief
native case by case and application by native, you need to directly control the financial officer, not a technologist. At
application. native resources. the end of the day, this is about costs,
Unfortunately, there are no pre- When this architecture is consid- not what is technically superior.
made checklists you can use to make ered in the design, development, and The biggest trade-off, as we covered
these assessments. You simply have to deployment of an application, the utili- already, is that you’re giving up porta-
get as smart as you can about the trade- zation of the underlying cloud resources bility for the advantages of being cloud
offs and make the best decisions you can be as much as 70 percent more native. Applications that are localized
can (https://www.cloudtp.com/doppler/ efficient. This cloud computing effi- or made native for specific cloud plat-
pros-cons-going-cloud-native/). ciency equals money. You’re paying forms are not portable to other cloud

This article originally appeared in
IEEE Cloud Computing, vol. 4,
no. 5, 2017.

applications truly cloud native, the

Traditional Cloud Aligned
Application Application architecture must be rethought before
Architectures Architectures you begin refactoring the code.
• Scale Up • Scale Out Does this new approach to applica-
• Monolithic • Distributed tion architecture suck resources and
• Stateful • Stateless
• Infra Dependent • Infra Agnostic money, as well as adding a great deal
• Fixed Capacity • Elastic capacity of risk? Yes. However, the risk-reward
• LAN, SAN • WAN, Location
• Latency intolerant transparency scale typically leans to the reward side if

• Tightly coupled • Latency tolerant the life of an application is 10–15 years

• Consolidated/ • Loosely coupled
(which it is for most enterprises). The
clustered DB • Sharded/replicated/
• Rich/chatty client distributed DB effort of both rearchitecture and refac-
• Commercial licenses • Mobile/thinclient toring for an application with long-term
• Infra Supported • Cloud PaaS/Open
Availability Source use will pay for itself many times over.
• Manual build/deploy • App Supported However, enterprises in the US
• Manual fault Availability
recovery • Automation are not wired for long-term investiture.
• Active/Passive/DR • Self healing Most enterprises opt for lift and shift
• Perimeter Security • Active/Active
versus refactoring for cloud native.
• Allocated costs • Defense in depth
• Metered cost Enterprises are rewarded based upon
The “Old World” The “New World” earnings, and the lower the cost of new
software development and IT, the more
FIGURE 2. Being cloud native means more than changing code. You need to change
your application architecture as well. Source: Cloud Technology Partners. earnings they can claim—short term.

platforms without a great deal of rewrit- good application design practices are not Do the Right Thing
ing or refactoring of the code. common; most developers exploit the The evidence is compelling for the cloud-
That being said, few cloud-to-cloud cloud-native interfaces systemic to the native application path when migrating
migrations have happened yet. This is application. Those are harder to change. applications to the cloud. The benefits
true even when rehosting or lift-and- My best recommendation is that outweigh the costs for most applications
shift migration paths are chosen. If the you at least consider cloud-native appli- that are picked to be moved to the cloud,
current trends continue, the argument cation development approaches and best but given that refactoring costs 30 times
against lock-in does not seem to be practices—if not for the efficiency, then simple rehosting, enterprises are reluc-
much of an issue for most enterprises. perhaps for cost or performance. Although tant to jump in with both feet.
But you should at least keep this issue there is a clear trade-off in portability, So, this will be another learning
in the back of your mind. the benefits seem to outweigh that cost process. Much like we saw when we
Cloud lock-in does expose the quickly if you’re going to run the applica- made applications platform native, such
enterprise to some risk if the cloud plat- tion for more than a couple of years. as Win32- and Unix/Linux-native APIs,
form provider becomes difficult to deal we had to fail first. In other words,
with, raises prices, or most likely, some Emerging Architectures working around the native platform fea-
of the services you’ve purchased and Cloud native is not about just changing tures led to applications that did not live
use get turned off. In those cases, you’re the code to follow the features of a spe- up to the expectations of the business,
going to have to bite the bullet and re- cific cloud; it’s also about changing your and IT had to go back to square one to
refactor those applications to be cloud approach to architecture design. As you redesign and refactor the applications to
native on another cloud. can see in Figure 2, what’s emerging meet the needs of the business.
Some developers are getting good at is a world of cloud-aligned application I suspect we’ll follow the same pat-
placing the cloud-native features of an architectures. terns here. In a few years, cloud native
application into a specific domain of the These cloud-aligned architectures will become the best practice. However,
application design. This allows them to can autoscale and are distributed, that won’t happen until we fall on our
more easily change the application for stateless, and loosely coupled, to name faces a few more times. Some things
other cloud platforms, if needed. These a few features. If you want to make never change.

14 ComputingEdge
I EEE CLO U D CO MPUTING W W W . C O M P U T E R . O R G / C L O U DFebruary
C O M P U T2018

Dana Ulery: Pioneer of Statistical

Computing and Architect of Large,
Complex Systems

Irina Nikivincze
Georgia Institute of Technology

From Grinnel to JPL though JPL hired female mathematicians in the 1940s
Dana Lynn Ulery (1938–present) did not grow up in a to 1960s to perform trajectory calculations, it was in an
typical 1950s family.1 Her mother, Meriam Mueller2 all-female unit.5 There was no precedent for hiring
(1908–2005), was a businesswoman and active volun- women for an engineering job. A man at JPL who inter-
teer in the local hospital. Her father, Harry Tanzer, died viewed Dana was amused by an enthusiastic and math-
when she was only 2 years old. Over the years the lone- minded young lady. Nevertheless, he offered her a job
liness that Dana felt was replaced by her love for learn- and took care of the paperwork, so she did not need to
ing and school. Graduating from Grinnel, a small go to Human Resources. Dana became the first and only
liberal arts college in Iowa, in 1959 as a double major in female engineer at JPL in 1960. JPL did not hire other
mathematics and English literature, Dana’s future women engineers for the next 7 or more years. One of
looked uncertain, but exciting. She was getting married Dana’s first assignments was to evaluate the systematic
and the young family was going to tour the West and pointing errors in the Goldstone Polar-Mount Antenna
move to Pasadena, California.3 As with many other using the star-track data.6 During the following three
women of that time her choices were limited—she years she worked on real-time tracking systems and
could be a secretary, nurse, stewardess, or teacher. Just algorithms for NASA’s Deep Space Network—the work
in case, she got a teaching certificate and quite unex- that was exciting and that she became very fond of.
pectedly got an offer to teach mathematics in a high The official title that Dana held was of a junior engi-
school in Burbank, California, minutes away from her neer and it meant that she was paid less than her col-
husband’s workplace. Her initial excitement soon dissi- leagues. Having learned that from a friend, Dana
pated after she learned that the class that she was teach- succeeded at changing her title (and the corresponding
ing was filled with kids who could not read. There was pay) to one of a Research Engineer. She greatly enjoyed
more to it. All of those kids were boys from a reform her job at JPL but not for long. In 1963, Harris was grad-
school—teenagers who already had gotten in trouble uating with his PhD and announced that he would be
with the law. A young female math teacher with her looking for jobs. Although, Dana wanted to stay at Pasa-
rules and homework was the least of their concerns and dena, the family weighted in, and the move was
perhaps a source for amusement. The boys refused to do imminent.
homework and Dana’s teaching aspirations vanished Among the three job offers that Harris received, the
day by day as her focus shifted from content to class dis- least “horrible” in Dana’s opinion, was one in Delaware,
cipline. One year was enough for Dana to realize that OH. Dana stayed at JPL as long as she could and then it
teaching was not what she wanted to do in her life and was time to move to the East Coast. Moving to a new
that she needed a new job. place meant facing the same obstacle—with no estab-
In Pasadena, her husband, Harris Ulery, just started lished networks, getting a new technical job would be a
his graduate studies in organic chemistry at the Califor- hurdle. She explored every opportunity: a job at Thio-
nia Institute of Technology. Since Caltech was a techni- kol, a chemical company that also contracted with
cal school, Dana was convinced that somewhere there NASA, and a programming job at Getty Oil. Her training
was a job for a math major. Having gotten her confi- and previous experience at JPL made her a superior can-
dence together, Dana walked the halls of Caltech, stop- didate for the programming job. Again, she became the
ping by open doors and inquiring about jobs. only woman in her team. Eight men who worked in her
“Somebody finally took pity at me and sent me to JPL,” group all had desks, but since they were not used to
she later recalled.4 women and did not know how to treat her, Dana
The National Aeronautics and Space Administration was given a private office. The unfortunate side of
(NASA) Jet Propulsion Laboratory (JPL), a large federally this arrangement was that she became isolated. There
funded research center, was located near Caltech. Even were no mental challenges, contacts, or discussions of

2469-7087/18/$33.00 © 2018 IEEE Published by the IEEE Computer Society February 2018 51
IEEE Annals of the History of Computing Published by the IEEE Computer Society 1058-6180/17/$33.00 c 2017 IEEE 91

interesting problems. The new job did not recalled. At home, Dana still took care of the
come close to the work she was doing at JPL. family and cooked dinners, and the family
The conflict at home was growing: Dana ate together. Not having a full-time mother
wanted to go back to school, while Harris meant that the family had to cope with late
wanted his life to continue as it was without dinners, accommodations for kids, and
any changes. Dana pursuing graduate work other “inconveniences.” Only with their
would put a burden on the whole family, and support and cooperation was Dana able to
especially on him. The family somehow man- get through school.
aged when he was a graduate student and In 1975, Dana had finished her disserta-
Dana worked, took care of the house, and tion “Computer Science’s Reincarnation of
their small son. However, Harris was not Finite Differences,” submitted in 1976 for the
ready to assume some of Dana’s responsibil- degree of Doctor of Philosophy in Applied
ities. Divorce was imminent and only the Science (Computer Science).” In 1976, her
birth of their second child, Terrie in March advisor was doing lecturing and research at
1965, made Dana halt her aspirations. the American University in Cairo in Egypt.
Not having other prospects, Dana accepted a
postdoc position at Cairo University and
Going back to graduate school at the became a visiting lecturer. At that time, the
University of Delaware subject of computer science intrigued aca-
Five years later, in 1970s, at the age of 32, demics around the world, including in Egypt.
Dana entered the graduate program at the However, that temporary position did not
University of Delaware. She felt revived to be resolve the question of what she should do
back in an intellectual environment. It meant next.
a great deal to her to be around people who Dana’s graduate training was grounded in
talked about interesting problems. One of mathematics, statistics, and computing. Her
them was a young faculty member Dr. Hatem graduate department trained students in
Khalil. His openness and confidence would “Applied Sciences,” although it was already
encourage Dana to stay in the program called the department of Statistics and Com-
beyond Master’s.7 Hatem Khalil defended his puter Science.8 Dana’s advisor, Dr. Khalil, was
dissertation the same year Dana entered uni- involved in curriculum reform at his depart-
versity under Dr. John H. Giese, Professor of ment. Unlike some of his colleagues, he gave
Mathematics and Computer Science and importance to the theory and practice of
Chief of Computing Laboratory at Aberdeen computing. In his view, programming was to
Proving Grounds, Maryland. Khalil was a computer science “what the laboratory is to
mathematician who was interested in com- the physical sciences,” and the study of com-
puters and the new emerging field of com- puter science that he defined as “the theory
puter science. He would become not only a and practice of programming computers”
teacher but also a colleague and friend. should include the study of algorithms—the
Khalil also was one of most difficult pro- main theoretical foundation for program-
fessors in numerical analysis. In the early ming.9 Thus, Dana’s graduate work was in
1970s, there were very few female students in the theory of programming—in algorithms.
the department. Men would talk a lot in As part of her Master’s degree she devised a
classes, and they appeared smart and knowl- language-oriented system that she called Lin-
edgeable. Dana got so intimidated that she eal for solving problems in linear algebra. In
took it to her advisor. She thought that she her dissertation she explored the techniques
was doing poorly as she could not think and for solving partial differential equations, in
follow the class discussion as fast, and she particular the method of finite differences.
was pondering if she should even stay in this The repetitive nature of this method begged
class. To her surprise, Khalil told her that she for a generalization and automation. That
was the best student in his class and even was exactly what Khalil and Giese were
though others were talking a lot and answer- doing—devising and exploring algorithms
ing his questions, they were giving wrong for generating families of difference approxi-
answers. These comments became a source of mations. In her work, Dana extended their
relief and encouragement. In the company of method to higher dimensions and explored
Dr. Khalil, Dr. Giese (Dana’s co-advisor), and its boundary conditions.10
her husband Dr. Harris Ulery, the work What to do with her PhD still remained
became fun. “If I would not have these men, I a question. As with many other women,
would not have stayed in the field,” she later Dana did not get much direction from the

9252 IEEE Annals of the

History of Computing February 2018
university about what to do next. Univer- designers, the role of statistical consultants
sities and organizations for the most part became one of system builders and organi-
were ambivalent towards women.11 zational optimizers.
By the time Dana entered DuPont the
company already had problems. The business
DuPont, 1977–1994 was in trouble because the quality of the
One day somebody called the department at products was inconsistent. A suggestion was
the University of Delaware looking for stu- made to put together a large system that
dents interested in high-level languages and could help to get the product quality back in
contract work. The department gave Dana’s order. Since 1972, DuPont was developing a
name. Don (Donald) Marquardt was a man- system for product quality management that
ager at DuPont who was highly respected and incorporated business philosophy, manage-
influential in statistics. Marquardt told Dana ment, and technology systems. The develop-
about the job, but she did not find it very ment of a product quality control system by
interesting. When Dana’s advisor found out Marquardt’s team was an important project
that she refused the job, he was furious. For- that required innovations in statistical meth-
tunately, DuPont called back and this time ods and its implementation as a computer
they had a more interesting offer that Dana program. If in a pre-computer age manufac-
enthusiastically accepted. turers could use Shewhart charts to monitor
Dana was working in the Applied Statistics process changes, but with increase in scale
Group of the Engineering Department of “E. and complexity of production, it was diffi-
I. du Pont de Nemours and Company” in Wil- cult to discover intermittent changes. Mar-
mington, DE first as a software engineer and quardt’s team proposed a cumulative control
later as a consultant. DuPont was a large strategy—one that they called Cumulative
high-technology company that also had Sum (CUSUM), which included additional
“large and conceptually complex prob- measures, such as warning limits, run tests,
lems.”12 It was one of the best places for basic and more sophistical detection of changes
research. The company was collecting the through average run length curves.15 The
Crème-de-la-Crème—technical people from development and use of such a complex
various places. Very prestigious PhD statisti- quality control system depended on the use
cians worked there. Donald Marquardt man- of computers and on the training of the
aged a group of approximately 30 people, of personnel.
whom 18 were statisticians, half had PhDs, The Applied Statistics Group was running
and the rest had MS degrees.13 simulations and modeling, but they did not
The Applied Statistics Group of the Engi- have many people who could program it on
neering Department wanted to do more work PDP-11. Dana worked on computer imple-
with computers. Working in industry, Mar- mentation of CUSUM quality control
quardt wanted statisticians to be more entre- schemes that were able to detect changes in
preneurial and business-oriented. His unit processes ahead of production defects.16 By
was providing “statistical consulting” serv- 1977 that project became the major consult-
ices at DuPont and serving close to 110 sites ing activity of Marquardt’s group: they
in the United States. He encouraged consul- trained over 15,000 people in product quality
tants to get involved in projects, learn the management and implemented over 10,000
lingo, but avoid doing the engineering work computerized CUSUM control loops.17 It also
or project management, and instead address launched Dana’s career in software quality
a client’s problem using statistical methods. management.
Frequently, the end product of statistician’s In the Research Division DuPont had very
work became “embodied in a computer pro- specialized personnel—some of whom had
gram that the client could use to handle PhDs specifically in color theory. Although
future instances of the same problem.”14 A they collected data and did experiments,
computer program was not only able to they did not solve nonlinear equations that
standardize and monitor manufacturing were Dana’s area of expertise. However, work-
processes, but also improve the organiza- ing with them did not get her closer to statis-
tion of operations. Maintaining some dis- tical work and the opportunity to contribute
tance from actual projects put consultants to that area. Instead, she moved to software
in an advantageous position to be able to engineering because it was “available” to her.
codify and improve the organizational pro- Together with new career advancement came
cesses. In addition to being quality control changes in family. Old arrangements were

April–June 2017 5393

not working. Dana divorced her husband in a lot of consideration and responsibility
1978 and married her colleague William Fell- because the systems that she was designing
ner, a consultant with DuPont’s Quality Man- were shaping users and their thinking.
agement and Technology Center in 1980, Having worked on many complex proj-
with whom she shared many interests and ects, Dana learned that technical and social
hobbies. aspects of software engineering were often
Dana excelled in development of large sys- intertwined. Problems appeared to be tech-
tems and applying new tools for the optimi- nical only at the beginning and eventually
zation of existing systems. She wrote a came down to dealing with people. Such
quality system named QFACS that facilitated systems often required streamlining exist-
data communication. As a result, product ing work processes while organizations
pigment quality improved, and she was asked resisted changes in business practices. Even
to do a major presentation. As the first quality the small-scale transactional systems that
system was gradually aging, DuPont wanted she worked on required improvements in
a new quality system, only much bigger. That work practices. The management of work
system included close to 150,000 people. She practices necessitated the management of
was working in the team with two other men organizational culture. By that point, Dana
and was responsible for software architecture became increasingly interested in social
of the system. At that time they did not have aspects of systems, growing semantic web
software architects but they had Dana. Such (internet) and policy issues. From the late
system had to handle not only operational 1980s, Dana worked on common data
complexity but also coordinate an influx of standards while serving on the Accredited
data from different business units. In 1991, Standards Committee X12 (ASC X12) of the
together with Donald Marquardt, Dana American National Standards Institute
edited a 600-page reference book entitled (ANSI) and as Pan American Delegate to the
Product Quality Management.18 United Nations Electronic Data Interchange
In the mid-1990s, DuPont, which was for Administration, Commerce, and Trade
made up of many businesses, started to out- (UN/EDIFACT). In 2007, she retired from
source many of its parts to other companies, her position at the United States ARL.
such as Computer Sciences Corporation
(CSC) and Anderson Consulting. They kept a Conclusion
small number of people on a contract basis. As many other pioneering women, Dana had
The outsourcing fever was only starting, but few expectations and little knowledge of
it was clear that sooner or later this move career options in technical fields. Although
would significantly reduce DuPont’s internal she overcame some gender stereotypes by
consulting unit. Dana lost her job in 1994. At entering a technical field and by working on
that time she was 56 years old. Even with her technical problems, she could not change the
experience, finding another job was not easy. environment or the behavior of other people.
Institutions treated women differently: they
often occupied separate spaces, were not
United States Army Research Lab (ARL) given client accounts and money that their
After many phone calls and networking, male colleagues had, and suffered the indig-
including with former advisors, she managed nity of having men even refuse to work with
to get a job at a senior position at the Aber- them. It was easy to be in “someone’s way,”
deen Proving Ground, MD. She became a and if a woman moved up to a position of
senior research scientist and Acting Chief of responsibility—somebody did not like it.
Intelligent Systems Branch at the United Men’s attitudes often were very wearing on
States Army Research Lab. As in DuPont, she technical women. If things were different,
continued doing applied research, except Dana thought that she might have gone
now her focus shifted to systems that helped higher, for example, became a manager or a
the United States military to do things better director of a larger part of the organization—
and faster: the use and impact of intranets,19 a dream that she partially realized at the ARL.
software prototype for purchasing (BuyIt)—a The biography of Dana Ulery provided a
part of the Corporate Business Application glance at the transformational role of com-
Software System (C-BASS), and finally, the puters in science, industry and military at
technologies for intelligent data/knowledge the end of the 20th century. It revealed that
fusion that facilitated actionable knowl- computers were indispensable in solving
edge.20 Dana approached system design with complex quality control issues in the 1970s,

9454 IEEE Annals of the

History of Computing
February 2018
optimizing both technical and organiza- 9. H. Khalil and L.S. Levy, “The Academic Image of
tional processes and, perhaps, facilitating Computer Science,” ACM SIGCSE Bulletin 10, no.
organizational outsourcing in 1980s. Dana’s 2, 1978, pp. 31–33.
unique skills, which combined the knowl- 10. See Ulery’s dissertation and J.H. Giese, H.M.
edge of statistics, programming, and system Khalil, and D.L. Ulery, “Multiparameter Families
architecture, came in handy in addressing of Differences Approximations for the First Initial
those problems. Dana had a successful career Boundary Value Problem for the Heat Equation
in industry. She followed the opportunities. in an Arbitrary Region,” J. Eng. Math., vol. 12,
One technical challenge led to another. Her no. 2, 1978, pp. 97–114.
career allowed her to see the breadth of tech- 11. This comment was made more than once in
nology much more than she would have interviews conducted by the author with
encountered in a classroom. She was able to women pursuing degrees in computer science
raise two children and finish graduate in early 1970s.
school. Even long hours at graduate school 12. D.W. Marquardt, “Statistical Consulting in
had a positive side effect—it allowed her kids Industry,” Am. Statistician, vol. 33, no. 3, 1979,
to see her as a full human being. Her daugh- p. 105.
ter later became a scientist with a PhD in 13. D.W. Marquardt, “Statistical Consulting in
Biology. Being one of the first in computer Industry,” Am. Statistician, vol. 33, no. 3, 1979,
science, Dana Ulery managed to have what pp. 102–107.
other women of that time could only dream 14. Marquardt, “Statistical Consulting in Industry,”
of—a loving family and the excitement of p. 103.
intellectual work. 15. D.W. Marquardt, “New Technical and Educa-
tional Directions for Managing Product Quality,”
Am. Statistician, vol. 38, no. 1, 1984, pp. 8–14.
References and Notes 16. D.L. Ulery, “Software Requirements for Statisti-
1. This biography is based on the interview with cal Quality Control,” IFAC Real Time Program-
Dana Ulery conducted by the author on Decem- ming, 1985, pp. 39–42.
ber 4, 2013 for the project “Careers and Contri- 17. Marquardt, “New Technical and Educational
butions of the First Doctoral Women in Directions for Managing Product Quality.”
Computer Science” sponsored by the ACM His- 18. D.W. Marquardt and D.L. Ulery. Product Quality
tory Committee. Management, E.I. du Pont de Nemours, Quality
2. “Meriam Mueller,” The News Journal (Wilming- Management & Technology Center, 1991.
ton, DE), November 7, 2005, p. 13. 19. D.L. Ulery, ARL Intranet Analysis and Development
3. “Dana Tanzer Married to Harris E. Ulery,” St. Study, ARL-CR-441, Army Research Laboratory/
Louis Post-Dispatch, August 30, 1959, p. 102. Georgia Institute of Technology, 1999, http://
4. D.L. Ulery, “Making Science: Careers and Con- handle.dtic.mil/100.2/ADA362869.
tributions of the First Doctoral Women in Com- 20. R. Scherl and D. L. Ulery, Technologies for Army
puter Science,” Interview by Irina Nikivincze, Knowledge Fusion, Army Research Laboratory,
Landenberg, PA, December 4, 2013. 2004, http://purl.access.gpo.gov/GPO/
5. E. Conway, “Women Made Early Inroads at JPL,” LPS125004.
March 27, 2007, https://www.jpl.nasa.gov/
news/news.php?feature¼1327. Irina Nikivincze is a Postdoctoral Researcher at
6. D. Ulery and J. Fearey, “Evaluation of Goldstone the Georgia Institute of Technology. Her research
Polar-Mount Antenna Systematic Errors from explores scientific careers, gender, achievement, and
Star Tracks,” Technical Memorandum 33-45 recognition in computer science. Contact her at
(Unclassified), Jet Propulsion Laboratory, May
5, 1961, Publications of the Jet Propulsion Labo-
ratory July 1961 through June 1962 (Bibliogra-
phy No. 39-3), S.L. Kresser and R.J. Sippel, eds.,
Jet Propulsion Laboratory, October 15, 1962,
This article originally appeared in
p. 47.
IEEE Annals of the History of
7. D.L. Ulery, “Computer Science’s Reincarnation
of Finite Differences,” PhD dissertation, Univ. of
Computing, vol. 39, no. 2, 2017.
Delaware, 1976.
8. H.M. Khalil, “A Two-parameter Family Of
Approximations to The Two-dimensional Heat Read your subscriptions through
the myCS publications portal at
Equation,” PhD dissertation, Univ. of Delaware,

www.computer.org/computingedge 55
April–June 2017 95

Computing in World War I

his year marks the 100th anniversary of America’s entry into World War I.
On 6 April 1917, both chambers of Congress voted overwhelmingly to grant
President Woodrow Wilson’s request to declare war on the German Empire.
Among the first US military formations to arrive in Europe was US Battle-
ship Division Nine, which comprised the battleships Delaware, Florida, New York, and
Wyoming and the destroyer Manley. The ships joined the Royal Navy’s Grand Fleet at
its base in Orkney on 9 December.
The fastest, most modern warships of the time were propelled by oil-powered steam
turbines. Short on oil because German U-boats were sinking British oil tankers, the
by Charles Day Royal Navy had asked the US Navy to send older, coal-powered ships instead. Despite
their relative age, the battleships of Division Nine were formidable war machines. All
of them survived the war.
When the US ships went on training exercises with British ships, it became clear
that the British could sustain higher and more accurate rates of fire. Part of the British
superiority arose from the three years of experience the Royal Navy had already gained
from fighting Germany and its allies at sea. But the Royal Navy also had a technologi-
cal advantage: the main guns of its largest battleships were controlled by a computer.
Computer control was an integral feature of a new type of battleship introduced
a decade before war broke out. Named after their archetype, HMS Dreadnought, the
new battleships were equipped with ten or so long-range guns of a single large caliber,
rather than a mix of small, medium, and large calibers. If a target’s range, bearing,
and speed were known, an aiming command could be transmitted to the gun turrets
to direct a single, devastating salvo at the target. And if the aim was off target, a single
adjustment sufficed for the second salvo.
The sequence of events that culminated in a Dreadnought salvo began with the use of
an optical rangefinder, a binocular-like device that used triangulation to determine the
range. Measurements from the rangefinder, including speed and bearing, were fed into
a mechanical computer and electrically transmitted to mechanical computers that used
lookup tables to determine the guns’ elevation and azimuth at the moment of firing.
The biggest naval battle of World War I, Jutland, took place in the North Sea between
31 May and 1 June 1916. One hundred and fifty-one warships of the Grand Fleet engaged
99 warships of the Imperial German Navy’s High Seas Fleet. Despite the Grand Fleet’s nu-
merical advantage and despite the High Seas Fleet’s lack of computerized fire control, Britain
lost more ships: 14 versus 6. Still, the German losses were deemed so heavy that the High
Seas Fleet remained at port for the rest of the war. The Royal Navy continued to blockade
Germany, while the Imperial German Navy resorted to unrestricted submarine warfare, a strat-
egy that contributed to the US decision to enter the war on the side of Britain and her allies.

O ne of the British battleships that fought at Jutland was HMS Warspite. The ship
not only survived the battle, it went on to serve in World War II with upgraded
weaponry and a faster, though still analog, computer control system. During the
This article originally Battle of Calabria on 9 July 1940, the Warspite’s fire control system succeeded in hit-
appeared in ting the Royal Italian Navy’s battleship Giulio Cesare at a range of approximately 24
Computing in Science km. The feat remains one of the longest-range hits in the history of naval gunnery.
& Engineering, vol. 19,
no. 4, 2017. Charles Day is Physics Today’s editor in chief. The views in this column are his own and not nec-
essarily those of either Physics Today or its publisher, the American Institute of Physics.

56 February 2018 Published by the IEEE Computer Society  2469-7087/18/$33.00 © 2018 IEEE
88 Computing in Science & Engineering 1521-9615/17/$33.00 © 2017 IEEE Copublished by the IEEE CS and the AIP July/August 2017
Tokyo, Japan 2018
Tokyo, Japan
Tokyo, Japan
July 23-27Japan
July 23-27
Staying Smarter in a Smartening World
Smarter in a Smartening World
Staying Smarter in a Smartening World
Staying Smarter in a Smartening World
Call for
for Papers
COMPSAC is the IEEE Computer Society Signature Conference on Computers, Software and Applications. It is a major
is the IEEEforComputer
and government
Conference to discuss research Software
on Computers, results andandadvancements,
Applications. Itemerging
is a major
challenges, and
international forumfuture
for trends in computer
academia, industry,and software
and technologies
government and research
to discuss applications. The theme
results of COMPSACemerging
and advancements, 2018 is
Staying Smarter
challenges, in a Smartening
and future World. and software technologies and applications. The theme of COMPSAC 2018 is
trends in computer
Staying Smarter in a Smartening World.
Computer technologies are producing profound changes in society. Emerging developments in areas such as Deep
Learning, supported
Computer technologies by are
producing powerful
profound andchanges
increasingly miniaturized
in society. Emerginghardware, are beginning
developments in areasto be deployed
such as Deep
in architectures, systems, and applications that are redefining the relationships between
Learning, supported by increasingly powerful and increasingly miniaturized hardware, are beginning to be deployed humans and technology. As
this happens, humans are relinquishing their roles as masters of technology to partnerships
in architectures, systems, and applications that are redefining the relationships between humans and technology. As wherein autonomous,
this happens, humans devices arebecome our assistants.
relinquishing What
their roles asare the technologies
masters enabling
of technology these changes?
to partnerships How farautonomous,
wherein can these
partnerships go?devices
computer-driven What will be our our
become future as we deploy
assistants. Whatmore andtechnologies
are the more “‘things”enabling
on the Internet of Things How
these changes? – to create
far can smart
partnerships go? What will be our future as we deploy more and more “‘things” on the Internet of Things – to createthese
cities, smart vehicles, smart hospitals, smart homes, smart clothes, etc.? Will humans simply become IoT devices in smart
cities, smartand if so, what
vehicles, smartwillhospitals,
be the social,
homes,and economic
smart clothes,challenges
etc.? Willarising
humans from thesebecome
simply developments? What
IoT devices are
in these
the technical challenges to making this all happen – for example, in terms of technologies such
scenarios and if so, what will be the social, cultural, and economic challenges arising from these developments? What are as Big Data, Cloud, Fog,
Edge Computing, mobile computing, and pervasive computing in general? What will be the role of the ‘user’ as the 21st
the technical challenges to making this all happen – for example, in terms of technologies such as Big Data, Cloud, Fog,
Century moves along?
Edge Computing, mobile computing, and pervasive computing in general? What will be the role of the ‘user’ as the 21st
Century moves along?
COMPSAC 2018 is organized as a tightly integrated union of symposia, each of which will focus on technical aspects
related to the ”smart” theme of the conference. The technical program will include keynote addresses, research papers,
COMPSAC 2018 is organized as a tightly integrated union of symposia, each of which will focus on technical aspects
industrial case studies, fast abstracts, a doctoral symposium, poster sessions, and workshops and tutorials on emerging
related to the ”smart”
and important topicstheme
relatedofto the
conferenceThe technical
theme. program
A highlight will conference
of the include keynote
will beaddresses,
plenary and research papers,
industrial case studies, fast abstracts, a doctoral symposium, poster sessions, and workshops
panels that will address the technical challenges facing technologists who are developing and deploying these smart and tutorials on emerging
and important
systems topics related
and applications. to the
Panels willconference
also address theme. A highlight
cultural of the
and societal conference
challenges for a will be plenary
society and specialized
whose members must
panels that will address the technical challenges facing technologists who are developing
continue to learn to live, work, and play in the environments the technologies produce. Authors are invited to submit and deploying these smart
systems and applications. Panels will also address cultural and societal challenges for a society
original, unpublished research work, as well as industrial practice reports. Simultaneous submission to other publication whose members must
continue to learn to live, work, and play in the environments the technologies produce. Authors
venues is not permitted. All submissions must adhere to IEEE Publishing Policies, and all will be vetted through the IEEE are invited to submit
original, unpublished
CrossCheck Portal. research work, as well as industrial practice reports. Simultaneous submission to other publication
venues is not permitted. All submissions must adhere to IEEE Publishing Policies, and all will be vetted through the IEEE
CrossCheck Portal. Chair: Sorel Reisman, California State University, USA
Standing Committee Important Dates
Steering Committee Chair: Sheikh Iqbal Ahamed, Marquette University, USA Workshop proposals
GeneralCommittee Chair:
Chairs: Shinichi Sorel Reisman,
Honiden California State University, USA
(NII, Japan) Due date: 15 Dates
Important October 2017
Roger U. Fujii, Fujii Systems, 2016 IEEE Computer Marquette
Committee Chair: Sheikh Iqbal Ahamed, University, USA
Society Prelident Notification: 15 November 2017
Workshop proposals
General Chairs: Due date: 15 October 2017
Program ChairsShinichi
in Chief:Honiden (NII, Japan) Main Conference
Notification: papers 2017
15 November
Roger U. Fujii, Fujii
Jiannong Cao (Hong Systems, 2016 IEEE University,
Kong Polytechnic Computer Society Prelident
Hong Kong) Due date: 15 January 2018
Stelvio Cimato
Program Chairs in(University
Chief: of MIlan, Italy) Notification: 31 March
Main Conference 2018
Yasuo Okabe
Jiannong (KyotoKong
Cao (Hong University, Japan)University, Hong Kong)
Polytechnic Due date: 15 January 2018
Sahra Sedighsarvestani (Missouri
Stelvio Cimato (University of MIlan, University
Italy) of Science & Technology, USA) Workshop papers
Notification: 31 March 2018
Yasuo Okabe
Workshop (KyotoKenichi
Chairs: University, Japan)
Yoshida (University of Tskuba, Japan) Due date: 10 April 2018
Sahra Sedighsarvestani
Ji-Jiang Yang (Tsinghua(Missouri University
University, China) of Science & Technology, USA) Notification: 1 May 2018
Workshop papers
Hong Va Leong (Hong Kong Polytechnic
Workshop Chairs: Kenichi Yoshida (University University, Hong Japan)
of Tskuba, Kong) Due date: 10 April 2018
Chung Horng Lung (Carleton University, Canada) Camera Ready1 and
Notification: MayRegistration
Ji-Jiang Yang (Tsinghua University, China) Due date: May 15, 2018
Leong (Hong Kong
Hironori Polytechnic
Washizaki (WasedaUniversity,
University,Hong Kong)
Chung Horng Lung (Carleton University, Canada) Camera Ready and Registration
Due date: May 15, 2018
Local Chair: Hironori Washizaki (Waseda University, Japan)

The IEEE Computer Society is launching INTERFACE, a new communication

tool to help members engage, collaborate and stay current on CS activities.
Use INTERFACE to learn about member accomplishments and find out how
your peers are changing the world with technology.

We’re putting our professional section and student branch chapters in

the spotlight, sharing their recent activities and giving leaders a window
into how chapters around the globe meet member expectations. Plus,
INTERFACE will keep you informed on CS activities so you never miss a
meeting, career development opportunity or important industry update.

Launching this spring. Watch your email for its debut.