MARCH 2018 EBOOK

AI
THE
RACE
FOR
The return of great-power competition is spurring the quest
to develop artificial intelligence for military purposes.

The Race for AI | Page 1

 THE RACE FOR AI
The return of great-power competition is spurring the quest
to develop artificial intelligence for military purposes.

CONTENTS

FOREWORD // Patrick Tucker

p. 3

‘WOULDN’T IT BE GREAT IF WE COULD HOW NOT TO WIN AN AI ARMS RACE

SHOOT SOMEONE IN THE FACE AT // Patrick Tucker
200 KILOMETERS?’ // Patrick Tucker p. 20
p. 4

RUSSIA IS POISED TO SURPRISE THE US WHAT THE CIA'S TECH DIRECTOR WANTS
IN BATTLEFIELD ROBOTICS // Samuel Bendett FROM AI
p. 8 // Patrick Tucker
p. 24

CHINA IS ON A WHOLE-OF-NATION PUSH AS AI AND CYBER RACE AHEAD, THE STATE

FOR AI. THE US MUST MATCH IT DEPARTMENT IS FALLING BEHIND
// Elsa B. Kania // Josh Kirshner
p. 13 p. 27

THE NSA'S NEXT LIKELY CHIEF WANTS TO CHINESE SUB COMMANDERS MAY GET AI
ENLIST AI FOR CYBER OFFENSE // Patrick Tucker HELP FOR DECISION-MAKING
p. 17 // Elsa B. Kania
p. 30

The Race for AI | Page 2

Cover: Rick_Jo, istock.com
Foreword
L
ast summer, the Chinese government published
their national plan outlining government
research and investment goals for artificial
intelligence, or AI. By 2020, they declared, research
facilities and institutions should be publishing
scholarly work and producing products on par with
those of the United States. By 2030, the goal is to
“become the world’s premier artificial intelligence
innovation center.”
Two months later, Russian President Vladimir
Putin remarked, “Whoever becomes the leader in this
sphere will become the ruler of the world.”
In the United States, a growing number of private-
sector companies, from Facebook to Google to
General Motors and Tesla, have robust and growing
investments in artificial intelligence and machine
learning. The U.S. military is also expanding research-
and-development spending on what many consider
the seminal technology of the 21st century.
Artificial intelligence is sometimes treated as a
breakthrough weapon. In fact, it’s more like a set
of features. AI makes the weapon more precise. It
doesn’t kill the target; it makes it easier for the analyst
to predict what the target will be and where it will go.
In the decade ahead, AI will change everything
from the search for new medical cures to the way
we interact with cars, cities, and institutions. Within
the national security and military space, its most
conspicuous effect will be a steady increase in speed,
mikiel, istock.com
and a shrinking of the observe, orient, decide and act
cycle, sometimes called the OODA loop.
The United States, unlike some peer-nation
competitors, intends to keep a human being in
the position of ultimate decider in matters of life
and death on the battlefield. The idea of human-
machine teaming remains the guiding thrust of
the U.S. military’s approach to integrating artificial
intelligence into both weapons and in operations.
It’s an approach that privileges human judgement
and accountability above mechanical efficiency.
But as the OODA loop continues to contract, and as
peer adversaries reep the benefits of a coordinated
and ambitious pursuit of new AI capabilities, the
military will see its resolve to keep humans in control
over machines challenged again and again. In the
virtual domain, away from the physical battlefield,
commanders are already poised to unleash the full
power of artificial intelligence for defense as well
as offense. The pace of cyber operations requires
nothing less.
This ebook reveals how artificial intelligence
is already shaping the battlefield and forecasts the
practical, ethical, and philosophical conflicts ahead.
Patrick Tucker
Technology Editor
Defense One
The Race for AI | Page 3
‘Wouldn’t It Be Great If We
Could Shoot Someone in the
Face at 200 Kilometers?’
What the U.S. Air Force
wants out of artificial
intelligence — and an
exclusive look into its next
project to fuse everything.
By Patrick Tucker
matejmo, istock.com
T
he U.S. Air Force is doubling down on efforts
to fuse text, video, and virtually every
potential source of data or information
together through artificial intelligence. The goal
is to change the way every commander, airman,
and even thinking machine makes decisions.
It’s a program called Data to Decision, or D2D.
Launched last year, the program will expand
significantly in 2018, service officials say.
What is D2D aiming for? “A complete cycle of
understanding, from predicting what we expect
to encounter, prescribe what can be done to
help understand the environment, then find, fix,
track, target, engage, assess, anything, anytime,
anywhere in any domain” Mark Tapper, the Air
Force’s Champion
for Data 2 Decision,
an experimentation
campaign
underpinning
the Air
Superiority 2030
flight plan, said via
email. “Our mantra is improve every decision –
those made by airmen or by machines.”
That includes decisions about where a piece
of information or data needs to go next. “How you
aggregate all the data, take the pieces that matter
for a mission, move it where you need to move it
(perhaps for different purposes) then think about
where it needs to move and why” will be key,
Tapper wrote.
The program is grand in its ambitions to
use a wide variety of data, extending well
beyond traditional aerial surveillance footage to
potentially include, well, everything: social media
posts, live-streaming diagnostic data off of jets,
drones, and other aircraft, attainable whether
data, pilot biophysical data from soldier-worn
sensors, and more. “We will use all data available”
including “unstructured, open and national
sources,” Tapper wrote. “We are shifting thinking
to focus on the object in question and looking
for any data that may be relevant. Machines
will help us determine relevance, especially as
we aggregate in ways never before considered.
The Race for AI | Page 4
 So all the potential data cited could be part of the
U.S. AIR FORCE ILLUSTRATION/STAFF
SGT. ALEXANDRE MONTES
relevance discussion.”
How do you take all of that data and use it
to output a dynamic and credible picture of the
future? Here, too, the strategy is: diversity in
approach. The potential artificial intelligence
tool kit the Air Force is using ranges from neural
nets and deep learning approaches to less exotic
machine learning methods, useful for tasks where
the data is structured or the variables are fewer,
like chess.
It’s not the first time the military has
experimented with fusing a wide variety of data for
better decision making. A 2014 Defense Advanced
Research Projects Agency, or DARPA, program
called Insight also sought to create a fuller picture
of the battlespace by combining and crunching
data. It used multiple neural nets to populate a
larger engine that then used Bayesian statistical
methods to output predictions and probability.
D2D uses some lessons from Insight. “AI and
Machine Learning can help by creating neural
networks through disparate data sources that
would then help us better understand the potential
to use the nets for Air Force mission execution,”
said Tapper.
The Race for AI | Page 5
 That’s key, because Bayesian statistical
methods don’t provide the most accurate answer
the first time you use them. You run the formula
over and over again, stirring in new data or
information that you receive, coming closer
and closer to a prediction in which you can
have confidence.
Modeling and simulation will play a big role in
the experimentation. The reason has everything
to do with artificial intelligence in 2018, and the
challenge of accepting outputs from artificial
intelligence tools, especially those outputs based
on opaque processes. It’s one reason why public-
policy makers were historically slow to embrace
artificial intelligence, and particularly neural
networks, for decision making even as social
media companies and high-frequency traders
have raced ahead.
Neural nets and deep learning approaches
have proven effective in chaotic, unstructured
environments and problems, such as helping
self-driving cars navigate city streets, helping
algorithms identify objects in YouTube videos, or
helping computer games win the high-variable
game of Go. But the underlying processes
behind them defy easy explanation, and that’s a
big drawback.
“The biggest part of the problem of artificial
intelligence is: they build these incredibly long
algorithms with all of these gates to go through.
They push all of this machine learning and data
through it. Frankly, we are not entirely sure how
all of that works, all the time,” said Dale Ormond,
the principal director for research in the Office
of the Assistant Secretary of Defense (research
and engineering,) speaking at a recent AUVSI
conference in Maryland.
“If someone sabotages your data that you're
feeding your algorithm for learning, we’re not
entirely sure what’s going to come out of the other
end. It’s going to take a while to figure out how to
organize these things to be successful,” Ormond
said at a recent AUVSI conference in Maryland. 
That’s not a big issue when the objective is
to show the best possible ad to the right user at
the right time a particular social network. Even
when the goal is to predict the future behavior
of a stock, a hedging strategy can mitigate losses.
But when war, peace, and human lives are on the
line, the stakes are a bit higher. Commanders and
policymakers need some way to better explain
why they arrived at the decision they did, besides
“the machine made me do it.”
Gen. Paul Selva, Vice Chairman of the Joint
Chiefs of Staff, expounded on that point during
a recent sitdown with reporters. “We haven’t
cracked the nut on man-machine teaming yet.
I don’t believe anybody has. The closest we’ve
gotten is the extremely high level of information
we’ve pushed to aviators in cockpits,” Selva said.
The Race for AI | Page 6
“Wouldn’t it be great if we could shoot someone in the face at
200 kilometers? They don’t even know you are there. That’s the
kind of man-machine teaming we really want to get after.”
GEN. PAUL SELVA, Vice Chairman of the Joint Chiefs of Staff
Nevertheless, the military has reached a new
level of comfort experimenting with artificial
intelligence for decision-making.
“The components of the Data to Decision
(D2D) Program have transitioned into operational
prototypes in a variety of mission areas and
include activities in video, image, and text
analytics. The D2D technical products and lessons
learned are critical elements of the DoD roadmap
for future capability in machine learning and
artificial intelligence,” Ormond said in an email
to Defense One.
On that roadmap, Selva has high expectations.
“You’ll see some modest investments in that space.
If you look inside what we are pushing to aviators,
it’s a hell of a lot of information to absorb. So it’s
about, can you actually push the information
to the human. Can you make it make sense to
the human, to help the human make decisions
at greater speed? That last part is the hard part.
The human has to decide faster. So parsing the
data and how you decide actually matters. That’s
a technology piece that’s really going to require
some experimentation,” he said.
Selva’s office later clarified that he was
not speaking about D2D specifically but more
broadly about Defense Department investment in
artificial intelligence.
“Some would argue that is man-machine
teaming. I’ll describe it differently: you take that
same person that’s working in that cockpit — with
all that information streaming to them — and
you augment it by a factor of ten by teaming
that airplane — that can push that amount of
information to an aviator who is making target
engagement decisions (with offboard sensors that
push ten times more data to that same person)
and give them the weapons that will allow them
to prosecute those targets. Now you are talking
about more than what fighter pilots call beyond-
visual-range shots with high confidence. You’re
talking about being able to prosecute the enemy at
hundreds of kilometers away from your airplane
with very high confidence that you’re actually
engaging the targets you intend to engage, because
we don’t engage targets unless we have high
confidence,” he said.
“Wouldn’t it be great if we could shoot
someone in the face at 200 kilometers? They don’t
even know you are there. That’s the kind of man-
machine teaming we really want to get after.”
The Race for AI | Page 7
Russia Is Poised to
Surprise the US in
Battlefield Robotics
How? It's a story of leaders'
unusual agreement, a
focus on fast-and-cheap
production, and a decision particularly advanced. Certainly, it trails Western
to field lethal robots
for combat.
By Samuel Bendett
N
o one would call Russia’s government
and budgetary bureaucracy particularly
nimble, nor its defense industry
economies in such key areas as communication
equipment, microelectronics, high-tech control
systems, and other key technologies. But in certain
aspects of the field of unmanned military systems,
Russia may be inching ahead of its competition in
designing and testing a wide variety of systems
and conceptualizing their future use.
In recent years, an unusually close alignment
of its executive leadership and the Ministry
of Defense on the importance of unmanned
systems has vastly streamlined their funding,
development, and deployment. (The defense
minister has a direct line to the president, and
final military decisions are often made by a very
small circle of individuals — a far cry from the
American budgetary process. As well, the Russian
defense budget will remain largely unchanged
over the next few years, give or take a few percent,
even as other government ministries fight for
budget share.)
Russia’s swift progress in unmanned systems
suggest that the United States and its allies
should prepare for battle against foes who
can put U.S. forces at a disadvantage by inhibiting
their operational capabilities.
Air, Land, and Sea
The vast military force that Russia inherited
from the Soviet Union was generally older
and less technologically advanced than
the U.S. arsenal, but it did include a relatively
good ISR UAV: the Pchela/Shmel, which has been
used in every major conflict from the Chechen
wars in the 1990s to today’s Syrian campaign.
In the 2000s, Russia compensated for its lack
of domestic UAV manufacturing capability by
importing Forpost and Zastava UAVs from Israel.
Today, the Eleron, Orlan, and Forpost trio of UAVs
are in widespread use by Russian forces, including
The Race for AI | Page 8

Russia's International Military-Technical
Forum 2017 expo in Moscow, August 2017. /
RUSSIAN MINISTRY OF DEFENCE
in domestic and be offered in a combat version in the coming
overseas combat years. Meanwhile, two storied Russian aircraft
operations, firms have announced UAVs with combat
together with a characteristics to start entering service around
growing list of 2020: the MiG Corporation claims an entire
other models lineup of heavy combat drones, while the Sukhoi
and platforms. Design Bureau is working on Ohotnik (Hunter),
Russia’s UAVs a large combat UAVthat may resemble Northrop
are generally Grumman’s X-47. As well, the Defense Ministry
smaller, has begun talking with Russian firms about
simpler, and producing swarming UAVs with artificial
cheaper than intelligence, with an eye on similar efforts by
their American the U.S. and China.
counterparts, On the ground, the Russian military
which frees the currently has unmanned systems clearing
military to use mines in Syria, starting with the Uran-6 and later
them without adding smaller Scarab and Sphere situational
worrying awareness systems. But its vision for the future
much about includes mid-to-heavy UGVs armed with a
loss. As well, variety of weapons. This flows from Russia’s
Russia has leveraged its capabilities in electronic status as a major land power that still plans to
warfare; for example, an Orlan UAV was fight with tank and armored vehicle formations
incorporated into a Leer-3 EW platform and backed by long-range artillery and air force
surprised American and Western observers assets. For now, a range of UGVs have been
with its effectiveness at manipulating cellular built as technology testbeds, including the tank-
communications in a contested environment. sized Uran-9 and Vihr, along with the mid-range
Now Russia is working to catch up to and smaller Soratnik, Nerehta, Platforma-M,
Western militaries in other areas. In August Argo, and other models. The Nerehta, built to
2017, the wraps came off the country’s first help develop AI concepts, is to be acquired by
domestically produced medium-altitude, the Russian military in the near future, while
long-endurance UAV, the Orion, which will Soratnik is undergoing testing and evaluation.
The Race for AI | Page 9
 The emphasis on armed robots underscores
the difference between U.S. concepts of
operations, in which unmanned ground systems
largely support ISR and augment warfighters’
capabilities, while the Russian military
contemplates small to large UGVs doing the actual
fighting in the near future alongside or ahead
of the human fighting force. Russian defense
firms are debating the proper role of AI in such
systems, and the current consensus is emerging
that a human will be present in the decision-
making firing loop, at least in the near future.
American defense and policy planners
may want to begin their own push for larger
unmanned vehicles that can fight as well as
perform a variety of functions — along with
the need for the development of relevant
concepts of operations. As the American
manufacturers  contemplate the role of a
human operator, certain new. battlefield
tactics may emerge in an environment where
man and machine have to make quick and
irreversible decisions.
At sea, Russia’s defense industry is fielding
a variety of smaller unmanned and tethered
submersibles and deep-water autonomous
gliders and mini-submarines, while a variety
of deep-sea platforms capable of long-term
autonomous operations are being designed and
tested. The military envisions using such UUVs
and USVs to guard Arctic maritime approaches,
perhaps in swarms and ideally for long periods
of time. While Russian unmanned naval
systems still trail those of the United States and
its allies, the prospect of Russian autonomous,
potentially-AI-driven swarms should prompt
the U.S. Navy to re-evaluate its ability to control
the world’s waterways.
Where the U.S. Army Is Going
Compared to Russia’s push to develop and
deploy armed unmanned ground systems,
the U.S. exhibits a certain reluctance, particularly
in how such weapons could be used. Even as
the U.S. Army develops its own UGVs, the process
of fielding such machines is expected to be “slow
and cautious,” according to American defense
officials and military experts. The key policy that
guides such developments is the Robotic and
Autonomous Systems Strategy, or RAS, released
in February 2017 by the Army’s Training and
Doctrine Command. This public document
details how the Army will incorporate emerging
technologies into its force structure, along with
the benefits these advancements provide. This
effort by the Army aims at “embracing
technological advancements and harnessing the
nature of modern warfare that traverses domains
of war, described in Army parlance as multi-
domain battle. With near-term priorities ranging
from increasing situational awareness for
The Race for AI | Page 10
 Above: gorodenkoff, istock.com; dismounted forces at lower echelons, protecting
below: farakos, istock.
forces with explosive ordinance disposal robots
and lightening physical loads for dismounted
forces, the Army hopes
to get at far-term
priorities 20 years in the
future.” Such priorities
will include “increased
situational awareness
with persistent
reconnaissance from
swarming systems,
improving sustainment
with autonomous aerial
cargo delivery, the need to
roboticize combat vehicles
performing dangerous tasks
and facilitating maneuver
with advancements to
unmanned combat vehicles.” Curiously, there
was only a cursory mention of developing
armored and combat UGVs to assist or augment
the warfighter’s capabilities in battle, although
the Army’s robotics strategy proposes that such
unmanned systems should be developed in the
future. In contrast, Russians are working on such
platforms today and are putting them through
rigorous trials.
U.S. Army officials are also pursuing the
next-generation combat vehicle, also known
as the NGCV. According to the Army Maneuver
Center of Excellence, “robotics and autonomous
systems are ... going to be a part of the concept.
When you start thinking about future possibilities
and the effect that robotics might play, it could
be a game-changer.” Still, caution prevails.
The Maneuver Center
hasn’t decided whether
the NGCV would ideally
replace the Abrams tank, the
Bradley Fighting Vehicle,
both, or neither. According
to the Army Capabilities
Integration Center, “the
Army needs to take baby
steps to achieve some type of
unmanned vehicle that can
operate with ... its mounted
armored formations.”
American caution is
understandable: there are concerns about UGVs’
ability to function, survive, keep operating, and
contribute in a complex battlefield environment
— not to mention ethical concerns about arming
machines to kill.
Russia’s Approach to
Lethal Autonomy
U.S. military leaders like to highlight that
Russia doesn’t have the same internal restrictions
on lethal autonomy as does the United States.
The Race for AI | Page 11
Recent Russian moves at the United Nations
seem to support that view. But Russia’s approach
to autonomy and especially lethal AI is more
nuanced than many in the West know.
Of special interest to U.S. civilian and military
decision makers should be the range of Russian
opinion on the use of AI for military purposes.
For example, the discussions by military
experts zero in on the notion that such systems
may take unpredictable actions. One line of
argument holds that “there would be no 100%
guarantee that [an unmanned system] wouldn’t
suddenly chose to act independently of its human
operators. Therefore, (at least at this stage of
development) we cannot speak of AI introduction
into military affairs.” Moreover,
the official Russian position with respect to
recently held United Nations conference on
banning lethal autonomous weapons highlights
the "inadmissibility of loss of meaningful human
control." In Russia, the AI debate seems to be
inching toward a requirement to have a human
in the loop of all lethal decisions, as witness
a recent announcement by the director of the
Russian manufacturing company that makes
unmanned ground vehicles about preventing
robots from “'rebelling’ against operators due to
programming errors." Russian military strategists
and decision makers are also contemplating the
role of AI in the nation’s version of net-centric
warfare. Still, President Putin has said that the
country that “leads in AI will get to rule the
world,” while other Russian politicians say that
the Russian military could soon use some form
of AI to replace a human in battle. In reality,
Russian President’s statement may have served
as a significant catalyst for the country’s military-
industrial complex to invest in and begin serious
deliberation of AI in the military role.
At present, the Russians are hardly “ten
feet tall” in their application of unmanned and
autonomous systems. The U.S. still leads in many,
if not all advanced technologies and systems.
But Russia — and others — are catching up fast
in important ways. American defense and policy
establishment should re-evaluate how it
measures threats and develops and acquires new
and promising technologies, and add flexibility to
conceptualizing and formulating how American
forces may fight in the future. In particular, they
should boost efforts to improve ISR, counter-UAS,
artificial intelligence, and electronic warfare
technologies — and rethink how these might
be used in combination with cyber and kinetic
weapons to secure battlefield superiority.
The Race for AI | Page 12
China Is On a Whole-of-Nation
Push for AI. The US Must
Match It
C
Beijing is harnessing hina has made no secret of its ambitions to
government and lead the world in artificial intelligence, nor
commercial entities of the military and geopolitical advantage
in pursuit of a it hopes to gain from this rapidly advancing
once-in-a-generation technology. A closer look at Beijing’s whole-of-
technological kingmaker. nation AI strategy shows the challenge to the
By Elsa B. Kania United States — and suggests what America must
do lest it be eclipsed in this latest round of great-
power competition.
China’s vision came into focus over
the summer with the release of the
New Generation AI Development Plan,
which articulates an ambitious agenda
to “lead the world” in the field. Chinese
leaders, no longer content to copy Western
technologies, are aiming to become the world’s
“premier AI innovation center,” advancing an
“innovation-driven” strategy for civilian and
military development.
The implementation of this agenda will be
a whole-of-government endeavor involving
15 central agencies and a growing number of
local governments. Their efforts will foster the
growth of a robust AI industry and ecosystem
and pour billions into longer-term research and
development of next-generation technologies.
The plan will tap the dynamism of national
tech champions, such as Baidu, Alibaba,
Tencent, and iFlytek, that have been leading
China’s AI revolution.
Under the national strategy of “military-civil
fusion,” their breakthroughs can also be put to
military use.
It is telling that the agencies responsible
for the plan include the Central Military-Civil
Fusion Development Commission and both
the Equipment Development Department
and Science and Technology Commission of
the Central Military Commission, or CMC. The
strategic advisory committee responsible for
supporting the new plan’s implementation also
includes several PLA civilians (in uniform). And
China’s AI agenda has the very highest support, in
the country’s leader — and CMC Chairman — Xi
The Race for AI | Page 13

PLA thinkers expect AI to
reshape the character of war
itself, from today’s ‘informatized’
(信息化) ways of warfare into
‘intelligentized’ (智能化) warfare.
Jinping, who has urged his country to advance
military innovation and highlighted the strategic
significance of AI.
PLA thinkers expect AI to reshape the character
of war itself, from today’s “informatized” (信息
化) ways of warfare into “intelligentized” (智能
化) warfare, in which AI is critical. According to
Lt. Gen. Liu Guozhi, who leads the CMC Science
and Technology Commission, AI could accelerate
military transformation, reshaping military units’
programming, operational styles, equipment
systems, and models of combat power generation,
ultimately leading to a profound military revolution. At such a point, human cognition might no longer
He warns, “facing disruptive technology, [we] must
... seize the opportunity to change paradigms. If you tempo of combat. That could require that humans
don’t disrupt, you’ll be disrupted!” So the PLA is
pursuing intelligent and autonomous unmanned
systems; AI-enabled data fusion, information
processing, and intelligence analysis; war-gaming,
simulation, and training; defense, offense, and
command in information warfare; and intelligent
support to command decision-making, among
other applications. In particular, the CMC Joint Staff
Department has called for the PLA to leverage the
“tremendous potential” of AI in planning, decision
support, and operational command. This fall, in
his report at the 19th Party Congress, Xi Jinping
himself called for the acceleration of this agenda of
military “intelligentization.”
Given this high-level focus on defense and
military innovation, the PLA is funding and the
Chinese defense industry is pursuing a range of
research and development, while Chinese defense
academics start to explore and experiment with new
potential concepts, including through war-gaming.
As AI and robotics start to become more pervasive
on the future battlefield, certain PLA thinkers even
anticipate the approach of a battlefield “singularity.”
be able to keep pace with the speed of decision and
no longer remain directly “in the loop” but instead
shift to command and supervisory roles. Certainly,
current limitations in the capabilities of AI systems
may preclude higher degrees of autonomy and
The Race for AI | Page 14
 In this image made on Monday, Feb. 16,
2015 from an interactive graphic from Baidu,
the Chinese Internet portal, lines tracing the
trips of individual travelers provide a "heat
map" of the massive migration home from the
big cities. / AP PHOTO/BAIDU
automation for the time being, but there will
be missions and contexts in which they are
desirable or imperative. This is already is the case
in air and missile defense and could soon be for
cyber operations.
Although there have been predictions and
expectations that authoritarian regimes may
opt for fully automated approaches, while
neglecting the human factor, PLA thinkers have,
in fact, highlighted the importance of human-
machine collaboration and manned-unmanned
teaming. Lt. Gen. Liu has even anticipated that
human-machine hybrid intelligence will be the
highest form of future intelligence. The PLA could
also be unwilling to remove humans from key
aspects of command decision-making because
of the preference for centralized authority and
concerns over controllability. Concurrently,
the PLA could face considerable human
challenges in its introduction of such new, highly
complex systems.
At this point, the future trajectory of U.S.-
China strategic competition in AI remains
uncertain. However, it is clear that
the U.S. military must recognize the PLA’s
emergence as a true peer competitor and
reevaluate the nature of U.S.-China military
technological competition accordingly. As
The Race for AI | Page 15
 the PLA attempts to overtake, rather than
just catch up with or match, U.S. progress in
these emerging capabilities, it will be vital to
understand and take into account the PLA’s
evolving approach and
advances. For instance,
The U.S. military since the locus of innovation
must also focus has shifted to the private
sector in these emerging
on leveraging its technologies, China’s
enduring advantage implementation of military-
in the human element civil fusion in AI could
provide the PLA a structural
of military power, advantage in rapidly adapting
which will face the latest advances for
even more complex military purposes.
U.S. competitive strategy
challenges in an age and defense innovation
of automation and initiatives should be
autonomous systems. informed by a more nuanced
understanding of the PLA’s
strategic thinking on and development of military
applications of AI. The U.S. military should also
continue to explore the risks and advantages
of developing “counter-AI” capabilities. While
seeking to develop appropriate concepts of
operations for the AI revolution, the U.S. military
must also focus on leveraging its enduring
advantage in the human element of military
power. This remains vital and faces even more
complex challenges in an age of automation and
autonomous systems.
The future trajectory of U.S. defense and
military innovation will depend upon a closer
partnership with the private sector and the
pursuit of long-term strategies to increase
national competitiveness in AI, especially
boosting spending on research and education.
U.S.-China military and strategic competition
could turn on their relative success in using this
disruptive technology to increase national power
and military capabilities.
The Race for AI | Page 16
Trump’s Pick for NSA/
CyberCom Chief Wants to
Enlist AI For Cyber Offense
A look at Lt. Gen. Paul
Nakasone’s public
statements about artificial
intelligence, offense,
and defense.
By Patrick Tucker
MF3d, istock.com
T
he Army general likely to be tapped
to head U.S. Cyber Command and
the NSA has some big plans for deploying
cyber forces and using artificial intelligence in
information attacks.
Lt. Gen. Paul Nakasone, who currently
leads U.S. Army Cyber Command, is expected to
nominated in the next few months to replace Adm.
Michael Rogers, as first reported by The Cipher
Brief (and confirmed by the Washington Post and a
Pentagon source of our own). (Update: On January
30, Nakasone's nomination
was further confirmed
by Politico.)
How does Nakasone
differ from his predecessor?
The Army general’s
public comments suggest
he would use artificial
intelligence aggressively in
offensive cyber operations,
(read hacking activities.) That could mean a
lot more offensive cyber activity to achieve an
overwhelming effect and that, in turn, is a big
shift from where the Defense Department was in
2016 when it treated public talk of cyber offensive
operations with great delicacy. 
Nakasone has pointed to
the 2016 DARPA Grand Cyber Challenge as
emblematic of how artificial intelligence will
change both offensive and defensive operations,
replacing human hackers with software
that can heal its own bugs and
vulnerabilities while simultaneously
searching for and exploiting bugs in
adversary systems.
“It’s really easy to say, ‘I’m going to
get on this network, achieve presence,
have a persistent ability to go after
whatever I do in the future.’ But
what if you had a machine that did
that? That was able to rapidly find a
The Race for AI | Page 17
 vulnerability? Identify a vulnerability? Have an
implant? Put down a persistence presence? And do
that at machine speed? You’ve taken away a ton of
the work we do today,” he said.
He said in June that he would send non-
commissioned officers to grad school in
artificial intelligence.
Lt. Gen. Paul Nakasone discusses the
Background
future of AI and US Cyber Command at the
second Defense One Technology Summit in Nakasone, who helped stand up U.S. Cyber
June 2017. / KRISTOFFER TRIPPLAAR
Command under Gen. Keith Alexander, “provided
indispensable leadership to help define, organize
and build the U.S. cyber forces, providing the
blueprint to build the workforce, to improve the
training and to expand the capabilities required to
conduct full spectrum operations,” a retired DoD
senior technology official wrote Defense One in
an email.
Under Nakasone, the Army’s Cyber Mission
Force teams achieved full operational capability
a full year ahead of schedule. But just how good
are the U.S. military’s information operators?
It depends on whom you are talking to. Last
year, unnamed officials quoted by the New York
Times and Washington Post described poor
performance against ISIS.
Nakasone told a different story at
June’s Defense One Technology Summit, where
he said,“We are contesting ISIS’s use of force
in cyberspace on a daily basis.” He expanded
the message in August at the Army’s CyCon
conference: “A year ago, the common narrative
was that ISISis uncontested across virtual space.
They could tweak their ideas and share their
propaganda rapidly. Over the past nine to 12
months, we’ve taken that on and that narrative is
no longer true.”
Nakasone has spoken favorably of the idea
of embedding cyber operators at the battalion
level and conducting work outside of traditional
network defense. “We’ve empowered brigade
combat teams with elements that go after social
media” along with more traditional roles like
network defense, he said at CyCon.
Training 41 Army Cyber Mission Force teams
helped Nakasone realize something about
information warfare operators: the smartest
The Race for AI | Page 18
person in a unit is rarely the one with the
highest rank.
“It’s interesting to command an organization
where the power is ... the lieutenants, sergeants,
the young captains who have this remarkable
talent,” he said in June. “The talent, the technical
ability is at the lower level ... It will be that way
for a generation.”
Like his predecessor, he’s a big fan of
information-sharing partnerships with the
commercial tech sector to gain “early warning” of
new threats, hostile actors and vulnerabilities.
Of course, his job is also to develop holes and
exploits into commercial computer products,
which can undermine those relationships. This
was illustrated by the disagreement between
Microsoft and the intelligence community on
the WannaCry virus, discovered by the NSA and
later (after wide revelation) used by
North Korea to launch ransomware attacks
around the world.
The next head of Cyber Command will also
take charge of NSA, but not for long. The two
are to be separated, as mandated by the 2017
National Defense Authorization Act, after the
new chief believes that Cyber Command is ready.
That recommendation will go first to the Defense
Secretary and then to the President. Look for
intense questioning on the topic during the
nominee’s Senate confirmation hearings.
The Race for AI | Page 19
How Not to Win an AI
Arms Race
T
A senator's proposal to he idea of a Chinese-U.S. arms race for
give feds new powers artificial intelligence conjures up images
protect an American of an army of swarmbots defeating self-
technological advantage driving tanks on a smoldering, depopulated
has more than a hellscape. It’s an idea so captivating that Sen. John
few critics. Cornyn, R-Texas, wants to make it harder for the
By Patrick Tucker Chinese to invest in U.S. technology development,
including in companies developing artificial
intelligence, out of fear that Beijing will use small
investment positions in Silicon Valley firms to
erode U.S. national security and technological
advantage. But tech entrepreneurs, academics in
the field, and former senior officials in the White
House and Pentagon think the proposal would do
more harm than good.
Cornyn pitched his idea at a Council on
Foreign Relations event on Thursday, citing a
2016 Defense Department report that explored
how various Chinese investment activities might
affect U.S. national security. The report, produced
under former Defense Secretary Ash Carter and
sometimes referred to as “the DIUx paper,” is not
classified but has not been made available to the
public. (Defense One has submitted a Freedom of
Information Act request to obtain it.)
Cornyn said the report found that “China is
aggressively investing in future technologies that
will be foundational to future innovations across
technology that will have commercial and military
applications,” including AI in particular.
The senator has dubbed his proposed
remedy the Foreign Investment Risk Review
Modernization Act. It would allow the Committee
on Foreign Investment in the United States,
or CFIUS, to let the Treasury Department, working
with other agencies, scrutinize and possibly
unwind tech deals that involve countries on a
special Defense Department list. Cornyn said
Treasury Secretary Steven Mnuchin supports the
idea, broadly.
A former senior Pentagon official who had
also seen the report offered a rather different
view of it. Sure, the report found that “there was
significantly more investment in those areas [like
The Race for AI | Page 20

Sen. John Cornyn, R-Texas, speaks during
the general session of the Texas Republican
Convention Friday, May 13, 2016, in Dallas. /
AP/LM OTERO
artificial intelligence] than in others.” But that
fact alone “was not that strange,” the official said.
Furthermore, “the idea that CFIUS is going to
be able to look intelligently on what investments
the Chinese should be able to make in Silicon
Valley firms is probably not a great idea,” he said.
“I’m super-skeptical of CFIUS being effective
for that.”
Moreover, it is neither new nor surprising
that Chinese investors are attracted to start-ups
focusing on artificial intelligence.
“Almost every early-stage Chinese fund
coming [to Silicon Valley] is heavily, heavily
into AI,” says Rui Ma, an independent angel
investor who has helped Chinese firms invest in
U.S. tech.
Rui said Cornyn’s proposal caught many
Chinese investors by surprise. “Almost every
early-stage Chinese fund coming to [the United
States] I’ve talked to is not aware of any political
implications of their money,” she said.
Chinese investors are relatively new to Silicon
Valley and “are not typically sought after for
the largest chunks [or] leads in the best deals,
currently,” she said. Pushing them out of the start-
up market might not have a large effect on the
funding stream.
Entrepreneur Sean Gourley echoed this.
“Given that there is already a large amount of
capital available to good companies, the extra
Chinese capital hasn’t changed too much,”
he said.
The Race for AI | Page 21
 But that could change in a few years. At some
point, Cornyn’s law could hurt startups’ ability to
raise funds.
Key venture funds with a foot in both China
and the U.S.— including Danhau Capital, Seven
Seas Partners, Hemi Ventures and GSR Ventures
— all have positions in AI startups.
The question is, will thwarting
Chinese investment in American start-
ups improve U.S. national security or
secure U.S. technological dominance in artificial
intelligence in the next 20 years? A former
Senior White House official, as well as
“If you really academics who spoke to Defense One,were
wanted to highly skeptical and said that the proposal,
as they understood it, represented a
buck up fundamental misunderstanding about the
the U.S. in AI, state of AI research and its potential.
just attach
This is Not the Droid Problem
a green card You’re Looking For
to every new
Ph.D.” Artificial intelligence will have a huge
impact on the military in the next 10
FORMER SENIOR years — well beyond drones, according
PENTAGON OFFICIAL to one former senior White House official
who was instrumental in crafting the
Obama administration’s technology
policy and guidance reports. The biggest game-
changer for the military is “not going to be the
sexy weapon systems. It’s going to be logistics
and supply chain optimization. That’s where
there will be massive efficiency gains. Just like in
industry,” he said.
“If you really wanted to buck up the U.S. in AI,
just attach a green card to every new Ph.D.”
This is not to say that surrendering
competitive advantage in artificial intelligence
to the Chinese, or any other nation, does not
carry serious national security risks. But, said the
official, those risks reflect how the Chinese might
use artificial intelligence within key areas — not
in the formulas, tools, or early-stage investment
they build or acquire.
“The places I get more concerned about,” he
said, are “what happens particularly around
health care when health care combines with AI,
and what happens around cybersecurity.”
Managing those two risks, as well as others
that emerge from the applications of AI, means
better identifying relevant data sets that
would feed AI programs or machine learning
algorithms, several researchers said.
Restricting Chinese money into early-stage
startups would have “literally zero impact” on
that, according to the official.
The former Pentagon official was even more
pessimistic about stifling Chinese investment
in early-stage start-ups. “You can hurt Silicon
Valley by trying to put a very DC-centric model on
[venture capital.] I think it’s something we need to
watch out for.”
The Race for AI | Page 22
 Cornyn acknowledged that industry Another inexorable trend that legislation
representatives he had spoken to were “skeptical” dealing with startup funding can’t address:
of the measure. Chinese academic expertise in AI is growing
One prominent Silicon Valley AI researcher far faster than their foreign direct investment.
and entrepreneur who had worked closely Chinese attendance at February’s Association
with top technology companies in both the for the Advancement of Artificial
United States and China Intelligence conference,
said Cornyn’s measure “will U.S. technological considered by many the
only hurt US AI efforts by field’s premier academic
restricting foreign investments dominance in AI is, conference in the field,
and therefore bringing indeed, in danger. was up 72 percent from the
less capital and know-how But dominance previous year.
to [the] US AI community. While the U.S. had the
China’s AI efforts are already in AI takes the most attendees, China
very advanced.” form of people, was close behind. Chinese
U.S. technological expertise and students also produced the
dominance in AI is, indeed, second-highest number
in danger. But dominance capital rather than of accepted papers at the
in AI takes the form of people, specific AI programs conference — 172, just
expertise and capital rather or even companies. behind the 189 U.S. papers.
than specific AI programs or “Limiting Chinese
even companies. investment in AI startups will lead to more money
The U.S. technological dominance in AI is, staying in China. There are many American-
indeed, in danger. But dominance in AI takes the educated Chinese nationals that would take jobs
form of people, expertise and capital rather than in China if the opportunities were comparable to
specific AI programs or even companies, said those in the United States,” said Sean McGregor,
officials, entrepreneurs, and researchers. While who recently earned a Ph.D. in AI from Oregon
it’s possible to restrict the sale of a port, a piece of State University.
physical infrastructure, or nuclear material, these “They want to stay in Silicon Valley,” the
kinds of covetous measures work less well against former Pentagon official said of Chinese students
a formula, method, or intellectual breakthrough, studying artificial intelligence. “If you really
which are far more exportable. wanted to buck up the U.S. in AI, just attach a
green card to every new Ph.D.”

The Race for AI | Page 23

What the CIA’s
Tech Director
Wants from AI
S
Dawn Meyerriecks says
staying ahead of Russia
and China isn’t as hard
as getting U.S. leaders to
listen to their own artificial become the ruler of the world.” That caught the
intelligence analysis
By Patrick Tucker
Opposite: High Performance
Computing and Storage Complex II
(HRSK-II) during the official opening of
the new data center of the Lehmann
Center (LZR) of the Dresden University
of Technology in Dresden, eastern
Germany, Wednesday, May 13, 2015. /
AP/Jens Meyer
hould the U.S. fear growing Russian
progress in artificial intelligence? Last
week, Vladimir Putin  told students,
“Whoever becomes the leader in this sphere will
interest of noted AI phobe / profiteer Elon Musk
who tweeted, simply and ominously: “It begins...”
But the CIA’s head of technology
development has a different take. Dawn
Meyerriecks is less worried about rival nation
states might use AI to outflank the United
States than about getting U.S. leaders to believe
what AI is telling them. “If I want to increase [
certainty in a particular AI-aided assessment]
what goes into it? What do I need in order to make
a really good assessment on the back-end because
that tells me what sort of collection I need to raise These “experiments” include everything from
confidence to go address national leadership?” automatically tagging objects in video (so analysts
The CIA currently has 137 pilot projects can pay attention to what’s important) to better
directly related to artificial intelligence, predicting future events based on big data and
Meyerriecks, the CIA’s deputy director for correlational evidence.
science and technology, told the Intelligence “Can we back into correlations with cause and
and National Security Summit in downtown DC. effect that will allow us to be more predictive with
The Race for AI | Page 24
what’s about to go down, like the North Koreans “The computing power is fundamentally
are about to launch this or about to do this. We important for deep learning,” Mikhail Burtsev of
have that in pockets,” she said. the Moscow Institute of Physics and Technology
When asked whether she was worried told Sputnik. “The more powerful hardware
that U.S. prowess in artificial intelligence we have, the more complex neural network
was falling behind that of China and Russia, architecture we will be able to work with. The
Meyerriecks said, “I bet on the innovation and complexity of the model often allows us to make a
on the systems engineering of the United States revolutionary breakthrough in solving practical
every time. problems,” he said.
One of the The hardware
things we talk “I just want to go faster than powering the
about with In- [Russia and China] can keep up. effort was the
Q-Tel [the CIA’s 170-teraflop DGX-1
venture capital If there’s a bear in the woods, you server from NVIDIA,
arm] is, as long just have to be faster than the the California-
as we’re going slowest person." based maker of
faster than graphics processing
everybody DAWN MEYERRIECKS, CIA deputy director for science unit chips. Jeff
behind us, I and technology Herbst, NVIDIA’s
don’t want to vice president
think about of business
how we find them off [from catching up to development, seated next to Meyerriecks, seemed
the U.S. in AI]. I just want to go faster than they amused by the news. “We sell those to everyone,”
can keep up. If there’s a bear in the woods, you he said.
just have to be faster than the slowest person.” Even if U.S. industry, if not the government,
Russia’s interest in AI is both well- retains a technological lead in artificial
established and overhyped. Consider, on Monday, intelligence development, there are real contrasts
Kremlin-backed news site Sputnik ran a story in the way the U.S. and other rival nations use
about a new Russian supercomputer for deep it. Russia has shown an increased willingness
learning, part of a “conversational” AI project to mix artificial intelligence and guns and lethal
called iPavlov.   ground robots.

The Race for AI | Page 25

 The U.S. government’s approach to AI is much
more conservative. For Meyerriecks, the biggest
challenge in applying cutting-edge AI products
and techniques to intel collection and reporting is
convincing leaders in government – including the
President – to accept intel that comes, at least in
part, from a robot, she said.
“We produce a presidential daily brief.
We have to have really, really good evidence
for why we reach the conclusions that we do.
One of the things that’s a challenge for the
current AI community, one of the things I’m
positive will get addressed, is ... you can’t go to
leadership and make a recommendation based on
a process that no one understands.”

The Race for AI | Page 26

As AI and Cyber Race
Ahead, the State Department
Is Falling Behind
Technology is reshaping
the global order. America’s
diplomats need to start
thinking ahead.
By Josh Kirshnet
T
here has been considerable interest in
the direction of Secretary of State Rex
Tillerson’s ongoing “reimagining” of the
State Department, but for all the conversations
about the future of American diplomacy, what has
been underaddressed is how our oldest Cabinet
agency is preparing to deal with the new, but
foreseeable, diplomatic challenges presented by
emerging technology.
While at the Department, I joined the effort to
establish a Coordinator for Cyber Issues reporting
directly to the Secretary, because no single
bureau was able to represent all of the interests
that the U.S. has in the cyber domain, including
economic, military, intelligence, and freedom
of expression-related issues. In 2011, the United
States became the first country to assign a senior
diplomat the task of focusing on cyber issues;
allies and competitors alike have since set up
similar positions, thus moving the international
community forward – albeit in fits and starts – in
developing norms and standards. Creating an
office to focus on international cyber policy was a
clear example of senior State Department officials
anticipating where our interconnected world
was headed and allocating resources to meet the
challenge in a truly strategic, long-term manner.
Secretary Tillerson eliminated the Coordinator
for Cyber Issues position earlier this year, but
Reps. Ed Royce, R-Calif., and Eliot Engel, D-N.Y.,
the chairman and ranking member of the House
Foreign Affairs Committee, along with 15 other
representatives, have co-sponsored the Cyber
Diplomacy Act of 2017 to re-establish it and
require the Secretary of State to develop an
international strategy for cyberspace. While
it is encouraging to see a coordinated push by
Congress to ensure the U.S. can fully pursue its
national interests in the cyber domain, this move
would merely allow us to get back to the former
status quo.
Meanwhile, technological innovation in
other realms continues at a furious pace. China
is spending billions of dollars to surpass the United
States as the world leader in artificial intelligence
by 2030, with an eye towards improving its
The Race for AI | Page 27

Secretary of State Rex Tillerson waits
to deliver an address at the Wilson Center
in Washington on Nov. 28, 2017. / SAIT
SERKAN GURBUZ/AP
economy and matching our military prowess.
Russian President Vladimir Putin believes the
country with the best AI “will become the ruler of
the world,” and Moscow recently made clear that
it opposes international efforts to ban lethal
autonomous weapons.
Yet there is little evidence that the
State Department is focused on how such
developments may shape America’s place in
the world. Certainly, the department has legal
and political-military experts on preventing
the development of inhumane weapons, and
they are participating in the international
discussions about the security ramifications of
autonomous weapons. But the Department’s lack
of foresight and senior-level attention to the issue
is troubling, especially considering the absolute
sprint underway around the world to develop
artificial intelligence and other society-changing
technologies, and the priority placed upon it by
other U.S. agencies.  The U.S. government should
be using its full diplomatic toolbox to ensure
that the development of these technologies
by countries around the world is in line with
The Race for AI | Page 28
 America’s interests shape the international community’s use of
The Secretary’s ambitious and values.  cyber tools and emerging technology to best
‘reimagining’ of State There are two steps advance U.S. interests. This will require the
provides the opportunity that the U.S. government recruitment of both experienced diplomats with
should take to prepare expertise in multilateral diplomacy as well as
to make U.S. leadership for the future: First, technology experts who want to get out of the lab
in emerging technology Congress should and into the policy community.
diplomacy a hallmark of consider broadening Secretary Tillerson, like previous secretaries,
Chairman Royce’s cyber is rightly focused on immediate, urgent threats
his legacy. bill to include emerging to the United States and the international order
technology beyond the United States has painstakingly helped to
cyber by creating a build. Today, that means dealing with North
Coordinator (or Ambassador) for Emerging Korea’s nuclear program and an increasingly
Technology reporting directly to the Secretary. assertive China, instability in the Persian Gulf, the
They should also work with appropriators fight against ISIS, and a disturbingly aggressive
to ensure the Department is resourced Kremlin. These challenges are all the more reason
appropriately to exercise both interagency and that at the same time, he needs a senior official
global leadership on this issue. and expert staff to prepare for the (barely) over-
Second, the State Department should the-horizon technological innovations that stand
develop a cadre of diplomats who are experts to change our world. The Secretary’s ambitious
in the sometime byzantine ways of multilateral “reimagining” of State provides the opportunity
diplomacy, sufficiently proficient in emerging to make U.S. leadership in emerging technology
technology, and equipped with the vision to diplomacy a hallmark of his legacy.

The Race for AI | Page 29

Chinese Sub Commanders
May Get AI Help for
Decision-Making
W
But can a recent news hat can we learn from a recent news
report be taken at face report that China is seeking to develop a
value? A CNAS fellow nuclear submarine with “AI-augmented
unpacks the intersection brainpower” to give the PLA Navy an “upper hand
of Chinese tech, in battle”?
messaging, and A February 4 piece in the South China Morning
naval power. Post quotes a “senior scientist involved with the
By Elsa B. Kania programme” as saying there is a project underway
to update the computer systems on PLAN nuclear
submarines with an AI decision-support system
with “its own thoughts” that would reduce
commanding officers’ workload and mental
burden. The article describes plans for AI to take
on “thinking” functions on nuclear subs, which
could include, at a basic level, interpreting and
answering signals picked up by sonar, through the
use of convolutional neural networks.
Given the sensitivity of such a project, it
is notable that a researcher working on the
program is apparently discussing these issues
with an English-language Hong Kong-based
newspaper owned by Chinese tech giant Alibaba.
That alone suggests that powers-that-be in Beijing
intend such a story to receive attention. The
release of this information should be considered
critically – and might even be characterized as
either a deliberate, perhaps ‘deterrent’ signal
of China’s advances and/or ‘technological
propaganda’ that hypes and overstates current
research and development. Necessarily, any
analysis based on such sourcing is difficult to
confirm – and must thus be caveated heavily.
Nonetheless, there is at least a basic
consistency between the article as reported
and the apparent direction of China’s pursuit
of military applications of AI, which has emerged
as a top priority in PLA defense innovation. In
addition, certain known lines of Chinese effort
do make this piece seem plausible, including
advances in submarine development undertaken
by the China Shipbuilding Industry Corporation,
or CSIC. At a basic level, the application of
machine learning to acoustic signal processing
has been an active area of research in China for
a number of years. As such, it seems feasible,
and even unsurprising, that the PLA would look
The Race for AI | Page 30

Chief of Naval Operations (CNO) Adm.
Jonathan Greenert crosses the brow and
boards a People's Liberation Army Navy
(PLAN) Type-39B submarine for a pierside
tour on Lushun Naval Base in 2014. / U.S.
NAVY / CHIEF MASS COMMUNICATION
SPECIALIST PETER D. LAWLOR
to use machine learning to help sub crews and
their commanders interpret the scarcity and
complexity of information available in the
undersea domain. “In the past, the technology
was too distant from application, but recently
a lot of progress has been achieved,” one
researcher at the Institute of Acoustics of the
Chinese Academy of Sciences told the SCMP.
“There seems to be hope around the corner.”
As China continues to develop more
advanced nuclear-powered and nuclear-armed
submarines, the PLAN will likely remain focused
on such new concepts and capabilities for this
force. For instance, according to Wu Chongjian (吴
崇建), a chief submarine designer at CSIC, China’s
next-generation conventional submarines could
leverage quantum communications, quantum
navigation, and intelligent unmanned vehicle
technologies. Concurrently, the PLAN is also
pursuing the development and deployment
of unmanned underwater vehicles, such as
the Sea Wing (海翼), which could support
submarines engaged in military missions. In
the future, the PLAN might seek to use UUVs in
The Race for AI | Page 31
 conjunction with submarines in an attempt to
advance its anti-submarine warfare capabilities
and shift the undersea balance. In this context,
as the deep sea battlespace becomes even
more complex and contested, the use of AI to
support commanders for at least acoustic signal
processing and underwater target recognition
in the near term – and perhaps providing more
direct decision support as the technology matures
– seems to be a plausible, and perhaps quite
impactful, application.
However, the potential existence of such
ktsimage, istock.com a PLA program also raises critical questions.
The SCMP article does not specify or clarify
whether these future AI systems would be used
only on nuclear-powered SSNs or also on nuclear-
armed SSBNs, such as the Type 096 that is under
development. Rather sensationally, the Chinese
Academy of Sciences researcher quoted in the
piece goes on to say, “If the [AI] system started
to have its own way of thinking, we may have
a runaway submarine with enough nuclear
arsenals to destroy a continent.” Certainly, it
is too soon to be alarmed that the PLA might
intend to put “superintelligence” on nuclear
subs or unleash ‘killer AI with nukes’ upon
the world. However, this ambiguity raises the
question of whether and under what conditions
the PLA might decide to use AI in ISR or decision
support systems that directly support its nuclear
arsenal, whether those under the control of
the PLA Rocket Force or its future SSBN fleet. The
lack of transparency – and resulting uncertainties
– are concerning, given the potential impact
of AI on cyber, nuclear, and strategic stability.
Although there have also been concerns that
the PLA – and other authoritarian militaries
that are disinclined to trust human personnel
– may choose to take humans entirely “out of
the loop,” that does not seem especially likely
in this scenario. It is true that PLA writings and
statements on these issues do not display the
visceral negative reaction that U.S. commanders
seem to have to the notion of doing so.
The Race for AI | Page 32
 Above: gremlin, istock.com; below:
Erikona, istock.com
Certain PLA strategists have also speculated
about the potential for a “singularity” on the
future battlefield, a point at which the human
mind simply cannot keep pace with the speed and
complexity of combat, necessitating that AI agents
take on greater responsibility in command. In
this case, the unnamed researcher reportedly
emphasized, “There must be a human hand on
every critical post. This is
for safety redundancy.” For
the time being, keeping at
least a basic level of human
involvement seems to be most
practical and effective option.
However, that alone is not a
guarantee of safety.
As the PLA seeks
to use AI to improve
its C4ISR capabilities, there
is a risk that it may rely too
heavily upon or overestimate the
supposed superiority of machine
intelligence and judgment over
that of humans. Although highly
automated systems might seem, at a superficial
level, to promise to lessen the burdens upon
commanders, past experience, including with
the Patriot air and missile defense system, has
demonstrated that such complex systems can, in
fact, create greater challenges for their operators,
necessitating nuanced understanding of their
advantages and limitations, often through
specialized training. In addition, the dynamic
of “automation bias” can cause compromised
decision-making when humans start to rely too
heavily on automated systems, at the expense of
their own judgment.
Inherently, the employment of AI on
the future battlefield will create new and
unexpected operational risks.
Those likely include potential
malfunction, adversarial
interference, or unexpected
emergent behaviors. The
unnamed scientist quoted in this
piece supposedly emphasized,
“What the military cares most
about is not fancy features. What
they care most is the thing does
not screw up amid the heat of a
battle.” However, that may be
much easier said than done, which
raises questions of how and to what
extent such systems would be tested
for safety and assurance. At this
stage in its development, AI remains brittle and
very vulnerable to spoofing or manipulation.
If the PLA chooses to introduce AI systems in to
its conventional submarines, let alone nuclear,
submarines, there will thus inevitably be not only
new capabilities but also new risks. As major
militaries start to rely more upon AI systems, this
The Race for AI | Page 33
will also place a premium upon the development
of “counter-AI” capabilities to disrupt them.
And even though the veracity of
the SCMP’s account cannot be verified
at this point, it is clear that the PLA is
prioritizing pursuit of decision superiority
through AI technologies. PLA strategists have
recognized—particularly since AlphaGo’s defeat
of Lee Sedol in the spring of 2016—that AI could
confer a critical advantage through its ability to
devise tactics and strategems that even the most
talented humans cannot equal. In particular,
AlphaGo triumphed over Lee Sedol – and
later Chinese champion Ke Jie – through its
capability to anticipate all potential options and
trajectories in the game of Go, which PLA thinkers
see as at least roughly analogous to warfare,
and formulate moves that can be novel, even
superior to those humans have invented in
thousands of years of playing the game. Since
then, AlphaZero has demonstrated even more
astonishing capabilities, beating the original
AlphaGo 100 to nothing. Although the battlefield
is considerably more complex than the game,
the PLA seems to aspire to create an ‘AlphaGo
for warfare’ that might support commanders –
or perhaps someday even replace them, some
have speculated.
While the PLA’s pursuit of decision support
systems is not new, the capabilities to develop
“intelligentized” (智能化) command decision-
making capabilities may advance considerably
with today’s rapid progress in AI technologies.
Indeed, the PLA’s pursuit of “intelligentized”
command decision-making capabilities appears
to be a high-level priority, even highlighted in
an authoritative article authored by the Central
Military Commission Joint Staff Department.
Beyond submarines, the PLA also appears to
be working on the development of systems to
augment command decision-making and, at the
tactical level, for the pilots of fighter jets. The
use of AI to enhance ISR – whether of video and
imagery intelligence, as in Project Maven, or
in acoustic signal processing for submarines,
as the PLA seems to intend – will be early and
impactful applications. In the future, with the
advent of AI in “intelligentized” (智能化) warfare,
the capability to leverage AI-enabled support to
command decision-making could become critical
to achieving decision superiority and dominance,
in the deep seas and beyond.
The Race for AI | Page 34
About the Authors
PATRICK TUCKER
Patrick Tucker is technology editor for
Defense One. He’s also the author of
The Naked Future: What Happens in
a World That Anticipates Your Every
Move? (Current, 2014). Previously,
Tucker was deputy editor for The
Futurist for nine years. Tucker has
written about emerging technology in
Slate, The Sun, MIT Technology Review,
Wilson Quarterly, The American Legion
Magazine, BBC News Magazine, Utne
Reader, and elsewhere.
SAMUEL BENDETT
Samuel Bendett is an Associate Research Analyst at CNA
and a Fellow in Russia Studies at the American Foreign
Policy Council.
ELSA B. KANIA
Elsa B. Kania is an Adjunct Fellow with the Technology
and National Security Program at the Center for a
New American Security, where she focuses on Chinese
defense innovation and emerging technologies. She
is the author of “Battlefield Singularity: Artificial
Intelligence, Military Revolution, and China’s Future
Military Power.”
JOSH KIRSHNER
Josh Kirshner previously served as special assistant
to the Under Secretary of State for Arms Control and
International Security, and as a professional staff
member on the House Permanent Select Committee on
Intelligence. He currently is Senior Vice President at
Beacon Global Strategies.
The Race for AI | Page 35