10 Charts That Will Change Your

Perspective On Artificial Intelligence's

Growth

Attendees play a virtual hockey game at the Internet of Things booth on the expo floor during
the Microsoft Developers Build Conference in Seattle, Washington, U.S., on Thursday, May
11, 2017. Microsoft said it will focus investments on Azure cloud services meant for the
Internet of Things, in which multiple sensors and smaller computing devices track data that
can be analyzed by the company's cloud and artificial intelligence tools. Photographer: David
Ryder/Bloomberg

 There has been a 14X increase in the number of active AI startups since 2000.
 Investment into AI start-ups by venture capitalists has increased 6X since 2000.
 The share of jobs requiring AI skills has grown 4.5X since 2013.

These and many other fascinating insights are from Stanford University’s inaugural AI Index
(PDF, no opt-in, 101 pp.). Stanford has undertaken a One Hundred Year Study on Artificial
Intelligence (AI100) looking at the effects of AI on people’s lives, basing the inaugural report
and index on the initial findings. The study finds “that we’re essentially “flying blind” in our
conversations and decision-making related to Artificial Intelligence.” The AI Index is focused
on tracking activity and progress on AI initiatives, and to facilitate informed conversations
grounded with reliable, verifiable data. All data used to produce the AI Index and report is
available at aiindex.org. Please see the AI Index for additional details regarding the
methodology used to create each of the following graphs.

The following ten charts from the AI Index report provides insights into AI’s rapid growth:

 The number of Computer Science academic papers and studies has soared by
more than 9X since 1996. Academic studies and research are often the precursors to
new intellectual property and patents. The entire Scopus database contains over
200,000 (200,237) papers in the field of Computer Science that have been indexed
with the key term “Artificial Intelligence.” The Scopus database contains almost 5
million (4,868,421) papers in the subject area “Computer Science.”

2017 AI Index Report, http://aiindex.org/

 There have been a 6X increase in the annual investment levels by venture capital
(VC) investors into U.S.-based Ai startups since 2000. Crunchbase, VentureSource,
and Sand Hill Econometrics were used to determine the amount of funding invested
each year by venture capitalists into startups where AI plays an important role in some
key function of the business. The following graphic illustrates the amount of annual
funding by VC’s into US AI startups across all funding stages.
 2017 AI
Index Report, http://aiindex.org/

 There has been a 14X increase in the number of active AI startups since 2000.
Crunchbase, VentureSource, and Sand Hill Econometrics were also used for
completing this analysis with AI startups in Crunchbase cross-referenced to venture-
backed companies in the VentureSource database. Any venture-backed companies
from the Crunchbase list that were identified in the VentureSource database were
included.
2017 AI Index Report, http://aiindex.org/

 The share of jobs requiring AI skills has grown 4.5X since 2013., The growth of
the share of US jobs requiring AI skills on the Indeed.com platform was calculated by
first identifying AI-related jobs using titles and keywords in descriptions. Job growth
is a calculated as a multiple of the share of jobs on the Indeed platform that required
AI skills in the U.S. starting in January 2013. The study also calculated the growth of
the share of jobs requiring AI skills on the Indeed.com platform, by country. Despite
the rapid growth of the Canada and UK. AI job markets, Indeed.com reports they are
respectively still 5% and 27% of the absolute size of the US AI job market.

2017 AI Index Report, http://aiindex.org/

 Machine Learning, Deep Learning and Natural Language Processing (NLP) are
the three most in-demand skills on Monster.com. Just two years ago NLP had been
 predicted to be the most in-demand skill for application developers creating new AI
apps. In addition to skills creating AI apps, machine learning techniques, Python,
Java, C++, experience with open source development environments, Spark,
MATLAB, and Hadoop are the most in-demand skills. Based on an analysis of
Monster.com entries as of today, the median salary is $127,000 in the U.S. for Data
Scientists, Senior Data Scientists, Artificial Intelligence Consultants and Machine
Learning Managers.

2017 AI Index Report, http://aiindex.org/

 Error rates for image labeling have fallen from 28.5% to below 2.5% since 2010.
AI’s inflection point for Object Detection task of the Large Scale Visual Recognition
Challenge (LSVRC) Competition occurred in 2014. On this specific test, AI is now
more accurate than human These findings are from the competition data from the
leaderboards for each LSVRC competition hosted on the ImageNet website.
2017 AI Index Report, http://aiindex.org/

 Internationally, robot imports have risen from around 100,000 in 2000 to around
250,000 in 2015. The data displayed is the number of industrial robots imported each
year into North America and Internationally. Industrial robots are defined by the ISO
8373:2012 standard. International Data Corporation (IDC) expects robotics spending
to accelerate over the five-year forecast period, reaching $230.7B in 2021, attaining a
Compound Annual Growth Rate (CAGR) of 22.8%.
2017 AI Index Report, http://aiindex.org/

 Global revenues from AI for enterprise applications is projected to grow from

$1.62B in 2018 to $31.2B in 2025 attaining a 52.59% CAGR in the forecast
period. Image recognition and tagging, patient data processing, localization and
mapping, predictive maintenance, use of algorithms and machine learning to predict
and thwart security threats, intelligent recruitment, and HR systems are a few of the
many enterprise application use cases predicted to fuel the projected rapid growth of
AI in the enterprise. Source: Statista.
https://www.statista.com/statistics/607612/worldwide-artificial-intelligence-for-enterprise-applications/

 84% of enterprises believe investing in AI will lead to greater competitive

advantages. 75% believe that AI will open up new businesses while also providing
competitors new ways to gain access to their markets. 63% believe the pressure to
reduce costs will require the use of AI. Source: Statista.
https://www.statista.com/statistics/747775/worldwide-reasons-for-adopting-ai/

 87% of current AI adopters said they were using or considering using AI for
sales forecasting and for improving e-mail marketing. 61% of all respondents said
that they currently used or were planning to use AI for sales forecasting. The
following graphic compares adoption rates of current AI adopters versus all
respondents. Source: Statista.
https://www.statista.com/statistics/737865/worldwide-specific-ai-use-cases-2017/

Enterprise Software Strategist with expertise in analytics, cloud computing, Configure, Price
& Quote (CPQ), Customer Relationship Management (CRM), Enterprise Resource Planning
(ERP).
The 10 most important breakthroughs in
Artificial Intelligence

“Artificial Intelligence” is currently the hottest buzzword in tech. And with good reason -
after decades of research and development, the last few years have seen a number of
techniques that have previously been the preserve of science fiction slowly transform into
science fact.

Already AI techniques are a deep part of our lives: AI determines our search results,
translates our voices into meaningful instructions for computers and can even help sort our
cucumbers (more on that later). In the next few years we’ll be using AI to drive our cars,
answer our customer service enquiries and, well, countless other things.

But how did we get here? Where did this powerful new technology come from? Here’s ten of
the big milestones that led us to these exciting times.

Getting the 'Big Idea'

The concept of AI didn’t suddenly appear - it is the subject of a deep, philosophical debate
which still rages today: Can a machine truly think like a human? Can a machine be human?
One of the first people to think about this was René Descartes, way back in 1637, in a book
called Discourse on the Method. Amazingly, given at the time even an Amstrad Em@iler
would have seemed impossibly futuristic, Descartes actually summed up some off the crucial
questions and challenges technologists would have to overcome:

“If there were machines which bore a resemblance to our bodies and imitated our actions as
closely as possible for all practical purposes, we should still have two very certain means of
recognizing that they were not real men.”
A portrait of René Descartes

He goes on to explain that in his view, machines could never use words or “put together
signs” to “declare our thoughts to others”, and that even if we could conceive of such a
machine, “it is not conceivable that such a machine should produce different arrangements of
words so as to give an appropriately meaningful answer to whatever is said in its presence, as
the dullest of men can do.”

He then goes on to describe the big challenge of now: creating a generalised AI rather than
something narrowly focused - and how the limitations of current AI would expose how the
machine is definitely not a human:

“Even though some machines might do some things as well as we do them, or perhaps even
better, they would inevitably fail in others, which would reveal that they are acting not from
understanding, but only from the disposition of their organs.”

So now, thanks to Descartes, when it comes to AI, we have the challenge.

The Imitation Game

The second major philosophical benchmark came courtesy of computer science pioneer Alan
Turing. In 1950 he first described what became known as The Turing Test, and what he
referred to as “The Imitation Game” - a test for measuring when we can finally declare that
machines can be intelligent.

His test was simple: if a judge cannot differentiate between a human and a machine (say,
through a text-only interaction with both), can the machine trick the judge into thinking that
they are the one who is human?
Alan Turing, British computing pioneer

Amusingly at the time, Turing made a bold prediction about the future of computing - and he
reckoned that by the end of the 20th century, his test will have had been passed. He said:

“I believe that in about fifty years' time it will be possible to programme computers, with a
storage capacity of about [1GB], to make them play the imitation game so well that an
average interrogator will not have more than 70 percent chance of making the right
identification after five minutes of questioning. … I believe that at the end of the century the
use of words and general educated opinion will have altered so much that one will be able to
speak of machines thinking without expecting to be contradicted.”

Sadly his prediction is a little premature, as while we’re starting to see some truly impressive
AI now, back in 2000 the technology was much more primitive. But hey, at least he would
have been impressed by hard disc capacity - which averaged around 10GB at the turn of the
century.

The first Neural Network

“Neural Network” is the fancy name that scientists give to trial and error, the key concept
unpinning modern AI. Essentially, when it comes to training an AI, the best way to do it is to
have the system guess, receive feedback, and guess again - constantly shifting the
probabilities that it will get to the right answer.
An image created by a Google Neural Network.

What’s quite amazing then is that the first neural network was actually created way back in
1951. Called “SNARC” - the Stochastic Neural Analog Reinforcement Computer - it was
created by Marvin Minsky and Dean Edmonds and was not made of microchips and
transistors, but of vacuum tubes, motors and clutches.

The challenge for this machine? Helping a virtual rat solve a maze puzzle. The system would
send instructions to navigate the maze and each time the effects of its actions would be fed
back into the system - the vacuum tubes being used to store the outcomes. This meant that the
machine was able to learn and shift the probabilities - leading to a greater chance of making it
through the maze.

It’s essentially a very, very, simple version of the same process Google uses to identify
objects in photos today.

The first self-driving car

When we think of self-driving cars, we think of something like Google’s Waymo project -
but amazingly way back in 1995, Mercedes-Benz managed to drive a modified S-Class
mostly autonomously all the way from Munich to Copenhagen.
A recent Mercedes S-Class

According to AutoEvolution, the 1043 mile journey was made by stuffing effectively a
supercomputer into the boot - the car contained 60 transputer chips, which at the time were
state of the art when it came to parallel computing, meaning that it could process a lot of
driving data quickly - a crucial part of making self-driving cars sufficiently responsive.

Apparently the car reached speeds of up to 115mph, and was actually fairly similar to
autonomous cars of today, as it was able to overtake and read road signs. But if we were
offered a trip? Umm… We insist you go first.

Switching to statistics

Though neural networks had existed as a concept for some time (see above!), it wasn’t until
the late 1980s when there was a big shift amongst AI researchers from a “rules based”
approach to one instead based on statistics - or machine learning. This means that rather than
try to build systems that imitate intelligence by attempting to divine the rules by which
humans operate, instead taking a trial-and-error approach and adjusting the probabilities
based on feedback is a much better way to teach machines to think. This is a big deal - as it is
this concept that underpins the amazing things that AI can do today.

Gil Press at Forbes argues that this switch was heralded in 1988, as IBM’s TJ Watson
Research Center published a paper called “A statistical approach to language translation”,
which is specifically talking about using machine learning to do exactly what Google
Translate works today.

IBM apparently fed into their system 2.2 millions pairs of sentences in French and English to
train the system - and the sentences were all taken from transcripts of the Canadian
Parliament, which publishes its records in both languages - which sounds like a lot but is
nothing compared to Google having the entire internet at its disposal - which explains why
Google Translate is so creepily good today.
Deep Blue beats Garry Kasparov

Despite the shift in focus to statistical models, rules-based models were still in use - and in
1997 IBM were responsible for perhaps the most famous chess match of all time, as it’s Deep
Blue computer bested world chess champion Garry Kasparov - demonstrating how powerful
machines can be.

The bout was actually a rematch: in 1996 Kasparov bested Deep Blue 4-2. It was only in
1997 the machines got the upper hand, winning two out of the six games outright, and
fighting Kasparov to a draw in three more.

Deep Blue’s intelligence was, to a certain extent, illusory - IBM itself reckons that its
machine is not using Artificial Intelligence. Instead, Deep Blue uses a combination of brute
force processing - processing thousands of possible moves every second. IBM fed the system
with data on thousands of earlier games, and each time the board changed with each movie,
Deep Blue wouldn’t be learning anything new, but it would instead be looking up how
previous grandmasters reacted in the same situations. “He’s playing the ghosts of
grandmasters past,” as IBM puts it.

Whether this really counts as AI or not though, what’s clear is that it was definitely a
significant milestone, and one that drew a lot of attention not just to the computational
abilities of computers, but also to the field as a whole. Since the face-off with Kasparov,
besting human players at games has become a major, populist way of benchmarking machine
intelligence - as we saw again in 2011 when IBM’s Watson system handily trounced two of
the game show Jeopardy’s best players.

Siri nails language

Natural language processing has long been a holy grail of artificial intelligence - and crucial
if we’re ever going to have a world where humanoid robots exist, or where we can bark
orders at our devices like in Star Trek.
And this is why Siri, which was built using the aforementioned statistical methods, was so
impressive. Created by SRI International and even launched as a separate app on the iOS app
store, it was quickly acquired by Apple itself, and deeply integrated into iOS: Today it is one
of the most high profile fruits of machine learning, as it, along with equivalent products from
Google (the Assistant), Microsoft (Cortana), and of course, Amazon’s Alexa, has changed the
way we interact with our devices in a way that would have seemed impossible just a few
years earlier.

Today we take it for granted - but you only have to ask anyone who ever tried to use a voice
to text application before 2010 to appreciate just how far we’ve come.

The ImageNet Challenge

Like voice recognition, image recognition is another major challenge that AI is helping to
beat. In 2015, researchers concluded for the first time that machines - in this case, two
competing systems from Google and Microsoft - were better at identifying objects in images
than humans were, in over 1000 categories.

These “deep learning” systems were successful in beating the ImageNet Challenge - think
something like the Turing Test, but for image recognition - and they are going to be
fundamental if image recognition is ever going to scale beyond human abilities.

Applications for image recognition are, of course, numerous - but one fun example that
Google likes to boast about when promoting its TensorFlow machine learning platform is
sorting cucumbers: By using computer vision, a farmer doesn’t need to employ humans to
decide whether vegetables are ready for the dinner table - the machines can decide
automatically, having been trained on earlier data.
GPUs make AI economical

One of the big reasons AI is now such a big deal is because it is only over the last few years
that the cost of crunching so much data has become affordable.

According to Fortune it was only in the late 2000s that researchers realised that graphical
processing units (GPUs), which had been developed for 3D graphics and games, were 20-50
times better at deep learning computation than traditional CPUs. And once people realised
this, the amount of available computing power vastly increased, enabling the the cloud AI
platforms that power countless AI applications today.

So thanks, gamers. Your parents and spouses might not appreciate you spending so much
time playing videogames - but AI researchers sure do.

AlphaGo and AlphaGoZero conquer all

In March 2016, another AI milestone was reached as Google’s AlphaGo software was able to
best Lee Sedol, a top-ranked player of the boardgame Go, in an echo of Garry Kasparov’s
historic match.
What made it significant was not just that Go is an even more mathematically complex game
than Chess, but that it was trained using a combination of human and AI opponents. Google
won four out of five of the matches by reportedly using 1920 CPUs and 280 GPUs.

Perhaps even more significant is news from last year - when a later version of the software,
AlphaGo Zero. Instead of using any previous data, as AlphaGo and Deep Blue had, to learn
the game it simply played thousands of matches against itself - and after three days of
training was able to beat the version of AlphaGo which beat Lee Sedol 100 games to nil.
Who needs to teach a machine to be smart, when a machine can teach itself?

 TechRadar's AI Week is brought to you in association with Honor.

Big Bets on A.I. Open a New Frontier for
Chip Start-Ups, Too

Credit Robert Beatty

SAN FRANCISCO — For years, tech industry financiers showed little interest in start-up
companies that made computer chips.

How on earth could a start-up compete with a goliath like Intel, which made the chips that ran
more than 80 percent of the world’s personal computers? Even in the areas where Intel didn’t
dominate, like smartphones and gaming devices, there were companies like Qualcomm and
Nvidia that could squash an upstart.

But then came the tech industry’s latest big thing — artificial intelligence. A.I., it turned out,
works better with new kinds of computer chips. Suddenly, venture capitalists forgot all those
forbidding roadblocks to success for a young chip company.

Today, at least 45 start-ups are working on chips that can power tasks like speech and self-
driving cars, and at least five of them have raised more than $100 million from investors.
Venture capitalists invested more than $1.5 billion in chip start-ups last year, nearly doubling
the investments made two years ago, according to the research firm CB Insights.

The explosion is akin to the sudden proliferation of PC and hard-drive makers in the 1980s.
While these are small companies, and not all will survive, they have the power to fuel a
period of rapid technological change.
It is doubtful that any of the companies fantasize about challenging Intel head-on with their
own chip factories, which can take billions of dollars to build. (The start-ups contract with
other companies to make their chips.) But in designing chips that can provide the particular
kind of computing power needed by machines learning how to do more and more things,
these start-ups are racing toward one of two goals: Find a profitable niche or get acquired.
Fast.

“Machine learning and A.I. has reopened questions around how to build computers,” said Bill
Coughran, who helped oversee the global infrastructure at Google for several years and is
now a partner at Sequoia, the Silicon Valley venture capital firm. Sequoia has invested in
Graphcore, a British start-up that recently joined the $100 million club.

By the summer of 2016, the change was apparent. Google, Microsoft and other internet giants
were building apps that could instantly identify faces in photos and recognize commands
spoken into smartphones by using algorithms, known as neural networks, that can learn tasks
by identifying patterns in large amounts of data.

Nvidia was best known for making graphics processing units, or G.P.U.s, which were
designed to help render complex images for games and other software — and it turned out
they worked really well for neural networks, too. Nvidia sold $143 million in chips for the
massive computer data centers run by companies like Google in the year leading up to that
summer — double the year before.

Intel scrambled to catch up. It acquired Nervana, a 50-employee Silicon Valley start-up that
had started building an A.I. chip from scratch, for $400 million, according to a report from
the tech news site Recode.

After that, a second Silicon Valley start-up, Cerebras, grabbed five Nervana engineers as it,
too, designed a chip just for A.I.

By early 2018, according to a report by Forbes, Cerebras had raised more than $100 million
in funding. So had four other firms: Graphcore; another Silicon Valley outfit, Wave
Computing; and two Beijing companies, Horizon Robotics and Cambricon, which is backed
by the Chinese government.

Raising money in 2015 and early 2016 was a nightmare, said Mike Henry, chief executive at
the A.I. chip start-up Mythic. But “with the big, aquisition-hungry tech companies all
barreling toward semiconductors,” that has changed, he said.

China has shown a particular interest in developing new A.I. chips. A third Beijing chip start-
up, DeePhi, has raised $40 million, and the country’s Ministry of Science and Technology
has explicitly called for the production of Chinese chips that challenge Nvidia’s.

Because it’s a new market — and because there is such hunger for this new kind of
processing power — many believe this is one of those rare opportunities when start-ups have
a chance against entrenched giants.

The first big change will most likely come in the data center, where companies like
Graphcore and Cerebras, which has been quiet about its plans, hope to accelerate the creation
of new forms of A.I. Among the goals are bots that can carry on conversations and systems
that can automatically generate video and virtual reality.

Researchers at places like Microsoft and Google, which has built its own chip just for A.I.,
“train” neural networks by extreme trial and error, testing the algorithms across vast numbers
of chips for hours and even days on end. They often sit at their laptops, staring at graphs that
show the progress of these algorithms as they learn from data. Chip designers want to
streamline this process, packing all that trial and error into a few minutes.

Today, Nvidia’s G.P.U.s can efficiently execute all the tiny calculations that go into training
neural networks, but shuttling data between these chips is still inefficient, said Scott Gray,
who was an engineer at Nervana before joining OpenAI, an artificial intelligence lab whose
founder include Tesla’s chief executive, Elon Musk.

So in addition to building chips specifically for neural networks, start-ups are rethinking the
hardware that surrounds them.

Graphcore, for example, is building chips that include more built-in memory so that they
don’t need to send as much data back and forth. Others are looking at ways of widening the
pipes between chips so that data exchange happens faster.

“This is not just about building chips but looking at how these chips are connected together
and how they talk to the rest of the system,” Mr. Coughran, of Sequoia, said.

But this is only part of the change. Once neural networks are trained for a task, additional
gear has to execute that task. At Toyota, autonomous car prototypes are using neural
networks as a way of identifying pedestrians, signs and other objects on the road. After
training a neural network in the data center, the company runs this algorithm on chips
installed on the car.

A number of chip makers — including start-ups like Mythic, DeePhi and Horizon Robotics
— are tackling this problem as well, pushing A.I. chips into devices ranging from phones to
cars.

It is still unclear how well any of these new chips will work. Designing and building a chip
takes about 24 months, which means even the first viable hardware relying on them won’t
arrive until this year. And the chip start-ups will face competition from Nvidia, Intel, Google
and other industry giants.

But everyone is starting from about the same place: the beginning of a new market.
How an A.I. ‘Cat-and-Mouse Game’
Generates Believable Fake Photos
To create the final image in this set, the system generated 10 million revisions over 18 days.

The woman in the photo seems familiar.

She looks like Jennifer Aniston, the “Friends” actress, or Selena Gomez, the child star turned
pop singer. But not exactly.

She appears to be a celebrity, one of the beautiful people photographed outside a movie
premiere or an awards show. And yet, you cannot quite place her.

That’s because she’s not real. She was created by a machine.

The image is one of the faux celebrity photos generated by software under development at
Nvidia, the big-name computer chip maker that is investing heavily in research involving
artificial intelligence.

At a lab in Finland, a small team of Nvidia researchers recently built a system that can
analyze thousands of (real) celebrity snapshots, recognize common patterns, and create new
images that look much the same — but are still a little different. The system can also generate
realistic images of horses, buses, bicycles, plants and many other common objects.

The project is part of a vast and varied effort to build technology that can automatically
generate convincing images — or alter existing images in equally convincing ways. The hope
is that this technology can significantly accelerate and improve the creation of computer
interfaces, games, movies and other media, eventually allowing software to create realistic
imagery in moments rather than the hours — if not days — it can now take human
developers.

In recent years, thanks to a breed of algorithm that can learn tasks by analyzing vast amounts
of data, companies like Google and Facebook have built systems that can recognize faces and
common objects with an accuracy that rivals the human eye. Now, these and other
companies, alongside many of the world’s top academic A.I. labs, are using similar methods
to both recognize and create.

Nvidia's images can't match the resolution of images produced by a top-of-the-line camera,
but when viewed on even the largest smartphones, they are sharp, detailed, and, in many
cases, remarkably convincing.

For example, look at the two photos below and see if you can figure out which person is real.
Like other prominent A.I. researchers, the Nvidia team believes the techniques that drive this
project will continue to improve in the months and years to come, generating significantly
larger and more complex images.

“We think we can push this further, generating not just photos but 3-D images that can be
used in computer games and films,” said Jaakko Lehtinen, one of the researchers behind the
project.

Today, many systems generate images and sounds using a complex algorithm called a neural
network. This is a way of identifying patterns in large amounts of data. By identifying
common patterns in thousands of car photos, for instance, a neural network can learn to
identify a car. But it can also work in the other direction: It can use those patterns to generate
its own car photos.

As it built a system that generates new celebrity faces, the Nvidia team went a step further in
an effort to make them far more believable. It set up two neural networks — one that
generated the images and another that tried to determine whether those images were real or
fake. These are called generative adversarial networks, or GANs. In essence, one system does
its best to fool the other — and the other does its best not to be fooled.

“The computer learns to generate these images by playing a cat-and-mouse game against
itself,” said Mr. Lehtinen.
10 MINUTES
8 HOURS
16 HOURS
1 DAY
2 DAYS
3 DAYS
4 DAYS
5 DAYS
6 DAYS
7 DAYS
8 DAYS
9 DAYS
10 DAYS
14 DAYS
16 DAYS
18 DAYS

This series of images shows the output of Nvidia’s system over the course of 18 days of
processing. With their method, called progressive GANs, the Nvidia researchers built a
system that begins with low-resolution images and then gradually progresses to higher
resolutions. This allows the training to happen more quickly, but it also in a more
controlled and stable way. The result: 1024- by 1024-pixel images that are sharp,
detailed, and, in many cases, very convincing. Source: Nvidia

A second team of Nvidia researchers recently built a system that can automatically alter a
street photo taken on a summer’s day so that it looks like a snowy winter scene. Researchers
at the University of California, Berkeley, have designed another that learns to convert horses
into zebras and Monets into Van Goghs. DeepMind, a London-based A.I. lab owned by
Google, is exploring technology that can generate its own videos. And Adobe is fashioning
similar machine learning techniques with an eye toward pushing them into products like
Photoshop, its popular image design tool.

Trained designers and engineers have long used technology like Photoshop and other
programs to build realistic images from scratch. This is what movie effects houses do. But it
is becoming easier for machines to learn how to generate these images on their own, said
Durk Kingma, a researcher at OpenAI, the artificial intelligence lab founded by Tesla chief
executive Elon Musk and others, who specializes in this kind of machine learning.

“We now have a model that can generate faces that are more diverse and in some ways more
realistic than what we could program by hand,” he said, referring to Nvidia’s work in
Finland.

But new concerns come with the power to create this kind of imagery.

With so much attention on fake media these days, we could soon face an even wider range of
fabricated images than we do today.

“The concern is that these techniques will rise to the point where it becomes very difficult to
discern truth from falsity,” said Tim Hwang, who previously oversaw A.I. policy at Google
and is now director of the Ethics and Governance of Artificial Intelligence Fund, an effort to
fund ethical A.I. research. “You might believe that accelerates problems we already have.”
Each of these images took about 18 days for the computers to generate, before reaching
a point that the system found them believable. Source: Nvidia

The idea of generative adversarial networks was originally developed in 2014 by a researcher
named Ian Goodfellow, while he was a Ph.D. student at the University of Montreal. He
dreamed up the idea after an argument at a local bar, and built the first prototype that same
night. Now Mr. Goodfellow is a researcher at Google, and his idea is among the most
important and widely explored concepts in the rapidly accelerating world of artificial
intelligence.

Though this kind of photo generation is currently limited to still images, many researchers
believe it could expand to videos, games and virtual reality. But Mr. Kingma said this could
take years, just because it will require much larger amounts of computing power. That is the
primary problem that Nvidia is also working on, along with other chip makers.

Researchers are also using a wide range of other machine learning techniques to edit video in
more convincing — and sometimes provocative — ways.

In August, a group at the University of Washington made headlines when they built a system
that could put new words into the mouth of a Barack Obama video. Others, including
Pinscreen, a California start-up, and iFlyTek of China, are developing similar techniques
using images of President Donald Trump.

The results are not completely convincing. But the rapid progress of GANs and other
techniques point to a future where it becomes easier for anyone to generate faux images or
doctor the real thing. That is cause for real concern among experts like Mr. Hwang.

Eliot Higgins, the founder of Bellingcat, an organization that analyzes current events using
publicly available images and video, pointed out that fake images are by no means a new
problem. In the years since the rise of Photoshop, the onus has been on citizens to approach
what they view online with skepticism.

But many of us still put a certain amount of trust in photos and videos that we don’t
necessarily put in text or word of mouth. Mr. Hwang believes the technology will evolve into
a kind of A.I. arms race pitting those trying to deceive against those trying to identify the
deception.

Mr. Lehtinen downplays the effect his research will have on the spread of misinformation
online. But he does say that, as a time goes on, we may have to rethink the very nature of
imagery. “We are approaching some fundamental questions,” he said.
This Cat Sensed Death. What if Computers
Could, Too?

Credit Photo illustration by Cristiana Couceiro. Cat: Iakov Filimonov/Shutterstock.

Of the many small humiliations heaped on a young oncologist in his final year of fellowship,
perhaps this one carried the oddest bite: A 2-year-old black-and-white cat named Oscar was
apparently better than most doctors at predicting when a terminally ill patient was about to
die. The story appeared, astonishingly, in The New England Journal of Medicine in the
summer of 2007. Adopted as a kitten by the medical staff, Oscar reigned over one floor of the
Steere House nursing home in Rhode Island. When the cat would sniff the air, crane his neck
and curl up next to a man or woman, it was a sure sign of impending demise. The doctors
would call the families to come in for their last visit. Over the course of several years, the cat
had curled up next to 50 patients. Every one of them died shortly thereafter.

No one knows how the cat acquired his formidable death-sniffing skills. Perhaps Oscar’s
nose learned to detect some unique whiff of death — chemicals released by dying cells, say.
Perhaps there were other inscrutable signs. I didn’t quite believe it at first, but Oscar’s
acumen was corroborated by other physicians who witnessed the prophetic cat in action. As
the author of the article wrote: “No one dies on the third floor unless Oscar pays a visit and
stays awhile.”

The story carried a particular resonance for me that summer, for I had been treating S., a 32-
year-old plumber with esophageal cancer. He had responded well to chemotherapy and
radiation, and we had surgically resected his esophagus, leaving no detectable trace of
malignancy in his body. One afternoon, a few weeks after his treatment had been completed,
I cautiously broached the topic of end-of-life care. We were going for a cure, of course, I told
S., but there was always the small possibility of a relapse. He had a young wife and two
children, and a mother who had brought him weekly to the chemo suite. Perhaps, I suggested,
he might have a frank conversation with his family about his goals?

But S. demurred. He was regaining strength week by week. The conversation was bound to
be “a bummah,” as he put it in his distinct Boston accent. His spirits were up. The cancer was
out. Why rain on his celebration? I agreed reluctantly; it was unlikely that the cancer would
return.

When the relapse appeared, it was a full-on deluge. Two months after he left the hospital, S.
returned to see me with sprays of metastasis in his liver, his lungs and, unusually, in his
bones. The pain from these lesions was so terrifying that only the highest doses of painkilling
drugs would treat it, and S. spent the last weeks of his life in a state bordering on coma,
unable to register the presence of his family around his bed. His mother pleaded with me at
first to give him more chemo, then accused me of misleading the family about S.’s prognosis.
I held my tongue in shame: Doctors, I knew, have an abysmal track record of predicting
which of our patients are going to die. Death is our ultimate black box.

In a survey led by researchers at University College London of over 12,000 prognoses of the
life span of terminally ill patients, the hits and misses were wide-ranging. Some doctors
predicted deaths accurately. Others underestimated death by nearly three months; yet others
overestimated it by an equal magnitude. Even within oncology, there were subcultures of the
worst offenders: In one story, likely apocryphal, a leukemia doctor was found instilling
chemotherapy into the veins of a man whose I.C.U. monitor said that his heart had long since
stopped.

Photo
Credit Photo illustration by Cristiana Couceiro. Hand: Westend61/Getty Images.

But what if an algorithm could predict death? In late 2016 a graduate student named Anand
Avati at Stanford’s computer-science department, along with a small team from the medical
school, tried to “teach” an algorithm to identify patients who were very likely to die within a
defined time window. “The palliative-care team at the hospital had a challenge,” Avati told
me. “How could we find patients who are within three to 12 months of dying?” This window
was “the sweet spot of palliative care.” A lead time longer than 12 months can strain limited
resources unnecessarily, providing too much, too soon; in contrast, if death came less than
three months after the prediction, there would be no real preparatory time for dying — too
little, too late. Identifying patients in the narrow, optimal time period, Avati knew, would
allow doctors to use medical interventions more appropriately and more humanely. And if the
algorithm worked, palliative-care teams would be relieved from having to manually scour
charts, hunting for those most likely to benefit.

Avati and his team identified about 200,000 patients who could be studied. The patients had
all sorts of illnesses — cancer, neurological diseases, heart and kidney failure. The team’s
key insight was to use the hospital’s medical records as a proxy time machine. Say a man
died in January 2017. What if you scrolled time back to the “sweet spot of palliative care” —
the window between January and October 2016 when care would have been most effective?
But to find that spot for a given patient, Avati knew, you’d presumably need to collect and
analyze medical information before that window. Could you gather information about this
man during this prewindow period that would enable a doctor to predict a demise in that
three-to-12-month section of time? And what kinds of inputs might teach such an algorithm
to make predictions?

Avati drew on medical information that had already been coded by doctors in the hospital: a
patient’s diagnosis, the number of scans ordered, the number of days spent in the hospital, the
kinds of procedures done, the medical prescriptions written. The information was admittedly
limited — no questionnaires, no conversations, no sniffing of chemicals — but it was
objective, and standardized across patients.

These inputs were fed into a so-called deep neural network — a kind of software architecture
thus named because it’s thought to loosely mimic the way the brain’s neurons are organized.
The task of the algorithm was to adjust the weights and strengths of each piece of information
in order to generate a probability score that a given patient would die within three to 12
months.

The “dying algorithm,” as we might call it, digested and absorbed information from nearly
160,000 patients to train itself. Once it had ingested all the data, Avati’s team tested it on the
remaining 40,000 patients. The algorithm performed surprisingly well. The false-alarm rate
was low: Nine out of 10 patients predicted to die within three to 12 months did die within that
window. And 95 percent of patients assigned low probabilities by the program survived
longer than 12 months. (The data used by this algorithm can be vastly refined in the future.
Lab values, scan results, a doctor’s note or a patient’s own assessment can be added to the
mix, enhancing the predictive power.)

So what, exactly, did the algorithm “learn” about the process of dying? And what, in turn, can
it teach oncologists? Here is the strange rub of such a deep learning system: It learns, but it
cannot tell us why it has learned; it assigns probabilities, but it cannot easily express the
reasoning behind the assignment. Like a child who learns to ride a bicycle by trial and error
and, asked to articulate the rules that enable bicycle riding, simply shrugs her shoulders and
sails away, the algorithm looks vacantly at us when we ask, “Why?” It is, like death, another
black box.

Still, when you pry the box open to look at individual cases, you see expected and unexpected
patterns. One man assigned a score of 0.946 died within a few months, as predicted. He had
had bladder and prostate cancer, had undergone 21 scans, had been hospitalized for 60 days
— all of which had been picked up by the algorithm as signs of impending death. But a
surprising amount of weight was seemingly put on the fact that scans were made of his spine
and that a catheter had been used in his spinal cord — features that I and my colleagues might
not have recognized as predictors of dying (an M.R.I. of the spinal cord, I later realized, was
most likely signaling cancer in the nervous system — a deadly site for metastasis).

It’s hard for me to read about the “dying algorithm” without thinking about my patient S. If a
more sophisticated version of such an algorithm had been available, would I have used it in
his case? Absolutely. Might that have enabled the end-of-life conversation S. never had with
his family? Yes. But I cannot shake some inherent discomfort with the thought that an
algorithm might understand patterns of mortality better than most humans. And why, I kept
asking myself, would such a program seem so much more acceptable if it had come wrapped
in a black-and-white fur box that, rather than emitting probabilistic outputs, curled up next to
us with retracted claws?