Google Aims to Make VR Hardware Irrelevant Befo

Even Gets Going

Smartphones have sidelined digital cameras and other specialpurpose devices. Now Google thinks mobile phones will shove
virtual-reality headsets like the Oculus Rift into the shadows, too.
By Tom Simonite on November 3, 2015
10 COMMENTS

The bright black sand at Diamond Beach, Iceland, is dotted with smooth gray pebbles and glassy chunks
of ice. The crash of foamy waves fills my ears as I turn my head to look up and down the beach and see
the dark strip of sand fade into white fog as if it goes on forever. I dont feel the cold, because Im not
really there. Moments earlier, I had been standing in a cable car climbing over Onomichi, Japan.
Im in a windowless room at Googles headquarters, holding a cardboard box against my face in a way
that positions a smartphone two inches from my eyes. Google Cardboard, as both the phone holder and
the accompanying app are called, transforms the computer in your pocket into a virtual-reality headset.
Its something of a gimmick. But its also a serious attempt to create a new mass communication and
entertainment medium. Virtual reality will have an important role to play in entertainment,
communications, work, and learning, says Clay Bavor, who leads Googles virtual-reality project.
Cardboard will be the way that we make these immersive experiences available for everyone.
Mark Zuckerberg, CEO of Facebook, also thinks virtual reality will change the world. He has predicted
that it is the next major computing and communication platform, a very serious claim in Silicon Valley
business-speak. But Zuckerberg and Google are trying to make it come true in almost opposite ways. In
2013, Facebook spent $2 billion to acquire Oculus VR, a company developing a sophisticated virtualreality headset called the Rift that is held to your face with a thick elasticated strap and attaches via a
cable to a powerful PC. The Rift goes on sale early next year. The price is unknown, but a version that
until recently was available to developers cost $350, and an Oculus-approved PC to power its
forthcoming headset costs at least $950. Facebook has also collaborated with Samsung on a $99
headset called the Gear VR, which can use some Samsung smartphones as a screen, and boasted that
it will widen virtual realitys audience this way. But Samsung is unlikely to make the Gear VR compatible
with its competitors devices, and Facebooks main project is the high-end Rift experience.

In contrast, Cardboard works with iPhones and almost any phone running Googles Android software.
Google has released the design of the Cardboard phone holder for free, and other companies sell
versions for $10, or more if you want one in plastic, metal, or wood. Google estimates that it and other
companies have sold or given away more than a million Cardboard kits. This weekend the New York
Times will send out over a million more to its subscribers, to promote a virtual-reality documentary on
children displaced by war. Google has also begun sending Cardboard kits to schools with a special
version of the app that lets a teacher take a class on a 3-D tour of a coral reef or Machu Picchu.

Students in a classroom at the Bronx Latin School in New York City tried Cardboard as part of Googles Expeditions Pioneer Program for
schools.

The Rift offers a far better experience than Cardboard and has spurred tech and media companies to
invest hundreds of millions in virtual-reality startups, content, and copycat headsets from companies
such as Sony and phone manufacturer HTC. But the technological tide that made the Rift possible also
works against itin Cardboards favor. Even as the smartphone industry slashed the costs of the
displays and sensors needed to build a good virtual-reality headset, smartphones have made people less
inclined to spend money on PCs or on single-purpose gadgets such as cameras or GPS devicesand
perhaps virtual-reality technology. Theres a set of enthusiastic users, me being one of them, thatll be
willing to charge and plug in and assemble these things, and were going to have a great experience,
says Googles Bavor. And indeed, many people on his team are working on applications for high-end VR
technologies, he says. But for virtual reality to go anywhere right nowand to answer the question of
what on earth its good forit has to spread beyond that small niche, says Bavor.

Mark Zuckerberg says virtual reality is the next

major computing and communication platform.
But Facebook and Google are trying to make that
statement come true in almost opposite ways.
Google has several initiatives designed to help its lightweight take on virtual reality take off. It is working
with phone makers to ensure that future smartphones support Cardboard better. It has invented a new
kind of camera to make it easier to produce virtual-reality movies, and it is developing YouTube into a
platform to deliver them. If it works, Googles gimmick might neutralize virtual-reality headsets before
they even get properly established.
No head strap required
Cardboard originated in a sequence of events that seem divinely chosen to help Google executives
boast of their companys openness to new ideas. In the spring of 2014, David Coz, a software engineer in
Googles Paris office, traveled to company headquarters in Mountain View, California, carrying the first
prototypes of Cardboard, which hed made with a colleague, Damien Henry, in the time Google allows for
side projects. Coz demonstrated it to a few people, who liked it and showed it to others. Within hours
somebody handed Cardboard to Google cofounder and CEO Larry Page after he stepped off stage at
the companys weekly all-hands meeting. He liked it too. The next day, Bavors boss, Sundar Pichai (who
is now Googles CEO), called him in to try it. Bavor lifted the cardboard box to his face and was
transported to Paris, thanks to imagery from Googles Street View service. When Bavors mind returned
to Mountain View, Pichai told him to turn the prototype into a real product, to be launched two months
later at Googles annual conference for developers.
That June, thousands of developers, Google employees, and journalists shuffling out of the almost threehour keynote were each handed a brown cardboard packet the size of a slim paperback. It opened to
reveal a set of cardboard goggles; a few Velcro patches, two plastic lenses, a magnet, and a rubber band
were also included. Simple instructions explained how to fold the device together, download an app, and
put your phone inside to jump into virtual space. Cardboards design was updated this summer, but it
remains essentially the same. The app splits a phones display in two, and the goggles lenses project a
slightly different viewpoint to each of your eyes, so your brain perceives depth. To interact with what you
see on the screento select from a virtual menu, say, or pause a movieyou press a button on top of
the goggles. It swings a piece of metalized fabric against the phones touch screen, which registers the
tap of a fingertip.

Cardboard is easy to fold and put to use with many kinds of smartphones.

Putting your expensive glass-and-metal pocket computer into a dowdy brown box you just folded
together and raising it to your face feels a little silly. But watch someone try Cardboard for the first time,
and you will usually see a grin quickly appear beneath the plain brown goggles. There is magic in the
difference between your expectations and what it delivers, says Bavor. You already have this device in
your pocket that, when you add some cardboard, turns into a good VR viewer.
He is both right and wrong. It is somewhat surprising that your phone can function as virtual-reality
headset. But right now Cardboard is only okay, not good. For a few minutes its great, but the discomfort
and even nausea that VR can causewhen your bodys sense of how it is moving conflicts with what it
is seeingstrike more readily than they do when you use a headset like the Rift. Current smartphone
hardware cant update the virtual world in close enough synchrony with the movements of your head as
you look around, which is what causes motion sickness. The Rift works well in part because it has cut
that lag to less than 20 milliseconds, and even then, viewing some kinds of content can still leave some
users nauseated. Cardboards latency is significantly greater.
Google designed Cardboard without a head strap in recognition of that deficiencyand to make it more
social. The idea is that your arms will get tired before you overexpose yourself or your friends get antsy
for their turn. But one reason the Google VR team is now large enough to occupy its own building on the
Google campus is that Bavor thinks smartphones offer a good enough experience for some purposes
and will soon get much better. Google is talking with manufacturers about how their devices could help
fix Cardboards limitationsfor example, by upgrading their motion sensors to be more accurate and to
update their readings more frequently, like those in Samsungs Gear VR devices. Because its Android
operating system powers 1.4 billion smartphones around the world, Google has significant influence on
the companies that make them and how they perform.

Qualcomm, which leads the market in smartphone processors, has made better support for virtual reality
a selling point of its new flagship chip, the Snapdragon 820, which is due to appear in many Android
devices in 2016. Manufacturers see support for virtual reality as a priority for high-end handsets, says
Jay Wright, who until this week was a vice president of product management at Qualcomm who worked
on virtual and augmented reality. He joined the 3-D design software company PTC, which bought a
Qualcomm group working on augmented reality software. Its been proven there is a viable experience,
he says. Now its about improving the performance.

You already have this device in your pocket that,

when you add some cardboard, turns into a good
VR viewer.
Cardboard could also benefit from a Google research effort known as Project Tango, which is
developing 3-D sensors and software to help phones and tablets accurately track their position in space.
Qualcomm and competing chipmakers Infineon and Intel are all on board. Google is unlikely to get much
help from its competitor Apple, although its devices currently work with Cardboard as well as high-end
Android devices do. Nonetheless, with the progress being made, add-on devices like Samsungs Gear VR
arent necessary, according to Bavor. We prefer to tune the software and components of the
smartphone to work well as opposed to adding complexity and things you need to charge, he says.
Strategy games
When you ask virtual-reality developers to predict what we all will do with the technology and why, you
often hear predictions about how we will physically engage with imaginary worlds and the objects and
people in them. The Oculus Rift will ship with a games controller from Microsofts Xbox, and Facebook
will also sell complicated controllers that can be used to grab and manipulate virtual objects. Google is
instead trying to make virtual reality into a new kind of casual, mostly passive, entertainment like
watching TV or surfing online videosone perhaps more likely to win a mass audience and thus become
a lucrative platform for ads or premium content.
The Cardboard app offers a handful of such experiences, including a version of the Google Earth 3-D
map program where you fly over cities or mountain ranges, and a cute animated short by a former Pixar
director. Other companies can create mobile apps to display their own content through Cardboard and
will take you climbing in Yosemite, freak you out with a short horror movie, or put you on stage next to
Paul McCartneys piano. And this summer, YouTube was upgraded to host what are described as 360degree videos, which dont present your eyes with 3-D imagery but give a sense of being somewhere
else by enveloping you with a panoramic video. They can surround you with frenetic dancers in a music
video, or take you to snowy Svalbard to watch a solar eclipse. Soon, much more impressive 3-D videos
like the one that transported me to Iceland will appear on YouTube. For this purpose, Google invented a
camera called Jump; its built from a ring of 16 small cameras positioned so that every point is captured
from three different perspectives, allowing software to merge them into a 3-D view.

Jumps basic design has been released for free, and GoPro will begin selling a version of the camera
later this year. Google will offer an assembler service that converts raw Jump footage into 3-D
experiences ready for viewing or editing at no charge. Bavor predicts that the cameras will become
standard for film crews large and small; indeed, major media companies such as ABC and the Discovery
Channel are already producing content for this nascent virtual-reality platform. Mercedes and Volvo have
made Cardboard apps for promotional purposes. Anyone with a smartphone and a Cardboard will be
able to see the most beautiful places on earth and participate in important events and moments in
history, Bavor says.

The students in New York viewed Aztec ruins through Cardboard.

Google is taking on a chicken-and-egg problem common to any company trying to establish itself as the
middleman in a new marketplace. Consumers wont buy into virtual reality unless theres something to
experience; content companies wont provide it without a ready audience. The only way to get around it
is to subsidize one side of the market, says Michael Cusumano, a management professor at MIT who
has studied how software and consumer electronics companies conjure up new businesses. Give away
3-D viewers, or pay software developers to create applications or other content. Facebook is trying to
fill the content gap by hosting 360-degree video as YouTube does, partnering with content companies
such as Netflix, and building an in-house production effort called Oculus Story Studio. Google is
effectively subsidizing both sides of the market. By making handsets more or less for free out of devices
we already own, its offering its content partners a potentially much larger audience.
Philip Rosedale, founder of the virtual world Second Life and now the CEO of High Fidelity, a company
that has raised $16 million to develop ways for people to socialize in virtual-reality environments, predicts
that both Googles and Facebooks projects will initially disappoint people who expect it to become
popular. Next year theres going to be a big push to make VR a success, and its not going to succeed,
he says. Rosedale thinks it will take a few years for Oculus to reach beyond a niche gaming audience, or

for smartphones to be capable of providing a widely acceptable experience. Referring to Bavors group,
he says: I have a lot of respect for that team. But I dont think that doing VR in software on top of a
mobile device can be compelling today.
Bavor acknowledges that Google and other companies investing in virtual reality have much to learn, but
he says the only way to do it is to get the technology to a significant audience. And the device for doing
that is alreadyperhaps all too oftenin our hands and in front of our eyes. The technologys here, he
says. Weve seen the proof that you can create wonderful VR experiences with todays smartphone
hardware; the sensors and the hardware and the content are only going to get better.

Credit: Illustration by Sophia Foster-Dimino, photos courtesy of Google

Tagged: Computing, Google, virtual reality
Reprints and Permissions | Send feedback to the editor

MIT Technology Review

2015

v1.13.05.10