Facebook Deploys Robots to Save Blu-ray From Extinction

By Cade Metz
02.06.14 |
6:30 am |
Permalink

Facebook will start storing data on Blu-ray discs, the same discs that let you play
high-def movies on your living room TV. Image: Kyle Owen

One day, your Facebook photos will sit in the hands of robots.

Behind the scenes at Mark Zuckerbergs social networking giant, Facebook engineers
have already built these robots, and one of them was on display last week in down-
town San Jose, at a gathering of companies dedicated to exploring new technology
inside the massive data centers that underpin the internet. This robot is little more
than a mechanical arm a device that moves up and down and side to side, grab-
bing things and carrying them from place to place but it works entirely on its own,
without human intervention, and its designed to grab things youd never expect to
find in the state-of-the-art computing centers operated by a company like Facebook.
The companys social network an online service now used by over one billion peo-
ple runs on custom-built computer servers packed with hard drives, DRAM mem-
ory, and the new breed of high-speed flash storage devices. But using its robotic
arms, Facebook will store information on something very different: Blu-ray discs, the
same discs that let you watch Stanley Kubricks 2001 on your living room TV in high-
definition.

Their latest project is to move some of the older Facebook data onto Blu-ray, which
is significantly cheaper than hard disks and flash and better suited to longterm
storage

As those one billion users generate increasingly enormous amounts of data an

endless stream of status posts, likes, comments, photos, and videos the team of en-
gineers working under Facebook vice president of infrastructure engineering Jay
Parikh are always looking for ways to more efficiently store all this digital informa-
tion. With tongue in cheek, they call themselves Jays commandos, and their latest
project is to move some of the older Facebook data onto Blu-ray, which is significant-
ly cheaper than hard disks and flash and better suited to longterm storage. The ro-
botic arms will help move the discs to and from the drives where the data is written
and retrieved. Yes, this is a rather slow process compared to storing stuff on hard
disk or flash, but it will only be used with data that rarely gets accessed, like very old
photos.

Facebook calls it cold storage, and its yet another way that the giants of the web are
pushing data center design into the future, seeking more efficient ways of processing
and, yes, storing your data. It could also signal a larger movement towards the use of
robotics in the data center, but at the same time, Facebook is venturing into the past
with this rather surprising project. Just when the Blu-ray seemed to be on the way
out undercut by Netflix and other services that stream movies over the internet
another part of the internet has thrown it a lifeline.

We went to Japan to understand the industry, and we were just embraced, says
Giovanni Coglitore, the Facebook director of engineering who dreamed up this cold
storage project. I dont want to call us a savior, but there are a lot of companies who
see us as the new opportunity for optical discs.

The Man Who Saved Google From the Fire Marshal

Though rarely written about in the press even the hardware trade press Gio-
vanni Coglitore is one of the key figures in the rapid evolution of data center hard-
ware over the last decade. As the co-founder of server maker Rackable Systems, he
helped build the seminal computing centers fashioned by Google at the turn of the
millennium, and at Facebook, he helps the company not only explore an even newer
type of hardware, but share its designs with the rest of the computing world through
the Open Compute Project, a Facebook creation thats rapidly changing the world-
wide hardware market.

Even in its earliest days, Google wasnt one to buy commercial servers from big-
name hardware makers like Dell and HP. Larry Page and company ran their search
engine on machines pieced together from dirt-cheap parts and motherboards placed
onto baking trays literally, baking trays that could be quickly slotted into racks.
Some called them bread rack servers. The problem is that these baking trays were
lined with cork board, and at one point, the fire marshal told the company that hot
motherboards on cork wasnt a good idea. So, Google called in Coglitore and his part-
ner to help them build some bread-rack servers that met the fire code. We built out
the first two years of Google, he says.

At the time, Coglitores hardware business eventually renamed Rackable was a

tiny operation, but it grew into a major supplier as countless other companies fol-
lowed Googles lead into the world of low-cost custom servers, including Amazon,
Yahoo, and Hotmail, the online email provider now owned by Microsoft. The new
breed of web startup, you see, needed a cheaper type of system to run their ever
growing services, and Rackable helped deliver it. They would say: Just build this,
Gio,' he remembers. And we would do it.

Taking this trend even further, Google soon started designing its own servers and
contracting with manufacturers in Asia to build them, and as Rackable was acquired
by big-name hardware maker SGI, it moved away from the sort of streamlined sys-
tems it helped pioneer. But Coglitore is now back in the game with Facebook.
Facebooks Blu-ray rack. Image: Kyle Owen

Like Google, Facebook designs its own gear in tandem with manufacturers in Asia,
but unlike its rival, it open sources these designs in an effort to encourage collabora-
tion with other web players and drive down costs even further. Coglitore is one of
the chief minds in the middle of this effort. He helps designs not only servers, but all
sorts of other gear, including entirely new ways of plugging servers into the network.
He has this unbelievable knack for connecting seemingly unrelated dots, says
Frank Frankovsky, the Facebook hardware guru who helps oversee the Open Com-
pute Project. Im just in awe when he brings up this stuff.

The cold storage project is a prime example. As the company explored various ways
to storing older data on the cheap, Coglitore suggested optical disks. Its a counterin-
tuitive suggestion. But Blu-rays are actually more reliable and last longer than hard
disks, and theyre significantly cheaper. Yes, as anyone who bought CDs in the 80s
and 90s will tell you, they scratch easily. But if you take out the human element,
thats not an issue, and through discussions with suppliers in Japan and other parts
of Asia, Coglitore eventually found a cost-effective way of juggling Blu-rays with
robots. Were really just repackaging old technologies, he says. But I think were
looking at a whole era of evolution for optical discs.
Why Robots Are the Future of the Data Center

Coglitore likens the project to the sort of thing he would do while working in the lo-
cal hardware store in high school. When a customer walked in with a problem, he
would grab stuff from across the store and piece together a solution. In solving Face-
books cold storage problem, he paired the Blu-ray discs and drives with a company
called HIT Archive Corp. This Asian manufacturer has long offered a small robotic
arm that helps store data on CDs inside hospitals and small offices. Working with the
company, Coglitore and crew refashioned this arm for use inside a data center rack.
We just scaled a robot that had been around for decades, he says. We made the
arm longer.

The companys new cold storage racks hold about 10,000 Blu-ray discs, dividing
them among dozens small trays. With each disc holding up to 50GB of data, you can
stuff about a petabyte, aka about a million gigabytes, into a single rack. Each tray
holds a handful of discs, and the robotic arm is programmed to find the right disk,
lift it from the tray, and move to a drive where data can be written and read. Its like
those CD jukeboxes that turned up in high-end stereos and cars in the late 90s only
much bigger.

Its like those CD jukeboxes that turned up in high-end stereos and cars in the late
90s only much bigger

Facebook yet to install these racks inside its data centers, but it soon will. It has al-
ready built a separate cold storage facility beside its data center on the high desert in
Prineville, Oregon, and this where the company will first make the move to Blu-ray.
When Facebook initially discussed the facility last year, Jay Parikh indicated that the
company could use Blu-ray to store older photos. But at least initially, it will use the
discs to store rarely used data that its legally required to keep. With legal stuff, you
dont care about retrieval time, Colitore says. You just want to have it there when
someones asks for it.

Some question whether this type of cold storage will ever make sense for photos. On
Facebook, even old photos get accessed from time to time, and if theyre stored on
Blu-ray, youll have to wait several minutes for the robots to retrieve them. With to-
days web services, some of the cold storage gets activated pretty often, says Mike
Yang, the vice president and general manager at Quanta, one of the Asian manufac-
turer helping to drive the new hardware revolution inside the data center. Is it really
practical to build someone just for cold storage?
But Coglitore believes this sort of cold storage is a real possibility not only for photos,
but all sorts of older Facebook data. Facebook will use a variety of cold storage meth-
ods, and the trick will be to pair the right method with the right data. When you con-
sider just how must data Facebookers generate on a daily basis and how little of it
still gets looked it it only stands to reason that Blu-ray will play a major role. The
amount of high performance data is growing, but the amount of tepid to cold data is
growing like crazy, says Frankovsky. Thats where the real opportunity is.

But Facebooks robotic arms are also a sign of even bigger things to come. Just as
robots have begun to remake the distribution centers that drive Amazons retail em-
pire, they will surely reinvent the data center as well. Ken Patchett, who oversees
Facebooks Prineville data center, has long seen robotics as the future of the data cen-
ter, and Giovanni Coglitore, the man dreamt up the companys Blu-ray robots,
agrees. I think our friends in Mountain View have shown that this is their desire,
he says, nodding to Google, which is buying up robotics companies left and right. I
would be foolish to say this isnt inevitable. Everyone wants to increase efficiency
and automation. It would be silly not to.

Image: Kyle Owen

2 Comments |
Permalink