Bruce Schneier

An American Idol for Crypto Geeks

By Bruce Schneier
Wired News
February 8, 2007
The U.S. National Institute of Standards and Technology is having a competition for a new cryptographic hash
function.
This matters. The phrase "one-way hash function" might sound arcane and geeky, but hash functions are the
workhorses of modern cryptography. They provide web security in SSL. They help with key management in email and voice encryption: PGP, Skype, all the others. They help make it harder to guess passwords. They're
used in virtual private networks, help provide DNS security and ensure that your automatic software updates
are legitimate. They provide all sorts of security functions in your operating system. Every time you do
something with security on the internet, a hash function is involved somewhere.
Basically, a hash function is a fingerprint function. It takes a variable-length input -- anywhere from a single
byte to a file terabytes in length -- and converts it to a fixed-length string: 20 bytes, for example.
One-way hash functions are supposed to have two properties. First, they're one-way. This means that it is
easy to take an input and compute the hash value, but it's impossible to take a hash value and recreate the
original input. By "impossible" I mean "can't be done in any reasonable amount of time."
Second, they're collision-free. This means that even though there are an infinite number of inputs for every
hash value, you're never going to find two of them. Again, "never" is defined as above. The cryptographic
reasoning behind these two properties is subtle, but any cryptographic text talks about them.
The hash function you're most likely to use routinely is SHA-1. Invented by the National Security Agency, it's
been around since 1995. Recently, though, there have been some pretty impressive cryptanalytic attacks
against the algorithm. The best attack is barely on the edge of feasibility, and not effective against all
applications of SHA-1. But there's an old saying inside the NSA: "Attacks always get better; they never get
worse." It's past time to abandon SHA-1.
There are near-term alternatives -- a related algorithm called SHA-256 is the most obvious -- but they're all
based on the family of hash functions first developed in 1992. We've learned a lot more about the topic in the
past 15 years, and can certainly do better.
Why the National Institute of Standards and Technology, or NIST, though? Because it has exactly the
experience and reputation we want. We were in the same position with encryption functions in 1997. We
needed to replace the Data Encryption Standard, but it wasn't obvious what should replace it. NIST decided
to orchestrate a worldwide competition for a new encryption algorithm. There were 15 submissions from 10
countries -- I was part of the group that submitted Twofish -- and after four years of analysis and
cryptanalysis, NIST chose the algorithm Rijndael to become the Advanced Encryption Standard (.pdf), or AES.
The AES competition was the most fun I've ever had in cryptography. Think of it as a giant cryptographic
demolition derby: A bunch of us put our best work into the ring, and then we beat on each other until there
was only one standing. It was really more academic and structured than that, but the process stimulated a lot
of research in block-cipher design and cryptanalysis. I personally learned an enormous amount about those
topics from the AES competition, and we as a community benefited immeasurably.
NIST did a great job managing the AES process, so it's the perfect choice to do the same thing with hash
functions. And it's doing just that (.pdf). Last year and the year before, NIST sponsored two workshops to
discuss the requirements for a new hash function, and last month it announced a competition to choose a
replacement for SHA-1. Submissions will be due in fall 2008, and a single standard is scheduled to be chosen
by the end of 2011.
Yes, this is a reasonable schedule. Designing a secure hash function seems harder than designing a secure
encryption algorithm, although we don't know whether this is inherently true of the mathematics or simply a
result of our imperfect knowledge. Producing a new secure hash standard is going to take a while. Luckily, we
have an interim solution in SHA-256.
Now, if you'll excuse me, the Twofish team needs to reconstitute and get to work on an Advanced Hash

Standard submission.
earlier essay: The Psychology of Security
later essay: Why Vista's DRM Is Bad For You
categories: Computer and Information Security
back to Essays and Op Eds
Schneier.com is a personal website. Opinions expressed are not necessarily those of BT.