O'Reilly Network: Steven Johnson on "Emergence"

http://www.oreillynet.com/lpt/a/1574

Published on O'Reilly Network (http://www.oreillynet.com/) See this if you're having trouble printing code examples

O'Reilly Emerging Technology Conference

Steven Johnson on "Emergence"

by David Sims and Rael Dornfest 02/22/2002 At the beginning of the Web era, Steven Johnson wrote Interface Culture, a book that found an enthusiastic reception among web folks, who appreciated its theme that wired culture is creating new metaphors for ways to see the larger world. Now Johnson, the founder and editor of one of the Web's earliest magazines, FEED, has written Emergence, which takes up the theme of bottom-up organization of complex systems. O'Reilly Network editorial director David Sims and Emerging Technology conference chair Rael Dornfest talked to Johnson about emergent theory and its relationship with the Web. Dave Sims. What is emergence? Steven Johnson. Emergence is what happens when the whole is smarter than the sum of its parts. It's what happens when you have a system of relatively simple-minded component parts -- often there are thousands or millions of them -- and they interact in relatively simple ways. And yet somehow out of all this interaction some higher level structure or intelligence appears, usually without any master planner calling the shots. These kinds of systems tend to evolve from the ground up. The book spends a lot of time with the ants as a great example of this. Colonies having this miraculous ability to pull off complex engineering feats or resource management feats without an actual leadership dictating what any ants should be doing at any time. They just follow a lot of local rules, and through those rules the intelligence of the colony comes into being. Sims. It was interesting, the sort of built-in need of observers to try to find the leader or master planner in those systems. Johnson. It seems like in many cases it's a useful strategy. The systems actually work as though they had leaders. So even if you're wrong in your assessment of them, it's not a bad guess to assume that the queen ant is in charge of the whole thing. Because they're so organized, they look like systems that have leaders. So it may not be that there's a neural mechanism to find leaders, but our brains may be skewed to look at things in top-down ways, because we grew up as social, hierarchical primates, or whatever the evolutionary psychology explanation of it is. You have to sometimes kind of push your head to think about things in an emergent way, or in a bottom-up way. Once you do it, it can be very illuminating. Sims. It reminded me in some ways of the Manhattan consulting firms who try to find fashion leaders before the trends are well identified, so they can pass that information back to their clients. It's another place where

1 of 7

5/2/2013 17:24

O'Reilly Network: Steven Johnson on "Emergence"

http://www.oreillynet.com/lpt/a/1574

people are trying to find leaders where it's not at all clear who the leaders are. Johnson. Yeah, it's one of the ways the book connects to Malcolm [Gladwell]'s book, The Tipping Point, which talked about that. In that context, leader is probably the wrong word for people who start trends, early adopters. It's not leader in the sense of a top-down broadcast role, where they have a big megaphone and they sit there and they say, "Okay, hooded sweatshirts, now! Everybody put them on!" They just are somewhere at a key point in the overall system of fashion, wherever that is, where they're connected to the right people, and what that core group decides ripples out very quickly through the whole system. So they're leaders in the sense that their ideas emanate from them, but they emanate in a much more distributed-network kind of way. Sims. You write about four stages of emergence. What are they? Johnson. I was trying to avoid the question of whether there would be sentient networks in our future. The first stage was people working on the problem without realizing they were working on the problem -people like [Alan] Turing, working on his morphogenesis paper, and to some extent, the story I tell about Engels in Manchester. Because the field hadn't really coalesced, it wasn't clear that they were working on a field that had more general applicability. The second stage is when it becomes a field in itself, and people understand that there is a connection between ants and cities, that you can study them both under this general interpretive rubric. The third stage is where people actually go out understanding the laws that run through these systems and start building things in a conscious awareness of those laws. Sim City is the first example of that, in some ways: "We understand how little self-organizing systems work, so I can create a little software program that will simulate that on the screen, and it'll be fun, and I'll do it as a deliberate work of culture, not as a model of doing it in a lab." And then the fourth stage was conceivably down the line when computer networks get to actually start having kind of a mind of their own -- about which I try to be pretty much agnostic throughout the book. Rael Dornfest. Probably a smart idea, given that the road to AI always wanders out into the wilderness. Johnson. Yeah. I wrote a long piece for The Nation this summer about AI, the Spielberg movie, using it as a launching pad to say that one of the most misleading things about all of the popular representations of AI is the idea that what will be so unsettling and uncanny will be how close it is to human intelligence. From Frankenstein up through Bladerunner, all the anxiety comes from the fact that the robot or whatever looks too much like a human, so much that you can't quite tell the difference. In fact what's much more likely to happen is that you will end up evolving the intelligence because you can engineer it. You'll wind up growing it from below rather than from above, and it'll evolve in ways you won't totally be in control of. It'll end up being intelligence that won't quite look like human intelligence. It'll have other properties in it, and it may be hard for us to pick up on the fact that it is intelligent because our criteria is different. The uncanny thing about it will be that it doesn't look like us, it looks like something radically different. In a way, all of the sci-fi films have it the wrong way around. Sims. You wrote about Danny Hillis and the genetic programs he evolved to solve a math problem.
2 of 7 5/2/2013 17:24

Steven Johnson presented the keynote, Emergence: From Real-World Cities to Online Communities at the 2002 O'Reilly Emerging Technology Conference.

O'Reilly Network: Steven Johnson on "Emergence"

http://www.oreillynet.com/lpt/a/1574

Johnson. He didn't understand them. It's one of the great stories. He looked at the code, and it may well have been that there was no actual explanation of what they did in that there was no description of what they did that was actually shorter than the actual code itself. Sims. It's still hard for me to understand how he could look at the results of a program he wrote and not understand how it works. Johnson. Yeah. It gets to the question of, you know, he didn't quite write it. He created an environment in which that program could grow. If he'd written it, he would have understood it. But he didn't. He set up the kind of bylaws that enabled it to evolve. Sims. Did you come out of this project with a definition of what emergent software is? Johnson. No, because there are different kinds. On the one hand, you have something like Sim City, which is ... the software itself was written in a top-down way. Somebody -- Wil Wright or whoever -- sat down and programmed a little game that is a complex system that has all these little interacting agents that are looking at each other and changing, based on what they see. The software itself was created in a more traditional way, but the tool that ends up being created is a little emergent, self-organizing system. Then you have what Danny Hillis did, where you're literally growing the program itself. That model is really closer to natural selection than to emergent software. And then you have this idea of emergent networks. The best way to describe that is networks that get more organized with use, that naturally structure themselves into orderly categories the more people use them. Which is sort of the opposite of networks that just get more chaotic. That's why the city example is a really important one, because cities have this great ability, if they're set up right, to organize themselves into neighborhoods as they get bigger. So they get these useful categories without anybody actually planning that; it makes it easier to navigate, better for storing and retrieving information, and all the things that cities have done so well over history. Dornfest. In the same manner, the mayor of that city would not necessarily know how some of that information crosses from place to place, just as a programmer of emergent software may not know where those connections are made. Johnson. Hopefully the mayor or the programmer can set up an environment where those connections can be made, but the actual connections are being made by the users of the system or the people in the city. Think about the way that the Amazon user-recommendation engines work. They've gotten really astonishingly good, if you have a long purchasing history with Amazon. It now has enough resolution to create these little galaxies of related books, and relationships between books, and it's gotten really refined. And it's been cultivated entirely by the purchase decisions of users. Sims. That's a situation where the cumulation of interactions benefits you, the user. But in Emergence, you point out the example of sidewalk interactions, which may or may not make life better for the individual, but certainly make things better for the larger system, the city. Johnson. I think the individual benefits, but it's a little more indirect. I was trying to make the point that the importance of the random interactions on the sidewalk, passing things on your way from X to Y, swerving into Z on your way there -- which is how the links of the city get more intelligent and more sophisticated -that that traditionally has been thought of by people who read [author and urban critic Jane] Jacobs as being an aesthetic or multicultural response to what makes cities great. You walk out on the street, and you see a lot of diversity, and that's exciting. And that's directly good for you, as a person. And that's a good point, but
3 of 7 5/2/2013 17:24

O'Reilly Network: Steven Johnson on "Emergence"

http://www.oreillynet.com/lpt/a/1574

there's another point, which is a more subtle one that I think Jacobs is trying to make, which is public space and those interactions are what enable cities to develop the neighborhoods and self-organizing clusters of like-minded people that give cities such great personality, give them their texture and flavor that are indescribable, and that make a city great. Dornfest. In New York, no matter what guidebook you use, it's often sort of hard to find what you're looking for, until you find the street that feels right. Then you can pretty much stay with it, or let it amble through the neighborhood that also has the right feel to you. Johnson. It has an incredible resolution to it. If you're on the wrong street, you're totally on the wrong street. And you go over five blocks and find, "This is exactly what I'm looking for, everything I want is right here. I'm guaranteed that the next 10 stores or five restaurants are all going to be in the general zone that I want." And I think the point is that this is ultimately good for the people in the city as well. The individual's interactions create the higher level shape of the city, which turns out to be enjoyable for the individual. Sims. You write that more is different, and that seems to be the case with a city the scope of New York. With smaller cities you don't really get those districts. Johnson. It's another way that it connects to Amazon. Early on there were a bunch of reviews of Amazon or systems like that where the reviewer wrote, "I went to Amazon and I told it I liked these two books and asked it to recommend a few more to me, and it was totally wrong." Well, there just wasn't enough data at that point. And Jacobs' refrain in her book, where she says, big cities are not like towns only bigger; they're a totally different beast. So more is different in both those cases. Sims. Slashdot seems like another example. It filled a niche, to become a sort of watering hole for the open source community. But you wrote about how, once it grew too big to fill its role, Rob Malda implemented a solution that was partly top-down, but partly community managed. Johnson. Yeah, when you get to a certain threshold of size, you start having signal-to-noise problems in an online community, particularly in an online community where you have people who have lots of opinions and ... some younger people [laughs]. And you can effectively solve that problem two ways. You can hire a lot of people to patrol the boards and delete spam and other useful information, or you can have the community do it for you. Malda and his crew didn't have the luxury of putting a bunch of people on staff to do it, and I don't think they were temperamentally inclined to do that anyway. They thought it would be better to let the community do it, and follow an open source model in developing a community itself. And so they built the karma system where everything was evaluated by other members of the community, and if you contributed a lot your karma increases. Moderation filters enable you to look at highly rated things and eliminate things that are not highly rated by the community. And it created a kind of currency within the system that enabled quality contributions to rise to the surface. It really worked, that was what was amazing about it. When you read Slashdot at the highest filter level, it's as good as many professionally edited tech sites. Dornfest. What's interesting is that Slashdot worked for me to a certain degree, and then to a certain degree it worked for me only to the extent that I was an average Slashdot reader -- which I'm not. Even if I turned off everything but the highest rating posting, I still find a lot of noise to signal. What has emerged in the weblog community is that I don't have to become an average Slashdot reader, I can say, I'm kind of like Cory, and I'm kind of like Steven, and I'm kind of like Dave Winer in a certain sort of way. I'll read their things, and they'll

4 of 7

5/2/2013 17:24

O'Reilly Network: Steven Johnson on "Emergence"

http://www.oreillynet.com/lpt/a/1574

point me to the appropriate things, including Slash articles. So you have this wonderful after-market community. And if I decide, for example, that Dave Winer's focusing too much on politics, I may stop reading his blog, but I'll still get stories from him, via somebody else. The result is that when I wake up in the morning, I get to see a lot of the stories that come through Slashdot or from the New York Times that are interesting to me, without having to wade through Slashdot to find them. Johnson. That's a great point. I know people are working on creating the meta-blogs, and I feel there's an incredible opening to create that -- the thing like Slashdot that sits on top of all the blogs, and is collectively filtered by all those bloggers and their readers. There are a lot of different versions of that, but I don't see one that's really solved the problem. To me, the thing that has to happen to the individual blogs is that they're still too centered around the personality of the blogger him- or herself. They're still too limited to emailing the blogger, or a crude bulletin board. What I would love to see is, one way or another, by force of personality or whatever, to have these clusters of 100 or 200 or maybe 1,000 people who offered real contributions and collectively owned the thing. Sims. A blog tribe. Johnson. Yeah. Dornfest. Metafilter comes closest to that of the things I've seen. It still has a personality, but it's very much a group dynamic. Sims. There are all these little pieces being figured out -- whether it's Slashdot's rating system or meta-blogs or the way file-sharing systems make the files that are most in demand easiest to get -- but can you see a point where all these little things add up to a system that wasn't planned, but fixes the problems by grouping together. Johnson. I had an interesting idea about that today, which is kind of a metaphor of where we need to go. I'm writing a new book that's entirely about the brain. Our frontal lobes differ dramatically from those of the other primates. It's disproportionately large, and one of the things that happens there is all the different specialized data processing going on through the rest of the brain gets brought there and kind of synthesized -what's going on in the visual cortex, the audio realm, the emotional realm. All that stuff is brought together. I was thinking that what the Web needs is a big neo-cortex. There are all these very specialized smart, focused tools being developed, and data that's being mined, and collective intelligence on specific problems. But we're not as good yet at, not just filtering all that stuff, but figuring out what belongs connected to what else. Google is, in a way, the beginning of that. It's letting the Web solve that pattern itself, looking at patterns and links of what should be connected to other things. But we need more of that kind of synthesis going on. I think XML is going to be a great platform for that. Once you have clear, simple markup for describing big chunks of data, it should be easier to do that as well. Sims. And it offers the potential of two-way linking. Johnson. Yeah, two-way linking is kind of essential to letting the Web evolve in an organic way. Dornfest. After reading Emergence, I went back and read Interface Culture again. Johnson. How does it hold up? Dornfest. It holds up well, actually. The thing that struck me was the talk of exaptations. I'm wondering as I look at the Web today, what are the interesting exaptations that are coming from the way people are trying to

5 of 7

5/2/2013 17:24

O'Reilly Network: Steven Johnson on "Emergence"

http://www.oreillynet.com/lpt/a/1574

extend the network, in ways that no one expected. What interesting exaptation are you seeing in the Internet today, the last six months, or year? Johnson. That's an excellent question, Rael. Do you have an answer? Dornfest. I don't yet. There was a real flurry between 1993 and 1996 in online communities, as well as how the Web was used. But I don't see, I have to admit, in the last year, much in terms of exaptations. I see in the Web services world people trying to control the evolution -- Microsoft, IBM -- trying to put more intelligence into the network. But I don't see a lot of exaptation coming from that, or coming at all. I worry that the Internet is becoming more like, it's so big we can do whatever we want with it. And the result is nothing evolving out of the Internet doing what it wants to do. Johnson. One place where I feel there's an interesting question -- and in Interface Culture I was pretty skeptical about this idea, but maybe I'm coming around to it -- is what's evolving out of the gaming community in terms of virtual worlds. Isn't Everquest supposed to have this economy that's the size of a small country? As these worlds that were designed to let people play Dungeons and Dragons in a virtual, networked environment, as they get more and more users, and as they develop an actual economy within that world, and as it translates into real-world dollars -- as people sell their characters on eBay and stuff like that -- are we actually, through that gaming design, is there an exaptation that leads to what the Internet was supposed to look like, a virtual world with commerce, a William Gibson-like system? A few years ago, people were saying that Quake was going to turn into a platform for virtual worlds, with little poetry readings in Quakespace. But I think something interesting may evolve in some of these other games. Dornfest. There were a lot of attempts to create destination spots around '94, '95, '96. What ended up happening is that Slashdot and places like that became the destination spots, but they weren't as planned. They ended up evolving. We're planning a lot today for web services, but I wonder what the next exaptations are of that. Blogging is obviously one piece of that. But it's more of a community around trading stories, rather than tapping the intelligence of the community as a whole. Johnson. Yeah, there needs to be some other thing that comes along that holds all of that information and turns it into some higher level structure that can actually make sense of it all. But there's enough innovation going along at the base level, and enough interesting people contributing to that, that I feel kind of optimistic that we're going to figure out interesting things to do with the tremendous amount of data that's being produced by all those people. David Sims was the editorial director of the O'Reilly Network. Rael Dornfest is Founder and CEO of Portland, Oregon-based Values of n. Rael leads the Values of n charge with passion, unearthly creativity, and a repertoire of puns and jokes some of which are actually good. Prior to founding Values of n, he was O'Reilly's Chief Technical Officer, program chair for the O'Reilly Emerging Technology Conference (which he continues to chair), series editor of the bestselling Hacks book series, and instigator of O'Reilly's Rough Cuts early access program. He built Meerkat, the first web-based feed aggregator, was champion and co-author of the RSS 1.0 specification, and has written and contributed to six O'Reilly books. Rael's programmatic pride and joy is the nimble, open source blogging application Blosxom, the principles of which you'll find in the Values of n philosophy and embodied in Stikkit: Little yellow notes that think.

6 of 7

5/2/2013 17:24

O'Reilly Network: Steven Johnson on "Emergence"

http://www.oreillynet.com/lpt/a/1574

Return to the O'Reilly Network.

Copyright 2009 O'Reilly Media, Inc.

7 of 7

5/2/2013 17:24