hyperpeople :: the world is my hard drive

hyperpeople :: the world is my hard drive :: the thrill of it all

Over the 1999 - 2000 academic year I received an appointment as Faculty Master of Annenberg House at the University of Southern California. The building was named after Walter Annenberg, the founder and publisher of TV Guide magazine, who had already become a major American philanthropist before he sold his publishing empire to Rupert Murdoch in 1998, for three billion dollars. Two schools in particular - the University of Pennsylvania and the University of Southern California - were showered with Annenberg’s largesse. Each institution received over a hundred million dollars to develop educational programs in media studies. At USC, part of this money went to redesign and refurbish a decaying dormitory, renamed in Annenberg’s honor. To gain a berth in the House, undergraduates had to be concentrating in one of three majors: engineering, communication or media studies. Although many of the communication and media studies students found Annenberg House unappealing, most of the engineering students loved it, because the entire residence had just been wired for very high-speed Internet access. In every suite within Annenberg House, you could plug a computer in and be connected to the Internet at 100 megabits per second - faster, by far, than today’s fastest broadband connections. As Faculty Master, my responsibilities were minimal - I just had to keep an open door to the students (who only rarely came knocking) - but I had a 2000 square-foot apartment, newly and fully furnished, with a study that can only be described as wondrous: enough bookshelf space for all of my thousands of books, comfortable, high-backed armchairs for private conferences with my students, and a network outlet, giving me access to that high-speed Internet service.

USC, along with a few other schools, can rightly claim to be the “birthplace of the Internet,” and even now, some thirty years later, USC has a strong relationship with the various organizations which manage its infrastructure. Many of the engineering students at USC go on to careers as network architects and engineers. All of these engineers-in-training are technically adept, and compete for “geek chic” by being the first to experiment with new technologies. Shortly after I moved into my residence, I had a late-night conversation with two of these engineering students; they were interested in my own engineering projects, so I told them about my early work on the World Wide Web; eventually the conversation turned, and I asked them what they were using - what had they found interesting? The answer was one word: “Napster.”

“Napster?” I asked. “What’s that?”

“Just go to Napster.com. You’ll see.”

I trotted back to my suite, launched my web browser, typed in the URL, and downloaded the software offered up free on Napster.com. When I installed the Napster software on my computer, it asked me to designate an area of my computer’s hard drive where it could store files, and another area, which would be used to share files. With that, I was up and running. From inside the Napster program I could type any text I wanted - such as the name of a band, the name of an album, or the name of a song - and I’d get a list of “matches,” files whose name which matched the words I had typed in. These files resided on other computers, all across the Internet - I didn’t know where, exactly, only that they were out there, somewhere. If I double-clicked on any of these matches, that file would begin to download to my computer. Some files downloaded slowly - probably because the computer on the other end was connected to the Internet through a dial-up connection. Other files downloaded almost instantaneously, because, between their high-speed broadband connections and my ultra-high-speed 100 Mbps connection, the bits could move very fast indeed.

Written over the first half of 1999 by Northeastern University engineering student Shawn Fanning (whose curly hair tagged him with a nickname which would become the name of his software), in an attempt to improve upon the barely workable techniques of Internet file-sharing, Napster was composed of two separate pieces: the “client” software (running on my own PC), and a “server”, which tracked all of songs available through every computer running Napster. By the time I installed Napster on my computer, in October 1999, there were tens of thousands of computers running the client software; the Napster server was running out of Napster’s new corporate offices in Silicon Valley. (Fanning’s uncle, recognizing the value of his nephew’s intellectual efforts, helped him to incorporate and funded the first stages of the company’s development.) Every time anyone launched the Napster software, it would scan the portion of their computer’s hard drive which had been designated as “shared” space, make up a list of the available files, sending that along to the Napster server, which added this information to its master list of available files. Then, any time anyone searched for a song, the Napster server would look through that master list, find the matches, and send them back. Once you had the list of matches - these came back with pointers to the computers they resided on - you could download the file directly from the other computer. The Napster server was a clearinghouse; it didn’t store the files themselves, only a list of the available files, and pointers to the computers they could be downloaded from.

In the years just before Napster’s release, PCs had grown into powerful machines, capable of playing a wide array of digital media. By 1999 nearly all PCs came with a drive which could read CD-ROMs and play audio CDs; audio CDs, introduced in 1983, provided nearly noiseless, digital high-fidelity. CDs proved so popular that the recording industry reaped windfall profits in the late 1980s and early 1990s as people converted their collections of vinyl albums to the smaller and more durable CD. The CD was a very high-technology item at its introduction; twenty years later it seemed almost antique. An audio CD can hold a fair amount of data - about 700 million bytes (700 megabytes), which translates into 72 minutes of music. (The storage capacity of a CD, determined by a Sony researchers, was set to the average length of a recording of Beethoven’s 9^th Symphony.) An audio CD holds all of digital information in a “raw” format - the bits on the CD are a perfect copy of the ones which have been mastered in the recording studio.

In the 20 years since the introduction of the CD a number mathematical techniques have been developed to “compress” raw music data into much smaller files. The most popular of these compression techniques is MP3 (MPEG-1, Layer 3, for Motion Picture Experts Group, version 1, Layer 3 Audio). MP3 recording uses a mathematical technique known as Fourier Transforms to break an audio signal into its constituent sound waves. It’s like a chord played on a guitar: you can think of a chord as a set of individual strings being played simultaneously. In the same way, MP3 takes digitized music and breaks it into the simple tones from which it is composed. That process of analysis and simplification makes MP3 an efficient way to store music digitally. Back in the early 1980s, computers weren’t powerful enough handle a conversion from MP3 format back into an audio signal. By 1999, they were more than fast enough. You could pop an audio CD into your computer and “rip” it - translating each of its hefty digital “tracks” into much smaller MP3 files. Using MP3 you could easily convert a 700 MB CD into a 35 MB MP3 file. That 20:1 savings in file size made it possible to transmit songs across the Internet in a reasonable amount of time, even over slow dial-up connections. So, by 1999 there were a lot of MP3 files on Internet-connected computers, but there was no way to find where these files were. Until Napster. Suddenly I had access to a global database of all of the MP3 files of every Napster user. When I first logged into Napster, the total count of available MP3 files stood at over half a million.

It is difficult to describe the feeling of joy that accompanied my discovery of Napster. There are certain technologies - such as the Web itself, or Google, or Friendster (which I will discuss in the second section of this book) - which are just so alluring, so seductive, they suck you right in. I know that when I “got it,” when I understood what Napster brought to me, I started vibrating with ecstasy. I had a lot of music in my personal collection - at least 300 CDs, collected over the previous 15 years - but even so, there were a lot of songs I wanted, but I didn’t have. If they were available on Napster, they soon ended up on my hard drive. Because I had a very high-speed connection to the Internet, many of these songs took no time at all to transfer. Within a week’s time I had 500 new songs on my hard drive - songs that I’d always wanted to own, songs that I’d always loved. I offered up these songs to be shared, and, through Napster, people downloaded these songs from my hard drive. Share and share alike. I even ripped several albums from my personal CD collection, putting them in the shared area of my hard drive, precisely because I couldn’t find them anywhere on Napster. It wasn’t enough to share what was already out there - it was important to add to the overall selection. Napster excelled at popular music - if a song was popular you’d find countless versions of it available for download. But if something was more obscure you might find only a single copy of it - or it might not be out there at all. This popularity contest is a natural feature of file-sharing networks; as something becomes more popular it becomes more available from more sources.

Although I was a relatively early user of Napster, much of the rest of the world soon caught on, beginning with America’s university students. By the end of 1999, many American universities had, like USC, provided their students with high-speed connections to the Internet, seeing this as essential for learning. Napster spread like wildfire across college campuses, and the number of available MP3 files grew, smashing through one million, then two, then four and eight million. By the time America’s college students had returned from their Winter recess, you could search through a list of ten million MP3 files. While much of this music was simply duplicates of the same few hundred hit songs (Eminem, Metallica, Pink Floyd, etc.), as the collection grew it also grew in diversity. Friends reported that they found recordings of songs they’d sought for years. One friend built up a collection of Tin Pan Alley recordings, another a collection of French-Moroccan “rai” music. As more people used Napster, the collection grew; as it offered something for everyone, it became ever more alluring. It was thrilling: I could foresee a day when every recording ever made by anyone, anywhere, would be available through Napster.

Of course, it was too good to last.

In December 1999, sensing an impeding disaster, the Recording Industry Association of America (RIAA) filed suit against Napster in a California court, claiming that the company was profiting from wholesale copyright violation. It is illegal, under US copyright law, to deal in materials under copyright without the permission of the copyright holder, and the RIAA argued in its legal filings that Napster was doing exactly that. Not so, replied Napster: we don’t store any of these files, we don’t provide them to the users of Napster. Napster argued that its users shared these song files with each other - something that is legal under US copyright law. While Napster acted as a clearinghouse for these files, it insisted that this, in itself, was not illegal under US copyright law. A court date was set, and lawyers prepared for the first legal showdown of the era of digital distribution: the RIAA vs. Napster. Meanwhile, the publicity generated by the lawsuit brought Napster even more users. Over the opening months of 2000, tens of millions of songs were regularly available through the service.

Napster reached a peak of 13.6 million users in February 2001. By that time perhaps as many as a quarter billion MP3 tracks were available through the service. Big name musicians like Madonna and Metallica filed suit against Napster - but Metallica took a beating for doing this, because their fans were more likely than most to be avid Napster users. Napster translated the world of recorded music into a gigantic disk drive - turning the business of recorded music upside down. And, although Napster might not have been directly responsible for massive copyright violation, it wasn’t hard to convince a judge that Napster provided the tools to make these violations possible - and that is an illegal act. It took eighteen months for the RIAA to triumph in the California courts; in July 2001 a judge ordered Napster’s server be shut down. In the hours before the order took effect at least 14 million file transfers were recorded. Then, as Napster went down, the wholesale file-sharing of the universe of recorded music apparently ceased.

If only things were that simple.

“Disruptive” technologies such as the World Wide Web or Napster are often incredibly seductive to their users. Once you’ve become accustomed to the World Wide Web, it begins to become difficult to imagine a world without it. It becomes incorporated into your thinking and behavior. If the Web suddenly went away, millions of us would suffer through a very painful change of mind. It’s not as though we’re “addicted” to these technologies; rather, we have incorporated them into our understanding of how the world works - we expect the Web to be there, much as we expect electricity or hot water in our homes. Napster was one of the most seductive technologies ever created, and its absence created quite a vacuum - a vacuum that was quickly filled.

Just three months after the RIAA sued Napster, Justin Frankel and Tom Pepper, two software engineers at Nullsoft, a tiny division of media giant AOLTimeWarner, posted an announcement to Slashdot, a website popular with the geek crowd. (Slashdot’s tagline is “News for nerds. Stuff that matters.”) The announcement proclaimed the release of a new piece of software, “Gnutella,” which could be used to create file-sharing networks between Internet-connected computers. Gnutella, they declared, differed from Napster in one very substantial feature: with Gnutella there is no centralized server, keeping a master list of all the files available on all the computers connected to the file-sharing network. Instead, the computers treat each other as peers (as is the case with BitTorrent), and use a process of discovery to find out what’s on other computers in the file-sharing network. My computer might send out a request for a song by ambient legend Brian Eno; in a Napster file-sharing network the server would take a look for all the songs by Brian Eno in its master list, then send back a list of matches. Within a Gnutella network, I send that request to the two or three other computers I am connected directly to; if any of these have matching files, they’ll tell my computer. They’ll also forward the request to another two or three computers they’re connected to. My request is forwarded again and again, slowly working its way across the large network of computers running the Gnutella software. It takes much longer for me to get an answer to my request, because the request has to be passed on to many thousands of computers, all over the Internet. Gnutella is less efficient than Napster, but, because there’s no centralized server (every computer on a Gnutella file-sharing network acts as both a server and a client) there’s no single point that can be shut down. Or sued out of existence.

Gnutella is a technical fix to the problem that plagued Napster: the RIAA. When the RIAA sued Napster, in December 2000, network engineers like Frankel and Pepper gave thought to the legal issues raised by file sharing. Consider: If it is illegal for a commercial organization like Napster to promote theft of copyright, but it is legal for you to share your music with your friends, can you design a network that allows you to share things with your friends without getting a commercial entity involved? At its essence, that’s what Gnutella is. It’s a file-sharing network with no center, just a loosely-affiliated network of peers (your friends), all of whom want to share music with one another.

Gnutella isn’t perfect; beyond a certain number of connected computers it won’t perform as well as Napster. But it is invisible, subterranean in a way that Napster could never be - making it all the more vexatious to the RIAA, and, more significantly, to the parent company of Nullsoft. Back in the mid-1990s, Nullsoft released Winamp, a very popular piece of software used to play CDs and MP3 files on PCs. AOL bought Nullsoft in 1999, fleshing out their portfolio of media software - but, in January 2000, at the height of the dot-com boom, AOL announced its merger TimeWarner. It was thought by many that the flagship online service provider and a media giant was a perfect marriage of new and old media. TimeWarner owned Warner Music, one of the largest companies in the record industry. This meant that AOL now had a substantial investment in copyrights. And one of its divisions had just released a tool which would aid in the theft of copyright.

It took AOL less than 24 hours to shut down the distribution of the Gnutella software; but, because of the announcement on Slashdot, hundreds of thousands of geeks had already downloaded and installed the software. I was one of them. In the earliest hours of the Gnutella network, you could actually watch the search requests as they passed through your computer. This was a unique experience - almost like having a window on the Internet. Many of the searches were for pornography - what else can be expected of a network populated principally by college-age men? But many other searches were for subjects far more obscure and interesting. The earliest version of Gnutella was almost like having a stock ticker on the collective mind of the Internet. What’s interesting? What’s important? What’s fun? All of that flashed by, several times a second, on my computer screen.

Despite the shutdown of the distribution of the Gnutella software (or, perhaps, because of it) many geeks took a good look at how Gnutella used the Internet to create its self-organizing file-sharing network. These observations were published - again on Slashdot - and lots of programmers around the world began to write their own versions of Gnutella, making a continuous series of improvements to the program originally developed by Frankel and Pepper. In short, all that Nullsoft had to do was to prove that it could be done. The rest took care of itself. When an idea as seductive as file-sharing comes along, people will line up to work on it, eager to help in the collective project to make it easier and easier to get to more and more information.

By the time the courts pulled the plug on Napster, most of the core community of file-sharing enthusiasts had already transitioned to Gnutella. It didn’t make any difference to any of them that Napster had gone dark; they had, through their own efforts, built their own file-sharing network. That network continues to grow to this day. The RIAA has lately taken to suing individuals who participate in file-sharing, believing that striking fear into the hearts of those who illegally trade in music is the best strategy toward keeping the practice to a minimum. But it is impossible for the RIAA to gain the upper hand; the decentralized, distributed nature of Gnutella makes it practically impervious to attack. All the RIAA can do is file lawsuits against the most egregious violators of copyright. This strategy has occasionally backfired, as in the case of a 68 year-old grandmother, who found herself on the wrong end of a subpoena. (She didn’t even own a computer.) The RIAA is also flooding the file-sharing networks with “bogus” versions of popular songs, hoping to make file-sharing more work, and more dangerous, than it’s worth. But these attempts are all just so many fingers plugging holes in a dike that’s being undermined by the built-up pressure of such a seductive technology.

The RIAA is engaged in a losing race; doing everything it can to shut down the file sharing networks, it is also applying the kind of pressure that forces a technological evolution. In this, the RIAA’s attempts echo the development of the Internet. Thirty years ago the Internet was very simple, and not very stable. As it grew, and more pressures were introduced, the protocols which create the Internet were modified to reflect the lessons learned along the way. Today, we have a highly reliable global Internet, because the pressures of the past have been incorporated into its design. When the RIAA attacks file sharing, they provide the pressure which forces these file sharing networks to evolve into forms which become increasingly invisible, increasingly pervasive, and ever harder to root out. The proof is as simple as this: although no one knows for sure, it is believed that many, many more songs are shared today than in Napster’s heyday. Far from being the enemy of file sharing, the RIAA has been its best, most loyal friend.

© Copyright 2005, Mark D. Pesce
This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/2.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.