Policy —

The essence of the ‘Net: a history of the protocols that hold the network together

The Internet is a loose arrangement of connected but autonomous networks of …

Johnny Ryan's A History of the Internet and the Digital Future has just been released and is already drawing rave reviews. Ars Technica is proud to present three chapters from the book, condensed and adapted for our readers. You can find Chapter 1 hereThe current installment is adapted from Chapter 3, "The Essence of the Internet," and it tells the story of the development of some of the fundamental technologies and protocols that underlie the Internet.

Chapter 3: The Essence of the Internet

The Internet is a loose arrangement of connected but autonomous networks of devices. Each device, a "host" in networking jargon, uses a "protocol" to communicate with other devices on the network. These protocols tie together diverse networks and govern communication between all computers on the Internet. Not only are the protocols elemental to the Internet and how it works, but the unique collaboration between their designers was the formative event of Internet culture. In as much as any single element of the whole can be, these protocols are the essence of the Internet. The remarkable manner in which a team of young collaborators developed these protocols set the tone for the future development of Internet culture. As their work on the protocols proceeded they began to establish the informal conventions that would characterize the tone of collaboration and discussion on the Internet thereafter. The process began in a bathroom, late on the night of April 7, 1969.

As BBN started building the IMPs for the ARPANET in 1969, an important piece of the network was missing: the software that would govern how computers would communicate. Graduate students at various facilities funded by the US Department of Defense Advance Research Projects Agency (ARPA) had been given the task in 1969 of developing the missing communication protocols. They formed an informal "network working group." Finding themselves working in a vacuum, the students connected to ARPANET, who had been given the task in 1969 of developing the technical protocols, also began to establish the informal protocols that would influence interpersonal communications on the Internet in general.

Uncertain of their positions within the hierarchy of the ARPANET project, the students issued notes on their protocols under the title "Request for Comments" (RFC). Steve Crocker, a graduate student who had received his bachelor's degree at UCLA only a year before, used the title Request for Comments to make the invitation to participate as open as possible, and to minimize any claim to authority that working on so crucial an aspect of the network as its protocols might imply. The first RFC document, which set the tone for the next half century of Internet culture and initiated the process to define the protocols that govern virtually all data exchange on the planet, was composed in humble circumstances. Its author recalls: "I had to work in a bathroom so as not to disturb the friends I was staying with, who were all asleep." The tone in which the RFCs were typed was distinctive.

Crocker was the de facto leader of the small group of six. He and two others of the group had been at the same high school in Los Angeles, Van Nuys High, and were graduate students of Leonard Kleinrock. (Kleinrock was under contract with ARPA to run the network measurement center at UCLA.) Crocker was writing a document that outlined some broad ideas on how the students would pass around ideas through "temporary, informal memos." Even as he drafted the document, the prospect of disapproval from far above in the academic hierarchy weighed heavily upon him:

In my mind, I was inciting the wrath of some prestigious professor at some phantom East Coast establishment. I was actually losing sleep over the whole thing.

Crocker was eager to open up the process to as many of his peers as possible:

Closely related to keeping the technical design open was keeping the social process around the design open as well. Anyone was welcome to join the party.

Vint Cerf, an early participant in the informal networking group (and now vice president of Google), sums up the approach and context:

Keep in mind that the original developers of the host level protocols were mostly graduate students. We adopted a humble and inclusive posture and a mantra that Dave Clark ultimately coined as "rough consensus and running code"—that means we don't really vote exactly, we just try to assess rough consensus among the group trying to agree on proposed standards.

RFC 3, released in April 1969, elaborated on the character and objectives of the RFCs (note that the word "Host" here refers to a connected computer):

These standards (or lack of them) are stated explicitly for two reasons. First, there is a tendency to view a written statement as ipso facto authoritative, and we hope to promote the exchange and discussion of considerably less than authoritative ideas. Second, there is a natural hesitancy to publish something unpolished, and we hope to ease this inhibition.

RFC 3 continues in the counter-hierarchical vein, establishing the principle that no text should be considered authoritative and that there is no final edit. This is a pivotal element of the "perpetual beta" described in the next chapter. Also implicit was that authority was to be derived from merit rather than fixed hierarchy. A later elaboration of this principle was:

We reject kings, presidents and voting.

We believe in rough consensus and running code.

Crocker's RFC, though penned in humble circumstances, set the open, inviting tone of the next half century of Internet culture and initiated the process to define the protocols that govern virtually all data exchange on the planet. Since Crocker's RFC there have been almost six thousand RFCs published, which maintain an open, collaborative approach in Internet-engineering circles. The meritocracy of the RFCs was exemplified by a generation of delinquent programmers at MIT from the late 1950s to the late 1960s, who in turn created the "hacker" culture that influenced much of what was to follow. The first fruit of the graduate students' labour was the NCP, the Network Control Protocols, which governed communications between machines on the Internet. The NCP, however, was merely the first protocol that allowed communications on the ARPANET. An "internetworking" protocol that could tie different machines and networks together was yet to come.

Radio and satellite networks

San Francisco features disproportionately in the history of the digital age. Little attention, however, has been given to one of its acknowledged landmarks: a public house called Zott's. Zott's (named "The Alpine Inn" since the mid 1950s) is a small, wood-panelled tavern and a historic focal point for the ne'er-do-wells of Silicon Valley. Its founder was Felix Buelna, a Mexican, who moved from Santa Clara in the wake of the gold rush when that area became crowded by would-have-been gold diggers in the mid 1800s. He built the inn on the site of a pony trail that had been used by rancheros and settlers to reach the coast. Buelna's inn was a place of gambling with a colorful clientele and, in the words of the US National Park Service's official survey, "a long string of colorful owners."

Regulars in the 1880s included the construction workers building Stanford University, whose entrepreneurs and technologies would propel the dot-com boom a century later. The inn also became the regular haunt of the new university's students. In 1908 the editors of the Stanford Sequoia lambasted their immoderate peers, writing that the student body had been "held up to the world as a community composed largely of drunkards." In January the following year the president of the university wrote in vexed mood to the county supervisors requesting that they not renew the inn's liquor licence because it was "unusually vile, even for a roadhouse, a great injury to the University and a disgrace to San Mateo County." Yet the humble wood-panelled structure remained a landmark through the twentieth century as the digital industry evolved around it. By early 2001 its car park accommodated the expensive sports cars of the young Silicon Valley millionaires. It was fitting, then, that more than a century after its establishment Zott's should be the site for an important event in the history of the Internet.

On 27 August 1976 a van parked in Zott's beer garden. It was part of the Stanford Research Institute's (SRI) packet radio experiment, conducted under contract for ARPA. The SRI team removed a computer terminal from the van and placed it on a wooden table in Zott's beer garden. A wire connected the terminal to the van, and radio equipment in the van connected it to ARPA's new packet radio network, PRNET, which in turn was connected to ARPANET. The team at Zott's sent a message from their terminal across the PRNET and thence to a distant machine connected to ARPANET. This was one of the more momentous events to have happened in any beer garden: it was the first ever packet data transmission across two networks using the new "internet" protocol.

The discoveries that made this transmission possible arose as part of an earlier project at the University of Hawaii in 1970. Norman Abramson, the Professor of Electrical Engineering and Computer Science, had faced a difficult problem. He wanted to network the University of Hawaii's seven campuses. This posed three problems. First, the campuses were physically spread across four islands. Second, the leased telephone lines that connected ARPANET facilities to each other were too expensive for his budget. Third, the line quality of the Hawaiian telephone system was too poor to carry networking data. The answer, Abramson decided, was to use radio. Thus from 1970 ARPA began to fund Abramson's attempt to develop a packet radio network.

Radio signals travel differently to electric signals across telephone lines. While telephone signals travel from point to point in an orderly sequence, radio transmits indiscriminately to all receivers within its broadcast range. Signals broadcast by different nodes to the receiver at the same time can collide and be destroyed. Abramson's team developed an elegant solution to this problem: when any node sent a packet but did not receive confirmation of successful delivery from the receiving node it would wait for a random period and then resend the message. Since all nodes would wait a random period before resending, the odds of repeat collisions were slight. Thus the network would quickly correct itself when it lost packets. Using this method Abramson's team built a functioning network called the AlohaNet that linked Hawaii University's campuses to each other and to the ARPANET. This method of dealing with collision between messages was called the "Aloha method," and ARPA used its example to build its own packet radio network, PRNET.

The discovery of the Aloha method for packet radio networking was particularly timely since the political tides in which ARPA swam had become slightly more turbulent. In 1969 Senate Majority Leader Mike Mansfield had signalled his intention to cut $400 million from the defence research budget. He was the author of Section 203 of the Military Procurement Authorization Act for Fiscal Year 1970, the so-called "Mansfield Amendment," which stipulated that all funded research must have a "direct and apparent relationship to a specific military function or operation." Packet radio was just such a project. The power of massive, expensive mainframe computers could be relayed to the battlefield by networks of radio, cable and, as ARPA was beginning to prove, satellite.

Channel Ars Technica