by John Horgan
Editor's note: This profile of Claude Shannon originally appeared in the January 1990 issue of Scientific American, and in the IEEE Information Theory Society Newsletter, June, 1990
Claude E. Shannon can't sit still. We're at his home, a stuccoed
Victorian edifice overlooking a lake north of Boston, and I'm trying to
get him to recall how he came up with the theory of information. But Shannon,
who is a boyish 73, with an elfish grin and a shock of snowy hair, is tired
of expounding on his past. Wouldn't I rather see his toys?
Without waiting for an answer, and over the mild protests of his wife Betty, he leaps from his chair and disappears into the other room. When I catch up with him, he proudly shows me his seven chess-playing machines, gasoline-powered pogostick, hundred-bladed jackknife, two-seated unicycle and countless other marvels. Some of his personal creations-such as a juggling W.C. Field mannequin and a computer called THROBAC that calculates in Roman numerals-are a bit dusty and in disrepair, but Shannon seems as delighted with everything as a 10- year old on Christmas morning.
Is this the man who, as a young engineer at Bell Laboratories in 1948, wrote the Magna Carta of the information age: "The Mathematical Theory of Communications"? Whose work Robert W. Lucky, executive director of research at AT&T Bell Laboratories, calls the greatest "in the annals of technological thought?" Whose "pioneering insight" IBM Fellow Rolf W. Landauer equates with Einstein's? Yes. This is also the man who invented a rocket-powered Frisbee and who juggled while riding a unicycle through the halls of Bell Labs. "I've always pursued my interests without much regard to financial value to the world," Shannon says. "I've spent lots of time on totally useless things."
From childhood on, Shannon was fascinated by both the particulars of hardware and the generalities of mathematics. Growing up in Gaylord, Mich., he tinkered with erector sets and radios given to him by his father, a probate judge, solved mathematical puzzles supplied by his older sister, Catherine, who became a professor of mathematics. As an undergraduate at the University of Michigan he majored in electrical engineering and mathematics.
His familiarity with the two fields helped him notch his first big success while he was still a graduate student at the Massachusetts Institute of Technology. In his master's thesis he showed how an algebra invented in the mid-1800's by the British mathematician George Boole - which deals with such concepts as "if X or Y happens and not Z, then Q results" - could represent the working of switches and relays in electrical circuits.
The implications of the paper were profound: engineers now routinely design computer hardware and software, telephone networks and other systems with the aid of Boolean algebra. Shannon downplays the discovery. "it just happened that no one else was familiar with both those fields at the same time," he says. He adds, after a moment of reflection, "I've always loved that word, 'Boolean."'
In 1941, a year after obtaining his Ph.D. from M.I.T., Shannon went to Bell Labs. During World War 11 his official responsibility was developing cryptographic systems, but on his own time he nurtured the ideas that were to evolve into information theory. Shannon's initial goal was simple: to improve the transmission of information over a telegraph or telephone line affected by electrical interference, or noise. The best solution, he decided, was not to improve transmission lines but to package information more efficiently.
What is information? Sidestepping questions about meaning, Shannon showed that it is a measurable commodity: the amount of information in a given message is a function of the probability that-out of all the messages that could be sent-it would be selected. He defined the overall potential for information in a system of messages as its "entropy," which in thermodynamics denotes the randomness-or "shuffledness," if you will-of a system. (Shannon once said that the great mathematician John von Neumann had urged him to use the term entropy, pointing out that since no one really knows what it means, Shannon would have an advantage in debates about his theory.)
Shannon defined the basic unit of information, which came to be called a bit, as a message representing one of two choices: heads or tails for example, or yes or no. One could encode great amounts of information in bits, just as the old game "20 questions" one could quickly zero in on a correct answer through deft questioning. A bit can be represented as a one or a zero or as the presence or absence of current in a wire.
Building on this mathematical foundation, Shannon then showed that any given communications channel has a maximum capacity for reliably transmitting information. Actually he showed that although one can approach this maximum through clever coding, one can never quite reach it. The maximum has come to be known as the Shannon limit.
How does one approach the Shannon limit? The first step is to eliminate redundancy. Just as a laconic suitor might write "I lv u" in his billet-doux, so will a good code compress information to its most compact form. The code then adds just enough redundancy to ensure that stripped-down message is not obscured by noise. For example, a code processing a stream of numbers might add a polynomial equation on whose graph the numbers all fall. The decoder on the receiving end knows that any numbers that diverge from the graph have been altered in transmission.
Shannon's ideas were almost too prescient to have an immediate practical impact. Vacuum-tube circuits simply could not calculate the complex codes needed to approach the Shannon limit. In fact, not until the early 1970's-with the advent of high speed integrated circuits-did engineers begin fully to exploit information theory. Today Shannon's insights help shape virtually all systems that store, process or transmit information in digital form, from compact disks to computers, from facsimile machines to deep- space probes such as Voyager.
Information theory has also infiltrated fields outside the communications, including linguistics, psycho- logy, economics, biology, even the arts. In the early 1970's, the IEEE Transactions on Information Theory published an editorial, titled "Information Theory, Photosynthesis and Religion," decrying this trend. Yet Shannon himself suggests that applying information theory to bio- logical systems may not be so far- fetched, because in his view common principles underlie mechanical and living things. "You bet," he replies, when asked whether he thinks machines can think. "I'm a machine and you're a machine, and we both think, don't we.?'
Indeed, Shannon was one of the first engineers to propose that machines could be programmed to play games and perform other complex task (see "A Chess-Playing Machine," by Claude E. Shannon; Scientific American, February, 1950). In 1950 he built Theseus, a mechanical mouse that - guided by a magnet and a mass of circuitry under the floor - could learn how to find its way out of a maze. The invention inspired the Institute of Electrical and Electronics Engineers to initiate a "micromouse" contest in which thousands of engineering students world-wide now participate.
He built a "mind-reading" machine that played the game of penny- matching, in which one person tries to guess whether the other has chosen heads or tails. A colleague at Bell Labs, David W. Hagelbarger, built the prototype; the machine recorded and analyzed its opponent's past choices, looking for patterns that would foretell the next choice. Because it is almost impossible for a human to avoid falling into such patterns, the machine won more than 50 percent of the time. Shannon then built his own version and challenged Hagelbarger to a legendary duel. Shannon's machine won.
Shannon left Bell Labs to become a professor at M.I.T. in 1956. Since his formal retirement in 1978, his great obsession has been juggling. He has built several juggling machines and devised what may be the unified field theory of juggling: if B equals the number of balls, H the number of hands, D the time each ball spends in a hand, F the time of flight of each ball and E the time each hand is empty, then BIH=(D+F)I(D+E). (Unfortunately, the theory never helped Shannon break his personal record of four balls at once.) He has developed various mathematical models of the stock market and tested them--successfully, he says--on his own portfolio. He even dabbled in poetry: among his works is A Rubric on Rubik Cubics, set-to the meter of "Ta-Ra-Ra-Boom-De-Aye."
Shannon has published little on information theory, however, since the late 1950's. Some former &U colleagues suggest that Shannon had "burned out" and tired of the field he created, but Shannon denies it. He says he continued to study various problems in information theory-at least through the 1960's-but did not consider his investigations good enough to publish. "Most great mathematicians have done their finest work when they were young," he says.
In 1985 Shannon and his wife decided on a whim to visit the IEEE International Information Theory Symposium being held in Brighton, England. He had not attended a meeting in many years, and at first no one noticed him. Then word raced around the conference: that shy, white-haired gent wandering in and out of technical sessions was Claude E. Shannon. At the banquet, Shannon said a few words, briefly juggled three balls and then signed auto- graphs for a long line of engineers. Recalls Robert J. McEliece of the California Institute of Technology, "It was as if Newton had showed up at a physics conference.