The soul of a new machine

Over the weekend, I took part in a panel at Windycon titled “Evil Computers: Why Didn’t We Just Pull the Plug?” Naturally, my mind turned to the most famous evil computer in all of fiction, so I’ve been thinking a lot about HAL, which made me all the more sorry to learn yesterday of the death of voice actor Douglas Rain. (Stan Lee also passed away, of course, which is a subject for a later post.) I knew that Rain had been hired to record the part after Stanley Kubrick was dissatisfied by an earlier attempt by Martin Balsam, but I wasn’t aware that the director had a particular model in mind for the elusive quality that he was trying to evoke, as Kate McQuiston reveals in the book We’ll Meet Again:

Would-be HALs included Alistair Cooke and Martin Balsam, who read for the part but was deemed too emotional. Kubrick set assistant Benn Reyes to the task of finding the right actor, and expressly not a narrator, to supply the voice. He wrote, “I would describe the quality as being sincere, intelligent, disarming, the intelligent friend next door, the Winston Hibler/Walt Disney approach. The voice is neither patronizing, nor is it intimidating, nor is it pompous, overly dramatic, or actorish. Despite this, it is interesting. Enough said, see what you can do.” Even Kubrick’s U.S. lawyer, Louis Blau, was among those making suggestions, which included Richard Basehart, José Ferrer, Van Heflin, Walter Pigeon, and Jason Robards. In Douglas Rain, who had experience both as an actor and a narrator, Kubrick found just what he was looking for: “I have found a narrator…I think he’s perfect, he’s got just the right amount of the Winston Hibler, the intelligent friend next door quality, with a great deal of sincerity, and yet, I think, an arresting quality.”

Who was Winston Hibler? He was the producer and narrator for Disney who provided voiceovers for such short nature documentaries as Seal Island, In Beaver Valley, and White Wilderness, and the fact that Kubrick used him as a touchstone is enormously revealing. On one level, the initial characterization of HAL as a reassuring, friendly voice of information has obvious dramatic value, particularly as the situation deteriorates. (It’s the same tactic that led Richard Kiley to figure in both the novel and movie versions of Jurassic Park. And I have to wonder whether Kubrick ever weighed the possibility of hiring Hibler himself, since in other ways, he clearly spared no expense.) But something more sinister is also at play. As I’ve mentioned before, Disney and its aesthetic feels weirdly central to the problem of modernity, with its collision between the sentimental and the calculated, and the way in which its manufactured feeling can lead to real memories and emotion. Kubrick, a famously meticulous director who looked everywhere for insights into craft, seems to have understood this. And I can’t resist pointing out that Hibler did the voiceover for White Wilderness, which was nominated for an Academy Award for Best Documentary Short, but also included a scene in which the filmmakers deliberately herded lemmings off a cliff into the water in a staged mass suicide. As Hibler smoothly narrates in the original version: “A kind of compulsion seizes each tiny rodent and, carried along by an unreasoning hysteria, each falls into step for a march that will take them to a strange destiny. That destiny is to jump into the ocean. They’ve become victims of an obsession—a one-track thought: ‘Move on! Move on!’ This is the last chance to turn back, yet over they go, casting themselves out bodily into space.”

And I think that Kubrick’s fixation on Hibler’s voice, along with the version later embodied by Rain, gets at something important about our feelings toward computers and their role in our lives. In 2001, the astronauts are placed in an artificial environment in which their survival depends on the outwardly benevolent HAL, and one of the central themes of science fiction is what happens when this situation expands to encompass an entire civilization. It’s there at the very beginning of the genre’s modern era, in John W. Campbell’s “Twilight,” which depicts a world seven million years in the future in which “perfect machines” provide for our every need, robbing the human race of all initiative. (Campbell would explore this idea further in “The Machine,” and he even offered an early version of the singularity—in which robots learn to build better versions of themselves—in “The Last Evolution.”) Years later, Campbell and Asimov put that relationship at the heart of the Three Laws of Robotics, the first of which states: “A robot may not injure a human being or, through inaction, allow a human being to come to harm.” This sounds straightforward enough, but as writers realized almost right away, it hinges on the definition of certain terms, including “human being” and “harm,” that are slipperier than they might seem. Its ultimate expression was Jack Williamson’s story “With Folded Hands,” which carried the First Law to its terrifying conclusion. His superior robots believe that their Prime Directive is to prevent all forms of unhappiness, which prompts them to drug or lobotomize any human beings who seem less than content. As Williamson said much later in an interview with Larry McCaffery: “The notion I was consciously working on specifically came out of a fragment of a story I had worked on for a while about an astronaut in space who is accompanied by a robot obviously superior to him physically…Just looking at the fragment gave me the sense of how inferior humanity is in many ways to mechanical creations.”

Which brings us back to the singularity. Its central assumption was vividly expressed by the mathematician I.J. Good, who also served as a consultant on 2001:

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.

That last clause is a killer, but even if we accept that such a machine would be “docile,” it also embodies the fear, which Campbell was already exploring in the early thirties, of a benevolent dictatorship of machines. And the very Campbellian notion of “the last invention” should be frightening in itself. The prospect of immortality may be enticing, but not if it emerges through a technological singularity that leaves us unprepared to deal with the social consequences, rather than through incremental scientific and medical progress—and the public debate that it ought to inspire—that human beings have earned for themselves. I can’t imagine anything more nightmarish than a world in which we can all live forever without having gone through the necessary ethical, political, and ecological stages to make such a situation sustainable. (When I contemplate living through the equivalent of the last two years over the course of millennia, the notion of eternal life becomes considerably less attractive.) Our fear of computers taking over our lives, whether on a spacecraft or in society as a whole, is really about the surrender of control, even in the benevolent form embodied by Disney. And when I think of the singularity now, I seem to hear it speaking with Winston Hibler’s voice: “Move on! Move on!”