The unfinished lives

Yesterday, the New York Times published a long profile of Donald Knuth, the legendary author of The Art of Computer Programming. Knuth is eighty now, and the article by Siobhan Roberts offers an evocative look at an intellectual giant in twilight:

Dr. Knuth usually dresses like the youthful geek he was when he embarked on this odyssey: long-sleeved T-shirt under a short-sleeved T-shirt, with jeans, at least at this time of year…Dr. Knuth lives in Stanford, and allowed for a Sunday visitor. That he spared an entire day was exceptional—usually his availability is “modulo nap time,” a sacred daily ritual from 1 p.m. to 4 p.m. He started early, at Palo Alto’s First Lutheran Church, where he delivered a Sunday school lesson to a standing-room-only crowd.

This year marks the fiftieth anniversary of the publication of the first volume of Knuth’s most famous work, which is still incomplete. Knuth is busy writing the fourth installment, one fascicle at a time, although its most recent piece has been delayed “because he keeps finding more and more irresistible problems that he wants to present.” As Roberts writes: “Dr. Knuth’s exacting standards, literary and otherwise, may explain why his life’s work is nowhere near done. He has a wager with Sergey Brin, the co-founder of Google and a former student…over whether Mr. Brin will finish his Ph.D. before Dr. Knuth concludes his opus…He figures it will take another twenty-five years to finish The Art of Computer Programming, although that time frame has been a constant since about 1980.”

Knuth is a prominent example, although far from the most famous, of a literary and actuarial phenomenon that has grown increasingly familiar—an older author with a projected work of multiple volumes, published one book at a time, that seems increasingly unlikely to ever see completion. On the fiction side, the most noteworthy case has to be that of George R.R. Martin, who has been fielding anxious inquiries from fans for most of the last decade. (In an article that appeared seven long years ago in The New Yorker, Laura Miller quotes Martin, who was only sixty-three at the time: “I’m still getting e-mail from assholes who call me lazy for not finishing the book sooner. They say, ‘You better not pull a Jordan.’”) Robert A. Caro is still laboring over what he hopes will be the final volume of his biography of Lyndon Johnson, and mortality has become an issue not just for him, but for his longtime editor, as we read in Charles McGrath’s classic profile in the Times:

Robert Gottlieb, who signed up Caro to do The Years of Lyndon Johnson when he was editor in chief of Knopf, has continued to edit all of Caro’s books, even after officially leaving the company. Not long ago he said he told Caro: “Let’s look at this situation actuarially. I’m now eighty, and you are seventy-five. The actuarial odds are that if you take however many more years you’re going to take, I’m not going to be here.”

That was six years ago, and both men are still working hard. But sometimes a writer has no choice but to face the inevitable. When asked about the concluding fifth volume of his life of Picasso, with the fourth one still on the way, the biographer John Richardson said candidly: “Listen, I’m ninety-one—I don’t think I have time for that.”

I don’t have the numbers to back this up, but such cases—or at least the public attention that they inspire—seem to be growing more common these days, on account of some combination of lengthening lifespans, increased media coverage of writers at work, and a greater willingness from publishers to agree to multiple volumes in the first place. The subjects of such extended commitments tend to be monumental in themselves, in order to justify the total investment of the writer’s own lifetime, and expanding ambitions are often to blame for blown deadlines. Martin, Caro, and Knuth all increased the prospective number of volumes after their projects were already underway, or as Roberts puts it: “When Dr. Knuth started out, he intended to write a single work. Soon after, computer science underwent its Big Bang, so he reimagined and recast the project in seven volumes.” And this “recasting” seems particularly common in the world of biographies, as the author discovers more material that he can’t bear to cut. The first few volumes may have been produced with relative ease, but as the years pass and anticipation rises, the length of time it takes to write the next installment grows, until it becomes theoretically infinite. Such a radical change of plans, which can involve extending the writing process for decades, or even beyond the author’s natural lifespan, requires an indulgent publisher, university, or other benefactor. (John Richardson’s book has been underwritten by nothing less than the John Richardson Fund for Picasso Research, which reminds me of what Homer Simpson said after being informed that he suffered from Homer Simpson syndrome: “Oh, why me?”) And it may not be an accident that many of the examples that first come to mind are white men, who have the cultural position and privilege to take their time.

It isn’t hard to understand a writer’s reluctance to let go of a subject, the pressures on a book being written in plain sight, or the tempting prospect of working on the same project forever. And the image of such authors confronting their mortality in the face of an unfinished book is often deeply moving. One of the most touching examples is that of Joseph Needham, whose Science and Civilization in China may have undergone the most dramatic expansion of them all, from an intended single volume to twenty-seven and counting. As Kenneth Girdwood Robinson writes in a concluding posthumous volume:

The Duke of Edinburgh, Chancellor of the University of Cambridge, visited The Needham Research Institute, and interested himself in the progress of the project. “And how long will it take to finish it?” he enquired. On being given a rather conservative answer, “At least ten years,” he exclaimed, “Good God, man, Joseph will be dead before you’ve finished,” a very true appreciation of the situation…In his closing years, though his mind remained lucid and his memory astonishing, Needham had great difficulty even in moving from one chair to another, and even more difficulty in speaking and in making himself understood, due to the effect of the medicines he took to control Parkinsonism. But a secretary, working closely with him day by day, could often understand what he had said, and could read what he had written, when others were baffled.

Needham’s decline eventually became impossible to ignore by those who knew him best, as his biographer Simon Winchester writes in The Man Who Loved China: “It was suggested that, for the first time in memory, he take the day off. It was a Friday, after all: he could make it a long weekend. He could charge his batteries for the week ahead. ‘All right,’ he said. ‘I’ll stay at home.’” He died later that day, with his book still unfinished. But it had been a good life.

The chance operation

If you were looking for insights into leading the life of an artist at a time of political turmoil, it would be hard to find a more intriguing example than Judith Malina, who spent five decades wrestling with the problem in public as the head of the Living Theatre. Beginning in the early fifties, Malina and her husband, Julian Beck, staged a series of shows in New York and Paris that appear to have been equal parts innovative, radical, and unbearable. In 1952, for example, their production of Paul Goodman’s Faustina seems to have strained the audience’s patience to the breaking point. As Erika Munk writes in a chapter of the book Restaging the Sixties:

Goodman—a great influence as anarchist thinker, Malina’s therapist, and gay-but-married exemplar to Beck—wanted theater that provoked: “Either the audience is terribly offended…or in two hours the play effects a character-change in the audience, more than all the manifestoes can accomplish.” At the end of Faustina, which concerned Rome in the time of Marcus Aurelius, the actor playing the title character was supposed to address the spectators directly, telling them they were responsible for its murderous climax because they didn’t leap onstage and stop the action. The actor playing Faustina refused to address the audience, and Malina took her place. They still didn’t leap.

Munk concludes: “Faustina was a disaster. No surprise, when sophisticated playgoers were asked to feel real guilty for not taking steps to stop a purely fictive crime.” And I can’t say that I especially wish that I had been in the audience that night.

Yet I also respect Malina’s lifelong obsession with exploring the relationship between society and art, which she expressed in the only way that she could—by forcibly transferring it to the actors and the audience. In the program notes, she wrote: “We are the creators in an art where every night hundreds of people are ignored, a pretense is made that they do not exist; and then we wonder that the actor has grown apart from society; and then we wonder that the art itself staggers lamely behind its hope of being part of life.” Her tentative solution was to explicitly acknowledge the audience’s presence, which led to a stark contradiction in itself. She was still imposing her will on the spectators, not to mention her reluctant actors, and her response was elegant, if not entirely convincing:

The play is a ritual. Any play is a ritual, but in this play the ritual is overt and we speak out about it with real brazenness…The I Ching says, “It is the Creative that begets things but they are brought to birth by the receptive.” And it says of me further, “The person in question is not in an independent position, but is acting as an assistant. This means that he must achieve something.” I consulted this book of ancient Chinese oracle not only for myself but for my audience. To allow ourselves to be led, all of us assistants in the ritual, which hasn’t any power, but from which power is derived. The play does not take place in pagan Rome. Believe me, believe me that it takes place in the theatre. In this theatre and tonight.

In other words, the play itself was in control, and both the performers and the audience had to surrender to its logic in order to achieve a deeper form of liberation.

It’s also revealing that Malina turned to the I Ching, which is the ideal vehicle for this kind of process. A year earlier, Malina and Beck had served as performers—or “operators”—at the debut of Imaginary Landscape No. 4 by their friend John Cage, which was presented by randomly changing the station and volume on twenty-four radios according to the instructions derived from the oracle. For Cage, these mechanisms were intended to serve as a means of surrendering control, as he explained in a passage that I quoted here a few months ago:

Chance operations are not mysterious sources of “the right answers.” They are a means of locating a single one among a multiplicity of answers, and, at the same time, of freeing the ego from its taste and memory, its concerns for profit and power, of silencing the ego so that the rest of the world has a chance to enter into the ego’s own experience, whether that be outside or inside.

And Cage expressly saw the use of chance operations as an artistic reaction against political dogmatism and oppression. Elsewhere in the same talk, “Lecture on the Weather,” he explains: “Now our government thinks of us also as the policemen of the world…The desire for the best and the most effective in connection with the highest profits and the greatest power led to the fall of nations before us: Rome, Britain, Hitler’s Germany. Those were not chance operations. We would do well to give up the notion that we alone can keep the world in line, that only we can solve its problems.”

And out of all forms of divination, the I Ching might be the best one to use when enacting the relationship between the individual and authority, precisely because its tone is bureaucratic and vaguely dictatorial. As Joseph Needham writes in Science and Civilisation in China:

The Book of Changes was a system for pigeon-holing novelty and then doing nothing more about it. Its universal system of symbolism constituted a stupendous filing-system. It led to a stylization of concepts almost analogous to the stylizations which have in some ages occurred in art forms, and which finally prevented painters from looking at Nature at all…The Book of Changes might almost be said to have constituted an organization for “routing ideas through the right channels to the right departments.”

This rigidity is part of the reason why I’ve never much cared for the I Ching as a creative tool—but if you’re looking to simulate the surrender of agency to an impersonal system of power, it’s perfect. (It probably isn’t an accident that Philip K. Dick turned to the I Ching to make narrative decisions while writing The Man in the High Castle, which imagines an alternate history in which the United States was divided between Germany and Japan after World War II.) And at a moment when issues of authoritarianism seem more urgent then ever, it might be a good time to revisit such tools, which force the artist to abandon the insidious inclination toward control itself. As the priestess says in Faustina: “It’s a lot of shit, but that’s how we do it.”

Science and civilization

Over the last week, I picked up two books—at the annual Newberry Library and Oak Park Public Library book sales, which are always a high point of my year—that I’d been hoping to find for a long time. One is a single volume, Civil Engineering and Nautics, of Joseph Needham’s landmark Science and Civilization in China, which currently consists of twenty-seven huge books that I all unreasonably hope to own one day. The other is a slim fascicle, or paperbound installment from a work in progress, from Donald Knuth’s The Art of Computer Programming, which, if we’re lucky, will release its fourth volume sometime in the next decade. These two projects are rather different in scale, but remarkably similar in their conception and incubation. Needham worked on his book for close to half a century without finishing it, and Knuth has been laboring on his for even longer, with no obvious end in sight. I’ve been intrigued by such grand projects for most of my life, but I’ve become even more interested after embarking on my own venture into nonfiction. Replace “computer programming” with “science fiction” and “discovered” with “written,” and what Knuth once said in an interview gets very close to my attitude two years ago when I started writing Astounding:

At the time, everybody I knew who could write a book summarizing what was known about computer programming had discovered quite a lot of the ideas themselves…By contrast, I hadn’t really discovered anything new by myself at that point. I was just a good writer…I had this half-conceited and half-unconceited view that I could explain it more satisfactorily than the others because of my lack of bias. I didn’t have any axes to grind but my own.

Knuth concludes: “Then, of course, as I started to write things I naturally discovered one or two new things as I went, and now I am just as biased as anybody.” Which pretty sums up my experience, too.

And what really fascinates me about both projects is how monstrously these tomes grew past their projected dimensions, both in space and in time. Both Needham and Knuth thought at first that their work would fit within a single volume, and although they each expanded it to the magic number of seven, neither seems to have grasped just how long it would take. Knuth recalls:

My original motivation was to write a text about how to write compilers, so I began drafting chapters. I was seriously planning to finish the book before my son was born…In June 1965, I had finally finished the first draft of the twelve chapters. It amounted to three thousand handwritten pages of manuscript…I figured about five pages of my handwriting would be about one page of a book.

As it turned out, he was a little off: the real proportion was one and a half handwritten pages to a single page in type, which meant that he had already written two thousand pages without even getting past the subject of compilers. Needham had a similar moment of clarity. As Simon Winchester writes in his biography The Man Who Loved China:

Needham had decreed early on in the process, as he watched each volume begin to swell and threaten to burst out of its covers, that no one volume should be “too big for a man to read comfortably in his bath.” But it was happening nonetheless…One book became two, three, or four. Volume V, a special case, became not five, but thirteen formal subsidiary parts, each one of them big and complicated enough to be made into a separate, self-standing, and equally enormous new volume of its own.

It’s frankly hard to imagine reading any of these expensive books in the tub, but Needham says elsewhere, more realistically, that critics found the volumes “too heavy and bulky for meditative evening reading,” which led to the work being repeatedly subdivided.

The Art of Computer Programming was released by a commercial educational publisher, Addison-Wesley, but it isn’t surprising that most such books tend to appear at academic presses, which are the only institutions capable of sustaining a project that lasts for decades. (Their sole competition here might be the Catholic Church, which has been underwriting a critical edition of the works of St. Thomas Aquinas since 1879. They’re about halfway through.) Winchester refers in passing to “the beleaguered Cambridge University Press, which was obliged to tolerate the constant expansion of the project,” and for the full picture, you can turn to the book A Skeptic Among Scholars, by August Frugé, the director of the University of California Press. He writes of The Plan of St. Gall, a three-volume monument of scholarship that is probably the most beautiful book I own:

The St. Gall manuscript…was said in 1960 to be in semifinal draft, about one hundred and fifty pages in length. When approved by the Editorial Committee and accepted by me in 1967, it came to several hundred typed pages, about right for a single quarto volume. As the work moved through the production process during the next twelve years, we paused every now and then to call for new estimates of size and cost, and each time discovered that new sections had been added, along with a few dozen new diagrams and drawings.

At one point, concerns about cost threatened to derail the whole enterprise, and Frugé retired before the three huge folios were published. James H. Clark, his successor, saw it to completion, writing later: “But what is a university press for if not to take these kinds of risks, make these investments, and publish books that make a difference?” Aside from Knuth, one of the few comparable examples on the commercial side must be Robert Caro’s The Years of Lyndon Johnson, which was originally planned as three volumes to be written over about six years. Forty years and four books later, Caro still isn’t done, and the fact that he has been allowed to keep working at the same methodical pace is a tribute to his longtime editor Robert Gottlieb at Knopf.

And if there’s one key takeaway from the examples I’ve mentioned, it’s that none of these authors set out to devote their lives to these projects—they all thought at first that they could finish it within a few years. Knuth recalls:

It gradually dawned on me how large a project this was going to be. If I had realized that at the beginning, I wouldn’t have been foolish enough to start; I wouldn’t have dared to tackle such a thing…[But] I had collected so much material that I felt it was my duty to continue with the project even though it would take a lot longer than I had originally expected.

You also realize that you can’t explain the subject at hand without covering a lot of other material first. Caro treats his books on Johnson as windows onto such vistas as local politics, Texas, and the Senate, which is a big part of their appeal. (The equivalent in my case would be deciding that I couldn’t write the life of John W. Campbell in a comprehensible form without telling the entire history of science fiction, too, which might well be true.) Frugé, perhaps to his credit, ventures a more cynical reading:

In my skeptical and perhaps scatterbrained way, I sometimes wonder how a research scholar can work on the same project decade after decade and retain faith in its intellectual importance. Perhaps some do not, and that is why their books are never completed. But we can also observe an opposite phenomenon. As the years go by the object or document for study may swell and expand in importance until—until, for example, “The Plan of St. Gall is…one of the most fascinating creations of the human mind.”

He makes a good point. The cycle feeds on itself, with the work expanding in scope to justify the amount of time it takes. It’s human nature, and there’s something a little absurd about it. But it’s also the only way we get art, science, or civilization.

The Book of Changes

If there’s a single theme to which I’ve repeatedly returned for the five years and more I’ve been writing this blog, it’s the importance of randomness in the creative process. I’ve always tried to systematically incorporate elements of chance into my work, in a large part because I’m temperamentally the opposite: I’m an architect, not a gardener, and nearly everything I’ve written—fiction and nonfiction alike—has been planned, outlined, and structured within an inch of its life. I adopted this approach as a kind of survival strategy: I figured out early in my career that I had a better chance of finishing a project, rather than abandoning it halfway through, if I had a blueprint to follow. And that’s still true. But the fact that I’ve always been a fundamentally rational writer has led me to think about creative randomness and serendipity to a greater extent, I suspect, than many of those who naturally take a more intuitive approach. An author who begins a story without a clear end point in mind, apart from a willingness to follow the narrative wherever it leads, doesn’t need to consciously worry about randomness: it’s baked into the process from the beginning. But because I’m predisposed to lay everything out before I type the first sentence, I’ve tried to be diligent about keeping that fertilizing aspect of chance alive.

As Gregory Bateson wrote: “Creative thought must always contain a random component. The exploratory process—the endless trial and error of mental progress—can achieve the new only by embarking upon pathways randomly presented, some of which when tried are somehow selected for survival.” Elsewhere, Bateson is reported to have said to his secretary: “I am going to build a church some day. It will have a holy of holies and a holy of holy of holies, and in that ultimate box will be a random number table.” And the search for productive forms of randomness has been one of the most absorbing parts of my writing life over the last ten years. I’ve written at length here about how I’ve tried most of the usual suspects, like Brian Eno’s Oblique Strategies, and how the most useful repository of random connections I’ve found has been Ted Hughes’s anthology A Choice of Shakespeare’s Verse, which helpfully provides more than two hundred numbered quotations that I pick out of a virtual hat whenever I’m trying to crack a creative problem. I’ve also dabbled with methods associated with divination, which, as a sources of symbols for inspiring unexpected trains of thought, can be genuinely valuable tools. As I once wrote about the tarot:

It’s really a portable machine for generating patterns…It results in a temporary structure—in the form of the cards spread across the table—that can be scrutinized from various angles. At its best, it’s an externalization or extension of your own thoughts: instead of confronting the problem entirely in your own head, you’re putting a version of it down where you can see it, examine it, or even walk away from it.

But there’s one obvious resource that I’ve never been able to use to my own satisfaction: the I Ching. I’ve always been a little surprised by this, since it’s probably the most famous of all oracular texts. I’ve toyed with various translations, notably the Richard Wilhelm edition, and I had a reasonable amount of success with The Portable Dragon by R.G.H. Siu, which pairs the original hexagrams with illuminating quotations from both eastern and western sources. But the results have always left me cold, and it’s taken me a long time to figure out why. I found a helpful clue in a discussion of the subject in Joseph Needham’s Science and Civilisation in China, the legendary seven-volume masterpiece that I was recently delighted to find is available for download at Monoskop. In his section on the I Ching, which he thinks had a negative influence on the history of thought in China, Needham writes:

The elaborated symbolic system of the Book of Changes was almost from the start a mischievous handicap. It tempted those who were interested in Nature to rest in explanations which were no explanations at all. The Book of Changes was a system for pigeon-holing novelty and then doing nothing more about it. Its universal system of symbolism constituted a stupendous filing-system. It led to a stylization of concepts almost analogous to the stylizations which have in some ages occurred in art forms, and which finally prevented painters from looking at Nature at all.

And I think he’s onto something. The I Ching has a way of closing off pathways of thought—unlike the tarot, which opens them up—because it’s almost too comprehensive and organized. The tarot is a mess, but in the best possible way: the patterns it generates are necessarily incomplete, and they require a secondary act of consolidation in the user’s brain. The I Ching feels more like a card catalog. (Needham shrewdly compares it to the bureaucratic organization of much of classical Chinese society, and says: “The Book of Changes might almost be said to have constituted an organization for ‘routing ideas through the right channels to the right departments.'”) And after trying valiantly for years to incorporate it into my writing routine, I set it aside: it seemed to have some of the same freezing effect on my work that Needham identifies in Chinese culture as a whole. This is all very subjective, of course, and it clearly doesn’t apply to everyone: the I Ching played an important role in the careers of such artists as John Cage and Philip K. Dick, and I wouldn’t discourage any writer from at least trying it out. But when I relinquished it at last, it was with something like relief. The central principle of the I Ching is resonance, but for whatever reason, it just never resonated with me. And if a tool doesn’t work, it has to be put away. Because the search for randomness is too important to be left to chance.