Science and civilization

Over the last week, I picked up two books—at the annual Newberry Library and Oak Park Public Library book sales, which are always a high point of my year—that I’d been hoping to find for a long time. One is a single volume, Civil Engineering and Nautics, of Joseph Needham’s landmark Science and Civilization in China, which currently consists of twenty-seven huge books that I all unreasonably hope to own one day. The other is a slim fascicle, or paperbound installment from a work in progress, from Donald Knuth’s The Art of Computer Programming, which, if we’re lucky, will release its fourth volume sometime in the next decade. These two projects are rather different in scale, but remarkably similar in their conception and incubation. Needham worked on his book for close to half a century without finishing it, and Knuth has been laboring on his for even longer, with no obvious end in sight. I’ve been intrigued by such grand projects for most of my life, but I’ve become even more interested after embarking on my own venture into nonfiction. Replace “computer programming” with “science fiction” and “discovered” with “written,” and what Knuth once said in an interview gets very close to my attitude two years ago when I started writing Astounding:

At the time, everybody I knew who could write a book summarizing what was known about computer programming had discovered quite a lot of the ideas themselves…By contrast, I hadn’t really discovered anything new by myself at that point. I was just a good writer…I had this half-conceited and half-unconceited view that I could explain it more satisfactorily than the others because of my lack of bias. I didn’t have any axes to grind but my own.

Knuth concludes: “Then, of course, as I started to write things I naturally discovered one or two new things as I went, and now I am just as biased as anybody.” Which pretty sums up my experience, too.

And what really fascinates me about both projects is how monstrously these tomes grew past their projected dimensions, both in space and in time. Both Needham and Knuth thought at first that their work would fit within a single volume, and although they each expanded it to the magic number of seven, neither seems to have grasped just how long it would take. Knuth recalls:

My original motivation was to write a text about how to write compilers, so I began drafting chapters. I was seriously planning to finish the book before my son was born…In June 1965, I had finally finished the first draft of the twelve chapters. It amounted to three thousand handwritten pages of manuscript…I figured about five pages of my handwriting would be about one page of a book.

As it turned out, he was a little off: the real proportion was one and a half handwritten pages to a single page in type, which meant that he had already written two thousand pages without even getting past the subject of compilers. Needham had a similar moment of clarity. As Simon Winchester writes in his biography The Man Who Loved China:

Needham had decreed early on in the process, as he watched each volume begin to swell and threaten to burst out of its covers, that no one volume should be “too big for a man to read comfortably in his bath.” But it was happening nonetheless…One book became two, three, or four. Volume V, a special case, became not five, but thirteen formal subsidiary parts, each one of them big and complicated enough to be made into a separate, self-standing, and equally enormous new volume of its own.

It’s frankly hard to imagine reading any of these expensive books in the tub, but Needham says elsewhere, more realistically, that critics found the volumes “too heavy and bulky for meditative evening reading,” which led to the work being repeatedly subdivided.

The Art of Computer Programming was released by a commercial educational publisher, Addison-Wesley, but it isn’t surprising that most such books tend to appear at academic presses, which are the only institutions capable of sustaining a project that lasts for decades. (Their sole competition here might be the Catholic Church, which has been underwriting a critical edition of the works of St. Thomas Aquinas since 1879. They’re about halfway through.) Winchester refers in passing to “the beleaguered Cambridge University Press, which was obliged to tolerate the constant expansion of the project,” and for the full picture, you can turn to the book A Skeptic Among Scholars, by August Frugé, the director of the University of California Press. He writes of The Plan of St. Gall, a three-volume monument of scholarship that is probably the most beautiful book I own:

The St. Gall manuscript…was said in 1960 to be in semifinal draft, about one hundred and fifty pages in length. When approved by the Editorial Committee and accepted by me in 1967, it came to several hundred typed pages, about right for a single quarto volume. As the work moved through the production process during the next twelve years, we paused every now and then to call for new estimates of size and cost, and each time discovered that new sections had been added, along with a few dozen new diagrams and drawings.

At one point, concerns about cost threatened to derail the whole enterprise, and Frugé retired before the three huge folios were published. James H. Clark, his successor, saw it to completion, writing later: “But what is a university press for if not to take these kinds of risks, make these investments, and publish books that make a difference?” Aside from Knuth, one of the few comparable examples on the commercial side must be Robert Caro’s The Years of Lyndon Johnson, which was originally planned as three volumes to be written over about six years. Forty years and four books later, Caro still isn’t done, and the fact that he has been allowed to keep working at the same methodical pace is a tribute to his longtime editor Robert Gottlieb at Knopf.

And if there’s one key takeaway from the examples I’ve mentioned, it’s that none of these authors set out to devote their lives to these projects—they all thought at first that they could finish it within a few years. Knuth recalls:

It gradually dawned on me how large a project this was going to be. If I had realized that at the beginning, I wouldn’t have been foolish enough to start; I wouldn’t have dared to tackle such a thing…[But] I had collected so much material that I felt it was my duty to continue with the project even though it would take a lot longer than I had originally expected.

You also realize that you can’t explain the subject at hand without covering a lot of other material first. Caro treats his books on Johnson as windows onto such vistas as local politics, Texas, and the Senate, which is a big part of their appeal. (The equivalent in my case would be deciding that I couldn’t write the life of John W. Campbell in a comprehensible form without telling the entire history of science fiction, too, which might well be true.) Frugé, perhaps to his credit, ventures a more cynical reading:

In my skeptical and perhaps scatterbrained way, I sometimes wonder how a research scholar can work on the same project decade after decade and retain faith in its intellectual importance. Perhaps some do not, and that is why their books are never completed. But we can also observe an opposite phenomenon. As the years go by the object or document for study may swell and expand in importance until—until, for example, “The Plan of St. Gall is…one of the most fascinating creations of the human mind.”

He makes a good point. The cycle feeds on itself, with the work expanding in scope to justify the amount of time it takes. It’s human nature, and there’s something a little absurd about it. But it’s also the only way we get art, science, or civilization.