The index fund

When the time comes to prepare the index for a nonfiction book, there are basically two schools of thought on how to proceed. One is that the author is the only person qualified to perform this particular task. You see this view expressed at its most eloquent by Douglas R. Hofstadter, who reveals in a long endnote in Le Ton Beau de Marot that completing the index for that book required him to work fifteen hours a day for an entire month. He explains:

My feeling is that only the author (and certainly not a computer program) can do this job well. Only the author, looking at a given page, sees all the way to the bottom of the pool of ideas of which the words are the mere surface, and only the author can answer the question, “What am I really talking about here, in this paragraph, this page, this section, this chapter?” To answer those questions takes total understanding of the book.

Hofstadter adds that going through the book one last time awakened him to deeper themes and concepts that he hadn’t known were there, including “conflation,” “colliding cultures,” and “Chopin.” He concludes: “Once the index was essentially done…I found it interesting to flip through it and, by comparing the sheer sizes of various entries, to get new perceptions of what my book is most centrally about.” At a point at which a writer might be expected to have looked at a manuscript from every angle, an index can be a fund of new insights.

Another vote in favor of the author comes from Isaac Asimov. For his first nonfiction book, the textbook Biochemistry and Human Metabolism, he unquestioningly prepared the index himself, despite having only “a vague idea of how it should be done.” He enjoyed the job—which consisted mostly of preparing a mountain of index cards, alphabetizing them, and typing up the result—and was annoyed by what he saw as a “more cavalier attitude toward indexing” among his collaborators. For the rest of the career, he aways insisted on doing his own indexes, and when A Short History of Biology was indexed without his knowledge, he wasn’t pleased:

I looked over the index, which had, presumably, been professionally prepared, to see if I could learn lessons in technique. I quickly found that the only lesson I could learn would be on the method of preparing a thoroughly inadequate index. Half the names in the book were not included. A number of subjects were not mentioned.

Asimov concluded that the index was “insupportable,” and after that, he was careful to make his preferences known to his editors: “It added just one more time-wasting task to the list. I had to see it that no publisher, either through ignorance or through forgetfulness, ever allowed a “professional” to prepare my indexes.”

Of course, there’s also a strong case to be made for the opposite point of view, which Asimov recalled hearing from Dick DeHaan, one of his editors at Basic Books: “I tried to explain that I liked indexing, but he kept saying that no writer could approach his own book with sufficient detachment to do a good index.” Asimov eventually acquiesced for The New Intelligent Man’s Guide to Science, and the outcome left him predictably outraged:

It was dreadful; simply dreadful. It left out a great variety of things that should have been put in. It was the slapdash job of someone working for money instead of for his own book, and never again was I fooled by any talk of expertise in indexing. When I later discovered that I had been charged five hundred dollars against royalties for the privilege of having that rotten index made, I was ready to choke DeHaan.

Yet you could also argue that this detachment is necessary, a perspective most famously expressed by Kurt Vonnegut in Cat’s Cradle, which includes a chapter titled “Never Index Your Own Book.” It features a former professional indexer who informs the narrator that “indexing was a thing that only the most amateurish author undertook to do for his own book.” She continues: “I’m always embarrassed when I see an index an author has made of his own work…It’s a revealing thing, an author’s index of his own work. It’s a shameless exhibition—to the trained eye.”

Speaking from a position of minimal experience, I’d suggest that the best approach is to split the difference, and to have an outside indexer make the first pass, after which the author is given the chance to make modest additions and corrections. I’m currently in the process of doing this for Astounding, and it certainly satisfies me. (I once planned to do it all on my own, like Asimov, but I decided to let somebody else handle it, despite the fact that the cost would be taken out of my advance. This was partially because I liked the idea of a third party going through the book with an objective eye, and also because nobody at my publisher seemed to have even considered the possibility that I would want to do it myself.) The index that they’ve provided is a nice piece of work, and although I’ve caught a few errors and omissions, I’m glad that I left it to a professional. This is the last major task that remains in the writing of a book that has taken up three years of my life, and seeing it through the eyes of an ideally attentive reader—which is what an indexer should be—allows me to engage for hours on end in what Hofstadter calls “a very curious activity, and perhaps overly introspective in some people’s eyes, but irresistible for at least a little while.” It’s as close as I’ll ever get to reading this book for the first time, and although my engagement with this index wasn’t as intensive or prolonged as his was, I can only echo Hofstadter’s conclusion: “Doing this index, painful though it was, afforded me one last pass back through the text, tying things together for a final time, saying goodbye to a work created out of love, and with love, for words, ideas, people.”

The art of the index

Earlier this week, as planned, I finished the bulk of the background reading for my book Astounding. I’m far from done with the research process: there are still unanswered questions, gaps that need to be filled, and mysteries that I’m not sure I’ll ever be able to solve. But I have a sense of the territory. I knew going in that I had to cover an immense amount of raw material in a limited amount of time, and from the beginning, I was forced to prioritize and triage based on what I thought would actually end up in the book—which doesn’t mean that there wasn’t still a lot of it. It included all of John W. Campbell’s published novels and stories; something like fifteen thousand pages of unedited correspondence; forty years of back issues of Astounding, Unknown, and Analog; and numerous secondary sources, including interviews, memoirs, and critical studies. I had to do much the same thing with Asimov, Heinlein, and Hubbard, too, but with an important difference: I’m not the first biographer to tackle their lives, so a lot of the structural work had already been done, and I could make educated guesses about what parts would be the most relevant. When it comes to Campbell, however, enormous swaths of his life have never been explored, so I had no choice but to read everything. In the words of editor Alan Hathaway, which I never tire of quoting, I’ve tried to turn every goddamn page. Whenever I see something that might be useful, I make a note of it, trusting that I’ll be able to find it again when I go back to review that section at greater length. Then I have no choice but to move on.

And it’s only recently that I realized that what I’ve been doing, in essence, is preparing an index. We tend to think of indexes as standard features of nonfiction books, and we get annoyed when they aren’t there. (I note with interest that a different John Campbell—a British politician of the nineteenth century, and apparently no relation to mine—proposed that authors who failed to provide an index would be fined and deprived of their copyrights.) In fact, indexes originated as working tools that scholars prepared for themselves, and they tailored them for their individual needs. What I find useful in a book may not interest anybody else, especially if I’m reading with a specific problem in mind, which is why it makes sense for readers to maintain indexes of their own. As Harold Nicholson, another British politician, once said in a commencement speech:

My advice is to go to France, direct from New York to Cherbourg, and to remain there for at least three months, if possible living in a French family. My second piece of advice is always to mark your books and write a personal index for yourself on the flyleaf.

He’s right, of course, and I’ve been doing this for years without thinking about it. Now I’ve started to do it more deliberately, and I’ve gotten into the habit of transcribing those notes into a searchable text file, as an index of indexes that I can use to consolidate my entries and keep the whole mess under control.

It’s hard to write about indexes without thinking of a famous chapter in Kurt Vonnegut’s Cat’s Cradle, which is titled “Never Index Your Own Book.” As a professional indexer says to the narrator, evaluating another writer’s index:

“Flattering to the author, insulting to the reader,” she said. “In a hyphenated word,” she observed, with the shrewd amiability of an expert, “‘self-indulgent.’ I’m always embarrassed when I see an index an author has made of his own work…It’s a revealing thing, an author’s index of his own work…It’s a shameless exhibition—to the trained eye.”

I read this passage again recently with greater attention than usual, because the odds are pretty good that I’m going to end up indexing Astounding myself. (Here’s a tidbit that you might not know: if a publisher wants an index, the author has the right to prepare it, but if he declines—or does an unsatisfactory job—the publisher can hire someone else. The cost is deducted from the author’s advance, which means that there’s a decent financial incentive for writers to do the job themselves.) I’m also uncomfortably aware that Vonnegut is correct in saying that you can tell a lot about an author from his index. For an example that’s very close to home, I don’t need to look any further than William H. Patterson’s two-volume biography of Heinlein. Its index tells you a lot about Patterson himself, or at least about how he saw his subject, and I don’t have any doubt that my index will reflect on me.

But I also don’t think that anyone but the author has any business preparing the index. I’ve spent the last eight months compiling an index for a book that doesn’t exist: the unimaginable book that would include all the known details of Campbell’s life in their original form. (If you want to get really deep, you could say that a biography is the index of the man.) It bears the same relation to its sources that a graphical projection does to the original object: it translates it to a two-dimensional surface, losing some of its properties, but becoming considerably more manageable. The reason I’ve put it together, aside from reminding me of where various facts can be found, is to produce a rough sketch of the whole that I can review in its entirety. It condenses the available material into a form that I can reread over a relatively short period of time, which allows for the iterative review process that tells you what a book is really about. As John McPhee said of his notes to The Paris Review: “I read them until they’re coming out my ears.” And this is only possible if you’ve boiled them down to a set of labels. The author is the only one who can decipher it: it’s a coded message he writes to his future self. But when the time comes to prepare an index for the general reader, it invisibly reflects that ideal index that nobody else will ever see. Only the author, who knows both the words on the page and the unseen words that made them possible, can make it. You can sense this in the indexes for books as different as Sir Richard Francis Burton’s Arabian Nights or Douglas R. Hofstadter’s Le Ton Beau de Marot. These indexes live. They tell you a lot—maybe too much—about the author. But that’s exactly as it should be. 

A marginal confession

Recently, I made a surprising discovery about myself: I’m less likely to buy a book that has been typeset with a ragged right margin. Over the weekend, I went to the winter sale at the wonderful Open Books store here in Chicago, and while I picked up a few nice discoveries—The Dymaxion World of Buckminster Fuller, Field Notes in Science and Nature, The Genius of the System—I also passed on a couple of promising books because I didn’t like the way they were laid out. (For the curious, these were Leonard Bernstein’s The Unanswered Question, a collection of his lectures at Harvard, and David Reck’s Music of the Whole Earth.) The price wasn’t an issue; they would have been just a few dollars each. And while I’m consciously trying to cut down on my book purchases, simply because I’m running out of space, I suspect I would have bought them both if their margins had only been justified. This isn’t an instance of the larger principle, which I still think is true, that shoddiness in design and typesetting is a sign that other compromises have been made on the editorial side; margins and all, these were handsome volumes. It’s a sign of a deeper, more idiosyncratic need on my part to read books that present themselves to me in a symmetrical column of text, and it means that I routinely judge books, not by their covers, but by their margins.

And it’s been an factor in my life for some time, both in my own writing and in reading the works of others. Early in my freshman year at college, I found myself obsessively writing my essays so that the margins came out neat on both sides. At the time, I was using a version of Word that had relatively primitive justification and hyphenation settings, so my only option was to rewrite the text itself, altering words here and there so that the margins were even. (I also liked a slightly tapering shape at the top of each paragraph, as the examples posted here illustrate.) Early on, I wrote my essays in monospaced 12-point Courier, which meant that not only did the lines need to be aligned to the naked eye, but they had to contain exactly the same number of characters, the occasional dangling comma or period aside. In my senior year, I switched over to Times New Roman, a proportional font, which made things easier, and I’ve been using it ever since. But my marginal obsession still remains, if in a somewhat attenuated form. I still justify and hyphenate all my own manuscripts—although I remove the hyphenation before they go out to readers—and I continue to revise the text if the spacing on a line seems loose. And if you’ve ever noticed that most of the paragraphs on this blog are roughly the same size and shape, with the right margin only slightly ragged, well, that isn’t an accident.

This naturally raises the question of why I go through all this trouble, especially for works that are eventually going to be published in a form that I can’t control. And I don’t really have a good answer. Writers, by nature, are obsessive creatures who have been known, as Norman Mailer once was, to devote an entire working day to changing a period to a comma and back again, and it shouldn’t be surprising that they’d be equally finicky about how their work appears on the screen or the page. Anecdotally, there’s a lot of evidence that writers who format their own work for publication fiddle with the wording in similar ways. In Le Ton Beau de Marot, Douglas Hofstadter writes:

I can clearly see the spacing as I type on my screen, and I rewrite and rewrite in order to make sure that no line is too tightly or too loosely spaced. In the course of such rewritings—here extracting a word, there using a shorter or a longer one, elsewhere inserting a word where none was—words and phrases that I would otherwise not have thought of pop to mind, suggesting ideas I would not have thought of, and those ideas suggest unexpected paragraphs, and those paragraphs are in turn linked to other ones, and so on…

Hofstadter’s story, incidentally, raises the question of why he didn’t just use hyphenation to deal with loose lines, since there isn’t a single instance of it in the entire book—I’ve always wanted to ask him about this. More recently, the graphic designer Chip Kidd wrote his novel The Cheese Monkeys in Quark, allowing him to revise it for formatting purposes as he went along. (When he told Thomas Harris about this, Harris is alleged to have replied: “I wish I could do that!”)

As a matter of fact, there’s one category of authors for whom these issues are of huge practical importance: screenwriters, who are essentially formatting their own work for the skeptical eyes of producers or studio readers. Not surprisingly, they’re all obsessed by margins, line spacing, and avoiding widows and orphans, often a way to fudge the page count, but also as a reflection of something larger. As Terry Rossio observes:

In retrospect, my dedication—or my obsession—toward getting the script to look exactly the way it should, no matter how long it took—that’s an example of the sort of focus one needs to make it in this industry…If you find yourself with this sort of obsessive behavior—like coming up with inventive ways to cheat the page count!—then, I think, you’ve got the right kind of attitude to make it in Hollywood.

And I sort of believe this. Deep down, I’d like to think that my obsession with margins has made me a better writer, not just because it reflects my meticulousness in other ways, but because of the discipline it enforces. As Hofstadter points out, keeping an eye on the physical appearance of your manuscript is a source of self-composed constraints, and reworking the text in this light isn’t all that different from making the lines of a poem fit a complicated form, like a sonnet or villanelle. (I almost wrote “like a sonnet or sestina,” but the line spacing ended up looking a little weird, so I changed it.)

The golden braid of Douglas Hofstadter

When I was growing up, if needed something new to read, I’d just head for the garage. My parents owned hundreds, possibly thousands of books, and there were never enough shelves for them all, so the same dozen cardboard boxes followed us from house to house, rarely, if ever, being unpacked. (Some of them are still there, untouched, after twenty years, and a visit to my parents’ house isn’t complete before I’ve had a chance to go through them yet again.) Rummaging through these boxes was like browsing through a great, if eclectic, used bookshop, and the quality of serendipity I love in such stores was multiplied tenfold—I just never knew what I was going to find. Quite a few of those discoveries have probably ended up on my own shelves, absorbed by now into the rest of my library, to the point where I no longer remember where they came from. And my inner life has been enormously shaped by the authors I found there, which only serves to illustrate the point that if there are books anywhere in a house, a true reader will always find them, like a junkie in search of a fix.

One book in particular sticks in my mind, if only because it influenced so much of what came afterward. When I was in seventh grade, my father was browsing in a carton of books—I can’t remember why—and came up with a copy of Douglas R. Hofstadter’s Metamagical Themes, which he said I might like, mostly because of Hofstader’s discussion of the Rubik’s Cube. I liked those chapters a lot, but loved the rest of the book even more, and it’s followed me on every move I’ve made since—I’m looking at my original copy as I write this. It’s pretty worn and tattered by now, and just leafing through it takes me back, as much as any book I own, to the period of ferocious reading that I wrote about yesterday. Metamagical Themas, a collection of Hofstadter’s columns for Scientific American, led me inevitably to Gödel, Escher, Bach: An Eternal Golden Braid and The Mind’s I, and what I found there dazzled me so much that I ended up dedicating my school project that year—an autobiography, printed on dot matrix paper, that ran a hundred pages or more—to Hofstadter himself.

Who was this guy, anyway? Then as now, Hofstadter was a professor of cognitive science at Indiana University in Bloomington, and his work was my first exposure to a kind of writing that I’ve since come to love passionately: the eccentric, unabashedly nerdy attempt to fuse science and the humanities into something that isn’t quite either, but draws on the best qualities of both. Even now, I don’t think I’ve quite made it through every page of Gödel, Escher, Bach, but what I found there, and in Hofstadter’s other work, has stuck with me ever since. Among other things, he was my first introduction to Zen, self-reference, the Codex Seraphianus, the Skeptical Inquirer, Alan Turing, Magritte, Nabokov’s notes on Eugene Onegin, James Falen’s translation of the same, and countless other authors and concepts I’ve been mulling over ever since, not to mention the larger subjects of consciousness and artificial intelligence. The range of his references is so rich, in fact, that he was later compelled to write another—and somewhat less interesting—book, I Am a Strange Loop, to clarify what he was trying to say in the first place.

I discovered Hofstadter in the same year as Umberto Eco, and they’ve acted on my life in similar ways, one on the side of science, the other of literature. (From an intellectual standpoint, it’s likely that ninety percent of what I care about as an adult was formed in middle school, although those aren’t exactly years I’d like to revisit.) Both are polymaths who opened me up to surprising influences and countless other books, and if my decision to major in classics in college was ultimately due to Eco, I’d be lying if I said I didn’t take also a long hard look at cognitive neuroscience. Since then, I’ve become more aware of Hofstader’s limitations—his own translation of Eugene Onegin is a misguided vanity project of the worst kind—but I remain in awe of his brilliance and intellectual omnivorousness. The book of his I treasure the most is Le Ton beau de Marot, which came out when I was a college freshman, leading to many late evenings in my dorm, with my roommate and I trading rival translations of “Ma Mignonne.” There are other writers I’ve come to love more, but few who fill me with such gratitude. If you haven’t read his stuff, you might want to give it a try—he might change your life, too.

Thinking in pictures

Last weekend, at the Printers Row Lit Fest in Chicago, I picked up a copy of Life: A User’s Manual by Georges Perec, a novel I’d been meaning to read for a long time. I’d been interested in Perec ever since reading about his work in Douglas Hofstadter’s Le Ton Beau de Marot, and while I’ve only begun dipping into Life, I’m already intrigued by the riches on display. As described in greater detail here, Life is an ambitious experimental novel, centered on a fictional apartment block in Paris, that Perec constructed using a system designed to generate a random list of items (an activity, a position of the body, a writer, even the number of pages) for each chapter, which he then had to incorporate into the narrative. The result, as Perec put it, is a “machine for inspiring stories.” Even apart from the merits of the novel itself, I find this premise tremendously exciting.

Regular readers of this blog know that one of my ongoing obsessions is finding new ways to insert randomness and constraints into the writing process. Writing a novel, at least as I tend to approach it, is such a left-brained activity that it’s necessary to create opportunities for the right brain to participate. Sometimes this happens by accident—while shaving, for example. But there are also ways of approaching randomness more deliberately. I’ve published stories based on juxtapositions of two unrelated articles from science magazines, used random selections from Shakespeare and the I Ching to guide chapters (although I’ve mostly dropped the latter, despite the fun of throwing the coins), and used mind maps to bind all these elements together. And I’m looking forward to applying some of Perec’s techniques to my own work, although probably in a much more limited sense.

Recently, I’ve also discovered another approach that might prove useful. In Origins of Genius (which, in case you haven’t noticed already, is one of the most stimulating books on creativity I’ve read in a long time), Dean Simonton describes a fascinating experiment by psychiatrist Albert Rothenberg:

[Rothenberg] and a colleague began by making up a set of visual stimuli that involved the superimposition of visual images. For example, one contained a photograph of an empty French four-poster bed placed in a period room superimposed over a group of soldiers in combat who were taking cover behind a tank. These highly incongruous homospatial images were then shown to writers and to artists, the latter including individuals selected in a national competition by faculty at the Yale School of Art. The writers were instructed to create new metaphors inspired by the stimuli, while the artists were instructed to make pastel drawings. In comparison with the control group (e.g., subjects who saw the images only separately), individuals exposed to these visual juxtapositions of unrelated images generated more creative products, as judged by independent raters.

In other words, juxtapositions of two unrelated concepts often result in ideas that would not have arisen from considering the two concepts separately, which only confirms one of my most basic convictions about the creative process.

What I find particularly interesting about Rothenberg’s experiment, though, is that the stimuli consisted of images, rather than words, which seems like an especially promising way of encouraging nonverbal, creative thought. With that in mind, I’ve started to incorporate a similar method into my own work, using images randomly chosen from three books that seem ideally suited for such an approach: Phaidon’s chaming little volumes The Art Book, The Photo Book, and The 20th Century Art Book. Each book consists of representative works by five hundred artists, one work to a page, arranged in alphabetical order—an arbitrary system that already lends itself to startling juxtapositions. For instance, in The Photo Book, by an accident of the alphabet, “A Sea of Steps” by Frederick H. Evans appears across from “Washroom in the Dog Run” by Walker Evans, exposing their haunting visual similarities. Two images, taken together, yielding a meaning that neither would have apart—that’s what art is all about, and why I’m looking forward to thinking more with pictures.