The next three years

No matter what your field might be, the most important factor in doing interesting work is often the selection of problems to tackle. We don’t always get to decide how we spend our time from one day to the next, but we occasionally arrive at decision points that will determine what we’ll be doing for years to come. Such moments tend to happen when we aren’t fully prepared for them, like when we have to pick a college major, and even as adults, we frequently fall back on instinct—and if some people have greater success than others, it might just be because they have better hunches. But we don’t always make such choices with the seriousness that they deserve. This might appear to go against the principle that ideas are cheap and execution is what really counts, but they aren’t as inconsistent as they seem. It’s true that there’s a big difference between having a bright idea and actually seeing it through, and that you should worry less about people stealing your ideas than about successfully bringing projects to completion. The world is full of good ideas, and if you lose out on one, there’s always another. But not every idea is equally suited for what you bring to it, and if you choose poorly, it can take you in the wrong direction for years. And it’s often the ideas that seem the most exciting at first that turn out to be the most misleading. (If I seem particularly interested in the subject right now, it’s because I’m about to deliver what I expect will effectively be the final draft of my book Astounding. The next few months will be taken up by the practical side of book publication, and I really need a break. But at some point, I’m going to have to figure out what to do next. And I’m writing this post to set down some guidelines for my future self about where to look.)

Not surprisingly, this issue gets a lot of attention in science and technology, which are fields in which the choice of subject can be crucial. In Advice for a Young Scientist, Peter Medawar has an entire chapter titled “What Should I Research?”, and he offers a good place to start:

It can be said with complete confidence that any scientist of any age who wants to make important discoveries must study important problems. Dull or piffling problems yield dull or piffling answers. It is not enough that a problem be “interesting”—almost any problem is interesting if it is studied in sufficient depth…In choosing topics for research and departments to enlist in, a young scientist must beware of following fashion. It is one thing to fall into step with a great concerted movement of thought such as molecular genetics or cellular immunology, but quite another merely to fall in love with prevailing fashion for, say, some new histochemical procedure or chemical gimmick.

In his fascinating, sometimes infuriating memoir Avoid Boring People, James D. Watson makes a similar point: “Mopping up the details after a major discovery has been made by others will not likely make you out as an important scientist. Better to leapfrog ahead of your peers by pursuing an important objective that most others feel is not for the current moment.” But he also qualifies this in a way that seems worth remembering:

I feel comfortable taking on a problem only when I believe meaningful results can come over a three-to-five-year interval. Risking your career on problems when you have any a tiny chance to see a finish line is not advisable. But if you have reason to believe you have a thirty percent chance of solving over the next two or three years a problem that most others feel is not for this decade, that’s a shot worth taking.

Watson knows what he’s talking about, but his own claim to fame—the discovery of the structure of DNA—was also due in part to luck and good timing. As Max Perutz, who won a Nobel Prize for his work on hemoglobin, recalled:

I sometimes envied Jim. My own problem took thousands of hours of hard work, measurements, calculations. I often thought that there must be some way to cut through it—that there must be, if only I could see it, an elegant solution. There wasn’t any. For Jim’s there was an elegant solution, which is what I admired. He found it partly because he never made the mistake of confusing hard work with hard thinking; he always refused to substitute one for the other.

Success, in other words, doesn’t just depend on choosing an important subject, but finding one in which you might hold an advantage. As Herbert A. Simon put it so memorably:

I advise my graduate students to pick a research problem that is important (so that it will matter if it is solved), but one for which they have a secret weapon that gives some prospect of success. Why a secret weapon? Because if the problem is important, other researchers as intelligent as my students will be trying to solve it; my students are likely to come in first only by having access to some knowledge or research methods the others do not have…In reviewing the record, I observe that I have always been pretty careful in setting the odds, and have usually behaved like an honest professional gambler…It is not unfair to have the experiences or to be at the places that provide one with a secret weapon.

Such weapons aren’t always obvious, and recognizing them can require a genius of its own. (For example, Simon writes that one of his secret weapons in the fifties was “a digital computer, and an idea—derived from contact with computers—that it would be used as a general processor of symbols,” which is hardly a trivial insight.) I’ve said elsewhere that I like to focus on areas where information is “available, but obscure,” and I often find myself thinking of an anonymous comment on a thread on Hacker News:

Find an unsexy domain that you have more access to than the average person. Start to build domain expertise in that area as quickly as you can (people are surprisingly willing to talk when you don’t want to sell them something, but just learn about how they do things)…Loop back with the people in the unsexy industry to get feedback. Remember, not all industries are bombarded with technology—you’ll need to strike a balance between showing them something sufficiently “fancy” to pique interest, and abstracting away your technology so they focus on a problem it solves…Build things in a low-cost way and use that to identify tangentially related problems until you think you’ve found a big enough pain point.

That’s basically how I wrote my book, and I’ve since come to realize how lucky I was to choose a subject that was neglected enough for me to do something useful and new, while also interesting enough to open doors. Frankly, I’m not sure if I’ll ever be able to do it again, although I’ll be thinking hard about how. I’ll make the best choice that I can. And I’ll know whether or not I was right in about three years.

Tennis and girls at Cambridge

They say all [James Watson] did in Cambridge was play tennis and chase girls. But there was a serious point to that. I sometimes envied Jim. My own problem took thousands of hours of hard work, measurements, calculations. I often thought that there must be some way to cut through it—that there must be, if only I could see it, an elegant solution. There wasn’t any. For Jim’s there was an elegant solution, which is what I admired. He found it partly because he never made the mistake of confusing hard work with hard thinking; he always refused to substitute one for the other. Of course, he had time for tennis and girls.

—Max Perutz