Coincidence and creativity

An off-hand comment of Neil Gaiman’s has shattered of my notions about the difference between science and literature. But first, a short pontification:

Science is, as any active researcher will tell you at great length, a very creative endeavor. There is a constant stream of problems — ranging from “Why do particles have mass?” to “How do I wire these three instruments together without blowing up the lab?” to “Can I word this paper to maximally piss of my rivals without having the referee make me change it?”. The solutions generally require agile and unconventional thinking. Far from the image of mindless button-pushing and number-crunching, a sizable part of even the day-to-day life is about constructing thoughts that no one has had before.

However, this creativity is also highly constrained. The question, “What is the universe made of?” has a huge range of possible answers. On the other hand, almost all of those are wrong. I could advance the theory that at the fundamental level everything is made of unicorn tears. While that may be an interesting (or possibly horrific) world, it’s not the one we live in. Scientific creativity is much more akin to that of a haiku writer than, say, a novelist.

The fact that the originality is in how one gets to a particular right answer leads to an interesting phenomenon: Quite often two (or more) people will come up with nearly identical ideas at the same time. My own personal experience came with my first solo paper. The day before I was set to publish the preprint another paper appeared on the preprint server outlining essentially the same idea. The reaction of my advisers wasn’t any sort of surprise, but a simple, “It always sucks when that happens.”

There are, of course, much more famous examples, such as Darwin receiving Alfred Russel Wallace’s manuscript on natural selection. A side-benefit of the Cold War (and there are so many) was to create two effectively isolated communities of scientists. This produced a lot of examples of similar ideas arising independently, such as the discovery of inflation by Alan Guth and Andre Linde.

The point is that, while a takes a lot of creativity to get to the right answer, the fact that there is a right answer makes it quite common for multiple people to get there. It’s slightly more mysterious why they tend to arrive at the same time, but not a lot. Often a new technique will become available, or popularized; sometime a new result raised a question that is fairly simple to answer; or possibly the question was just “in the air”.

I had thought that this sort of situation wouldn’t appear in the arts, and that it’s absence was one of the key differences between the kinds of creativity needed for the different disciplines. That somehow the space of possible novels, say, was so large that two people wouldn’t ever land on the same concept.

All of that brings me finally, to what Neil Gaiman had to say:

Sometimes I think that ideas float through the atmosphere like huge squishy pumpkins, waiting for heads to drop on. I remember back in 1989 Terry Pratchett and I plotting a novel once about a serial killer who kills serial killers, and we had most of the pieces in place, and then both of us realised we’d have to actually write it, which seemed like less fun than making it up, and so we left it.  I would have put him in the Serial Killer’s convention in Sandman, but he just didn’t fit. And I was pleased when I saw the Dexter books that that pumpkin had finally landed on the head of somebody else, who wanted to write them.  Sometimes you’re just lucky that the pumpkin lands on you first.

It’s not entirely the same as what I described. He goes on to explain

But the truth is, it’s not the idea, it’s never the idea, it’s always what you do with it. I remember Jonathan Carroll telling me to “Write it new”, when we talked about how I had thrown out a whole Sandman storyline on reading Bones of the Moon. And I’m pleased I went back and wrote A Game of You….

Different outcomes (and scientists get hit by sliderules, not pumpkins) but as a defining difference: not so much. Yet another theory goes Kaboom.