I think I understand now why Einstein worked as a patent clerk (or perhaps why he discovered what he did by working as a patent clerk, since I think his reason was simple inability to find a job). It took me a while away from my own research because it was really something I needed to solve on my own, but I think I have it now.
This is aside from the issues I’ve already identified with modern academia (“hot fields”, funding, bureaucracy, “publish-or-perish”, closed-mindedness, etc. etc. etc., ad nauseum), which I’m not going into any further here.
Academia is dominated by a single paradigm: look at problems for a very long time and eventually find a solution. This is great except for the fact that it’s a novelty seeking approach that doesn’t have any novelty itself. Everyone solves problems this way, which means everyone likely follows similar thought paths. I’m willing to bet Einstein himself was aware of this, because he himself stated that idiocy was doing the same thing and expecting a different result (Update: People attribute this quote to him, but I don’t know whether it’s actually his).
Anyway, the emphasis is always on logic, never on imagination, but imagination and creativity, not rigid deduction or calculation, are the driving forces behind the best science. Something very revolutionary seems to have more in common with a work of art or music than a mathematical proof. You can’t simply deduce something like relativity, even if you could deduce something like the true mass-energy equivalence equation e = mc^2 / sqrt(1-v^2/c^2) (the one everyone learns to recognize is just the special case of rest velocity) once the framework was in place. It says a lot that the actual framework was made by people who framed their thoughts intuitively, like Einstein and Poincare. And I think we can say it’s axiomatic that there’s always a better mathematician than you*, so however much talent you have at making those deductions, someone else will almost certainly have more.
*Unless you’re the best, in which case, carry on.
Your discoveries then become a matter of luck and precedence: can you discover something before someone else does? If you have to even ask this, you’re probably not doing truly revolutionary science. It also tends to make academics “idea misers”, zealously guarding their ideas lest someone steal them or independently arrive at the same discovery. Sometimes very revolutionary things pop up independently at around the same time as well, but if you’re always racing for credit, your only accomplishments are the instances in which you finish first.
Now, I’m not saying incremental advances are bad. The majority of progress is made incrementally. But my guess is that most scientists do not start out aspiring to mediocrity. They seem to lose their ideals and acquire a drive towards small discoveries during their training. Since this is essentially what their training requires them to do to graduate, it’s not too surprising. To be honest, it’s the lesson that I refuse to learn and it sums up a very great deal of the conflict of ideals between myself and academia. The work I do for Temple is an example of such incremental work, but it is merely a temporary compromise for the purposes of finishing my degree. My ultimate scientific goals remain unaltered.
Anyway, this is the fundamental clash between the schools of rational and intuitive thought, with academia fairly far in the rationalist camp. Einstein found a job outside of academia, however, which probably made a big difference in his discoveries. There’s nothing special about being a patent clerk. It’s probably a fairly mind-numbing job for someone of Einstein’s talent.
And I think that may be the idea. By freeing his own mind to simply wander, Einstein wandered onto something big. Actually, he wandered onto quite a few big things, because his mind was working differently. He wasn’t calculating; he was daydreaming about riding on light beams. The ideas all bubbled up to the surface in 1905, but so many revolutionary ideas don’t hit at once – they almost certainly previously existed as subconscious notions which had yet to be fully developed, and thus the “Annus Mirabilis” was probably just the year when Einstein decided to formally write down everything he had already figured out, possibly months or years earlier.
That’s not to discount academia entirely – one has to differentiate problem solving activities with actual training, and surely Einstein could not have derived his formulas knowing nothing of physics. But to equate training with rigidity is the mistake many seem to make in academia.