Category Archives: Research

Classification of population density by satellite images

People have sort of brushed on this idea before at http://www2.cr.chiba-u.jp/symp2005/documents/Postersession/p002_Ketutwikantika_paper.pdf, but it doesn’t look like there’s any significant progress towards actually having done it yet. I bet you can use vector quantization to do this – feed in some urban, suburban, and rural areas, extract texture descriptors using VQ, and feed them into a classifier.

This is one I’m going to pursue. The barrier to entry is very low and fits nicely into my area of expertise.

Lorentzian clustering

Gravitational clustering in machine learning, a previous idea of mine, has been done before, but it seems that you can also cluster points based on the Lorentz transform spacetime undergoes in general relativity. I’m wondering whether a data clustering algorithm based on the stress-energy tensor (simulated at relativistic speeds, of course) would be feasible. My dissertation is on using tensors in data mining, so it could be a useful example.

Academic publishers, be afraid…

The academic publishing model has always been a strange one: scientists publish articles for free, journals insist on taking the copyrights from these authors, and then have the audacity to charge viewers for access to work they had no part in creating!

This seemed so wrong to me that I started writing a long essay on the problem back in 2004, which I never finished because the open access movement started to take hold soon after (so there was no need).

Despite the growing popularity of the open-access movement, many journals still remain closed. Well, what happened to music seems about to happen to academic publishing.

The Pirate Bay is launching a new site called The Student Bay. I can’t read Swedish, but my guess is that it will trade academic material, copyrighted or otherwise.

Now, I don’t condone piracy in and of itself. However, I do embrace making learning material universally accessible, and I view the universal right to learn as a higher right in my moral hierarchy than copyright. I’m not simply saying this from the student’s end, either: I have plenty of IP of my own, which I’ve always given away freely. This includes my academic papers, which I’ve usually posted on my own site for people to read following publication.

Thus, it shouldn’t be surprising that I view this as a very positive development. I suspect that authors themselves will upload their works here; certainly I am considering it. And that is going to make things very hard for journals, because there’s a good chance that the first lawsuit brought against an author for infringement on his own work will provoke a sweeping reform of the system itself. This would be suicide for the journals.

They’re parasites. I can’t say I’d be sad to see them go.

Some financial risk measures simplified.

I just waded through a bunch of papers on the STARR and Rachev risk ratios. These seem to have vastly overcomplicated what you actually need to do, which is unfortunately typical of many mathematical papers (it’s a consequence of the logicians outnumbering the intuitives). I think I finally figured out what they were trying to say, and it turned out to be simple. I might still be wrong about this (after all, it was only an hour or so of deciphering), but here’s how I think the concepts can be summed up easily and intuitively for someone who understands a bit of statistics:

First, get a distribution of excess return by subtracting your returns from the value of a risk-free investment.

Value at risk: Find the qth percentile/quantile of the return; that is, q% of the time, you’ll make less than the value you find. If you assume a normal distribution, you’re just finding the z-score from the p-value q (hint: the Excel NormInv function will do this for you, or you can grab a normal table if you’re old-fashioned). If q = 0.05, that’s 1.96 standard deviations below the mean using a two-tailed test (one-tailed, it’s a bit less).

Conditional value at risk/Expected shortfall/Expected tail loss: Average of everything in the distribution where returns fall below the value at risk. Since the value at risk represents bad things with an unlikely probability (q%) of occurrence, this is the average of all of the really bad, really unlikely things that could happen to your portfolio. Oh, and it’s a loss, so if you’re dealing with a distribution of returns, you’ll want to negate the result.

STARR ratio: Excess returns over expected tail loss. It seems to measure how much you typically make vs. how much you can possibly lose. Average over worst case.

Rachev ratio: Tail loss of losses (just negate the returns distribution!) over tail loss of returns. Loss of losses (let’s call it “gain”) is a good thing, so the numerator represents what you can gain in the best q% of cases, while the denominator represents how much you can lose in the worst q%. Essentially, you’re judging the best possible case against the worst.

When you get down to it, the intuition behind the concepts is simple. It’s just dressed up funny. Unless I’m totally wrong about this 🙂

You can probably use these sorts of ratios in other fields as well, particularly when the terms “best case”, “average case”, and “worst case” have meaning, such as in the analysis of algorithms.

Some mathematical thoughts…

For some reason, fixed points in multiplicative functions fascinate me. I was thinking about one of my favorite functions, the divisor function (sigma), and figured out an interesting minor mathematical tidbit, which I’m recording now so I don’t forget it.

If x is an (even) perfect number, x = 2m-1 * (2m – 1), where m is a Mersenne prime.
σ(x) = 2x by definition, but σ(σ(x)), that is, σ(2x) = 2m * (2m+1 – 1).

"Putty" classifiers

I’ve had this idea for a while, but was constantly debating in my mind whether it was the same thing as an SVM with a kernel. I finally came to the conclusion that it wasn’t:

Start with a hyperplane boundary and then split it into warp/deformation points. Optimize the warping by minimizing MSE using gradient descent (or something) and it should begin to take on the form of the points. Impose some sort of regularization to prevent overfitting. Since warping can modify the plane in three dimensions, it’s no longer a hyperplane, but neither is it equivalent to passing a hyperplane through a kernel.

It should be a very powerful means of performing regression (and thus classification). I might research it later; I have too much on my plate now.

4 for 4

The IWDM paper has been accepted, which brings my total count for this round up to 4. There’s no way the fifth paper we submitted to ICIP will be accepted (the results are terrible and we know it), so we’ll just end it at that 🙂

3 for 3

I submitted 5 papers to conferences last month. I’ve heard back on 3 of them, all of which were accepted to their respective conferences. I should hear back on one more today, which I think stands a good chance. I’ll be extremely surprised if the fifth is published, however, since it’s not a good paper.

I still think the peer review system is broken (and will continue to do so irrespective of what fate my own papers meet).

Rationalism vs. Intuition in the Context of Academia

I think I understand now why Einstein worked as a patent clerk (or perhaps why he discovered what he did by working as a patent clerk, since I think his reason was simple inability to find a job). It took me a while away from my own research because it was really something I needed to solve on my own, but I think I have it now.

This is aside from the issues I’ve already identified with modern academia (“hot fields”, funding, bureaucracy, “publish-or-perish”, closed-mindedness, etc. etc. etc., ad nauseum), which I’m not going into any further here.

Academia is dominated by a single paradigm: look at problems for a very long time and eventually find a solution. This is great except for the fact that it’s a novelty seeking approach that doesn’t have any novelty itself. Everyone solves problems this way, which means everyone likely follows similar thought paths. I’m willing to bet Einstein himself was aware of this, because he himself stated that idiocy was doing the same thing and expecting a different result (Update: People attribute this quote to him, but I don’t know whether it’s actually his).

Anyway, the emphasis is always on logic, never on imagination, but imagination and creativity, not rigid deduction or calculation, are the driving forces behind the best science. Something very revolutionary seems to have more in common with a work of art or music than a mathematical proof. You can’t simply deduce something like relativity, even if you could deduce something like the true mass-energy equivalence equation e = mc^2 / sqrt(1-v^2/c^2) (the one everyone learns to recognize is just the special case of rest velocity) once the framework was in place. It says a lot that the actual framework was made by people who framed their thoughts intuitively, like Einstein and Poincare. And I think we can say it’s axiomatic that there’s always a better mathematician than you*, so however much talent you have at making those deductions, someone else will almost certainly have more.

*Unless you’re the best, in which case, carry on.

Your discoveries then become a matter of luck and precedence: can you discover something before someone else does? If you have to even ask this, you’re probably not doing truly revolutionary science. It also tends to make academics “idea misers”, zealously guarding their ideas lest someone steal them or independently arrive at the same discovery. Sometimes very revolutionary things pop up independently at around the same time as well, but if you’re always racing for credit, your only accomplishments are the instances in which you finish first.

Now, I’m not saying incremental advances are bad. The majority of progress is made incrementally. But my guess is that most scientists do not start out aspiring to mediocrity. They seem to lose their ideals and acquire a drive towards small discoveries during their training. Since this is essentially what their training requires them to do to graduate, it’s not too surprising. To be honest, it’s the lesson that I refuse to learn and it sums up a very great deal of the conflict of ideals between myself and academia. The work I do for Temple is an example of such incremental work, but it is merely a temporary compromise for the purposes of finishing my degree. My ultimate scientific goals remain unaltered.

Anyway, this is the fundamental clash between the schools of rational and intuitive thought, with academia fairly far in the rationalist camp. Einstein found a job outside of academia, however, which probably made a big difference in his discoveries. There’s nothing special about being a patent clerk. It’s probably a fairly mind-numbing job for someone of Einstein’s talent.

And I think that may be the idea. By freeing his own mind to simply wander, Einstein wandered onto something big. Actually, he wandered onto quite a few big things, because his mind was working differently. He wasn’t calculating; he was daydreaming about riding on light beams. The ideas all bubbled up to the surface in 1905, but so many revolutionary ideas don’t hit at once – they almost certainly previously existed as subconscious notions which had yet to be fully developed, and thus the “Annus Mirabilis” was probably just the year when Einstein decided to formally write down everything he had already figured out, possibly months or years earlier.

That’s not to discount academia entirely – one has to differentiate problem solving activities with actual training, and surely Einstein could not have derived his formulas knowing nothing of physics. But to equate training with rigidity is the mistake many seem to make in academia.