Monthly Archives: November 2007

Social Aggregate Optimization – Society as a Neural Net

In general, individuals can be seen as optimizing a specific (very complicated) constraint function. That is, people in general desire security, health, wealth, fame, etc. and will actively work towards these goals. Now, this is interesting because not everyone is optimizing on the same constraint; however, there are a set of common traits that will on average always factor into the optimization. At the level of an individual, aspects of the constraint that do not fall within this average area are noise, and will have little or no significant impact on the overall optimization.

So we have a bunch of people essentially performing regression on some unknown but deterministic constraint. What happens when we connect them (a “social network” formed by interaction with others)?

We get a neural network, of course! Thus society is, in a very strong sense (because humans are so much better at intelligent behavior than computers are at the moment), intelligent.

Treating society as a neural net, we can extend some properties of neural nets formally to society:

1. The “Social Limit Theorem” – as more people interact and participate in a society, the society becomes capable of modeling more and more complex problems; its appearance becomes more “intelligent”. It’s merely an extension of a well-known property of a neural network, but can be rigorously proven with bias-variance decomposition. The consequence of this is overfitting and “brittle” behavior, as in a traditional neural net; the society becomes unable to adapt to new situations / patterns easily. This leads to the rather pervasive and positively deplorable social inertia that we are unfortunately exposed to on a daily basis. It is the reason an entrenched sociological philosophy, of any sort (political, economic, ethical, environmental, etc.) cannot easily change. It also explains why the ideals of one society (in effect, the pattern it has learned) do not necessarily work as well in other societies; the model does not generalize well to new problems due to the complexity of the fit.

2. Formalization of the “linking postulate” (and others among my sociological postulates) – There is a clear dependency between the overall behavior of society and the behavior of the individual nodes with high weights (influential people) because the individual variance of the optimization will be more clearly expressed as the node’s weight factors more into the overall decision of the network. This has the same type of effect on overall weight propagation as changing an influential node in an abstract neural network from linear to sigmoid would, for example.

3. If the constraint can be discovered, the overall behavior of the society could conceivably be represented as an abstract neural network (with a degree of error proportional to the overall variance from the mean, probably modeled by a normal distribution), though this may be computationally intractable due to the sheer size, number of interactions, and overall complexity of the optimization. Still, it may be possible to obtain a practical approximation.

4. This answers my previous question of how a society composed of primarily individualistic members could exhibit a fairly optimal behavior on the scale of the entire society while simultaneously fulfilling the individuals’ goals fairly well. The weights are modified as necessary for the optimization of the entire network; this optimization is performed by the individuals attempting to optimize their own goals. For example, people going to work do so to achieve financial stability and monetary gain. However, the amount of pay they receive depends on their benefit to their employer, which itself depends on the profitability of the organization, which depends on the organization’s benefit to the society. Thus, so long as society’s constraint ties local optimization to global optimization, the society will continue to progress.

There are some other consequences of this as well, but I have to get back to my dissertation.

Researcher's Golden Rule no. 2

These rules are good research conventions that I’ve adopted based on both their intuitive appeal and the observed consequences of not applying them. The first is “it always needs more study”, which refers both to the perfectionism that can keep people from ever accomplishing anything as well as the convention of stating this in papers. I only intended one, but then I realized that there are a number of unstated rules that lead to good research productivity. That said, the second can be given as follows:

“Don’t be sloppy.”

The methodology / algorithm should be clean and easy to understand. So should the way the data is formatted. Make sure that the function of each file is immediately clear and that the entry point to running the experiments is easy to spot (something like run_classification_experiments.m is a good idea). Program generically, as your dataset and analysis will probably change at some point. Don’t program only for yourself, because at some point, someone else is going to need to run your analysis. That person will not think highly of you if you make his life difficult. Don’t program unless you know how to program well; it is a vital skill in computer science research and you should be as proficient in it as a professional programmer would be.

I spent the majority of this weekend wrapping data up from over a thousand different .hdr / .img files into one matlab “data” structure. The fields of the structure correspond to properties of the data. For example:

data.Source //”DVD 1″
data.task //”Left Squeeze”
data.subject //”John Doe”
data.volume //Raw image data.
data.foregroundIndices //Indices into volume that represent foreground voxels.
data.wavelets //Wavelet descriptors of the volume.

etc.

This is neat. Any researcher just joining the project could easily follow what is going on in this structure.

Fractional Tensor Modes

Today’s random thought: tensors have an integer rank, but what would happen if we extended the notion of a rank into the entire domain of reals (or even to complex numbers)? What would it mean for a tensor to have a rank of 2.5? Would the tensor have a fractal structure? What about a tensor of rank i?

Not the sort of question I have the time to chase, but an interesting one.

Content lock-in is becoming ridiculous

So apparently I now need to hack my phone in order to transfer ringtones of my own songs that I performed to my own phone.

Verizon claims that copying ringtones on the SD card is not permitted because it would violate artists’ rights.

What artists? What rights?

I’m thinking of returning the phone.

Update: BitPim to the rescue!

Dissertation – Week 4

I’m playing catch-up from last week, due to the hefty machine learning workload I was given then. I’ve finished my 10 pages for this week, so the remainder is simply make-up work.

I’m done with the background, though I’ve just decided to add CUR and CANDECOMP to the mix. I’m performing the wavelet experiments now; with luck we will be able to apply tensors to them after this week is over. I also found a few problems with the way SVD is described in our grant proposal; I’ve made sure to avoid replicating those mistakes in my paper.

The paper is starting to get very… verbose. But I guess that’s to be expected after writing 30 pages on something that really doesn’t need it.

On the upside, I’m more than 1/5 of the way done, according to the number of pages I’ve written. Yay 🙂

Sometimes I miss being a programmer…

As I sit 30 pages deep into the maze of English, mathematics, and mathematical English (which is a language in its own right) that is my dissertation, I can’t help but reminisce about the days when I just used to code all day. It didn’t matter what I was writing; every project became a labor of love, though it was eked out in a battle for mastery against a mercilessly correct machine and the equally merciless ambiguities of the human mind. Receiving an interview feedback form from Google brought me back for a time, forced me to remember all of my victories – and defeats – as I tried to impart the thoughts that flitted through my mind at the interview.

I’ve spoken of my childhood already: of the early victory that was Metasquarer, of the elation and superlative mastery that breathed life into Final Aegis, and of the zero-sum victory in the PlanetSourceCode contest that firmly embedded a non-competition principle into my code of ethics.

My primary thoughts today did not trace over those paths so much as my more recent evolution as a programmer: the culmination of my long years of study, the final self-acknowledgment of mastery (I’m always the last one to), and the associated conclusion: it was no longer a challenge worthy of being my primary activity. The evolution of programming from the desktop to the web simply served to reinforce these concepts; “programmers” these days are more likely to use languages such as Javascript and HTML (which I still consider a markup rather than programming language) than C++ and Java. Fun as that is, that’s web development, and its practitioners tend not to understand either the elegance of – or need for – a good computer program. “Why compute squares on a board in O(n) when you can do it in O(n4) by scanning the whole board for each point?” sums this attitude up. “Computers are getting faster, so who will notice?” (well, you might if your program becomes popular and your server goes down in flames as the number of users grows). I even proposed a new paradigm that built classes bottom-up (by their behavior) instead of top-down (by their structure), which was promptly, since most people can’t see the point and prefer to work top-down (a study which I can no longer find concluded that despite top-down programming being encouraged and perceived as being more efficient, the best programmers tended to work bottom-up, which is true of the way I generally code as well, though I’ve become more amenable to top-down approaches as I’ve grown).

In the end, I just decided that I should move on from programming. So I decided to study algorithms in grad. school.

Well, fast forward through all of the application drama (the righteous indignation still hasn’t faded; it probably never will, since my entire life plan was essentially derailed and had to be rebuilt) and I am now at Temple studying biomedical data mining, and the last people I want to work with are the ones who study algorithms. I’ve never met such an unhappy yet demanding group of people in my life. Instead of focusing my efforts on programming, I am now focusing them on… well, everything, but especially research at the moment. I still code enough to keep my skills sharp, but only in support of my other activities. Coding for the sake of coding has been lost.

It’s something I miss from time to time, but it almost seems as if the world itself has moved past the need when I wasn’t looking – or perhaps I’m now content to describe the solution without expending the effort of implementation, since I know no one will bother with it anyway. Whatever the reason, I sometimes feel orphaned from the first thing I was really really good at.

I’m thinking about taking a job that primarily involves programming when I graduate. I started the doctorate with the notion that I was doing it more for the training than the degree, and I meant it, but I badly misjudged the research community and thus I now spend most of my time writing about concepts that anyone who cared could find in a textbook, just so I can present my new idea while meeting some sort of expected page limit (they call it “scope”) on my dissertation. I don’t know if I want to deal with this for the rest of my life. I love coming up with new ideas, but… there’s so much meaningless work that accompanies it! So much bureaucracy, so much conformity, even some hypocrisy… just to maintain a job that isn’t even particularly rewarding to begin with. I love research, but I can’t stand the way research is practiced, while I also love programming and can at least tolerate the way programming is practiced.

The idea of taking an easy job and doing my research independently looks more and more intriguing…

Manifold learning in AI

Manifold learning techniques such as SDE have the ability to extract data from a high dimensional space and describe it in terms of its degrees of freedom. Thus low level concepts such as “collection of pixels” become integrated into higher-level concepts such as “teapot rotated at this angle”.

In other words, this is how you teach a system abstraction. Thus, use of something of this nature may be a necessary component of an artificially intelligent system. The only problem is that current methods may be computationally infeasible for this use. Of course, approximation would be a good idea here.

"Fit", "Broader Impacts"

The world would be a much better place if everyone stopped worrying about whether ideas “fit” the purposes of their specific organization / community and simply accepted them on their perceived merit (again, my philosophy holds that the absolute merit of an idea is inestimable).

And I’d love it if I could stop having to explain how my theoretical computer science research helps every minority under the sun (but not white males; that’s taboo). First of all, it’s very difficult to explain how developing a streaming kernel PCA algorithm helps starving children in Africa. Second, my research ultimately helps everyone (by adding to knowledge, which can then be used in all sorts of ways) or no one (if nothing is ever developed on them). Which is the case depends entirely on how these ideas are used.

If these things are more important than the quality of the research, it’s no wonder the USA is losing its technical edge!