Category Archives: Sociology

Four-Stage Development of Technical Ecosystems

Ecosystems appear to progress through four stages as they grow and mature:

(Note: I think the research ecosystem follows this as well…)

1. First come a handful of pioneers to the ecosystem – the ones who show everyone that something is possible or popular. These companies either fail or become entrenched market leaders – there is no middle ground, as the winner will become synonymous with the market. It is possible for a company to “win” a small market but fail to grow it into a new paradigm – in that case, the market will remain relatively stagnant until a pioneer with a larger vision or better ability to execute grows it. At least one pioneer must achieve both a strong market presence and a solid trajectory for the ecosystem to achieve the wild growth and optimism characteristic of the next stage.

These companies are distinguished by the novelty of their markets, and by their brand’s synonymous association with those markets. Their primary form of innovation is opening up and capturing completely new market segments.

Friendster, MySpace, Facebook, Groupon, and YouTube are examples of these types of companies. So are RIM and Palm, which captured the early mobile market but were unable to grow mobile beyond a niche product (ultimately, I categorize these as failures, and Apple as the first successful pioneer in this market).

2. Next come the clones. The successes of the pioneers spawn explosive growth, both general-purpose “me too” offerings and niche-targeted variations. These will very rarely displace the entrenched players; more often they become secondary offerings with lesser brand value, leading to fragmentation of the market. Some niche players, such as LinkedIn, can become surprisingly large – often this is the case when the leader’s niche poorly overlaps with the newcomer’s. It’s common for established players in adjacent markets to see the growth of the new area, and enter with their own offerings at this stage.

These companies are primarily valued for their brand recognition by consumers (since the space is becoming increasingly crowded at this point). They innovate by addressing specific market needs which were suboptimally fulfilled by market leaders, as well as leveraging their existing audience to gain traction in the new space.

Amazon Local, Gilt Groupe, Google+, LinkedIn, and Etsy are examples.

3. So many clones establish themselves that there’s now market value in aggregating/centralizing the results. So market plays at unifying the entire ecosystem are next to emerge on the scene.

These companies are distinguished by the comprehensiveness of their feeds or offerings. Their primary form of innovation lies in establishing the partnerships and integrations necessary to unify an at-this-point extremely diverse ecosystem. Occasionally one will emerge with a value-add enabled by the integrated nature of the data.

This focus on comprehensiveness typically leaves aggregators little room for optimizing consumer interaction/*delivery* of content. They become providers of unified data streams, and eventually provide platforms for accessing those streams.

Mamma.com and other “meta-search” engines are examples. Yipit and 8coupons fall into this role, as do Gigya, TweetDeck, Trillian, and Zillow.

4. Finally, the delivery experience is optimized. The core value proposition of the industry is at this point saturated and nobody wants to hear of “another X”, but the centralization of the ecosystem affords newcomers a clear path to the market’s table stakes, allowing them the luxury of focusing entirely on optimizing *how* consumers interact with the data or product.

These companies are distinguished by their user experience, consumer reach, and new angle on an old technology. They innovate by reaching consumers (both qualitatively and quantitatively) more effectively than other players in the industry and by seeding new adjacent markets which spin off around the delivery technology itself – which can open up new markets and cause the cycle to repeat.

“Demand Side Platforms” in advertising such as Invite Media and MediaMath are examples. So are Square, Shopify, Wix, and Tinder.

Economics and Computability

You can prove results in computability on economic systems. There are classes of problems that are insoluble in a given economic system. In a system such as capitalism where money is held by individuals and spent at discretion in order to maximize the holder’s return, there are two: “refactorings”, which make systems more efficient but have no direct short-term benefit, and “synergies”: problems that raise the standard of living of an entire group significantly more than they raise the standard of living of any given individual. Money is not zero sum because new value can be created, but spending decisions (taken individually at a single point in time) are zero sum, because you have a fixed amount to allocate among a collection of spending choices.

Game theoretically, a rational player in a capitalist economy will always try to spend money in a way that provides the greatest return for himself, monetary or otherwise. A player with knowledge of other player’s strategies will attempt to tie his own monetary gain to the monetary gain of as many other individuals as possible given his resources (this is, for example, where VCs came from). However, this is not the same as providing a benefit to the group as a whole; the holder must reach out and touch everybody directly, because only individuals can make spending decisions. (Corporations, for the purpose of this exercise, can be treated like wealthy individuals, since a corporation’s members usually have common motives).

It’s possible (indeed, easy) to construct a problem which has little individual benefit to the individual members of the group while having an overall benefit to the group’s welfare: for example, reducing carbon emissions, or cutting the use of pesticides in produce to prevent bee populations from collapsing, or becoming a space-faring civilization while there are still few extraterrestrial resources to exploit. These problems all require major changes in behavior, which require investment (as in physics, no Work gets done in the economy without an expenditure of energy/capital), but their benefit to individuals does not justify the investment, even though their benefit to the group does.

These problems are literally outside of what a rational capitalist economy can solve (similar arguments can be constructed for other economic systems). Since we inhabit a mixed economy, and since not all players behave rationally in a game-theoretic sense, the lines are fuzzier in the real world. But the rational strategy will always point to the mean behavior of the system.

Intriguingly, you can bin the ranges of problems available to be solved into classes (much like we do in computational complexity), and use class coverage as a measure of an economic system’s efficiency.

Should we continue to inhabit a mixed economy, the government has then found a just task to take up: aggregate economic interests so that efforts which provide the greatest benefit to group welfare also provide individual returns (and then recoup the difference proportionally from the groups that benefit most, preventing a Robin Hood economy from emerging). This would then increase the coverage of soluble problems within the economy (and the diversity of economic activity taking place), and quantitatively increase its efficiency if measured according to the paragraph above.

Feudocapitalism

Capitalism is an instance of feudalism, with money replacing land, corporations replacing fiefs, wealthy individuals replacing lords, skilled workers replacing warriors. I’m convinced that the feudalist pattern represents the intrinsic form of government that humans will form when left to their own devices, so this was inevitable.

However, the genius behind capitalism is that it tied acquisition of resources and power to human advancement. You can’t take money by force; you can only acquire it through consent, in exchange for providing a good or service which the buyer values more than the money. People who would otherwise be fighting wars over territory are instead doing their hardest to create and sell products. Sure, all sorts of underhanded and disgusting things happen on a routine basis, but ultimately, effort that would be put into killing is instead put into trade – that’s a tremendous accomplishment.

Social movements as Fisherian Runaway

Social networking, for instance, is a good example of an analogue to Fisherian runaway: the desire for the trait and the trait itself coevolve, until everyone has social because everyone wants social, rather than because it adds any functionality, value, or relevance. I suspect most social movements, and in fact traditions, are like this.

Publication bias

There has been recent talk about a consistent finding of scientific studies initially overstating effects, which become weaker and weaker with additional scrutiny. This to me is an artifact of both the difficulty of designing a truly unbiased experiment and the publication process. Mind you, there’s no foul play at work here; science is just a little less “free to explore” than most people believe it to be. But the insistence that results always be “good” to be worthwhile ultimately harms its objectivity.

Statistical methods have become very good at reducing variance, and most researchers make a large concern out of achieving adequate sample sizes and creating complex statistical models in an attempt not only to demonstrate an effect, but to demonstrate that it cannot possibly be due to chance (more accurately, we think it’s not more than 5% likely, anyway :))

Unfortunately, while variance is usually fairly easy to detect and deal with, there is another contributor to statistical error: bias. Each experimental setup introduces its own bias. If we assume that the biases of independent experiments are roughly random (i.e. the biases are unbiased :)), then we would expect a possible over or understatement of an effect in the beginning, with a gradual regression to its true prevalence as additional models are “averaged into” the literature.

But the biases which have been observed are positive only. Effects are found to be *stronger* initially rather than weaker.

Here is where publication bias enters the scene: it is acceptable to publish *new* work only if the results of that work are “good”, whatever that may mean in a given field, and experimental parameters and models will be adjusted until such results are achieved (resulting in more than a few cases of statistical overfitting, I’m sure). Negative results are not published; they are either improved or abandoned.

But while this is true for pioneering work, it is NOT true for subsequent “reviews” of said work, which can be published simply on the basis of picking apart an existing effect.

This virtually guarantees that the initial bias will be an overstatement, and the subsequent direction with more study will push it in the negative direction, presumably until – eventually – the true effect is approximated.

!Intelligence

Humanity’s primary selective advantage is not intelligence, but our social structure and intellectual *variance*. Every individual does not replicate every other individual’s discoveries; rather, one unusually bright person discovers something novel and shares it, enabling everyone else to benefit as well. This is part of our evolutionary heritage.

This explains a huge amount of human behavior which appears irrational under the assumption that intelligence is humanity’s primary selective mechanism. This is both why people tend to have an altruistic streak and why stupidity still exists. It is why tradition still exists, and thus why religion was naturally selected for and culture evolves. It is why war is made on a *group* rather than individuals – it is why genocide is practiced. It is why capitalism works (it provides an individual incentive to further foster this behavior). It is why the masses behave not like sheep, but like dogs, and why a very large number of people can be persuaded by individual demagogues and movements.

Memetics: Culture is to humanity as flight routes are to geese

The first and second factors are of equal importance in the psyche and the same evolutionary tendencies which support survival also support culture (and all of the baggage that comes with it). This is because for a long period in our evolution the two were identical – culture was our memetic means of survival, as things like migratory flight routes were those of geese. What’s more, you can apply the same evolutionary rules and look for the same evolutionary patterns in culture as you can in biology. Some mass movements are actually excellent examples of Fisherian runaway applied to the propagation of ideas rather than genes.

Yes, religion is a genetic trait

Lacking science, people had no way to explain the laws of nature or why certain consequences were associated with certain actions – nevertheless, the causes and effects themselves were understood (as I said earlier, science explains cause+effect+mechanism; mysticism explains cause+effect). It was an advantage to codify the actions that allowed people to stay alive and to integrate this deep into the psyche and the theology came along for the ride as a plausible explanation, if not a particularly grounded one. The result: religious people stayed alive, non-religious people died out. This not only created a selective pressure for religion, but convinced the religious that *the unbelievers really were being punished*, strengthening the belief in the religious population as well. (So this is a trait that’s both genetically and memetically reinforced).

Thoughts on the Anthropocene Extinction

While it is true that humanity is killing off species at an alarming rate, I don’t think this trend will continue indefinitely. The previous mass extinctions were driven (or at least initiated) for the most part by external events to the ecosystem, with reductions in the sustaining energy of the ecosystem and other consequences lasting for millions of years.

The rate at which we destroy ecosystems, on the other hand, is kept in check by our own population. Unless we pass some dire tipping point and cause the destruction to spiral out of our control, we will eventually hit a population limit, beyond which the planet can’t sustain us. It’s possible that we have already passed this limit; in that case, much like the current recession was caused because people borrowed money that didn’t actually exist and corrected by a return to the amount of real money left in the economy, the human population will be forced to decline, either through some sort of saturated-ecology problem (hunger is a big one; war could also be considered a limiting factor when resources become scarce) or simply through lower birth rates. Either way, the current mass extinction will not be as dire as the previous ones because, even at a faster rate of extinction, it will last for a much shorter period of time.

If I’m wrong and Earth becomes an ecumenopolis, it would instead bode well for humanity’s continuous expansion to other planets and we would nevertheless have the room to save what species remained extant.

…Barring a runaway process which takes matters entirely out of our hands. Watch those greenhouse gases!