Perfect internet, algorithms and long tails

"There is no standard Google anymore", "invisible, algorithmic editing of the web", "The internet is showing us a world we want to see, not what we need to see." This is what we all glory in, that whatever we want is available to us, the long tail in action. As in the post below, however, the tail has to be attached to something. The long tail says that you can profit from deviating from a norm, but the result of the algorithms that choose for us is that new norm are created. Thus we have the phenomenon of your Facebook page becoming an echo chamber. Such algorithmic editing has swallowed wholesale the arguments behind generation me, as well as the result of the culture wars where all norms are negative. What we have, then, is software as a crypto-morality. "There cannot be such a thing as a norm", this narrative tells us, "because nobody has a right to tell me what's normal!" There is a confusion in our conceptual apparatus, because a quantitive norm is being interpreted as a qualitative norm. There is a case where there is a cross-over, as Hegel tells us, but basic exchange of information requires this standard in terms of brute numbers. There is not a political or ethical agenda to this concept of the numerical norm. 

The assumption behind algorithmic editing is that there is a perfect search result. It is a cybernetic version of Plato's forms. There cannot be such a perfect internet experience, however, and we mislead ourselves in even imagining it is impossible. Perfect, in the sense of Lt. perfectus implies completion, of something being finished. This is nonsense, and so perhaps the ideology of perfection that underlies the thought and practice of  information technologies should be reminded of the irreducible reality of noise, of that which does not necessarily communicate a message, but without which the message cannot be communicated. We need to be able to go for a stroll with no particular destination in mind. We need that element of play in the hard sense, not hippy-dippy "y'know like whatever man". StumbleUpon curiously tries to inhere play into an algorithm, but for me it lacks that spookily magic sense of achievement when you find something that hasn't been linked by a thousand others already. We must realise there is a fundamental and structural contradiction in terms by attempting to make our online environment perfect for us in all ways because this leads to the isolation everyone on their own personal desert island. This simply establishes us in our own limits. Where then do we meet communally? Where do we have arguments? Where do we hear new stories, or jokes, or gossip? We need to have a norm from which we deviate, we need to have a same for there to be difference. 

See also, Web 3.0 : http://en.wikipedia.org/wiki/Semantic_Web

Is open source inevitable? II

[If open source is to have its day, some implications must be examined]

Technologies of privacy:

• Old style : passive, reactive, the default position. 
• New versions?: Opt-in, active privacy; specifically designed ways of deciding what we share, and with whom.

Transition away from previous economic models. Manufacturing and the mass ownership of capital has been on the wane for generations. Consider the MIT model of spinning-off industries. According to this study "Entrepreneurial Impact: The Role of MIT", if one were to regard companies developed by MIT alumni, collectively they would form the 11th largest economy in the world. Technologies must be proven, thus they must be peer-reviewed as well as tested by the market. If everybody can use the same ideas (goodbye proprietary anything), if open source and the intellectual commons get their day, then the matter of 'economic viability' is set aside in favour of "technical viability" and 'environmental viability'. The new models will have to incorporate recognition that there are diseconomies of scale, and what we called economies of scale were all too often a fetishization of size. This is a realization from the realm of network theory, which brings the long tail to bear on our everyday lives. It is not merely a niche element - the long tail is not long tail, as it were. (E.F. Schumacher had an intimation of this in his collection of essays, Small is Beautiful.)

How do we then differentiate between old and new? It becomes a matter of service. Will digital mean that we are all eventually a part of the service economy? We may be able to set our businesses apart according to how we deal with our customers. It may be a matter of approach rather than cost. A fully open source world, with respect for the intellectual commons, is utopian. Too much seems to stand in its way, but elements of this can be used to consider alternatives to developing nations making the mistakes that the industrial and post-industrial nations have made. Consider the principle of the long tail applied to national economies. Of course there will factors that lead a country to be wealthy by virtue of some natural resource, as long as unsustainable practices are maintained, but an open principle towards information will in theory allow innovation to take place anywhere. We see this in the emergence of 'regional hubs' and 'centres of excellence', but the best example yet, in terms of something that will actually last (unlike Dubai), is Singapore. CNN has fifty reasons to account for this (about 20 are compelling, but that's enough for me). For long enough have people considered the first part of McLuhan's "Global Village". We need now to give greater attention to the 'village' part. That is the locus of differentiation, and of what we can manage, to make our actions environmentally and socially sound.

Inevitability of irony

There is a structural inevitability to all discourse, such that as it progresses, it inheres learning within itself, creating more, growing, until ultimately it begins to query its conditions of possibility. This is irony, and it is not necessarily purely out of some sort of cognitive paralysis, as irony is all too often dismissed. It is not some disease of modern thought. It is a part and parcel of all discourse. It is in fact a permanent fact of all expert systems (Fernand Hallyn, The Poetic Structure of the World: Copernicus and Kepler, pp. 20-2), literature, and indeed to the vast complex of a culture's discourse in toto. Irony is an effort to reconsider the structural presuppositions that a discourse relies upon, most likely because of a perceived need for improvement and an awareness that the discourse one is using is coming up to the limits of its usefulness. It is the intuition that a discourse could operate better, could be more precise and efficient.

Often irony proceeds as a knowing and saying too much, as a show that one has mastered the vast amount of materials available in a given field (one cannot be ironic without this overflowing of knowledge), to the extent that jokes, puns and other diversions not only become possible but may seem necessary to maintain interest in a given topic. With this, interest ceases to be "constructive", and rather becomes a performance, and aesthetic. One is 'knowing' rather than knowledgeable. If we consider the pyramid of David McCandless below, it seems that in irony we slide back down from knowledge into information. We no longer have coordinating ideas to pull information together into a coherent structure. Irony is a kind of decoherence rather than incoherence. If we use the analogy of the stock market, then irony is a symptom of massive overvaluation, and any backlash against it such as "The New Sincerity" (see here for the neutered and here for the enthusiastic versions) is a crash, and an inevitable one (or necessary depending on your view). What emerges after this is that which has run a gauntlet of functionality, as it has demonstrated its worth via its robustness in the onslaught of critique.

The Whig interpretation of information: Is open source inevitable? I

The following is an attempt to trace some of the outlines of an element of the ideology of technology, it is not a detailed sociological analysis. The standard political model favoured by many interested in information technologies point to the supposed inevitability of openness. It is a techno-libertarianism that hovers over all discussion, our standard right-on refrain. Certainly it can be regarded as a goal, but more along the lines of a regulating ideal, rather than a Five Year Plan. This is a polite way of saying we don't back any of this up with action. The argument for openness is in its essence a historical one, which notes that once there had been an information technology that somebody attempted to control, there followed a breakdown of this control, with a free and open exchange following. I draw a parallel with Herbert Butterfield's thesis of the Whig interpretation of history, which recasts past history with a view to the present. Accordingly, an apparent chain of progress can be easily discerned from the chaos of the past, if you know what you are looking for. Such is the confirmation bias. This, however, misses the structures through which information is made available, those other technologies that have an influence - and there are many of them. In one set of terms this is how information and indeed an entire information technology is encoded. We need to confront the problem of context.

Reading the above outline (and I also think of Stephenson's Cryptonomicon mentioned previously) we must see that not all information is (or should be?) available as the ideal might hold. Indeed, openness is not a default. Were it so, should codes from history spontaneously decay into sense via some entropy of coherence? Openness is not a default because encoding is a technology. So too must we consider its recoding into openness (not decoding, which is then a contradiction in these terms). The myth of openness is based in reality, as all myths are, but it forgets TANSTAAFL, there ain't no such thing as a free lunch. This is a governing principle of information theory as much as it is thermodynamics.

According to the model I am proposing to dismiss, communication works as follows. whereby we speak openly to communicate a message, and then write this down so it is intelligible to both sides, and then it is encrypted. What this picture (or straw man, if I am totally off the wall) does not allow is that which a thoroughgoing techno-functionalist position can account for, namely the following. Speech is a form of encoding, long accepted and widely available, but a technology nonetheless. So too is writing. So too is transforming that writing into a second level of sense, namely an encoded (recoded according to my view) piece of writing, though of course it need not be writing. Codes do not degrade, they are broken. We invent a technology to extract sense from the encoded text, and thus we make it appear as though we have reversed the arrow of time, stepping into a past where the message we seek had not yet been put into that secret code which cloaks it. It almost appears as though we got that free lunch. We did not. If encryption is but another form of encoding. Encryption is as much a technology as is any other encoding. Openness too is a technology.

Is this an invalid conflation of discussing open source in the same breath as encryption? I do not believe so, and I do not compare them say they are equivalent, who cares. I do not propose indifferentism. I am saying that because there is no arrow of inevitable openness, one must actively fight for openness (with open source but the latest example of this). Pace the egotism of Julian Assange, we cannot have a knee-jerk ideology where we have a right to know everything at all times. Sissela Bok's Secrets from 1983 (currently out of print, but a contemporary New York Times review gives a fair outline) examines the issues surrounding secrecy, and notes that openness in the realm of science (which is broadly speaking where we might place the open source movement) has quite a different function to the role of secrecy in investigative journalism, whistleblowing, undercover police operations, and secrets of state (these and other issues each get their own chapter).

Lego: a threat to open source.

Open source as an idea as well as a movement needs to realise that because it is inextricably linked with trade and corporate secrecy (another chapter of Bok's), then for this to be more than a flash-in-the-pan, it must become part of an ecology of openness, interacting with problems such as copyright, and the knowledge economy. Indeed, regarding the former, there has been for some time an awareness of this, found in the idea of copyleft, which enacts a philosophy of pragmatic idealism, as outlined by Richard Stallman. This is not the time for micropolitics, but for a statement of intent, a declaration whereby we might make our unthought and unspoken assumptions real. One more effort, technologists, if you would become opens source!

See also, "Is open source inevitable? II"

Ideas and criticism

If an idea is presented, and elements of it are questioned in a manner that parries but does not thrust, if it is purely surface, does this in fact have any merit as criticism? A criticism ideally has a purpose, which is basically to be a form of troubleshooting. It must be specific, it must engage with the topic. To not do this is equivalent to saying "well I haven't used the software you developed, but it's probably rubbish" (we might call this The Troll's Refrain). Stand back and ask how does this help us to actually improve the idea. Even thinking 'us' is useful, because if there's a conversation, then there is a joint effort. David McCandless (author of the excellent Information is Beautiful) has a useful breakdown of how we think about the world, in the following pyramid:

In this, there is a progressive hierarchy of whittling out what is irrelevant, the old effort of separating the wheat from the chaff. This means that the bottom is a field somewhere, and so the top is presumably a Weetabix. Ideas are about turning information into knowledge, they move us from one level to the next. The non-criticism I am thinking of above seeks to move us back down, from knowledge back to information, or data. Instead of information, it focuses on noise.

There is a temptation in all debate to ask for greater and greater precision, but at the same time you have to step back and ask what level of perfection you want. If ideas are to be abstract, then they of course cannot account for everything. What we are searching for is a degree of functional 'robustness'. This is a term which Kevin Kelly makes continuous reference to in his writing (I am mainly thinking about 1995's Out of Control, which has dated far less than I would have expected), denoting that capacity for enduring networks of communication to cope with unpredictability. It is a nice antidote to management horseshit about "best practice" because it factors into its very structure that we start from least worst, and go on from there. We trade off a small degree of efficiency for a considerably greater degree of structural toughness. There will ever be noise, but not every blip is a threat to the entire system. Ideas are supposed to be solutions, and they are not to be rejected on aesthetic grounds. If it works it works, and from there we can begin the work of making our solution elegant.