Open Source Intelligence (original) (raw)

Flows for Democracy, or Monopolies of Knowledge?

Open Source Intelligence, n. 1. a) The ability to learn or understand from open and changing environments; a cognitive ability derived from the presence of open networks. b) The collaborative, transparent gathering and analysis of information by a distributed group; the connection of independent flows of information for collaborative problem solving.

By Jesse Hirsh

jesse@openflows.org

For Ronald Deibert

The global capital markets have been obsessed recently with the rather false distinction between what is called the old economy (of bricks and mortar) and the new economy (of clicks and bandwidth). Of course what recent �market corrections� have demonstrated is that the �new� are fools, since market reality seems to insist, that the �old� still rule. The more volatile the markets become, the greater the ability for large and institutional capital to move through and corner new markets. Riding the waves of market volatility requires the wind and power of deep capital flows, which only the biggest players can command.

This somewhat false dichotomization of the economy mirrors an earlier discourse that surrounded the politics of new technologies. Explicitly it concerns the (redefining) nature of the state in a world of simultaneously expansive and contractive communication networks that wrap the planet into a shared electronic media environment. While these networks are not the unified system they are sold as, we will for the purposes of this paper, assume that the Internet, will over the next decade or two, subsume all networks into itself.

On one extreme, lie Libertarian zealots who herald the end of all states, as if this were a prophecy embedded into the architecture of the Internet�s code. On the other extreme, are paranoid technophiles and technophobes, who feel that 1984 has come and gone, and in its aftermath, we find ourselves in a surveillance society, where big brother exists and rules, if not via the proxy of countless little brothers. Both of these visions, while somewhat ridiculous, speak to a sense of uncertainty, that is widely and overwhelmingly shared, when it comes to the impacts or long-term effects of the adoption and employment of network-based communications technology.

This paper does not align itself with either of these extremes, but instead, situates itself in between the two, as a consideration of either as a potentiality. Instead the focus is on the state in the surveillance society, and the role that Open Source Intelligence (OSI) plays in breaking the monopoly on intelligence that the imperial or nation state has traditionally held. This paper envisions a new era (regime) of network states that emerge as part of a new globalism, which is fostered by multilateral institutions and treaties such as the WTO, OECD, NAFTA, EU, and other ambitions acronyms. Speaking to a larger role for civil society in these emergent states, OSI allows non-governmental organizations (NGOs) to engage in what was traditionally and restrictively the domain of States. This paper explores this concept and practice in five sections:

1. Monopolies of Knowledge; A Tale of Two Systems

2. Open Source Intelligence

3. Open Source Networks (case studies)

4. Open Source Software (available tools)

5. The Intelligent Internet (trends, technology, and social strategy)

Monopolies of Knowledge, A Tale of Two Systems

A number of different factors have combined to create a dynamic period here at the end of the Common Era�s second millennium. Explicitly, it was the engine of the Cold War in the second half of the 20thcentury that generated the rapid development in communications technology that set the stage for today�s political uncertainty. In fact, one might argue that the most substantial byproduct of the Cold War was the �Global Village� of electronic media that now focuses all of our attention on the most trivial and traumatic events anywhere around the planet at anytime. While we may recognize this as the society of the spectacle, what is of consequence to political economy, is in fact the culture of speed, which is today the blood of states and markets.

Driven by the globalization of capital, and fueled by the globalization of politics, the culture of speed is that which makes or breaks companies and countries. To be nimble, and to be quick, is the only way states can survive the candlestick of electronic media. It is all too easy for politicians and regimes to be toppled if not discredited by the deconstructive power of the global press and freelance paparazzi. It is of no surprise to then observe that over the last two decades, one of the primary activities or focus of the state has been divestment: explicitly in the form of deregulation and privatization. While this has occurred in many areas, having a diverse and varied effect, the overall onus has been to shift substantial amounts of responsibility and control away from the public sector and instead to the markets. The primary justification or rationale for these policies has almost always been technology, and the rapid development and deployment of electronic network communications.

While it is not within the scope of this paper to evaluate the merits or value of said policies, it is important to examine the way in which functions traditionally held (and monopolized) by the state, are now being adapted and adopted by the private sector for use within the global markets. Explicitly, when one looks at the area of Intelligence, a clear migration of state activity and expertise has manifest within the corporate (market) sphere. The following table illustrates this relationship:

While these two systems have always co-existed, power and agency in our society is increasingly being invested in the Enterprise model rather than strictly in Empire. Our language now employs a duality, with the market and the state as two concurrent systems that together provide governance. Traditionally the state had the power and resources to regulate and control commercial activity, but increasingly, this is neither possible, nor entirely desirable. Instead, it appears as if the traditional state-based monopoly of knowledge that controlled the facilities of Intelligence, now resides scattered across the communications sphere, which is largely inhabited by private (corporate) interests. This is not to say that those Intelligence facilities do not in the end serve state-based priorities, rather it is not at all clear what priorities they serve, if any. Instead what we do have are two distinct systems, one focused on pragmatic needs (of profit), the other on statist needs (of regimes).

The Internet is the bridge between these two systems. With its origins in the military, and its current commercialization, the Internet has served to obfuscate the relationship or even distinction between the two. On the Net, anything can be commercialized, and everything is considered by the military as part of their domain. So its not so much a matter of mutual exclusion, but rather parallel co-existence, if not mutual reinforcement.

In between these two systems, under the (communications) bridge that joins them, is our society. In time we will rise to be that bridge, in the same way that we will rise to be the Internet. In the meantime however, the Internet is but a projection of our socio-techno capabilities. Thus the real question that needs to be addressed in the interim, is what happens to society when it is situated between those two systems of power? While this may seem like a banal and ancient question, it is relevant to our time due to the tension that results from the commodification of imagination that is Intellectual Property. The pursuit of Intellectual Property is essentially the corporate model of Intelligence, the same way that the pursuit (or deterrence) of violence is the state model of Intelligence. In this regard applied Intelligence is the maintenance of a monopoly (of knowledge). Thus in pursuing democracy, civil society may appropriate and deconstruct both of these models, which we will refer to as a process of Open Source Intelligence. This process works against monopolies of knowledge and property while seeking a world free of violence and domination.

Open Source Intelligence

Perhaps one of the most redeeming qualities of the Internet is its ability to develop collaborative environments for the gathering and analysis of information for and by heterogeneous communities or networks that share an area of interest.� Open Source Intelligence (OSI) is the organizing process that enables social networks to engage in these activities. This allows net-based cultures to analyze themselves and their peers in terms of their impact on and relationship with society at large, offering the potential to create a more open society. In many regards, OSI becomes a culture adjacent to the technology of the Internet, but in so being a culture, reflects and responds to the needs of its actors. The following, is the definition of OSI created by myself (Jesse Hirsh) and my colleague Felix Stalder:

Open Source Intelligence, n. 1. a) The ability to learn or understand from open and changing environments; a cognitive ability derived from the presence of open networks. b) The collaborative, transparent gathering and analysis of information by a distributed group; the connection of independent flows of information for collaborative problem solving.

OSI in this regard is comprised of five key dynamics: facility, technology, context, content and community. These dynamics play off of each other and tend to be addressed concurrently. In examining relevant applications of OSI, this paper will use these five dynamics to observe how these phenomena have manifest on the nets. What is at issue is the means by which these groups (actors) employ OSI (as an organizing culture), and are able to establish and sustain flows of knowledge that either values their organizing, or provide value to the public (interest). If this is the case, then perhaps the monopoly of knowledge that has controlled the practice of Intelligence is indeed over, and we are at the early phase of a new democratic society. Certainly it is still too early to fathom what state-based Intelligence agencies are doing in response or in conjunction with OSI, since disclosure in that area usually takes a few decades. Thus we are limited to observing that which is already public.

Open Source Networks

The easiest and most direct way to observe OSI is via �Open Source Networks� (OSN). These types of networks are (much like OSI) native to the emergent electronic communications networks. They have arisen out of the rag tag world of electronic media to form sprawling collaborative spaces within the electronic networks. This paper will examine seven different networks, all of which are �open� and �transparent�, yet fall into four distinctive sub-categories: corporate, cultural, technical, and political. While the agendas and mandates of each differ, the processes they use to organize are the same, all falling under our definition of OSI.

Stratfor.com � Open Source Corporate Intelligence

Stratfor is an excellent example of corporate, political, �free market� intelligence. They are the mercenaries of OSI, which makes them rather unique, since most mercenaries, or freelance corporate intelligence outfits would never be open enough to earn the OSI label. What sets Stratfor apart, is the use of openly published sources which they use to conduct analysis on world events, which they then publish openly on their website and various email lists. They are able to finance this activity with a mix of subscribers, corporate clients, and mass media circulation.

It�s not unreasonable to suggest that Stratfor engages in open activity primarily as a vehicle for marketing their services. Regardless, it is very effective, and they consistently demonstrate how OSI is widely available and in abundance. They are what a traditional state-based intelligence agency would be if it operated in the public while also not overtly loyal to any state (i.e. in a free market).

IndyMedia.org � Independent Media Centres

In the last twelve months, a new network has emerged in the United States. Organizing under the domain name indymedia.org, non-corporate media practitioners have come together in collaboration on the coverage of specific events or geographic areas. The most significant events so far, have been the WTO meetings in Seattle, GMO meetings in Boston, and the IMF meetings in DC. Future IMCs are planned for Philadelphia and LA around American presidential conventions. Indymedia.org quite literally has no structure or organization. Rather it is a fluid network that is based on situational formations. Adopting a traditional American libertarian position, the network exists as set of email lists, with archives coverage of events posted on their web site.

While Indymedia.org is supported by a great number of (primarily American) grassroots organizations, it exists because of the benevolence of larger well-funded groups like FreeSpeechTV (www.freespeech.org). At this point it is unclear as to where and how Indymedia.org will develop, but in the past, it has offered incredible perspectives on the demonstrations it has chosen to cover. As an example of spontaneous OSI, indymedia.org demonstrates just how easy it is to replicate OSI, given the will and mobilizing power of traditional protest movements.

The main detraction of the IMC movement however is its lack of structure, and its lack of editorial capabilities, vision, or control. This is what keeps this practice of OSI from being anything more than spontaneous, if not entirely accidental.

Nettime.org � Collaborative Text Filtering

To quote directly from their self-description:

�Nettime is not just a mailing list but an effort to formulate an international, networked discourse that neither promotes a dominant euphoria (to sell products) nor continues the cynical pessimism, spread by journalists and intellectuals in the 'old' media who generalize about 'new' media with no clear understanding of their communication aspects. We have produced, and will continue to produce books, readers, and web sites in various languages so an 'immanent' net critique will circulate both on- and offline.�

In many respects, Nettime was one of the pioneers of OSI. It has over the last half-decade pursued the project of �Collaborative Text Filtering�, and has in the process, attracted an impressive array of intellectuals, artists, writers, and hackers. Organized by a discordant trans-national editorial collective, nettime describes itself as �slightly moderated�. As a set of email lists, one is never too sure what is going to come through nettime, however after certain tumultuous or intriguing periods, the editorial group releases a newspaper or book publication that summarizes the discourse from the lists.

Nettime is very European, and is generally supported by organizations like the Open Society Institute (Soros), as well as various European Cultural Agencies. With affiliations to other European cultural and political networks, Nettime is a manifestation of OSI that reflects the role opening for civil society in the EU. In many respects, Nettime is kind of an informal techno-cultural early-warning system for European social groups. It offers a vision of OSI that is both autonomous, yet still situated within a broader collaborative (in this case European) culture, which is not present in the case of Indymedia.org.

Slashdot.org � News for Nerds

Slashdot, subtitled �News for Nerds, Stuff that Matters�, is the most widespread and substantial application of focused OSI. Hundreds of thousands of users, in addition to scores of corporate media journalists, come to Slashdot everyday for intelligence on technology and other issues of concern to American geeks. Slashdot is the most successful implementation and practice of a user driven open network news service, which also has a discussion facility with meta-moderation capabilities. Run from the �Geek Compound� in Holland Michigan, Slashdot was started and is still somewhat controlled by Rob Malda and his friends.

However what made Slashdot into an OSI application was the automation of the distribution of moderator status to users of the Slashdot system. Using a Perl-based interactive web environment, regular readers of Slashdot periodically receive moderator status, which enables them to rank comments and stories throughout the site. After a while, there were so many moderators, that a third tier of meta-moderation was created, that sought to rank the performance of moderators. All registered users of Slashdot are enticed to engage in meta-moderation. Rob and his friends post the stories, and the readers then regulate the environment.

As a result of this intimate involvement of the users in the system, Slashdot has accumulated considerable amounts of praise and credibility, and is as a result, one of the most authoritative and even legitimate sources for all things �geek�. As an example of OSI, Slashdot demonstrates how transparency and distributed decision-making can accelerate and augment the practice and purpose of an organization or constituency.

Freshmeat.net � Free and Fresh code

Freshmeat is the most active distributed relational database of free and open source software. It achieved this status by practically being the first to do so, but much like Slashdot, the open nature of the freshmeat.net database allowed its OSI qualities to emerge, engage, and entice the development communities to use it regularly. As a mix between a directory and a news service, freshmeat.net allows any coder to post a listing or update about their work, as well as any user to post comments about said code. Much like an Agora, freshmeat.net is perhaps the most comprehensive categorization and catalog of the myriad of software that thrives and diversifies in computer networks. Unlike tucows.com, which is focused on the consumer using proprietary tools, freshmeat.net is run by and for developers, and focuses pretty heavily on networking, although it does cover every aspect of free and open source computing. In fact much of the recent attention given towards the Linux platform is in part due to freshmeat.net and its ability to co-ordinate thousands of software projects amongst hundreds of thousands of users and developers.

The Internet is vacuous, and of transient form, Freshmeat.net however is an example of OSI bringing coherence and co-ordination to this obfuscation, which is otherwise a dispersed and shapeless culture. Civil Society could learn much from the use of distributed relational databases, explicitly in terms of organizing that which is already organized (such as credit histories or other data trails). Freshmeat.net offers an image of OSI as reclaiming the characteristics that are resident in one�s organizing, yet not inherent in ones practice. Freshmeat.net articulates a unity amongst free software developers that was largely invisible beforehand.

HackerNews.com � Network and Computer Security

The infamous L0pht crew, who recently merged with Forrester Research to form @stake, a corporate security firm, started the Hacker News Network. HNN serves as a dynamic source of intelligence on computer security and the political aspects of technology. It operates as a multi-nodal news and alert network, taking advantage of a large set of underground and counter-cultural networks. HNN shares its news and analysis openly, and over the years has come to be known as the most accurate and timely source of computer security related bulletins. As a model of OSI, HNN attracts a large amount of contributors, readers, and active promoters, mostly because of the quick turnover of news and analysis. In fact, it has gained so much credibility and established such a distributed network of affiliates; HNN has become the official voice of the computer network sub-culture, which is remarkable given its recent corporatization.

A-Infos (www.ainfos.ca) � International Anarchist News

Originally a postal network in northern Europe, A-Infos become the intelligence infrastructure for the international anarchist movement, since it migrated to computer networks in 1995. Described as a �multi-lingual news service by, for, and about anarchists� A-Infos is democratically self-managed by an international organizational collective. With dozens of items every day in multiple languages, A-Infos is reprinted in publications around the world, read aloud over community radio, and forwarded all across the nets.

It does not have any funds, and has a rag-tag of volunteer workers who provide light moderation and systems administration. What makes A-Infos thrive is the desire amongst its readers to actually be part of the actions and discourse that are described in the stories. Quite often, A-Infos carries first-hand and direct reports from world events and scenes of social unrest that are neither understood nor carried by the corporate media.

As a model of OSI, A-Infos is an excellent example of how immediate intelligence sharing can be extremely beneficial to specific movements. Although its focus is specific (anarchism), its subject matter is infinite, due to the appetites of anarchist activists. As anarchism increasingly comes to absorb the attention of North American civil society, it should be noted that A-Infos was the facility that provided this initial, internal, and self-recognized sense of a vibrant and active movement.

Open Source Software

In all of these examples however, the facility and technology that enables the content, context, and community of OSI, is underemphasized, if not completely taken for granted. This is perhaps the single most evident example of why the myth of the Internet is inappropriate in an age of immature networks. It�s not to say that technology determines structures or outcomes, but that combined with facility, technology is employed as part of a larger social agency that shapes the content, context, and communities.

The key component to OSI is of course Free (GPL) Open Source Software (OSS). The tools and systems described in the table below offer some indication of the incredible gifts available to initiates of the networks willing to wade into the world of system administration. If the rise of the Internet were to be described as a battle between an old academic co-operative culture and a new corporate commercial culture, OSS would be the armaments of this old (native to the networks) culture.

The Intelligent Internet

What�s remarkable about the technology is the way in which it instinctively revolves around a process of self-organizing. People code because they see the need or have the desire. As broadband networks continue to be deployed, the potential of distributed computing lies in the ability of clusters of diverse interests to spontaneously form networks of support and co-ordination. Coupled with the random and unpredictable dispersions that characterize the nets, that are then mixed with the catalytic coincidences and consequences of living at the speed of light, and one can get an image of a political arena, which is both unforgiving and at times unknowing. Buyer Beware!

The networks themselves require a certain level of social facilitation. Left to their own accord they tend to become absorbed by a weird mix of pornography and violence (gaming). Yet there persists a certain potential for something else, a public sphere that in its existence facilitates a public education, a networked discourse towards a democratic society.

Somewhere in this euphoria of seeing the new world, we forgot that with rights to freedom of speech, come rights to be heard, listened to, and respected. Thus the role of responsible moderation is the role of democratization, the process of shared realities and mutual understanding. Open source intelligence tries to facilitate this type of organizing, enabling this type of moderation, and a shared body of knowledge that can be freely exchanged.

The specter of big government that was left as a skeleton from the closet of the cold war has haunted any thoughts towards regulation, and detracts from the real role of social and cultural integration. Government regulation of the Internet should exist, and it should be in the provision of public network space, not the deterrence of net-based activities. The culture and code are freely available, however the facilities to apply and modify them are not. The Media Monopoly is very real, especially when it comes to telecommunications infrastructure, which is inevitably required when operating advanced communications networks.

The issue of access however is not one of technology but of culture, for in the �online� world of the electronic networks, the only thing that counts is whether or not you�re �there�. It is a binary and digital sense of political power and social standing. You are either present, or you are absent. Unless you are affluent enough to have a proxy, then you just don�t count. The concept of the �social safety net� has to not only be extended to the networks, but it also has to be completely refurbished and reworked to reflect our increasingly bi-polar world of those in the know with the loot and those still confused and out of the loop.