Simon Waldman on Wikipedia's success (original) (raw)

There have been no shortage of insane, over ambitious ideas on the internet. Most of them never make it further than the pub they are conceived in. Some generate hype but quickly fall flat on their face. Others survive, but prove to be minnows rather than the giants they set out to be. However, every so often, one sneaks through.

Wikipedia is one of the rare ones that made it. Even by the admission of its founder, the 38-year-old technology entrepreneur Jimmy Wales, it was a "completely insane idea": a free online encyclopedia that anyone can contribute to and anyone can edit. There is no editor, no army of proof readers and fact checkers; in fact, no full-time staff at all. It is, in other words, about as far from the traditional idea of an encyclopedia as you can get.

There are dozens of reasons why it shouldn't work, and it is still far from perfect, but in less than four years, it has grown to have more than 1 million entries written in 100 languages from Albanian to Zulu.

To its fans, it is a fantastic research resource - albeit one that you should use with caution; and an incredible example of what can be achieved by collaboration and cooperation over the internet. To its detractors - mostly those from the traditional world of encyclopedias and librarianship, it is barely worthy of the label "encyclopedia".

To put Wikipedia's achievements in numerical context, at the same time it was celebrating the publishing of its one millionth entry (a Hebrew article on the Kazakhstan flag) in less than four years, the Oxford Dictionary of National Biography launched its latest edition. It had taken 12 years to complete, yet contained a comparatively tiddly 55,000 biographies. It also cost some £25m to create. Wikipedia has so far been bankrolled by Wales, but the total cost so far is still around £300,000.

The current Encyclopedia Britannica has 44m words of text. Wikipedia already has more than 250m words in it. Britannica's most recent edition has 65,000 entries in print and 75,000 entries online. Wikipedia's English site has some 360,000 entries and is growing every day.

But numbers mean nothing if the quality is no good. And this is where the arguments start.

"Theoretically, it's a lovely idea," says librarian and internet consultant Philip Bradley, "but practically, I wouldn't use it; and I'm not aware of a single librarian who would. The main problem is the lack of authority. With printed publications, the publishers have to ensure that their data is reliable, as their livelihood depends on it. But with something like this, all that goes out the window."

Wales responds by acknowledging that Wikipedia's model leaves it anything but error free (something they make clear on the site) but he also points to an article in a German technology magazine this month, which compares Wikipedia with two established, traditional digital encyclopedias: Brockhaus and Microsoft's Encarta. All three were tested on breadth, depth, and comprehensibility of content, ease of searching, and quality of multimedia content. Wikipedia won hands down.

Dan Gillmor, the Silicon Valley commentator and author of We the Media, is one of many independent fans: "I don't think anyone is saying Wikipedia is an absolute replacement for a traditional encyclopedia. But in the topics I know something about, I've found Wikipedia to be as accurate as any other source I've found.

The roots of Wikipedia lie in the open source software movement, which took off in the early 90s with the development of Linux, a free version of the Unix computer operating system. This was developed in the mid-90s and posted on to the net, giving anyone the right to adapt and improve it. It has become the only real challenger to Microsoft's dominance of the market.

The programmer and author Eric Raymond analysed the Linux phenomenon in his 1997 paper, The Cathedral and the Bazaar, which has since become the manifesto for the open source movement. Raymond compared the traditional school of software development to cathedrals, which meant they were "carefully crafted, by individual wizards or small bands of mages (magicians) working in splendid isolation".

The open source community he wrote, "seemed to resemble a great babbling bazaar of differing agendas and approaches ... out of which a coherent and stable system could seemingly emerge only by a succession of miracles."

The critical law of the bazaar school of publishing, Raymond claimed, was that "given enough eyeballs, all bugs are shallow". And this is at the heart of how Wikipedia works. Because it has so many people swarming over it all the time errors and abuses of the system are all quickly picked up.

Indeed, research by IBM showed that acts of "vandalism" (ie people deliberately writing either rubbish or abusive comments) were normally cleared from the site within five minutes.

For example, at 23.23 on September 27 this year, a user under the name Hitler edited the entry on Judaism by proposing that it be put forward for a "Vote For Deletion" (ie it should be removed entirely from Wikipedia). He summarised his edit as "HOLOCAUST LOL" (Laughing Out Loud). At 23.29 another user removed this from the page. And at 23.44, "Hitler" was been banned indefinitely from using the site.

But, this isn't always the case. One blogger, who goes under the name of Frozen North, made a point of deliberately making a number of minor errors on a number of entries at the start of September. He made five changes and it took at least 20 hours for them to be changed. Some of his implanted errors were there for five days before he removed them himself.

As Wales admits: "The average quality of the entries is pretty good, but on any given day, any given entry might be up or down."

The truth is that Wikipedia is continuously evolving. There are now around 3,000 new entries being added each day (about 700-800 of which are in English); and as the site has got bigger, so has the amount of editing that takes place on it. In September this year, there were an average of 11 edits per article. The entry on the Israel-Palestinian conflict has been edited more than 250 times this year alone.

The most contested entry, though, is that of George Bush, which has caused so much controversy that it has been frozen from editing. It had had more than 500 edits between August and the start of October, and there are more than 13 pages of discussion about the entry.

The truth is that Wikipedia reveals what is normally hidden in an encyclopedia: the countless decisions that lie behind each entry. The only difference is that in Wikipedia, the decision-making never stops and the debates are often robust to say the least.

Wales says an entry in his encyclopedia is "like a sausage: you might like the taste of it, but you don't necessarily want to see how it's made".

It might seem like anarchy. And, given the lack of central control, it technically is. However, there are three elements that make it work.

The first is its ownership, and lack of commercial imperative. The site is manned by volunteers, and now owned by a foundation, which means people willingly give their time and intellectual property to the venture. It manages to run on less than $100,000 a year.

The second is that the overriding editorial principle that all articles should show a "Neutral point of view" (NpoV). According to Wales, this is as much "a social technique for getting people to work together" as it is an editorial policy.

The third - and perhaps most critical - has been the evolution of an incredibly intricate and democratic social order to keep this vast sprawling project in order.

There are hundreds of thousands of "Wikipedians" who have contributed or edited articles. But the core community, according to Wales, is about 200 people who, by now, know each other quite well. Outside this inner-inner circle is a core of about 2,000 people who make more than 100 edits each in the last month. Beyond this, you have around 10,000 people who will have made more than five edits. There are administrators, bureaucrats, stewards and developers all with different levels of technical and administrative authority.

Where individuals get into a dispute, such as an "edit war" where they are constantly undoing each others changes, or just an endless debate over a particular point, they can hold a vote on the issue. If this doesn't work, there is a formal mediation process, and if this fails, the case can go to an arbitration committee. In very extreme cases, Wales himself intervenes in his role as "benign dictator".

Random acts of violence - such as Hitler's activities above - are not a major problem, ac cording to Wales, because they are clear to spot and obvious to redeem.

"When someone just writes 'fuck, fuck, fuck', we just fix it, laugh and move on," he says, "but the difficult social issues, are the borderline cases - people who do some good work, but who are also a pain in the neck."

One such case was a contributor called Wik. "He violated some rules, but he also had a lot of supporters, so there was a huge debate over what we should do. Eventually, the arbitration committee decided he should be banned for a week."

There are rewards as well as punishments. Those who have done good work - either for editing, or for dealing with vandalism - are awarded "BarnStars" to put on their profile page.

This entire order is all there for everyone to see online. Policies and strategies are posted online, discussed and voted on. While the content develops automatically and anarchically, nothing happens to the social structure of Wikipedia without the consent of the core community.

All of this, however, fails to impress their rivals over at Encyclopedia Britannica. During a 20-minute conference call, two of their most senior editors and their head of publicity, acknowledge that Wikipedia is a remarkable phenomenon, before getting their knives out.

Executive editor, Ted Pappas, is the most extreme in his criticism: "The premise of Wikipedia is that continuous improvement will lead to perfection, that premise is completely unproven ... with many of the pieces you don't know who it's written by, and who the administrators are ... one of the administrators over seeing the political coverage openly encourages people to vote for John Kerry ... 30,000 articles were created by a bot [an automated program that goes round causing havoc] ... hyperlinks, bulletpoints and cut-and-paste press releases do not an encyclopedia entry make."

Editor-in-chief Dale Hoiberg is no less damning: "People write on things they're interested in, and so many subjects don't get covered; and news events get covered in great detail. The entry on Hurricane Frances is five times the length of that on Chinese art, and the entry on Coronation Street is twice as long as the article on Tony Blair."

Wales is quite sanguine when confronted with this assault. Pointing out that Wikipedia is still less than four years old and still improving. And, that he is looking at new techniques to improve the quality.

His current plan is to create a "stable version" of some entries: in other words, one that has been fact-checked by one of a number of approved editors (although they are still working on the details of this). It is a radical change, but Wales is willing to do whatever it takes to give his project the authority it needs.

The truth is that Wikipedia is far from finished, but even now, flaws and all, it is already one of the wonders of the digital age; and a pin-up for a growing movement that sees the internet evolving as a true "citizens' medium". However, Wales and his community of volunteers are not resting on their laurels. He is in negotiations to print part of the content, and distribute it in Africa as part of their ambition to "put a free encyclopedia in the hands of every person on the planet". Insane and over-ambitious? Definitely. But after everything they have achieved in the last three-and-a-half years, you would be equally insane to bet against them.