2012-03-24 How Emacs Wiki Works (original) (raw)

(TL;DR: People that don’t like the wiki as it is ought look at the official Emacs documentation instead. I wrote this so that I’d have something to link to in the future. This post was inspired by 2012-03-20.)

Every year or so, I read about suggested changes to the Emacs Wiki. The complaints are the same, year after year.

The pages are confusing.
The code snippets are wrong.
The site is badly organized.
The information is out of date.

The solutions invariably have nothing to do with the problem.

Switch to Mediawiki, the software used to run Wikipedia.
Use a database or a distributed version control system as the backend.
Change the text formatting rules to Markdown, Mediawiki markup, or something else that is better known.
Separate discussion from the main page.
Delete stuff that is outdated.
Fix errors.
Organize.
Moderate.

Why are these suggestions not helpful?

The first problem is the mistaken belief that technology can substitute for social change. Yes, the wiki is badly organized and many of the pages are outdated. Changing the wiki engine, the backend or the formatting rules will not change this, however.

The backend used by the wiki engine can influence performance and resource use, it can make the software harder or easier to maintain and backup—but it will not induce somebody to edit a messy page and fix it.

The second problem is the mistaken belief that moderation can be commanded. You can complain about bad editing and a lack of moderation all day. But since nobody is paying people to do a boring job, we must rely on obsessive compulsive people to fix typos and tag pages.

Maybe we could attract more people by gamifying the experience—offer rewards, badges, scores. But Stack Overflow already does this. It’s the best social question answering machine currently known. The wiki doesn’t need to imitate something better. The wiki needs to do what it does best. We’ll come to that.

The third problem is the mistaken belief that quality control and volunteers go well together. Just compare Wikipedia and Citizendium and consider the animosity generated by Deletionism on Wikipedia. How will you encourage authors to contribute if you are telling them that their contributions are lacking the quality you are looking for instead of simply accepting their text and working on it?

You fight spam, you rework text occasionally, you encourage others, you welcome newbies, you lead by example. That’s how you lead.

An abrasive personality, radical change involving a lot of work—those are not the tools you are looking for.

Let me return to the issue of commanding change. Things people have said:

“the content editing should be one with the goal of creating a comprehensive, coherent, article that gives readers info or tutorial about the subject.” – Xah Lee (2008)

“I favor a major reorganization of the wiki material.” – Neil Smithline (2011)

“The articles are littered with crappy advice confusing beginners, have little structure and are filled with ridiculous questions” – Bozhidar Batsov (2012)

“Wiki is a hydra you cannot cut enough heads off to make it die. I tried, and failed miserably. I suggest you don’t waste your efforts and time on that.” – Eli Zaretskii (2014)

“What I propose is starting anew and getting rid of a few common complaints at once.” – wasamasa (2015)

The critics can be unhappy about it all they want, and they can complain about it all they want—but in the end, one needs to understand the forces at work, here. There is no chain of command.

It works just like a free software project. If it doesn’t scratch someone’s itch, nobody is going to add it. I think it’s a fundamental issue with our business model: there is no pay for boring stuff. Plus, documentation is of no direct use for anything—unlike code. Thus, people are mostly motivated to keep their own code and its documentation up to date. I don’t think there is anything we can do about that. That’s why the Emacs Wiki Mission Statement does not mention organization and quality. It cannot be commanded.

Once we accept that this is the sand upon which we are building our house, we necessarily need to scale down our expectations. Personally, I think the wiki exists somewhere between the official documentation, Stack Overflow, the FAQ, the newsgroups, the mailing lists, and IRC. It’s certainly nowhere near the quality of organization and writing that the Emacs documentation has—and I don’t think this is the right medium to aim for this level of quality. I think the people willing to invest that amount of energy to write quality stuff ought to be writing the real Emacs documentation—and they probably are.

What remains are the people using Emacs Wiki for their own pet projects, questions asked, answers given, sometimes organized, sometimes rewritten, sometimes linked to the rest of the site.

Wikipedia works because of its universal appeal. When I added an image to an obscure Indian temple we visited when I was staying in Mysore, the photo was terrible. But it was a start, and enough people cared about the page and it grew, and it found people to tend it, and now it’s big and beautiful.

There just aren’t enough Emacs users and authors out there and the best of us will be contributing to the official Emacs documentation. The wiki exists somewhere between the official documentation and the mailing lists. Lower your expectations.

Given all that, why does the wiki exist at all?

When I started it, I had several reasons:

The wikis I knew, C2 and Meatball Wiki, had attracted a particular community and they had created a particular subculture I liked. We talked about the Wiki Now and many other things that made wikis work. The medium itself was interesting.
I had been posting on the newsgroups for a long time, and slowly I realized that the same questions kept being asked again and again. The newsgroups and mailing lists were failing as a medium because they were ephemeral. Sure, we kept telling people to search the archives. But the medium afforded asking questions instead of searching.
When I looked for Frequently Asked Questions, I found a document online, maintained by a single person. This person was a bottleneck. The FAQ updated slowly.
At the time I was getting into Internet Relay Chat. On IRC, conversation is even more ephemeral than on the mailing list. This time, however, “searching the archives” was out of the question. We needed our own archive. And thus I started answering questions on IRC and posting the answers on the wiki.

I think this last point bears consideration: I was creating pages or adding information to pages because it was pertinent on IRC. An index, linking to the page, categorization, returning to the page later and reworking it, all these quality related tasks were not pertinent on IRC. All I needed was a pastebin that I could go back to and rewrite if I felt like it. Often I did not—and I still don’t.

The wiki being on the web, updated every now and then, with pertinent answers to specialized questions, unorganized and raw, ended up being a good resource for the search engines out there. These search engines bring new people to the site. People that don’t understand how wikis work in general and how this wiki grew to be where it is in particular. They are shocked. So many pages outdated! Such a mess in style and quality!

I think those people are better served reading the official documentation. They don’t want this mess, they don’t benefit from it’s loose rules, they don’t understand how cool it is to have a site with no login required. They are better served elsewhere.

I’m sure that one day the Emacs Wiki will have become irrelevant. But just like the old newsgroups never disappeared entirely, so will the wiki transform into something else and remain part of our information landscape.

Perhaps one of the Emacs Wiki critics will one day set up an alternate site, pull all the pages (more than 8500 pages last time I checked), extract the quality content—or rewrite it from scratch—and produce something better. Perhaps they will build an organization that can keep the quality up, encourage new authors to join, provide more value to their readers. But I don’t think complaining about the existing Emacs Wiki is a step in the right direction. Build it, and they will come—elsewhere.

(Please contact me if you want to remove your comment.)

⁂

the mistaken belief that technology can substitute for social change

Well, simply having a “talk” page for each wiki page (with a big link at the top) would let people have conversations about those pages without cluttering them up. And having an actual, complete revision history would help someone figure out what happened to a completely messed-up page. But doesn’t seem to have those things. Its technology is too primitive to reasonably support – much less encourage – a workable society.

the mistaken belief that moderation can be commanded

As Wikipedia has amply demonstrated, there are plenty of obsessive-compulsive people online. If anything, moderation needs to be limited.

– SeanO 2012-03-24 03:44 UTC

One of the critics already started an alternative wiki - http://wikemacs.org 😃

Other than that - I’m with SeanO on this one. And with 8500 pages or so - it easier to save the worthwhile articles than to revisit all the material.

– Bozhidar 2012-03-24 05:48 UTC

Sean, if the lack of a complete revision history is what held you back from reworking and reorganizing, then I guess you’re right. I am keeping all the logs, but discarding all the older revisions after two weeks and never felt the lack.

The addition of Talk pages was discussed a while ago—it was one of the items on the suggestions page—but at the time we had two votes in favor and two votes against them. Nobody else seemed to care. I’ll be surprised if their mere existence improves the pages. But I guess we’ll see, now. Good luck to you all!

After a quick look at the site I’ll suggest that you should add licensing terms as quickly as possible. As it stands, you cannot copy anything from Emacs Wiki. That would require a copyleft license.

– Alex Schroeder 2012-03-24 07:16 UTC

Alex – I’d say my laziness and your sneering condescension were greater obstacles. But if I don’t check in for more than 2 weeks (quite likely, given how little time I have for Emacs these days), I’d still like to be able to see what happened if something I cared about (e.g. the Perl page or my homepage) went bad. (How much, to an order of magnitude Euros and hours’ work, would it cost you to keep a complete revision log, by the way?)

I guess the Talk page issue came and went in a revision window when I didn’t have time to hang out here.

I’m also tired of your pseudo-legal threats toward the content of the wiki (which is under GPL2, not GFDL).

(This comment is protected by the GFDL, I guess. Whatever that means.)

– SeanO 2012-03-24 08:14 UTC

I wonder where you felt my sneering condescension. Perhaps in my replies to people I felt were trying to tell me what to do in my free time and with my money? I also don’t think I threatened you in any way. Perhaps you missed the problems I had with changes to the Emacs Wiki license in its early days. I was trying to help you or Bozhidar not make the same mistake.

The logs are there for all to see if you follow the links. Here’s the the SiteMap history, for example. Keeping all the old revisions would cost me nothing – it’s simply a setting. I prefer it this way. As I said, I think keeping the old revisions provides no benefit and adds a number of small drawbacks such as needing administrators to permanently hide particular revisions that contain material deemed problematic from a legal perspective. As it stands, I can undo these edits and with time, they are gone. Another issue is that I like the idea of a right to be forgotten. The original C2 wiki kept no revisions at all. I think that the only reason old revisions need to be kept at all is peer review and anti-spam and anti-vandalism measures. For those tasks, a small time window is sufficient. After all, wiki pages are not code. We don’t need to look through the history of a page to find when bugs were introduced and by whom.

– Alex Schroeder 2012-03-24 10:30 UTC

P.S.: EmacsWiki:WikiDownload links to a CVS repository of the source files hosted on the wiki, a subversion repo with daily snapshots of all the wiki pages, and a new, up to date git repo of all the pages with full history. I guess in a way my preference regarding the right to be forgotten is already moot since deleted stuff can be pulled out of the archives. This just hides deleted info from casual visitors.

Excellent riposte, Alex. We who criticize the wiki should pay attention. I would like to thank you for this consistently useful resource and your dedication. I certainly don’t detect any sneering on your part. Having said that, I do think there are a lot of real and fixable problems with the wiki, the worst being the hosting of code with neither “proper” SCM nor automatic notification of changes. Once you’ve used LaunchPad and (especially) github, this is just not tolerable. So I’m going to see what if anything I can do to help any new project.

– PhilHudson 2012-03-24 11:29 UTC

I see the problem! I think people like me don’t feel bad about keeping code that consists of a single file on the wiki because we usually don’t think of these files as requiring maintenance. After all, that’s how gnu.emacs.sources used to work. The wiki has the benefit of providing a stable URL, but the process remains essentially the same: post & forget, possibly have discussions with other people via email, followed by another post & forget.

To me, creating a separate project on Savannah or Source Forge is an unacceptable overhead for files like EmacsWiki:rcirc-color.el or EmacsWiki:rcirc-controls.el. But if somebody else felt like taking those files, putting them up on some other site – excellent! At first, color-theme.el was hosted on Emacs Wiki. Eventually somebody took it, moved it elsewhere, and started a real project. Great!

I still think that the Emacs Wiki can act as a low barrier-to-entry incubator for all those small little files that need a place on the web. I don’t read gnu.emacs.sources anymore, and I don’t think many other people do. At the same time, I think there still are a lot of people without their own web pages out there. They can’t post code on Facebook or Google+ and I imagine uploading code to Wordpress and Blogspot sites is also unwieldy. For all those people, the Emacs Wiki offers an alternative. It’s a bit better than gnu.emacs.sources and Lisppaste but a far cry from a software forge.

If people would take popular code from the wiki to a forge, repackage it as a real project, that would be great.

– Alex Schroeder 2012-03-24 12:11 UTC

Phil, I just remembered EmacsWiki:Git repository. Maybe that helps? I know Jonas is very enthusiastic about it and has been pestering me for weeks when I dragged my feet. ;) I’m sure he’d appreciate help or some nice words.

– Alex Schroeder 2012-03-24 19:07 UTC

“the mistaken belief that technology can substitute for social change”

Github was an example of technology that brought about a change in social behaviour.

– Phil Jackson 2012-03-26 12:18 UTC

Excellent post, Alex. Long live the ! 😄

– Edward O’Connor 2012-03-26 23:35 UTC

Thanks, hober!

Phil, regarding Github: I’m not much of a github user. I see that git and github bring a lot of relevant new features to the table. Compared with other version control systems they facilitate forking on a grand scale. Do you feel that using Mediawiki introduces a similar set of new features that will revolutionize how wiki pages are edited and organized? I don’t see it, which is why I cannot imagine that Xah’s and Bozhidar’s idea of switching to Mediawiki will in fact help solve the quality issues they have with Emacs Wiki.

– Alex Schroeder 2012-03-28 15:32 UTC

Just saw this: The Wikemacs Experiment: 300 Days Later.

– Alex Schroeder 2013-01-22 11:57 UTC

We are grateful for the wiki. It’s the best we have. Thank you for your work.

– Visitor 2014-05-08 17:02 UTC

These days I tend to think that Emacs Stack Exchange will make Emacs Wiki obsolete in the long run.

– Alex Schroeder 2014-11-16 09:18 UTC

Never.

This comment was extracted to my blog: https://alexdaniel.org/2016-01-21_Suckoverflow

This might be different in emacs stack exchange, but it just shows how such systems are defective by design. And I believe that emacswiki will stay as good as it is now, even if thousands of idiots start migrating to it.

– AlexDaniel 2014-11-16 17:24 UTC

Hm, you are clearly much better informed than I am. I just type a ton of Java and Oracle questions into Google every working day of the week and so many of these answers are found on Stack Exchange it’s amazing. This works so well because many of my questions are simple and stupid and even those questions get asked and answered. I guess that’s where the low hanging fruit are if you are into the points.

I’ll keep my eyes open and watch out for this kind of development.

– Alex Schroeder 2014-11-16 19:31 UTC

And I see no reason why something like Ask Page Extension couldn’t be installed on emacswiki. Even tagging extension can be integrated with it. You will get the same ask button, questions feed and no scoring system – PERFECT. And as a bonus this will be highly integrated with the wiki, so that you can link stuff back and forth without throwing users to another website.

– AlexDaniel 2014-11-29 21:14 UTC

And I’m not alone: http://michael.richter.name/blogs/why-i-no-longer-contribute-to-stackoverflow/

– AlexDaniel 2015-01-31 23:48 UTC

The section on poor community aptly describes what I feel towards Stackexchange and Wikipedia. But I still participate. Maybe less enthusiastically than before, and avoiding the authoritarian feedback, but I still write and vote. Simple answers to stupid questions are still useful. I will learn or not, on my own terms. Not every question needs to be a learning experience. But poor community bites. 🙁

– Alex Schroeder 2015-02-01 10:06 UTC

↓ ↓

Now and then I stumble upon thoughts like this:

“I don’t chase reputation points any longer: it interferes with the writing of a good answer.” – Borodin ¹

But then I start thinking – how can we undo this mess? It is very unlikely that someone running Stackoverflow will decide to make it better, these guys are doing their business there. If they wanted it to be good they wouldn’t implement their score-based system. First of all, they need something that attracts users in any way, and a game-like experience works best right now. Great for them.

But since the whole thing will become a big pile of mess eventually, they will probably start deleting old answers. Yeah. Since they have a stable flow of questions (which are repeated over and over) this is not a big problem for them – every question out there still will have an answer. If not – ask it again and get an answer. And this kinda makes sense, in their system, improving old answers is hard and not appreciated. Therefore deleting old answers will solve everything.

Do you think that deleting old answers is OK? Or can you see any other way to keep their income without doing that?

But for the moment, let’s list mentioned emacswiki problems (copied from this page!):

The pages are confusing.
The code snippets are wrong.
The site is badly organized.
The information is out of date.

What about stackoverflow?

The pages are confusing. – One page has a question and a bunch of answers. Every answer answers the same question. You don’t know which one is better, there is a scoring system but it does not always reflect the quality. Even worse, new answers are placed on top (like if they were somehow better), but later they fall down anyway (like if they became worse or what? Of course, they are trying to fix their lack of Recent Changes by doing that, but maybe implementing Recent Changes can fix their lack of Recent Changes?). Some pages have useful answers but are locked (you cannot contribute and fix problems even if you want to). Some pages are turned into something weird, where every contributor name is hidden. It is called “community wiki”. WTF is that I still don’t know. However, it works OK most of the time, but that’s pretty confusing, don’t you think?
The code snippets are wrong. – yes, the code snippets are wrong. For example, think about Bash answers – instead of giving a link to well written wikis people write their own shitty code. Most of the time, these snippets have errors. It is not better with other languages – although these answers might work, the solutions are frequently very ugly.
The site is badly organized. – not organized at all.
The information is out of date. – yes. Some of the stuff is already out of date, but we will see it becoming much worse a bit later. Stackoverflow is very young, not enough time has passed until we can say “it is out of date”, but there is nothing else you can expect from a system that does not encourage improvements and refactoring by design.

There are many bad things our mankind has went through. We will manage to go over this one as well, eventually.

– AlexDaniel 2015-04-23 00:02 UTC

Perhaps you are right and eventually Stackoverflow will degenerate into a “worse is better” than Usenet. Old answers will disappear. The same questions will be asked again and again. But at least we’ll have rudimentary scoring. Perhaps this would be a simple thing for a wiki to add. Like little +1 or I like buttons – but for every section, or paragraph, or code snippet. Essentially, this would make it easier for people to keep the good stuff and delete the useless stuff.

– Alex Schroeder 2015-04-23 07:43 UTC

Oh my, people are still unhappy! As seen on Reddit, complaining about the website but also mentioning Emacs Wiki…

– Alex 2017-11-20 09:47 UTC