“the free encyclopedia that anyone can edit”
Wikipedia tag line

Should you ever highlight poor writing or errors on Wikipedia, you’ll likely hear the retort, “change it then!”, in reference to the open-to-all technology. Layered on top of this technology – which, it should be said, is not particularly friendly to the average user – are a complex set of community and individual processes, checks and etiquette. Taking these things into account, how easy is it to actually make a change on Wikipedia?

This article will examine the question by analyzing recent activity to the List of Content Management Systems English Wikipedia page. A short disclaimer and conflict of interest should be noted: until April 2010, I was director of a company that creates a content management system, which is one of many affected by the activity discussed. Empirical evidence is therefore presented, where available, to discuss points as neutrally as possible. This is certainly not an anti-Wikipedia article; rather, an open discussion that I hope will inform positive changes to what has become an invaluable resource.

Wikipedia Contributors

You might imagine that a diverse set of amateur subject-experts and experienced educationalists edit the online encyclopedia. A 2008 United Nations University survey of 130,000 Wikipedia users exposes a surprising profile: the average age of a contributor is 26.8 years (10 years younger than the average age of the general population in ‘more developed’ countries), 87% are male, and at least 46% are not university educated. Even with this relatively young age and education profile, 70-90% of contributors self-identify as “experts”.

The results hint towards a particular type of person becoming a Wikipedia user: young, male and self-assured. Might this lack of diversity affect the subjectivity and content of the encyclopedia?

Ownership of the List Of Content Management Systems

There were 500 edits to the List of Content Management Systems page (hereafter: the page) between 23 September 2009 and 24 May 2010. This is the range of edits we’ll limit our analysis to.

Previously (as of 23 September 2009), there were 40 unique products listed as “Proprietary” or related, and 92 as “Free and Open Source”. 500 edits later, the number of proprietary products dropped 70% to 12 products (28 less). Conversely, Open Source products dropped 4%, to 88 products (4 less).

There are a number of reasons why Wikipedia contributors/editors might remove an entry, but in this particular case, the vast majority of deletions are based on the non-notability guideline. This 1,500 word guideline – which is annexed with 10 subject-specific additional notability guidelines – states that content doesn’t necessarily need to be notable enough to have a stand-alone Wikipedia page to be included, but where a “Stand Alone List” is concerned (such as the Content Management System list in question), the Wikipedia editors may decide it is necessary to limit the length of the list.

Given this notability requirement, to be considered, content must have “significant coverage in reliable sources that are independent of the subject”, where “significant coverage is more than a trivial mention but it need not be the main topic of the source material“, and, “sources of evidence include recognized peer reviewed publications, credible and authoritative books, reputable media sources, and other reliable sources generally”.

During the timeframe of this analysis, 42 of the 500 edits are described as removing or reverting entries due to non-notability. Some edits remove more than one product; others remove the same product that has been re-added. In total, 46 unique products were removed in the 42 non-notability edits.

Given this constant scrutiny, it would be reasonable to expect that the products that make the grade – especially those that have remained on the list over all 500 edits – are notable – and that those that don’t are either unknowns, or have no clear proof of notability

Certainly, this is true for many of the deleted spam entries submitted over the time period, which come as both proprietary and open source entries. But someone working in the content management industry, or anyone who reads the major independent literature from Forrester, Gartner, CMS Watch or Gilbane, might spot some startling decisions.

Ektron, EPiServer and Interwoven TeamSite are all well-known proprietary products in the industry, and all that are significantly featured and reviewed in the third party literature previously mentioned. Yet all have been casualties of (or are still battling) ‘non-notability’ on Wikipedia; even though, in some cases, this well-known third party literature has been supplied as evidence. A quick Google search for “Interwoven TeamSite”, for example, reveals numerous third-party books on Amazon and articles on both the Adobe and IBM websites.

The question therefore is: who is making this decision? Is it a random group of temporary users who have no big-picture of the list, and therefore cannot apply consistent rulings?

Actually, it turns out that only two users have been responsible for all 42 non-notable edits: 35 by Haakon (who boasts, “I have deletionist proclivities” in his Wikipedia profile), and seven by Greenman. Although Wikipedia may claim that “anyone can edit” an entry, the reality is that to get a product on the list, you have to pass these two gatekeeper users. It could be argued that above these two supervisory users, a wider community is responsible. However, to date, the reality is that the majority of users who question the initial decisions of Haakon or Greenman do not have their edits reverted (with clear reasoning in some cases).

Let’s re-cap those figures again: there are 12 proprietary products listed (with major omissions) and 88 “Free and Open Source” products. “Notable” open source products that have passed the test include PivotX, whose current Wikipedia entry is shown in the screenshot below.

An old revision of Haakon’s user page lets us know that he’s “an avid believer in the Free software ideology”, and his blog states that his likes include “Linux, free culture”. Likewise, Greenman’s LinkedIn profile lists his passions, that include “Free and Open Source software”, and his blog details the articles and book he has written on PHP, MySQL and other open source technologies.

These aren’t just two people who use open source software; they are passionate advocates who mention free and open source software in short 2-3 sentence summaries about themselves – it partly defines who they are.

Recognizing the highly-charged, politically-edged nature of open source advocacy, permitting two open source ‘avid believers’ to moderate a page that lists proprietary and open source software is the equivalent of having a list of fast food restaurants moderated entirely by vegans. Neutrality is difficult. In some ways, the situation is horribly ironic: “conflict of interest” has been stamped onto many of the proprietary listings as a case for removal, but rarely do reputable Wikipedia moderators undergo the same scrutiny, or are made to answer for the same conflicts of interest.

These two users are not the problem, though: they are accomplished Wikipedia editors who have made thousands of edits. They deserve credit and thanks for the spare time they commit to improving the encyclopedia. Open source is not the issue either; it is responsible for most websites on the Internet – including this one – and deserves the fervent support it receives.

The problem stems from the lack of process that allows a narrow cross-section of users to moderate potentially contentious Wikipedia pages. The openness and anonymity that brings Wikipedia success also enables a certain degree of lack-of-scrutiny, coupled with self-certified autonomy. In theory, the wider community is responsible for decision-making, but the reality is page and niche-level micromanagement with great community deference to the self-built reputation of managers – in effect, an oligarchy.

What Needs to Change

This article isn’t really about the particular page in discussion, but a wider problem: it is highly likely that the same “open source/proprietary” issue affects many tens, hundreds or thousands of pages. Beyond open source, many other political and subjective issues will influence tens of thousands more pages.

The biggest problem is the incredibly narrow profile of the average Wikipedia user: young, male and (an assumption, based on the Wiki technology complexities) highly technically proficient. This is compounded by the 2005 statistic that 50% of all Wikipedia edits were made by just 0.7% of users, and 72% of articles were written by 1.8% of users.  Out of the top ten Wikipedia contributors, I could determine the sex of eight users: all were male.

To become a better, more well-rounded resource, Wikipedia must re-balance this profile.

First, the editing interface needs to be radically updated. The MediaWiki software that drives the website is incredibly powerful, but equally confusing for non-technical and even some technical users. Clicking ‘Edit’ on a Wikipedia page brings up a mish-mash of warnings, links, and a box full of code that can be hard to decipher.

After the interface has undergone rigorous usability testing by the ‘average’ person, and corresponding updates are made (e.g. WYSIWYG editing, section-specific templates), the rules need to be substantially edited and simplified.

Policies such as Neutral point of view and Verifiability are important as reference articles, but too convoluted for a first-time contributor who may want an introduction to the basics. Conversely, the Five Pillars are too obtuse for a first-time user. As soon as a new user clicks the ‘Edit’ button, they should have easy, clearly signposted access to a concise list of editing guidelines. There’s nothing more frustrating for a first time user than to commit time to an article only to be notified that their changes are unacceptable due to a rule that wasn’t made clear.

Finally, once the interface and guidelines have been significantly simplified, the Wikimedia Foundation must ensure that they continue to undertake positive-discrimination activities to attract and retain a diverse, balanced contributor base.

How to Help

On a positive note, all three of these changes – improved interface, simpler guidelines, and diversity outreach – are underway, and have some momentum behind them.

To help with interface improvements, check out the discussions about a WYSIWYG editor for MediaWiki. To help with training for new Wikipedia users, see the Bookshelf Project. Most importantly, to help the non-profit Foundation continue their efforts to increase diversity, please consider making a donation.

Appendix – Further Reading

Minutes before publishing this article, I found a similar blog post titled “Wikipedia, Notability and Open Source Software” which – although more personally worded – tells an interesting, similar story. The comments also make for good reading.

Be Sociable, Share!

• 
•