quality and authority

This page considers debate about quality in wiki authoring and editing.

It covers -

• introduction
• a media phenomenon?
• authority
• quality

     introduction

As the preceding page indicated, wiki has been hailed as an embodiment of open source - or merely open access. It has been promoted as necessarily better than commercial publishing because it is collaborative, community-based, updated on an ongoing basis and the embodiment of fashionable notions such as the 'wisdom of crowds' (in contrast to work by practitioners of scholarly disciplines and intervention by elitist editors).

Critics have complained that -

• contrary to claims about 'self-correction' - the quality of much wiki output is problematical,
• the coverage of much wiki publishing is very uneven and
• (contrary to the claimed communitarian ethos) it is driven by the same status seeking and clique formation evident in potlatch communities.

There has been little quantification and, as discussed below, many claims and counter-claims accordingly have an anecdotal basis.

Debate about wiki as "faith-based publishing" has featured many of the claims about open source software - examined here and here. It has also featured the passions evident in open source, with critics comparing some discussion to the enthusiasm - or merely intolerance - evident in religious wars.

     a media phenomenon

The wiki movement is as much a media phenomenon as a substantial advance in information production and distribution.

Reception of wiki in the general media, in lifestyle publications such as Wired and in online fora such as Slashdot and Whirlpool initially echoed coverage of blogging, with largely uncritical restatement of assertions that wiki was unprecedented, "the future" and necessarily better than 'old economy' models of publishing.

Disquiet spilled over from specialist media in late 2004, at around the same time that wiki was being assimilated by business strategists (the 'corporate wiki' to complement the corporate blog as a tool for knowledge management) and academia.

Much of that disquiet concerned questions about acceptance of wiki hyperbole and the authority of wiki publications such as Wikipedia. Critics noted, for example, that the rapid self-correction lauded by wiki proponents did not take place or did not not flow through to the numerous sites that lift wikipedia text - wiki clangers are digitally embalmed across the net.

     authority

Questions about the authority of Wikipedia - and more broadly about wiki publishing - have taken three forms.

The first, noted above, relates to mirroring of wiki text across the web. A correction or qualification is not necessarily picked up (and picked up quickly), with critics accordingly sniffing that self-correction is offset by error-spawning. Wiki proponents retort that traditional encyclopedias and other reference works are frozen in institutional/municipal libraries and personal collections, with the supersession of errors being a function of an institutional or private budget ("no family buys a new edition of a print encyclopedia every year").

A second question concerns issues of attribution and responsibility. In practice much wiki content is the responsibility of everyone and no-one, or as the 'WikiProject Countering systemic bias' frets, the responsibility of the wrong (white, male, tech-savvy, Christian and educated) people.

What James Surowiecki in The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations (New York: Doubleday 2004) lauded as collective common sense - populists are more reticent in lauding collective paranoia, xenophobia and sheer stupidity - may not be an effective substitute for the quality control exercised by professional editors and publishers in selecting authors and reviewing text. Anonymity is claimed to be antithetical to an 'acknowledged' author's investment in reputation, with the populist ethos discouraging contributions by experts.

Andrew Orlowski argued that accuracy cannot be separated from readability.

Even when a Wikipedia entry is 100 per cent factually correct, and those facts have been carefully chosen, it all too often reads as if it has been translated from one language to another then into to a third, passing an illiterate translator at each stage.

Simson Garfinkel asked

how do the Wikipedians decide what's true and what's not? On what is their epistemology based?

Unlike the laws of mathematics or science, wikitruth isn't based on principles such as consistency or observa bility. It's not even based on common sense or firsthand experience. Wikipedia has evolved a radically different set of epistemological standards--standards that aren't especially surprising given that the site is rooted in a Web-based community, but that should concern those of us who are interested in traditional notions of truth and accuracy. On Wikipedia, objective truth isn't all that important, actually. What makes a fact or statement fit for inclusion is that it appeared in some other publication--ideally, one that is in English and is available free online. "The threshold for inclusion in Wikipedia is verifiability, not truth," states Wikipedia's official policy on the subject.

The final question concerns perceptions of the way that Wikipedia is received by readers, with critics variously lamenting that hype in the mass media has encouraged uncritical reliance by students and other readers.

More broadly, some readers are claimed to believe that "online and self-correcting equals true" and that "free and collectively-written equals unbiased". Observers have commented that Wikipedia - and all other texts - should be read critically, with the reader being alert to potential bias, errors of fact, omissions and mistatement of interpretation as fact.

Can you trust Wikipedia or other wiki text? One response to that question is that wiki results in content of an unknown quality, with few indications of accuracy or bias. A more hyperbolic assessment is that "wikipedia is the least reliable source of information since the ouija board" and that 'wikiality' is

the reality that exists if you make something up and enough people agree with you.

     quality

In 2005 Wired burbled that Wikipedia

is the largest encyclopedia on the planet. Wikipedia offers 500,000 articles in English - compared with Britannica's 80,000 and Encarta's 4,500 - fashioned by more than 16,000 contributors. Tack on the editions in 75 other languages, including Esperanto and Kurdish, and the total Wikipedia article count tops 1.3 million.

A more meaningful measure might perhaps involve an analysis of the quality of those articles and their completeness, given that a large number are 'stubs' (ie placemarkers).

Former Encyclopaedia Britannica editor-in-chief Robert McHenry, in sniffing at "the faith-based encyclopedia", commented the Wikipedia "method" is that

1. Anyone, irrespective of expertise in or even familiarity with the topic, can submit an article and it will be published.

2. Anyone, irrespective of expertise in or even familiarity with the topic, can edit that article, and the modifications will stand until further modified. Then comes the crucial and entirely faith-based step:

3. Some unspecified quasi-Darwinian process will assure that those writings and editings by contributors of greatest expertise will survive; articles will eventually reach a steady state that corresponds to the highest degree of accuracy.

With an eye to the soundbite he compared Wikipedia to a public toilet -

The user who visits Wikipedia to learn about some subject, to confirm some matter of fact, is rather in the position of a visitor to a public restroom. It may be obviously dirty, so that he knows to exercise great care, or it may seem fairly clean, so that he may be lulled into a false sense of security. What he certainly does not know is who has used the facilities before him.

In 2006 the Britannica offered a tart rebuttal (PDF) to a Nature article that argued Wikipedia compared favourably with the traditional encyclopedia.

In discussing quality control one observer had earlier commented that

Wikipedia isn't really a fact-checking mechanism so much as a voting mechanism. If someone reads an entry, unless something sounds blatantly false, he or she will likely accept what it says. If there is disagreement about the facts, an edit war could break out until a consensus view develops.

McHenry's comments provoked a spirited, if not altogether convincing, response from Aaron Krowne on The FUD-based Encyclopedia, asserting that

McHenry's points are contradictory and incoherent and that his rhetoric is selective, dishonest and misleading.

... sins that never afflict practitioners of commons based peer production.

Nicholas Carr, author of the perceptive Does IT Matter? Information Technology and the Corrosion of Competitive Advantage (Boston: Harvard Business School Press 2004), savaged "hive-mind" rhetoric by critiquing Wikipedia entries on Jane Fonda and Bill Gates.

Carr commented that

This is garbage, an incoherent hodge-podge of dubious factoids that adds up to something far less than the sum of its parts. ... Something that aspires to be a reference work ought to be judged by the quality of the worst entry. An encyclopedia can't just have a small percentage of good entries and be considered a success. I would argue, in fact, that the overall quality of an encyclopedia is best judged by its weakest entries rather than its best. What's the worth of an unreliable reference work?

Wiki proponent Jimmy Wales agreed that the

The two examples he puts forward are, quite frankly, a horrific embarassment. Bill Gates and Jane Fonda are nearly unreadable crap. Why? What can we do about it?

Andrew Orlowski's answer was, essentially, nothing much.

He commented that

Traditionally, Wikipedia supporters have responded to criticism in one of several ways. The commonest is: If you don't like an entry, you can fix it yourself. Which is rather like going to a restaurant for a date, being served terrible food, and then being told by the waiter where to find the kitchen. But you didn't come out to cook a meal - you could have done that at home! No matter, roll up your sleeves.

As a second line of defense, Wikipedians point to flaws in the existing dead tree encyclopedias, as if the handful of errors in Britannica cancels out the many errors, hopeless apologies for entries, and tortured prose, of Wikipedia itself.

Thirdly, and here you can see that the defense is beginning to run out of steam, one's attention is drawn to process issues: such as the speed with which errors are fixed, or the fact that looking up a Wikipedia is faster than using an alternative. This line of argument is even weaker than the first: it's like going to a restaurant for a date - and being pelted with rotten food, thrown at you at high velocity by the waiters.

US parodists The Onion naughtily lampooned Wikipedia in 2006

Wikipedia, the online, reader-edited encyclopedia, honored the 750th anniversary of American independence on July 25 with a special featured section on its main page Tuesday.

"It would have been a major oversight to ignore this portentous anniversary," said Wikipedia founder Jimmy Wales, whose site now boasts over 4,300,000 articles in multiple languages, over one-quarter of which are in English, including 11,000 concerning popular toys of the 1980s alone. "At 750 years, the U.S. is by far the world's oldest surviving democracy, and is certainly deserving of our recognition," Wales said. "According to our database, that's 212 years older than the Eiffel Tower, 347 years older than the earliest-known woolly-mammoth fossil, and a full 493 years older than the microwave oven." ...

The special anniversary tribute refutes many myths about the period and American history. According to the entry, the American Revolution was in fact instigated by Chuck Norris, who incinerated the Stamp Act by looking at it, then roundhouse-kicked the entire British army into the Atlantic Ocean. A group of Massachusetts Minutemaids then unleashed the zombie-generating T-Virus on London, crippling the British economy and severely limiting its naval capabilities. ...

While other news and information websites chose to mark the anniversary in a muted fashion, if at all, Wikipedia gave it prominent emphasis over other important historical events from the same day, including the independence of the nation of Africa in 1847, the 1984 ascension of Constantine to Emperor of the Holy Roman Emperor, and the 1998 birth of Smokey, a calico cat belonging to Mark and Becky Rousch of Erie, PA.

That scepticism has not deterred the naive, with Pete Blackshaw of Intelliseek for example enthusing in 2005 -

For bloggers, it's almost like a badge of credibility to embed Wikipedia in their blog references. There's something about Wikipedia that confers a degree of respectability, because multiple Web users have converged on it.

Much of writing about wiki centres on values of 'community', 'free' and digital technology as a transcendent good. It also features 'us and them' hyperbole and the information populism questioned in works such as The Myth of the Rational Voter: Why Democracies Choose Bad Policies (Princeton: Princeton Uni Press 2007) by Bryan Caplan. That is considered in the following page of this profile.

   next page  (ideology)