Current events

From EvaluationWiki

  April 8th, 2007: EvaluationWiki Bulletin Board is launched.

We've added a bulletin board to allow for discussion about EvaluationWiki.org and evaluation in general. Login and join the conversation.

  September 20th, 2006: EvaluationWiki Has Struck a Nerve

Wow. the past several days have been interesting. I first announced EvaluationWiki.org on EvalTalk last Saturday (three days ago). Since then nearly 1000 people have visited the site from around the world, including United States, Canada, Germany, Australia, United Kingdom, Netherlands, Brazil, Israel, Portugal, Peru, Nigeria, New Zealand, France, Spain, Mali, Puerto Rico, Jamaica, Iceland, Nicaragua, and South Africa. I’ve been emailed and called by many people from evaluation organizations around the world expressing their interest in the project. EvaluationWiki has clearly struck a nerve with the evaluation community. People are asking: how can I contribute?

The most direct answer is: contribute to the Wiki. That is, begin entering evaluation resources into EvaluationWiki. If you are a professor, incorporate the wiki project into your class. Involve your students. If you work for an organization whose mission includes the dissemination of evaluation information and resources, include the wiki in your processes. Obviously EvaluationWiki is still evolving and the very nature of the wiki will change, but the work that you do now will help to build energy and begin to make the site a valuable resource.

The most salient issue that I have been discussing with people since Saturday is why I feel the need for EvaluationWiki to be separate from Wikipedia. I strongly believe that it does need to be separate because EvaluationWiki will differ from Wikipedia in several important ways. I've mentioned some of these previously, and so has Tom Shuster, but I wanted to add to Tom Shuster's comments.

Wikipedia is, for the most part, a totally open-source knowledge system. You or I can go there right now and make any changes we see fit, anonymously. I don’t believe that the value of a completely open-source, anonymous, collaborative, democratic knowledge system (like Wikipedia) has been empirically evaluated. Please, correct me if I am wrong. As I mentioned in a previous email, the journal Nature published a study that suggested that Wikipedia is about as accurate at the Encyclopedia Britannica. But the methods used in that study may be suspect. Read the articles below for more information about this issue:

Britannica Rips Nature Magazine on Accuracy Study

Nature mag cooked Wikipedia study

There is an abundance of anecdotal evidence suggesting that the quality some of the information in Wikipedia is not that good (not that these anecdotes are necessarily any more conclusive than the Nature study). I think that Wikipedia is a great experiment that has taught (and will continue to teach) us a lot about how open-source knowledge systems function and how they can be improved.

Larry Sanger, one of the co-founder of Wikipedia, has openly acknowledged some of the serious limitations of Wikipedia. The points that he makes in the quote below are exactly why EvaluationWiki must be separate from Wikipedia:

Sangers said, "The community does not enforce its own rules effectively or consistently. Consequently, administrators and ordinary participants alike are able essentially to act abusively with impunity, which begets a never-ending cycle of abuse.

Widespread anonymity leads to a distinguishable problem, namely, the attractiveness of the project to people who merely want to cause trouble, or who want to undermine the project, or who want to change it into something that it is avowedly not--in other words, the troll problem.

Many now complain that the leaders of the community have become insular: it has become increasingly difficult for people who are not already part of the community to get fully on board, regardless of their ability or qualifications.

This arguably dysfunctional community is extremely off-putting to some of the most potentially valuable contributors, namely, academics. Furthermore, there is no special place for academics, so that they can contribute in a way they feel comfortable with. As a result, it seems likely that the project will never escape its amateurism. Indeed, one might say that Wikipedia is committed to amateurism. In an encyclopedia, there's something wrong with that."

I believe that the evaluation community must take what has been learned from the Wikipedia experiment and move forward. What the science of evaluation needs is a Wiki-like knowledge system that incorporates the most successful features of Wikipedia (but uses those features somewhat differently) and adds new features not currently supported in Wikipedia but that meet the specific needs of the evaluation community.

To re-iterate what Dr. Shuster has said, I think that EvaluationWiki should not allow anonymous editing. People around the world will always be free to read and consume the information provided by EvaluationWiki. But those wanting to contribute should be required to register and provide some information (as yet to be determined) about their credentials, background, etc. This will accomplish several things. First, it will hold contributors responsible for what they do on EvaluationWiki. In addition, consumers of the information on the wiki can judge its quality by the content itself but also by examining the characteristics of the individuals that have contributed the information. For example, what degree(s) does that person hold and from where, what is their academic publication history, what is their EvaluationWiki publication history, etc.

Thanks to all who have sent me their words of encouragement. Let’s continue the dialogue. R. Shawn Edmondson, Ph.D.

Trustee

Evaluation Resource Institute

  September 8, 2006: The first EvaluationWiki article is created.

  September 5, 2006: EvaluationWiki goes online.

  September 1, 2006: ERI begins the formal process of forming as a 501(c)3 organization.