Recent discussions regarding content, quality, applicability, and utility of diaries have re-ignited my interest in something I touched on a while ago in a comment thread...I have talked with several folks about this, and they suggested I hold off on this discussion until later.
Well...now is later, and here is my idea, short and sweet:
DailyKos can and should push the boundaries and expand the envelope of blogospheric communication. Tagging is one example of this. Diaries and recommended diaries - the "blogs within a blog" concept - are other examples of Kos leading the way in communication techniques.
Love Kos, hate Kos, love the front pagers or hate them, love the BlogMeister or hate the man, Kos has led the way and is copied widely by allies, friends, enemies, critics.
What is the next step?
I think it is peer review, and I will explain my idea, and some ideas for how to implement it on the flip side.
Peer review embodies one of the primary reasons why modern science has been able to progress so fast and spread successful research results so widely and quickly across the globe and across cultures.
Peer review engages all levels of inquiry, requires that reviewers familiarize themselves with the research, the material, the techniques, and the background knowledge base used by the authors of any scientific/technical/professional work.
Peer review is used in basic science, in social science, in engineering, in software, and in many, many other fields.
I believe that the diaries should be peer reviewed.
I believe that the community here is diverse enough, intelligent enough, committed enough, and dedicated (read fanatical) enough to accomplish thorough, efficient, comprehensive, and rapid peer review.
I believe that the benefits of peer review would be substantial:
- Peer review would eliminate a large percentage of the idiotic chaff that clutters the diaries, often pushing more deserving, better works (even better works in the same vein) off the visible list and into oblivion.
- Peer review would provide a means of automatic exposure of EVERYONE'S writing to a minimum number of people.
- Peer review would provide a means of sharpening and honing and revamping arguments or theories or ideas beyond the initial, rough, sloppy state that they all-too-often take.
- Peer review would eliminate many, many repetitive diaries.
- Peer review would do a lot to eliminate "cliques" and subgroups that band together to get their particular groups' members elevated to the recommended list regardless of the worthiness or utility of their particular work, idea, critique, or message.
How to do it? How to make peer review work to increase quality without limiting access or hampering the free-flow of ideas?
Here is my suggestion as to a "workflow" - keep in mind that I am NOT a programmer or Scoop expert, and if this idea is useful, I hope that it would be sent to those who are for implementation, modification, or whatever.
- An author writes his or her diary as usual, with as many or few words, links, and etc as they desire.
- The auther clicks "preview" and "submit" just as is the normal procedure...
- Instead of getting the "Your diary has been posted" message, the author is sent to a "Review Page" - in this page, the diarist is presented with a list of OTHER diaries that are "in the queue." In order for the diarist to have their diary added to that queue, they MUST review, rank, rate, correct, and add suggestions to a minimum number of OTHER diaries that are already in the queue. That minimum number is a topic for later discussion...I think it should be 4. Diaries in the queue are ANONYMOUS! You do not have any idea, based on the title of the diary, who wrote it. Once you choose to review a diary, you cannot back out.
- The diarist reads, rates (in a special "diary review ratings" checkbox), and adds critical commentary to the minimum number of diaries. Should they wish to review MORE diaries, then they could start amassing "Reveiwer's Mojo" and in return they could: not have to review as many diaries the next time, get a standing reputation as a prolific reviewer and therefore gain friends and influence people, possibly get "Trusted Reviewer" status which would allow them to gain more "Reviewing Clout" or any number of other such incentives.
- Once a diary has received a minimum number of reviews (20? 50? Traffic dependent? I don't know??!!). Those reviews (which consist of rankings on one or more "checklists") are averaged - averages could be "weighted" based on "Reviewers' Mojo" or simply averaged....perhaps "Trusted Reviewers" would have the ability to revisit the reviews on a diary and expunge spurious, vindictive or insane reviews that have besmirched an otherwise exemplary diary, or whatever...
- Those averages are then used to "Pre-Rank" a diary, and determine its initial placement. Example 1: A diary is written and is reviewed by 100 people over the course of 3 hours. The average "Rating" of that review is a 9 out of possible 10 - a truly outstanding diary! It is automatically placed at or near the top of the recommended list due to its excellent reviews. It's recommendation status then begins to decay just like every other diary...but non-reviewers can recommend it (or not) just as before...if the reviewers were nuts and the diary stinks, it will quickly fall off the reco list. Example 2: A diary is written and is reviewed by 100 people over the course of 3 hours and receives an average "Rating" of 4 out of 10. It is published and is placed on the normal "Recent diaries" list in chronological order.... diaries are placed on the diary list in the order they exit review. Example 3: A diary is written and reviewed...it is ranked by 100 people in three hours, and is given a "Rating" of 2 out of 10. It is sent back to the author with attached rankings, comments and suggestions from the 100 reviewers.
- Reviewers' identities remain anonymous unless they specifically CHOOSE to be identified.
Can such a system be gamed? Of course - but that gaming would be more complicated and take more effort than is currently the case.
Can such a system bog down and be too slow? Of course, but I think that possibility can be dealt with through tweaking. Given the traffic, the reading rate, and the number and dedication of the participants here, I think that any given diary will receive at least 25 reads and reviews within a VERY short period of time - especially since ANYONE wishing to publish a diary MUST review a minimum number of OTHER diaries.
Is the anonymity of author and reviewer a bad idea? I don't really know....the upside is that we can bypass cliques and cabals and potential groups of saboteurs from the Right trying to post chaff and bullshit (or loons from planet zorg trying to alert us of the dangers of scalar weaponry)...on the other hand, we often rely on name recognition of highly respected authors to guide our reading habits (Meteor Blades, Maryscott, and so on...).
In any case, that's my suggestion.
What do you think?