Ratings Abuse: Here's How We Can Fix It

The ratings system here is being abused constantly.  I see people giving out 1s and 2s simply because they disagreed with the content of the post.  For example, one poster gave out a 1 to another simply because they disagreed with how they thought a certain senator should respond to a question.  It's becoming clear that people simply aren't paying attention to the guidelines.  Thus, it's time to enact more forceful measures.  Below the fold are my suggestions - some are more drastic than others.  Feel free to add further suggestions.

Also, take the poll on which suggestion, if any, you like most.

1.  Don't allow people to rate threads in which they have posted.  Too often you see people down-rating posters with which they are having a disagreement.  People who are doing this are abusing the system and using the ratings to try and make their post seem more valid.

2. Don't allow people to rate posts to which they have responded.  By responding to a post, you're indicating one way or another how you felt about that post - there's no need to do it twice.  Moreover, your rating is likely self-serving at that point - you're either trying to validate or question the content of the post.

3. Don't allow people to rate posts until they have spent a certain amount of time on the boards.  Arbitrary?  Sure.  But it's not uncommon for many boards to install this feature.  Moreover, it doesn't hurt new posters to have to wait a bit and learn about the board before they start rating people.

4.  Install a meta-moderating system.  Those of you who visit slashdot will be familiar with this system:  Posters who have been around a bit and have achieved enough points are allowed to judge whether a rating is fair or unfair.  Posters who rate unfairly often enough lose their privileges.  You can read about the slashdot system here.

These are just four suggestions I have, though I'm sure that people have other good ideas.  If you don't like mine, what do you think can be done to improve the system?  Or, do you think the current system is fine?

•  a christmas story, part 2

  "no, you can't have a ratings system - you'll shoot your eye out."

  by bsmcneil on Sun Dec 05, 2004 at 02:05:18 PM PDT

•  Problem?

  I don't find ratings abuse to be that much of a problem.  Show me some evidence of "ratings abuse" and then I'll consider recommending this thread.

  by gsg on Sun Dec 05, 2004 at 02:05:20 PM PDT

  •  asdf

    I prefer not to call out individual posters. because I don't think that would be helpful.  But, it at least seems to me that ratings have become more a way to indicate agreement/disagreement with a post rather than whether the post is above-average or a troll.

    Those who feel otherwise are more than welcome to vote to leave the system as it is.

    by patagonia on Sun Dec 05, 2004 at 02:09:02 PM PDT

  •  Here's a good example

    http://www.dailykos.com/comments/2004/11/28/01814/141/12#12

    I choose this one mainly because it includes a discussion of the ratings.

    by JenAtlanta on Sun Dec 05, 2004 at 02:48:28 PM PDT

  •  JenAtlanta gave an excellent example

    In a sense you are right it's not a big issue. But it is one that comes up regularly. People feel strongly about it the few times it happens to them/us. If you are a borderline trusted user, one drive-by TU abuse will knock you out of TU status.

    The status is over-rated, but carries a lot of emotional whollop. I favor the, what I call, the court of appeals approach.

    by wegerje on Sun Dec 05, 2004 at 03:41:25 PM PDT

•  only 2 ratings

  There should only be 2 ratings- "hat tip" and "alert!".  Hat tip (or something similar) should be what 4's do now.  Its just a post you thought was particularly good and deserved some recognition.  The alert button should only be for trusted users and it should be for posts that are disruptive to the point that they should be remove from the site and the user banned if the behavior continues.  These descriptions are much more accurate for how the ratings should be used and would help a great deal.

  by Damon on Sun Dec 05, 2004 at 02:11:45 PM PDT

  •  2 ratings

    I think that's a pretty good idea.  I'm fairly new, about two months, and the only rating I give is a 4.  Giving a 4 to say "good job," somehow does not seem judgmental in the same way that giving a 1 does.  I have given a few 1's, but decided that I don't want to be the ratings police.  That is not the reason that I come to this site.

    by elveta on Sun Dec 05, 2004 at 02:17:27 PM PDT

  •  hat tip to you

    for that suggestion

    by Joe B on Sun Dec 05, 2004 at 02:54:57 PM PDT

  •  Excellent Idea

    on the 2 rating system. It makes a lot of sense.

    I also think that those who do give 1s and 2s should post a comment as to why they gave the writer a low score. The onus should be on the rater to substantiate it. This way, the person who receives the score doesn't post the proverbial "why did you troll rate me?" post subsequently.

    I haven't found the rating system on this site to be a big problem. But I do enjoy giving 4s to the writers of comments who really deserve them. At least, I try. It's why I think a 2-tiered system will solve some problems: it's simple and doesn't really allow for loose subjectivity. A 2-ratings system would be more cut and dry.

    by juniper on Sun Dec 05, 2004 at 04:15:00 PM PDT

•  Not fond of ratings nor discussion therof

  Leave Troll rating up to trusted user only.
  Post ratings simply
  [1] agree
  [0] disagree
  Average shows clear response of ratings.

  Avoid nasty discussion on rating substance as it's not that important. Let people comment a post.

  Ratings offer insight in personalities of KDos member and is useful in establishing contact with their diary and contribution. Links to a certain humor.

  Born a Liberal, voting Liberal, dying as a Liberal: á la Vie á la Mort
  

  by creve coeur on Sun Dec 05, 2004 at 02:30:19 PM PDT

  •  qwerty

    I agree with what creve coeur says above.

    by jayatRI on Sun Dec 05, 2004 at 02:44:06 PM PDT

  •  how do you decide

    who gets trusted user status with this system?

    by yankeedoodler on Sun Dec 05, 2004 at 04:00:25 PM PDT

    •  I have no knowledge of DKos method

      I imagine it's a proven track record
      

      • in time and activity on DKos
        
      • not getting troll rated
        
      • community decides
        

      Surprise, it just evolves.   Any listing of % DKos is TU?

      I think no one new on blog should start with giving troll ratings. Good manners to your host and earning respect is how society [should] work.

      Born a Liberal, voting Liberal, dying as a Liberal: á la Vie á la Mort

      by creve coeur on Mon Dec 06, 2004 at 04:55:47 AM PDT

•  Some of your suggestions

  would create the potential for more problems than we currently have.  Ratings abuse happens but it really is not widespread, not a big problem, and is self-regulating over time because the majority of people won't stand for it.

  by diplomatic on Sun Dec 05, 2004 at 02:43:47 PM PDT

•  a plus - minus rating system might be okay

  it would probably be good if you had to accompany any negative rating with an explanatory comment. preferably longer than "you suck!"
  in general, though, this is the best blog on the internets - take care tampering with what works.

  by rasbobbo on Sun Dec 05, 2004 at 02:54:33 PM PDT

•  another few ideas

  I am sure that kos is working on the question of the ratings and considering the various ideas that have been floated here.

  Unlike some people, I do think that there is a problem of ratings abuse, and this diary has gotten me thinking about it.

  One of the difficulties, with regard to all of the rules and customs of this blog is that they are not compiled in one, easy to find and refer to place. Since I am sure that this will be rectified, here are a few ideas about how to infuse them more into the life of the community.

  1. It might be helpful, that as part of the registration process, new members be requried to check a little box, conventinetly located near the rules themselves, that says "i have read the rules and agree to them."

  2.  Anyone who has had a post removed from the site cannot rate, post, or diary again until they have gone back and read the rules and indicated in the little box that they agree to abide by them.

  3. Once the new rules are revised and located in one easy to find place, all registered users should have to go back, read the new rules and check the little box before they can rate, comment, and post diaries.

  by Frederick Clarkson on Sun Dec 05, 2004 at 03:22:07 PM PDT

•  Disagreement/Disruption

  I see people giving out 1s and 2s simply because they disagreed with the content of the post.

  This is not justification, it's explanation:

  I suspect that very few people intentionally give out 1s and 2s just for disagreement; they're giving out the 1s and 2s to posts (and posters) they honestly think are disruptive. The 1 doesn't mean "I disagree with you, and don't care whether you expressed it well or not"; it means "I don't think you belong here; most any strong opinion you express I'd consider trollish."

  If true, the solution isn't at easy as more reminders that rating isn't about agreement with the opinion.

  by Garrett on Sun Dec 05, 2004 at 03:31:54 PM PDT

•  It's not a big problem but...

  big enough that some solutions should be looked at.

  I like the technological approach that slashdot has implemented. That's a bit of coding though and it's either paid for or waited for.

  Another way to go that would be less technological (stiil some required)  would be a dKos Court of Appeals.

  In it certain users would be given the powers and responsibilities of reviewing abuse cases and meteing out justice. As the article says, it may well be overkill. I'm not happy with the abuse, but so far it's been manageable. It just hurts the dKos community spirit when it happens.

  While most of us hand out way more 4's than anything else, I know at least one "Trusted User" who hands out way more 0's than anything else. That's "Marginal" behavior, as the saying goes.

  by wegerje on Sun Dec 05, 2004 at 04:04:05 PM PDT

•  I agree with all but #2.

  You write

  2. Don't allow people to rate posts to which they have responded.  By responding to a post, you're indicating one way or another how you felt about that post - there's no need to do it twice.  Moreover, your rating is likely self-serving at that point - you're either trying to validate or question the content of the post.

  The drawback I see to this suggestion is that it prevents people from rating a post and then explaining why they rated it.  I find that this practice is helpful, and would also be blocked.  

  by osterizer on Sun Dec 05, 2004 at 04:26:34 PM PDT

•  How is the ratings system not working?

  I look in the hidden comments, and I see CRAP. From Bob Johnson's witty rejoinder "Fuck off, asshole" (which is a bit out of character for him, but whatever), to lots of antagonistic drivel by blues, tenaj, PhillipG... for example this by tenaj:

  BUSH GOT WHAT % OF THE VOTES FROM THE TROOPS IN IRAQ? IT WOULD BE KARMA IF PARENTS WHO VOTED FOR BUSH LOST THEIR CHILDREN IN HIS WAR

  Removing the crap content is the main purpose of the ratings system, and it does it well as is. And, it provides a solid disencentive to trolls, because their crap just vanishes.

  What problem are you trying to solve? Is it systematic, or is it the behaviour of a few users? I don't see any systematic problem.

  The only change I would recommend is this: Let everyone, trusted or not, see the hidden comments, but only let the trusted users vote on them once they are hidden (to prevent a Cabal of Trolls from detrolling each others posts). It wouldn't solve anything, but it might make people stop complaining if thy see the crap being pulled from the comments.

  by Gustavo on Sun Dec 05, 2004 at 04:29:43 PM PDT

•  asdf

  Don't allow people to rate threads in which they have posted.  

  Don't allow people to rate posts to which they have responded.

  Unfortunately, I think that both of these suggestions would make things worse.  I want people to explain troll-ratings, unless the reasons are obvious, as they often are.  I don't like it when random people, who haven't participated in the discussion, troll-rate a post with no explanation.

  by dtj on Sun Dec 05, 2004 at 07:00:40 PM PDT