This diary elaborates on suggestions that I offered in ek hornbeck's diary of this past Wednesday in response to one of Markos's recent diaries dealing, in part, with changes to site moderation.
Successful site moderation seems almost impossible -- but I think that it can be done reasonably well -- and reasonably automatically. Markos's description of the development of prospective changes, described below the gnocchi, makes many of us self-styled "moderation analysts" nervous. So I want to put this alternative onto the table now.
The secret is this: HRs serve several different purposes which must be disaggregated. Once you separate the "instructions" of an HR into its component parts, it becomes easy to apply different rules to them.
The Discussion So Far
Here are Markos's comments that precipitated ek's diary (in two separate posts):
We are in the process of sketching out a new community adjudication process for problem comments. The "hide" rating is not long for this world. It'll be replaced with a process that allows individuals to flag potential problems. The broader community would then adjudicate the comment.
The end result would be real punishment: if the community ruled against the commenter, then that commenter would receive a time out (exponentially increasing in length each time a new comment was hidden). If the community ruled FOR the commenter, the people flagging the comment would suffer their own time outs.
This is still a very early concept. At some point in the next month I'll have wire frames and a more fully developed plan, and I'll bring them to the community so we can all discuss the full concept and have you guys offer your feedback. I just wanted to make sure you all knew that we are working hard to come up with a way to better moderate the community.
As to concerns over the new community moderation scheme, the big one is fear that organized cliques will manipulate the system. Trust me, we're aware of that problem and are trying to design this new scheme in a way that minimizes their ability to game the system. More details will be forthcoming as we flesh the idea out, but I wanted to make sure you guys know that we're aware of that problem and are trying to account for it.
To this, Moderation Maven ek made a suggestion: allow an unlimited quantity of HRs. If this seems absurd to you, it isn't; refer to his diary above for his justification.
My response to him was here (presented in somewhat edited form and with emphasis added):
Disapproval of comments or diaries should be decoupled from disapproval of commenters or diarists.
People want -- even need -- venues in which to express both of these opinions. But they are different decisions. A "hide rating" against a diary or comment should be able to be made freely -- it just says "I disagree with this so much that it should be removed from our discourse." But, beyond that, people want to say "I find you so disagreeable that you should be removed from the community." That is the sort of action that should lead to the sort of adjudication (with real stakes in terms of participation) that Markos is promising.
Years ago, "Troll Rating" was changed to "Hide Rating." In retrospect, that was a mistake: because we need both. Hide rating should be relatively free, as you state, although should still reflect more than casual dislike. "Troll Rating" should be a momentous decision with consequences on both sides.
You'll see how, over the course of that diary and since, I've refined this idea.
First, let me note that ek has field tested something like this -- and it works.
In sites I administer we have a Wrong! rating that fulfills some of your observations.
I deliberately didn't introduce it to this conversation in the interests of simplicity.
by ek hornbeck on Wed Aug 24, 2011 at 01:47:16 PM PDT
Three negative buttons would probably cover the full gamut of objections to be expressed: "wrong," "hide," and "troll." [Note: in this diary, I'm suggesting a fourth].
by Seneca Doane on Wed Aug 24, 2011 at 01:57:34 PM PDT
Markos was unimpressed -- which is fine. It pushes me to refine my product. That's fair, Lord knows I do this often enough to him.
Ahh -- so you think it's a good idea to find new and wonderful ways for people to be rude to each other?
by kos on Wed Aug 24, 2011 at 03:27:07 PM PDT
Thanks for asking, Markos. As I'm sure you'll understand, no I don't. I hope that you'll find this post useful.
When confronted with a noxious post, site users may want to send one or more or three different and separate messages in response:
(1) I disagree with it (without necessarily saying that it should be hidden or that the user should be banned).
(2) I think that it is injurious to the site and should be hidden (without necessarily saying that I disagree with it -- unlikely, but it happens -- or that the user should be banned).
(3) I think that the user should be banned (without necessarily saying that I disagree with the post or that it should be hidden -- as happens, for example, with "hide on site" HRs).
Usually, choosing #3 will entail #2 and #1 and #2 will entail electing #1 -- but it doesn't work in the opposite direction. #1 doesn't entail #2 or #3; #2 doesn't entail #3.
People want to blow off steam and express their view with judgment #1. This shouldn't require administrative review.
Judgment #2 is more serious -- and really gets to the nub of site moderation, as with "look, I know that you didn't really intend to threaten Eric Cantor's life there, but that's how it read and so for everyone's sake I want it to be hidden." This should rarely require administrative review, mostly only if it is happening repeatedly and unjustly.
Judgment #3 is most serious, and should require administrative review -- even perhaps with the stakes you suggest.
So: three different judgments, three different consequences, three different requirement for review. Solution: three different buttons -- and three different warnings to raters (or just two, because #1 won't really warrant a warning.) MUCH LESS WORK FOR SITE ADMINISTRATORS.
I know that you and I are not bosom buddies, but I think that you know how much I respect and value this site. If I can do anything behind the scenes on a volunteer basis to help think through these problems, just ask.
by Seneca Doane on Wed Aug 24, 2011 at 04:03:57 PM PDT
Long-time user Agathena raises another useful objection:
Deja vu all over again: O 4, O 3, O 2, O 1, O 0.
4 very good
3 good
2 fair
1 not useful
0 troll
I actually forgot what the middle ones were for, just took a guess.
I remember if you got a 3 on your comment it would take 100 or something 4's to get the total up to 4.
by Agathena on Wed Aug 24, 2011 at 04:30:56 PM PDT
That was with them all lumped into one judgment. I'm suggesting that there be three separate "negative" radio buttons. It's totally different -- although one could certainly cut it down by one and make the first button an "agree/disagree" judgment.
by Seneca Doane on Wed Aug 24, 2011 at 04:47:05 PM PDT
A separate discussion arose over ek's proposal for unlimited hide ratings:
Unlimited hide ratings? Already had that. Didn't work.
by kos on Wed Aug 24, 2011 at 03:36:27 PM PDT
Always linked to the threat of autoban. It's autoban that doesn't work.
by ek hornbeck on Wed Aug 24, 2011 at 05:29:07 PM PDT
The problem with unlimited troll ratings (wasn't this before the change to "hide ratings"?) is that disagreement was not disaggregated from the effect of hiding (and, perhaps, from serving someone up to the autoban gods. In the "0 to 4" rating scale, "0" was not like the other numbers.
If one button merely expressed disagreement -- and, hell, that could be a -2 to +2 scale, frankly, or -1 to +1 -- its being unlimited would not be a problem. So people disagree -- big whoop! It's when you start adding more consequences to that rating that it becomes worth fighting over.
Do you really think that people would use other facebook pages to get people together just to express disagreement rather than something more consequential? Well, maybe they would, but who cares? Their names are recorded; it's easy to recognize a coordinated gang and ignore them. The problem you have to deal with here is people setting up such gangs to get comments hidden and users banned. Again: disaggregate them.
The other thing you'll need to do is to establish mores regarding things like sexism, racism, etc., when the author doesn't think that their comment is sexist. A system like this:
"minor infraction": go ahead and click disagree
"misdemeanor": go ahead and hide and wait for eventual adjudication if the fighting continues
"felony": go ahead and try to ban and then suffer the consequences if wrong
would help this work. But it only works if you disaggregate the choices!
by Seneca Doane on Wed Aug 24, 2011 at 04:17:57 PM PDT
So that brings us to the present moment.
A wag, a slap, a club -- and a plea
A negative response to a comment or diary -- whether metaphorically a wag of the finger, a slap on the wrist or cheek, a clubbing on the head, or a plea for intercession by the moderators -- carries with it a message about what should be done. It's a message to the writer, to other site members, to site administrators, and to the outside world that may or may not be taking note.
That negative message may range from "I disagree and I want others to know that I disagree" to "this should be made invisible to anyone but Trusted Users" to "this person should be punished with either suspension or banning" to -- and this is the new one I offer here -- "this comment creates a policy problem that requires adjudication." In other words: "whatever is done with this comment or commenter, we need to figure out how we deal with this kind of comment or commenter generally."
Here's how a comment might look under the sort of system I propose. (I'm not trying to match precisely the font or the color.) Yes, it's more complicated -- but it would make site moderation much easier.
A few initial observations on the above:
(1) There is currently no user named "poxcaster." (Damn right, I checked!)
(2) The above comment does not represent my personal opinion.
(3) The final closing bracket got cut off, among other possible crimes against graphic design. The "[ ]" boxes on the bottom row would be rendered as squares.
(4) Some elements of this layout are not critical elements of the proposal. I hope that people will focus on the "core" elements:
(a) changing "recommend" to "agree"
(b) adding a "disagree" radio button
(c) allowing people to click to hide a comment, separate from agreement
(d) allowing people to click to suspend a user, separate from anything else
(e) allowing people to say that the comment requites a clarification or determination of site policy, separate from anything else
(5) Among the "non-core" elements are:
(a) introduction of an "acknowledge" radio button
(b) what data is presented next to the comment header? Agree and Disagree only? Should number of Hides be shown? (I say "no.") Should the number of "acknowledged" responses be shown? (I lean "yes," but haven't included it here.)
(c) whether there should be "boxes" for the "Report" items or anonymous or by-name polls of TUs favoring each possible remedy
(d) the specific names I've proposed for the "Suspend User" and (especially) "Make Policy" options -- which, I suspect that many will agree, may sort of suck
(e) all of the "Report" items taking the user off of the diary page to a separate page, keyed to the comment in question, where they can discuss these application of these "meta" issues
This will take some explaining, so I'll put it all in a separate section.
The rationales for various aspects of this proposal
The overarching philosophy of this proposal is that it gives people the opportunity to present their opinions about posts and comments, which is something that they appear desperately to want to do, while minimizing the disruption caused by their doing so and facilitating responses to legitimate concerns.
1. Changing "Recommend" to "Agree": Right now, you can recommend a comment because you agree, because you don't know what you think but you think others should read it, because you want the author to know that you saw it without having to comment in return, etc. If we have a "Disagree" option then it makes sense for its opposite to be "Agree" rather than "Recommend."
2. Adding an "Acknowledged" radio button: This was actually a proposal made by Meteor Blades, way back when Hunter was first asking for ideas for modifications to be included in DK4. I think that he was right then and remains so. Sometimes you don't want to say "I agree," but "I read what you have to say, I don't agree enough to click "agree," but want you to know that I don't plan to comment again." The "I don't agree but think others should see this" would also fit here.
3. Adding a Disagree radio button: This would satisfy people's need to publicly condemn that which they find offensive without carrying with it the additional burden of "I think that the comment should be hidden" or "I think that the user should be suspended (or banned)" and such. This is pure disagreement and, unlike Hide Rates, should not set the recipient aflame with anger. We disagree often here; it's OK. People will soon enough learn that you don't have an obligation to click "disagree" even if you do disagree; a small number of "agree" clicks will speak eloquently to the lack of support for a proposal.
4. Creating a separate section for Reporting: This would only be available to Trusted Users. A TU could click any or all of the "Remedies" available in this section. This would, I hope, keep a lot of meta out of the comments and concentrate those who are fighting about which remedies are appropriate in discussion without a broader audience (although aspects of it could also be included in comments, if people prefer.) Currently, I don't provide for TU's to see the number of "Hide," "Suspend" and "Policy" requests without clicking the links, but this could be changed. For "Hide" and "Suspend," checking the box would register as a vote to act; clicking on the link without checking the box would be a vote not to act. It could be structured differently -- for example with a poll regarding the remedy on that page. In that case, no box for the TU's would be necessary at all. The newly proposed line might say, listing number of comments under each category:
TU Discussion [# comments]: Hide [12], Suspend [3], Policy [19]
5. Creating a separate link for "Hide this comment": This would take the TU to a meta-page (or to a section within a meta page) that would not be created until and unless someone clicked one of these boxes for a given comment where the specific remedy could be discussed. Discussions on "Hide" would be restricted to "Should we allow non-TUs to see this?" This will also, I hope, help to hone our policies in this domain. This could also include a drop-down menu that would require the TU to indicate what the reason was for hiding (or that it's a miscellaneous reason.) This would be a good place to discuss "HR Abuse" -- itself a common expressed basis for HRs -- although that may better be relegated to "policy" discussions.
6. Creating a separate link for "Suspend this user." I don't think that we need a link for "ban"; a ban is just a suspension intended to be permanent. Again, going to this "Suspend" link would lead to a discussion and possibly a "roll call" poll, which the Administrators could then address as they wish, including by imposing consequences on those who keep calling for suspensions without good cause.
7. Creating a separate link for "Make policy." Sometimes my own answer to whether someone or something should be suspended or hidden depends on what the site policy is, which is not always clear. This would allow users, without penalty to discuss both the applicable facts and rules -- and possibly to suggest refinements of the latter. This would be a good resource for whoever does moderation -- and a time sink without much gratification for people who just want to sound off. For example, we have supposed policies here that say that you can't say "Obamabot" or "professional left" as an attack -- and I'm not even sure that those policies even exist! This sort of approach would force and focus the issue.
8. Should summaries of the number of votes (and the names of TUs voting) be available to non-TU members (as hide rates, I think, are now?) I think that there are arguments to be made on both sides on this issue; I don't have strong feelings about it at the moment. We want to inform interested observers of what's going on without encouraging grandstanding.
How my proposal will hurt me personally, and why that's good
I'm the kind of user who will probably be hurt most by this proposal -- and I think that that's good. Let me use my own experiences this year as an example.
I am a rather notorious free speech advocate (a broader concept that "First Amendment advocate") here on DKos. (Of possible interest, I'm more of a free speech advocate here than I am in many "meat space" situations, because in my opinion the anonymity and group dynamics of a political blog make the temptation and pressure to squelch opposing opinion especially great.) As a result, I tend to bend over backwards to defend the ability of users to speak freely when I think that there is any realistic possibility that they are doing so in good faith. (In part, I take this view because my sense is that it is what Markos himself wants here.)
This has led me, this year, to substantial conflicts with the GLBT community, with African American users, and with some Jewish users (who don't like to be described as "pro-Israel," so I'm not sure how to describe them, as I consider myself to be as "anti-anti-Semitic" as they are) over the past half year or so. While I consider myself to be a supporter of all three demographic groups, many of them would disagree, at least so far as the issues that have arisen in various diaries (my own included) are concerned. I'm not eager to rehash those battles here; I want to note how they would be better under this sort of proposal.
My posts in their diaries have often been extremely controversial, simply for my not accepting their judgments of what constitutes bigotry. (In their shoes, I recognize, I might be pissed off at me as well.) Except when one person with whom I have a long history of clashes is involved, I rarely get hide-rated -- because either I successfully steer clear of saying anything that bad, or because I am perceived as an immensely powerful force in the blogosphere, or because I have the power to cloud people's minds, or because they are afraid of hurting my fragile feelings. I don't really know. I do know that sometimes I certainly engender a lot of negative comments, though -- which I accept as the cost of speaking my mind.
The problem with the lack of hide rates, I think, is that this makes it look like my comments are in some way being accepted -- and leaving that impression pisses people off. Sometimes, after my (9+ / 0-) comment, someone will post a comment slamming me with might end up (56+ / 0-); that's one solution, but it often isn't taken -- or isn't considered satisfactory.
I would be much happier -- and I think that my critics would be much happier -- if they had a way to say flat out that they disagree with what I have to say, right up there next to the header of my comment itself. They may not think that they can justify hiding the comment; they may not think that I should be suspended over it; but, dammit, they want to have their say in condemning me.
And they should have their say. It's only fair.
They should have their say, moreover, in a way that does not put their own ability to participate on the line, as I fear Markos's developing plan might do. They are not doing anything wrong; they're not trying to suppress my ability to speak; they are just disagreeing with me. That's fine. For those of us who do generally agree that the cure to bad speech is good speech, in fact, it's perfect.
So, by disaggregating the negative feedback they want to give me, we are all better off. It would take some programming time, I'm sure -- sorry, Hunter and elfling and ct -- but it would leave us with a site where people could have their say and where policies would be more easily created. Furthermore, it would "leave a mark." My critics would have the satisfaction of knowing that if I defended something that they considered bigoted, even on free speech grounds, I would have to bear the reputational damage of having done so.
And that's only fair, too. I should have to expect to stand up for what I say. I think that it would lead to a healthier environment here -- one that we could use.
Conclusion
We've got problems here on the site. They're bearable -- but they poison the atmosphere and they retard (used here as a verb, people!) our wider influence. As one who thinks that DKos is important and wants our voices more broadly heard, but who thinks that these sorts of fights impede that end, I'm not satisfied with just letting people continue to slam and plot against each other. I've long thought that maybe moderation just can't work here. With these sort of changes, though, I have some hope that it could.
I look forward to your views -- whether you agree, disagree, or just acknowledge the effort.