Yesterday, I wrote this diary about the accurate use of statistics and why self-selecting blog polls cannot be used to predict the opinions of the DKos population as a whole http://www.dailykos.com/...
As a follow-up today I would like to go into why it is important to use statistical analysis correctly and not make claims that are false about your blog poll results. I will use some comments from recent conversations to illustrate my points.
******Update***** 1/1/2009 3:50pm EST
The Diarist Chimes in with more pithy comments defending his use of blog polls that have self-selected samples as statistically valid to predict the views of the Kossacks overall...here are some of his recent comments for your consideration:
They are statistically significant...
and can predict what the Kos community thinks. That's why Kos uses them.
Hey Kos, this guy actually thinks that you believe that blog polls have some statistical significance to the total population of Kossacks...do you have a comment on that?
You're obsessed, and nobody believes it's about statistics.
Well he is right about 1 thing I am obsessed with the misuse of statistical analysis...but does anyone think this diary is about anything but statistics?
You have no credibility on this topic
other than with your 33%.[who are in the minority in his blog poll]
So who thinks I have no credibility on the topic the ability to draw a statistically valid conclusion on a poll...Anyone?
Why would I bother with your diary?
It doesn't deserve to be legitimized by anyone but your 33%.
The diarist believes that all the commenters here are only supporting my statistical analysis contentions is that you agree with my I/P position...can that possibly be true...?
just shows how obsessed you are with this I/P debate.
We already know your position.
And the diarist still thinks this is about the I/P debate...which if he read my diary would know it has nothing to do with that...
Final update, now the diarist has resulted in calling me names and a Troll...oh no...I am really scared now...
****** Update Completed *******
First of all, as many of you know here, I have a history of being a minority POV at DKos on a whole host of issues from are we a progressive blog to supporting Hillary, liking WJC, supporting Obama's reason for voting for the re-authorization of FISA and on the list goes...you can look at my 5 year record and see that I am not the most popular commenter or diary writer on the block...that being said...I have never been banned either and rarely been Hide Rated for negative comments.
So I have a long history of being in the minority both at DKos and IRL (see my profile) http://www.dailykos.com/...
I have no issues being in the minority...in fact I rather enjoy it and the challenges of being in the minority...so this discussion is not about changing people's minds about any current issue including the recent debates at DKos about the I/P conflict.
This is a discussion about being reality and fact based when applying statistical analysis. Why is it important to apply statistical analysis accurately and factually according to the laws of probability and statistics? Because so many organizations outside of DKos misuse polling and statistics to make a false reality that we need to combat that as a "Reality Based" site. So with that being said...lets get into some details...
This all came about from my comment on this small portion of a diary...
We also seem to believe that the actions Israel has take over the last week are wrong.
By a 2 to 1 margin.
And if this were a democracy rather than a blog run by a former member of the US military, that would mean we have a mandate here on Kos to condemn Israel's actions of the past week.
Which is based on the poll results in this diary...
http://www.dailykos.com/...
and the conclusion that the diarist made was:
I think that at 220 votes, we've hit our statistical level. 66% No. 33% Yes.
It looks like the majority of us here on Kos, a blog run by a former member of the US military, think that what Israel is doing this week is not right.
So without commenting on the substance of the diarist's conclusion, I simply and nicely commented to the diarist as follows:
My only quibble with your diary...
is falsly representing that Kossacks do not approve of Israel's actions by a 2:1 margin in a non-scientific self-selcting on-line poll...that only had 372 responses out of nearly 200K Kossacks...
Please clarify that in your diary for accuracy...Thanks...
This is a typical minor correction request to diaries that Kossacks regularly make of diarists so that we can be accurate in our assertions in our diaries. As most Kossacks know and believe, a factually correct diary is much more powerful and persuasive than a diary with factual inaccuracies...so I assumed the diarist would want to correct the assertion that a blog poll which is derived from a self-selected sample can be used to statistically project the opinions of the DKos population as a whole.
This was all I was looking to accomplish...no more no less.
Of course the diarist thought I was trying to attack his position on the I/P conflict and a host of other interesting attacks. To illustrate, lets go a little deeper, shall we.
So the initial response to my request from the diarist was this...
327 is more than enough to get a statistically relevant poll here on Kos.
Personally, I'd love to see Kos have a frontpage vote.
I won't hold my breath though.
It seems that gauging Progressive's opinions here on Kos with polls on issues is important unless it's about the most important issue in the world right now.
It'd be tragic for about 33% of Kossacks to have a clear picture of what Progressives really think about I/P.
Now depending on the population size 327 may be enough to obtain a statistically valid sample with a MoE of +-7% but that is not the point. The point is that a blog poll by its very nature is a self-selected sample of the population and therefore cannot be statistically relied upon to predict with any accuracy the whole population's opinion on a topic.
Then there were various exchanges which I had hoped to end with this friendly non-attacking comment:
A self-selecting poll is not statistically...
significant because it is based on the poll taker deciding wheter to take the poll...so even the FP polls are not statistically significant and should not say so...what they can say about their poll is that out of x number of respondents Y said this and Z said that...they cannot draw a conclusion about the DKos population at large either...take it from someone who has taken statistics...
This is not a knock on you...it is just basic statistics theory...the selection from the population has to be randomized where no item from the population has anymore a chance for selection than another...since a self-selecting poll answer does not have that feature because the respondents are selected based on who is on-line at the time and who is interested enough in the topic to respond...not every person in the population has an equivalent chance of participation and response and therefore the result cannot reflect the population at large...
This was a comment that I thought put in a way that non-statistically educated people could understand why making a statistically based claim about the Kossack community was not possible from a self-selected population. But no that just increased the resolve of the diarist to make the claim even more forcefully.
but it is statistically significant in that we can extrapolate the views of DK users from its results.
That's why DK puts polls on the frontpage of its website... because they are statistically significant.
So to my knowledge, no FPer has made claims that their FP polls are statistically valid for predicting the opinions of the nearly 200K Kossacks...and to say this from a self-selected population violates the principles of statistical analysis...but we continued for several days on this here are some more exchanges...
I tried putting in links that explained why self-selected samples cannot be used to predict with any statistical reliability the population as a whole...like this one...http://ellisonresearch.com/Articles/Article18.htm
Then I got this response:
Your argument has to be that every single
poll on DK's frontpage is statistically irrelevant.
That's nonsense. Arguing about the semantics here won't enable you to disguise the fact that this poll, like any poll on DK's frontpage is statistically useful for gauging DK users views on the issues.
To which I responded...while the polls do tell us something about who read the diary and is so inclined to respond a self-selected sample cannot statistically predict the results of the population as a whole because the sample may not be a representative sample of the population overall. This happens in self-selected samples. It does not matter if it is a FP diary with 10,000 responses or a diary with 400 responses...self-selected samples cannot be relied upon to predict the population overall because of sampling error rate is not able to be calculated.
And then the diarist simply declared...
Nope
They are statistically significant
And when asked for proof that this is true just repeated the statement over and over again and then attacked the motives of the commenters nicely trying to get the diarist to make one small correction to the diary.
So at the end, the diarist wanted to believe that a self-selecting blog poll could be used for drawing conclusions about the population as a whole of Kossacks...We all know as part of the reality based community that is not the case. We can talk about the poll results and draw conclusions about what it means and make inferences that are not statistically driven...all of those conclusions are fine...In fact, based on ancedotal evidence regarding the number of diaries and comments on the topic, I agree with the diarist's conclusion that a majority agree with his opinion...but we cannot conclude that based on a statistical analysis of a self-selected blog poll.
So I hope that all Kossacks will join me in making sure that we do not use the term statistically in describing the results of our blog polls that by their very nature have a self-selected population and cannot be relied upon to predict the population of Kossacks overall.