Skip to main content

View Diary: Science Friday: Real Climate (73 comments)

Comment Preferences

  •  The Shills Have Arrived! (none)
    Where to start? A common tactic of shills (a good example is the stunt pulled by shill for hire Steve Milloy) is to truncate a quote so as to completely distort its original meaning.

    Let us consider what the Schneider (2001) paper actually said (emphasis added):


    ...there are no general, problem-independent criteria according to which the optimality of a method for ill-posed problems can be established (Linz 1984). Hence, any claim that the regularized EM algorithm or any other technique for the imputation of missing values in climate data is ``optimal'' in some general sense would be unjustified.

    In other words, Schneider(2001) was making a very general, trivially true point that no method can be claimed to be a priori superior to any another in some context-independent generality. It depends on the application at hand. The commenter is probably fully aware that the Mann et al (2005) paper cited above specifically tested the performance of the RegEM method in the context of paleoclimate reconstruction based on application to a long-term model simulation where the answer is known beforehand, and the performance of the method can be precisely tested for synthetically designed proxy data with a range of possible signal-to-noise ratios. In this context, the RegEM method was shown to give the correct result within the estimated uncertainties.

    The commenter is also either extremely misinformed (i.e., didn't actually read Schneider 2001) or just plain dishonest when he/she claims that the Schneider (2001) "RegEM" algorithm provides "no defined error model".
    In fact, Schneider (2001) spends a good deal of the paper describing the iterative procedure (based on the statistical principle of Generalized Cross Validation) by which the estimated data matrix is explicitly and objectively separated into a signal and residual error component.

    So, the "RegEM" method is both "regularized" (obviously) and explicitly models the imputation error, pre-empting the two central criticisms of Burger and Cubasch(2005).

    But why trust either of us? The Schneider (2001) paper (and algorithm) are publically available anyway. Or didn't the commenter know that? Readers (warning, some background in statistics required) ought to take a look themselves, and decide who is giving them the straight story, and who might simply be lying to them.

    As for the agenda behind Burger and Cubasch, I'll leave it to others to speculate. But the agenda of the commenter--to disinform the readers of this thread--seems quite obvious.

    •  Keep cool (none)
      Does your comment exemplify what you understand as being "ad hominem"?

      Now to the point.

      1.

      ...that the Mann et al (2005) paper cited above specifically tested the performance of the RegEM method...

      Yes, but didn't I asked for a paleo study that compares RegEM with the original MBH98 (EOF) approach?

      2.

      ...he/she claims that the Schneider (2001) "RegEM" algorithm provides "no defined error model".

      The extrapolative error described by Bürger and Cubasch depends on the error of the model coefficients (of the regression) and not of the data. No such thing for RegEM, as seen here:

      [Schneider 2001]...The uncertainties about the adequacy of the regression model (1), of the regularization method, and of the regularization parameter all contribute to the imputation error, but the error estimate [...] does not account for these uncertainties.

      Consequently,

      Covariance matrices estimated with the regularized EM algorithm and statistics derived from them must therefore be interpreted cautiously, particularly when the fraction of missing values in an incomplete dataset is large.

      In Schneider, that fraction is 3%. You (you?) apply the method to 2000+ unknown grid points times 1000+ years back in time and don't ever even mention to be cautious. That is really strong!

      •  the last time we're going to discredit your claims (none)
        This is getting tiring, and we won't encourage you on any further than this.

        We wil discredit your main new point: You now ask for a study that compares the MBH98 and RegEM approaches. Why don't you take another look at the Mann et al (2005)paper provided above, and actually read it. What does figure 2 show? Ah yes, a comparison of applications of RegEM and MBH98 approach to the same precise data set, showing the two methods give very nearly the same result, well within the mutual uncertainties.

        Now, we could pick apart everything else you just said, just as we did in the first round(especially your comment about the multiple contributions to the estimated error term which is a strength, not a weakness as you seem to imply, of RegEM). But this is now just getting semantic and boring.

        We'd rather spend our time helping to educate the thousands of visitors a day that visit RealClimate who are often genuinely interested in learning about the science.

        Your cherry-picking and deceptive quotation are probably not welcome by the DailyKos readers, and we're not going to encourage you by responding further.

        •  Ah no (none)
          I'm sorry, but I really can't see how Figure 2 could prove that RegEM is superior to MBH98 - which was the outstanding issue, wasn't it?

          It is becoming confusing now, I agree.

          Good luck with RealClimate! I hope not too many people have been turned off that site by the general tone of this discussion.

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site