Skip to main content

View Diary: Researchers finally replicated Reinhart-Rogoff, and there are serious problems. (124 comments)

Comment Preferences

  •  here's the thing (13+ / 0-)

    these are hot-shot Harvard economists who don't need no stinkin' peer review of their data!

    but in all seriousness i think the answers breaks down to two things<:

    1) I'm pretty sure the journal referees (the guys who edit journals and manage what gets accepted) don't actually take the data and re-inpupt and calculate it all to verify regressions were run correctly.

    2) R&R has refused to give their data out until they made an exception for H-A-P. But for years, since the study came out, progressive economists at the EPI and CEPR, particularly Dean Baker, have been screaming about this because they wanted to reproduce and check the math and logic of the paper that has been used to justify society-destroying austerity policies across Europe and being touted in the US by the Republicans (predominately).

    hopefully this will be a wakeup call for journals and academic community in Economics to really buck up and stop worshipping big-name Ivy Leaguers and to hold everyone accountable.

    Deficits don't matter, jobs do.

    by aguadito on Tue Apr 16, 2013 at 01:30:00 PM PDT

    [ Parent ]

    •  How the f-- is it science if the data is secret?!! (20+ / 0-)

      Why didn't that alone discredit it?

      Try this crap in any other 'science' field and see how fast you're laughed out of the room.

      Cold fusion, anyone?

      Another f-- job based on 'secret evidence' iow.

      •  Here's what I know about this: (3+ / 0-)
        Recommended by:
        ozsea1, greenearth, Involuntary Exile

        -- projects spend lots of money to check the data entry of single entry points by low-level staff

        -- projects tend to spend no money on checking the coding for database management or statistical analyses, partly because there's often only one person on the team who could catch the error, and it's the same person who did the original programming

        -- database management and statistical analysis coding errors are common.  I found major ones in every project that I either inherited from someone else or collaborated with.

        -- these errors are sometimes due to simple typos, sometimes due to lack of understanding.  The first can be caught by the original programmer; the second can't.

        -- peer review doesn't review the coding.  Neither, usually, does the principal investigator, who takes responsibility for the work published under his or her name

        -- as much of a problem is the non-reporting of analyses that have no publishable results (because the "significance" of an analysis is based on the probability that the findings would occur by chance).  This greatly biases the results that get published.  Every principal investigator I've worked with has done what you're not supposed to do: run one analysis after another and discard everything but the publishable result.

        It's a wonder that we know anything at all.

        •  Not responsive to my pt. They refused to publish t (2+ / 0-)
          Recommended by:
          Involuntary Exile, Sychotic1

          he data but published the results and 'analysis'.  I'm not talking about errors or not publishing some results or anything you mentioned.

          The most basic rule of science is 'show your work'.  That includes you data.  So others can review your analysis, but also try to replicate the data.  If you refuse to reveal the data, you might as well just make crap up.

          That is not science.

          •  They did make this crap up. nt (0+ / 0-)

            Might and Right are always fighting, in our youth it seems exciting. Right is always nearly winning, Might can hardly keep from grinning. -- Clarence Day

            by hestal on Wed Apr 17, 2013 at 05:49:25 AM PDT

            [ Parent ]

          •  Whatever the merits of this particular (0+ / 0-)

            study and the reasons for its conclusions/errors, there is no basic rule of science to publish the data.  There's no format for it, nor would it be feasible.  Nor would it guarantee accuracy, because data itself often has to be extrapolated and collated.  Your statements make me wonder about the basis for your convictions about how science is practiced, reviewed, and published.  My only concern is that more people don't have the same misconceptions.

            The replicability of data is based on sharing the methods of the research, not the raw data of it.  Even when research is based on data that can be sourced, that doesn't ensure that there isn't an error along the lines that I've outlined.

            There's a presumption of integrity and competence among scientists.  When one of those fails, it's a big deal.  But errors get made all the time in research and one can only hope that they get caught before the work is published.

            •  in an economics paper such as this (0+ / 0-)

              you DO publish the raw data. I totally agree that there is no basic rule of science because different studies vary. And your points about not guaranteeing accuracy are well-taken. But when using prior historical data in economic studies, you absolutely include either the data or specific references where the data can be found.

              For that reason, the journal that published this is as much at fault as anyone. Undoubtedly, the authors' reputations contributed to how that went down.

              Want a progressive global warming novel, not a right wing rant? Go to and check out New World Orders

              by eparrot on Wed Apr 17, 2013 at 09:53:13 AM PDT

              [ Parent ]

              •  Internet Has Improved Availability Of Data Sets (0+ / 0-)

                It is more common now to have the data sets available for download on a journal's website.

                This data set was so small it could have been included as a table of fine print in the text version of the article.

                There’s always free cheddar in a mousetrap, baby

                by bernardpliers on Sun Apr 21, 2013 at 10:11:43 AM PDT

                [ Parent ]

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site