For the last decade, the science of psychology has been in what is often termed a ‘crisis.’
In the biggest project of its kind, Brian Nosek, a social psychologist and head of the Center for Open Science in Charlottesville, Virginia, and 269 co-authors repeated work reported in 98 original papers from three psychology journals, to see if they independently came up with the same results. …
According to the replicators' qualitative assessments, as previously reported by Nature, only 39 of the 100 replication attempts were successful.
Widespread concern over the failure of some well-respected psychological research to stand the test of replication went back to at least 2010, when reports of errors, bad methodology, and plain old fraud roiled the academic community. Nosek’s 2015 review wasn’t the first of its kind, but the scope of the issues he found, affecting a broad area of the field, only increased the uproar. These weren’t just just any academic papers. Some of them were foundational to many other studies. Among the research that failed to meet the reproducability test, were some that were behind assumptions about which people were suited for particular jobs, about how soldiers would behave on the battlefield, even about how children should be taught in school.
The results of this and other surveys of the field tossed psychology on its ear. Some, including many non-scientists who had an interest in seeing the field degraded, felt vindicated. Some of those involved in the studies that had not been verified took the paper not just as an attack on their live’s work, but as a personal challenge to their honesty and integrity. But many scientists were happy that this fight was being waged … over there. In psychology. One of those soft sciences where results were all too often dependent on interpreting responses and reactions. It wasn’t something that could affect biology. Or chemistry. Or the squeaky clean realms of physics.
Except, of course, it could. As scientists increasingly press forward in areas where the “easy” work was wrung out decades ago, or conduct experiments that take the resolution of their measuring instruments to the limit, even the hardest of hard science can fall prey to subjective interpretation and wishful thinking. Reproduction of results when the tiniest change in experimental design can swamp results can generate hair-pulling frustration. Ask the people who have dealt with “cold fusion” for decades. Or those attempting to verify announcements of reactionless “EM drives” today. Or ask any of the researchers who have lately be caught, if not falsifying results, at least cherry picking for success.
The truth is that all fields of human science are subject to … humans. Who don’t always quash all the variables, or think through every aspect of experimental design, and who may or may not be perfectly clinical when searching for a lump of gold among statistical dross.
So several of this week’s papers are about scientists looking to do better science. And if that sounds boring … nope, not at all. Come on, let’s go read some papers.
Meta Science
Daniele Fanelli’s background is in the study of behavior among social wasps … and the behavior of scientists. His paper addresses something that a lot of others have ignored: Is there actually a reproducability crisis? With the emphasis being on the word “crisis.” That is, are scientists so affected by this idea that it’s become more important than doing good research?
Is there a reproducibility crisis in science? Many seem to believe so. In a recent survey by the journal Nature, for example, around 90% of respondents agreed that there is a “slight” or “significant” crisis, and between 40% and 70% agreed that selective reporting, fraud, and pressures to publish “always” or “often” contribute to irreproducible research.
Powell’s concern is that the “crisis narrative” has set in to the extent that it can be paralyzing. Science is an international endeavor where millions of working scientists readily adapt, absorb, and build on the work of others. How does that happen if the basic trust is published research falls to nothing? This concern is also being whipped up by the already open floodgates of online “research publications” that are the scientific equivalent of vanity presses. Not only are they lacking in traditional peer review, some of these new “journals” are intended to do nothing but provide a outlet so poor scholarship — for example, climate change denial or creationism — can gain the imprimater of publication.
Put simply, this new “science in crisis” narrative postulates that a large and growing proportion of studies published across disciplines are unreliable due to the declining quality and integrity of research and publication practices, largely because of growing pressures to publish and other ills affecting the contemporary scientific profession.
However, Powell argues that the biggest crisis may actually be the crisis narrative. Broad studies suggest that research integrity across most fields — including psychology — is high. As always, research that returns surprising results, or which presses the bounds of measurability needs to be viewed with scrutiny, but Powell sees no sign that science is about to go down under a flood of charlatans and fools. Not only does reproducability not seem to be any worse than it was in the past, it seems to be getting better — and the process of science has been a well-tested winner at this point. Throwing up our hands and declaring that the system is broken would be the ultimate in bad results.
A pair of Johns Hopkins researchers argue that narrowing the standards by which tests are conducted and reported would be an important step in reproducibility.
Although existing research standards have improved both research and its reporting, we need to unify existing standards and to fill the gaps between steps throughout the research process. Existing gaps include implementation of standards and links between standards for study registration (to know about all studies undertaken), study protocols (to identify the preplanned study design and methods), data collection (to assess outcomes that are important and comparable across studies), dissemination of findings (to know the results of previous studies), data sharing (to make best use of existing data), and evidence synthesis (to draw appropriate conclusions from the body of evidence). The scientific community must work together to harmonize existing standards, to ensure that standards are kept up to date, to check that standards are followed, and to develop standards where they are still needed. A unified system of standards will make our work more reproducible.
This might seem like the simplest thing to do, but scientists are not without their own foibles in experimental design or in how results are reported. Making both ends of research more standard will seem to some confining, but it will also result in research that’s both easier to reproduce and easier to build on.
Kathleen Hall Jamieson from the Annenberg School for Communication looks at how much of the “crisis in science” is a media creation, designed to fit a narrative that feeds into an underlying distrust of science and scientists. What she finds is that news stories about science tend to fall into simple categories — most of which are designed to create the idea that something is wrong.
After documenting the existence and exploring some implications of three alternative news narratives about science and its challenges, this essay outlines ways in which those who communicate science can more accurately convey its investigatory process, self-correcting norms, and remedial actions, without in the process legitimizing an unwarranted “science is broken/in crisis” narrative. The three storylines are: quest discovery, which features scientists producing knowledge through an honorable journey; counterfeit quest discovery, which centers on an individual or group of scientists producing a spurious finding through a dishonorable one; and a systemic problem structure, which suggests that some of the practices that protect science are broken, or worse, that science is no longer self-correcting or in crisis.
So … hero scientist, crooked scientist, crooked science. When you wonder why so many people are unswayed by scientific results on things like climate change, or so readily accepting of the idea that all scientists are in some kind of global conspiracy, keep in mind that these are common narrative structures in most media accounts.
Physics
Metamaterials is one of those words that’s starting to show up not just in scientific literature, but in the occasional news article about such exotic things as an ‘invisibility cloak.’ And since today is meta science day, it seems only right that this week brings a background article on just what this exotic stuff is about. And if you’re expecting to find that metamaterials are still confined to secret labs … surprise.
The soles on Nike’s recent line, Free footwear, offer an illustration. The bottoms of these sneakers don’t look particularly special, just polymer cut into triangles. But the shapes are informed by science. Every time wearers take a step, their foot widens from the impact, stretching the shoes. Nike’s sole responds by both widening and lengthening, to help absorb impact.
That shouldn’t happen. Think of a rubber band pulled outward between your fingers; it narrows in the middle. Soft materials stretched wider should shrink in length. But the shoes exhibit bizarre behavior, thanks to materials made of simple patterns of repeating geometric shapes. And shoes are just the beginning, says Daraio. “A new field is emerging that creates unusual materials using simple geometrical architectures,” she says.
It’s a fascinating piece. Go read it and be amazed by what can by done just through how material is organized.
Environment
One of the key features of China’s “Great Leap Forward,” other than millions of deaths, was the implementation of a program to vastly increase China’s production of steel. Much of this took the form of small “beehive” ovens using coking coal. The resulting steel was, and is, of extremely uneven quality. But the resulting pollution was intense.
China has been moving to get rid of these artifacts of a time that was dark on a number of fronts, and the potential benefits are … pretty great.
Here, we present the results of a quantitative evaluation of the health effects of historical [Beehive Coke Oven] BCO operation, the health benefits of the ban, and the adverse impacts of the poor implementation of the ban. ... We demonstrated that more than 20% of the [benzo-a-pyrene] in ambient air was from BCOs in the peak year. The cumulative nonoccupational excess lung cancer cases associated with BaP from BCOs was 3,500 (±1,500) from 1982 to 2015. If there was no ban, the cases would be as high as 9,290 (±4,300), indicating the significant health benefits of the Coal Law. On the other hand, if the ban had been fully implemented immediately after the law was enforced in 1996, the cumulative cases would be 1,500 (±620), showing the importance of implementing the law.
Think of these ovens as thousands of tiny, poor-regulated coal plants. Just as with the “fogs” of Victorian London, which were not fogs but deadly smogs composed mainly of trapped smoke from coal stoves all over the city, these small sources can have an enormous effect on their immediate area. It may seem easier to target power plants, but these small sources can actually contribute more to city-choking smogs.
Anthropology
74,000 years ago, a super-volcano called Toba in the area that’s now Sumatra blew its stack, bringing on a period of drastic climate change akin to a nuclear winter. This event seems to have depressed populations of many animals, and represents a bottleneck that’s etched in not just the fossil record, but the genetic code of survivors. Based on the many shared gene sequences of modern people, there was an assumption that Toba also took people down to a near-extinction point. But new evidence suggests that, in one area at least, human beings cruised through the Toba event.
Here we describe the discovery of microscopic glass shards characteristic of the Youngest Toba Tuff—ashfall from the Toba eruption—in two archaeological sites on the south coast of South Africa, a region in which there is evidence for early human behavioural complexity. ... Humans in this region thrived through the Toba event and the ensuing full glacial conditions, perhaps as a combined result of the uniquely rich resource base of the region and fully evolved modern human adaptation.
If there really was a restricted number of people left on Earth after Toba’s ash filled the sky, then maybe this community was it. Maybe we’re all not just African, but South African.
Astronomy
With the passing of Stephen Hawking in the last week, it seems only right that one of the morning’s research papers concern a surprising revelation about black holes. It’s a paper written by … Stephen Hawking, from 1974 when Hawking was working at Cal Tech. This abstract may seem a little dense, and you can feel free to ignore a few equations, but stick with it.
Quantum gravitational effects are usually ignored in calculations of the formation and evolution of black holes. The justification for this is that the radius of curvature of space-time outside the event horizon is very large compared to the Planck length (Għ/c3)1/2 ≈ 10−33 cm, the length scale on which quantum fluctuations of the metric are expected to be of order unity. This means that the energy density of particles created by the gravitational field is small compared to the space-time curvature. Even though quantum effects may be small locally, they may still, however, add up to produce a significant effect over the lifetime of the Universe ≈ 1017 s which is very long compared to the Planck time ≈ 10−43 s. The purpose of this letter is to show that this indeed may be the case: it seems that any black hole will create and emit particles such as neutrinos or photons at just the rate that one would expect if the black hole was a body with a temperature of (κ/2π) (ħ/2k) ≈ 10−6 (M⊙/M)K where κ is the surface gravity of the black hole. As a black hole emits this thermal radiation one would expect it to lose mass. This in turn would increase the surface gravity and so increase the rate of emission. The black hole would therefore have a finite life of the order of 1071 (M⊙/M)−3 s. For a black hole of solar mass this is much longer than the age of the Universe. There might, however, be much smaller black holes which were formed by fluctuations in the early Universe. Any such black hole of mass less than 1015 g would have evaporated by now. Near the end of its life the rate of emission would be very high and about 1030 erg would be released in the last 0.1 s. This is a fairly small explosion by astronomical standards but it is equivalent to about 1 million 1 Mton hydrogen bombs.
In many ways this is the Hawking paper, it came at the end of a few years in which Hawking raced away from the field on black holes and both created, and destroyed ideas with seemingly lightning quickness. The accomplishment he made was showing that the tiny effects of quantum could define one of the most massive objects predicted by relativity. It’s a wonderful bit of magic that honestly isn’t beyond the mathematical capabilities of many — but was beyond the conceptual powers of all but a few. Hawking brought together the big and the small, and made them collide in one of the most awesome structures we know.
In memorium of Stephen Hawking, Nature has brought together a special collection of his works, and works about him. Included in this is the obituary written by UK Astronomer Royal Martin Rees.
By linking quantum theory and gravity, Hawking showed that a black hole would not be completely black, but would radiate with a well-defined temperature that depended inversely on its mass. Black-hole entropy was more than just an analogy. The implication was that the radiation would cause black holes to ‘evaporate’. This process would be unobservably slow, except in ‘mini-holes’ the size of atoms — and these are thought not to exist. Yet Hawking radiation — and the related issue of whether information that falls into a black hole is lost or is somehow recoverable from the radiation — was a profound issue, and one that still engenders controversy among theoretical physicists. Indeed, theorist Andrew Strominger at Harvard University in Cambridge, Massachusetts, said in 2016 that one of Hawking’s papers on the subject had caused “more sleepless nights among theoretical physicists than any paper in history”. …
Stephen’s expectations when he was diagnosed dropped to zero; he said that everything that had happened since had been a bonus. And what a bonus — for physics, for the millions enlightened by his books and for the even larger number inspired by his achievement against all the odds.
As usual, this week’s infographic comes from Andy Brunning at Compound Interest. If you’d like a larger, easier to read version, you can find it here.
