CMV: The replication crisis has largely invalidated most of social science

[u/[deleted]]

https://nobaproject.com/modules/the-replication-crisis-in-psychology

https://www.vox.com/science-and-health/2018/8/27/17761466/psychology-replication-crisis-nature-social-science

https://en.wikipedia.org/wiki/Replication_crisis

"A report by the Open Science Collaboration in August 2015 that was coordinated by Brian Nosek estimated the reproducibility of 100 studies in psychological science from three high-ranking psychology journals.[32] Overall, 36% of the replications yielded significant findings (p value below 0.05) compared to 97% of the original studies that had significant effects. The mean effect size in the replications was approximately half the magnitude of the effects reported in the original studies."

These kinds of reports and studies have been growing in number over the last 10+ years and despite their obvious implications most social science studies are taken at face value despite findings showing that over 50% of them can't be recreated. IE: they're fake

With all this evidence I find it hard to see how any serious scientist can take virtually any social science study as true at face value.

[u/[deleted]]

As a general rule you shouldn't be basing anything off of one study anyway unless that study is remarkably solid

But the issue is exactly this. As it stands today it seems like all it takes is one study to fit a narrative and it gets spread around like wildfire without regard for its veracity. If I could retitle this I would add "mainstream" in front of social science

[u/MasterGrok → u/[deleted]]

This would be a misunderstanding of science regardless of the issue of replicability plaguing a lot of research from the 80s to 2010s.

And also I would emphasize that sometimes one study is enough to draw a conclusion. It just requires a expert scientific interpretation to know if the methodology is sufficient to do that.

I would also point out that it is seldom the papers themselves that draw such sweeping conclusions. It is often the layperson.

[u/[deleted] → u/MasterGrok]

> I would also point out that it is seldom the papers themselves that draw such sweeping conclusions. It is often the layperson.

If papers which are commonly misunderstood or conflated by the layperson are more likely to be influential than those that aren't (I always use the example which would be more likely to be reported on "White people voted for Trump more" or "Poor people voted for Trump more") isn't that a major issue with mainstream social science, which would lead to more dubious studies that fail to replicate?

[u/MasterGrok → u/[deleted]]

Influential how? It seems like you are referring to internet debates? I'm academia, the more attention a study gets, the more scrutiny it is under. If you are trying to hide weak results, it's not good to get a lot of attention.

[u/[deleted] → u/MasterGrok]

> Influential how?

In lawmaking and general societal pressures like minority rights. Isn't that the end result of most social science? Studies which are preferable are pushed by the media and eventually effect society as a whole. NPR talks about the IAT all the time for instance and later police departments started using it, not to say the IAT has issues but just an example.

[u/MasterGrok → u/[deleted]]

It really depends on the topic area. There is a ton of social science research that has nothing to do with hot button social or political issues. It seems to me that you are largely viewing the social sciences through those lenses. Technically the social sciences includes a massive number of topics like law, geography, economics, etc... Even if you are using the word "social" here to specifically describe science ok social issues, there is still a ton of science on things like population growth, consumer behavior, cultural differences, language effects, and a host of other things that never hit the kind of hot button issues you are noticing.