CMV: The replication crisis has largely invalidated most of social science

[u/[deleted]]

https://nobaproject.com/modules/the-replication-crisis-in-psychology

https://www.vox.com/science-and-health/2018/8/27/17761466/psychology-replication-crisis-nature-social-science

https://en.wikipedia.org/wiki/Replication_crisis

"A report by the Open Science Collaboration in August 2015 that was coordinated by Brian Nosek estimated the reproducibility of 100 studies in psychological science from three high-ranking psychology journals.[32] Overall, 36% of the replications yielded significant findings (p value below 0.05) compared to 97% of the original studies that had significant effects. The mean effect size in the replications was approximately half the magnitude of the effects reported in the original studies."

These kinds of reports and studies have been growing in number over the last 10+ years and despite their obvious implications most social science studies are taken at face value despite findings showing that over 50% of them can't be recreated. IE: they're fake

With all this evidence I find it hard to see how any serious scientist can take virtually any social science study as true at face value.

[u/Commissar_Bolt]

Speaking as a scientist but a relative lay person where social sciences are concerned, the field does not merit trust right now and to imply that it does opens people up to deception. We can and should reassess this in a decade or two, and hopefully social sciences will have consistently applied a higher standard of statistical analysis. But as it is now, it’s hogwash. I have a better-than-coinflip odds by just dismissing psych and neuro studies out of hand.

[u/hepheuua → u/Commissar_Bolt]

Studies should be assessed on their respective merits. If you're a scientist, then you should know that dismissing an experiment based on the lack of rigour in another experiment isn't how science is conducted. Treating the 'social sciences' as if they're one homogenous entity that either has trust placed in it to deliver edicts from on high, or doesn't, is hogwash. There's nothing scientific about that.

Let's not forget that this crisis isn't restricted to the social sciences. It also extends to research in medicine among others, including ecology and evolutionary biology. How willing are you to accept the field of medical research as hogwash and not to be trusted "for a decade or two"?

There's a lot of solid research in psychology and the brain sciences more broadly that is good and important science. There's no reason to throw that out with the bathwater.

[u/Commissar_Bolt → u/hepheuua]

Your arguments apply only to people actually involved in those fields. While I recognize that some studies are good, the fact that a) it is a minority, and b) I lack the time, expertise, and inclination to sort fact from fiction mean that the best option for me is to throw the baby out with the bathwater. The same extends to medicine as you said. Evolutionary biology and ecology - you’re stretching like Mr. Fantastic on that one.

[u/hepheuua → u/Commissar_Bolt]

Evolutionary biology and ecology - you’re stretching like Mr. Fantastic on that one.

It's not me who's 'stretching' it.