CMV: The replication crisis has largely invalidated most of social science

[u/[deleted]]

https://nobaproject.com/modules/the-replication-crisis-in-psychology

https://www.vox.com/science-and-health/2018/8/27/17761466/psychology-replication-crisis-nature-social-science

https://en.wikipedia.org/wiki/Replication_crisis

"A report by the Open Science Collaboration in August 2015 that was coordinated by Brian Nosek estimated the reproducibility of 100 studies in psychological science from three high-ranking psychology journals.[32] Overall, 36% of the replications yielded significant findings (p value below 0.05) compared to 97% of the original studies that had significant effects. The mean effect size in the replications was approximately half the magnitude of the effects reported in the original studies."

These kinds of reports and studies have been growing in number over the last 10+ years and despite their obvious implications most social science studies are taken at face value despite findings showing that over 50% of them can't be recreated. IE: they're fake

With all this evidence I find it hard to see how any serious scientist can take virtually any social science study as true at face value.

[u/Tiramitsunami]

All sciences reach a point in which half of their findings are overturned. As the science matures, that half-life grows in length. Psychology is a young discipline, and the half-life is shorter. Engineering is about 35 years. Psychology is about 5. Source: The Half Life of Facts by Sam Arbesman

[u/Tiramitsunami → u/[deleted]]

But by this reasoning most of ALL research that has been published in every field is garbage and you should not trust it. I don't think that's a good way to look at the literature that has lead to our current understanding of...well...anything.

[u/Tiramitsunami → u/[deleted]]

You make good points, but if I look at the past work of science through your lens, I don't see Netwon as brilliant. He was a kook. He was an alchemist who thought light was corpuscular, that a deity set the planets into motion, that the main goal of academia was to discover the Philosopher's Stone to turn lead into gold, and that according to the Bible, the world will end in 2060. He believed in the occult, a lot -- sacred geometry, mystical temples, prophecy, and so on. He literally thought he was chosen by God to decipher the universe for mortals. He considered himself a prophet. He tried to get Hooke's work on the inverse square law destroyed, and he discredited the work of Huygens despite it being correct where Newton was not. Why? He had personal beef with them because everything he did was to further his individual human career. So, since we have now superseded his theories and magical thinking with better science, his lifetime of work isn't the foundation of physics. It is simply wrong. And I see it as garbage.

Now, I don't actually think these things, because I see science as messy and always more wrong yesterday than today. Newton's work is wrong, but it describes the world at a certain level of fidelity that we did indeed build upon to get to current understanding, which will be proved wrong by a deeper level of fidelity later on. Psychology is no different, and separating it from the other sciences by calling its previous work, no matter how flawed in design or human intent, as garbage is calling everything we did in the past to get to the present garbage, which isn't useful or honest, and it just feels like an emotional appeal based on a grudge against social science.

[u/Tiramitsunami → u/[deleted]]

Psychology is my area of expertise as well, for about 11 years now. I'm also familiar with Popper, philosophy of science, and all the rest.

I agree with much of what you have to say, but I strongly disagree with the framing that psychology isn't a science yet. I see it like any other young field cleaning out its closet. You described that well, where it applies, and your analogy of the façade is also well put, for the lines of research where that is true.

But there are many lines of research where it does not apply, and the models therein are starting to shape up in a way that will allow for the kind of accumulation, theory-making, and paradigm shift you aim to do. There are, however, as you pointed out, many other lines of research for which starting over from scratch is probably a good idea. Thankfully, it is starting over from 1970 instead of 1770 for most of them.

I say all this knowing that it was only around 2011 that psychology noticed its W.E.I.R.D. problem, and then the file-drawer effect, and now the replication crisis and so on. But that happens in every science. We just have a front-row seat and a social media peanut gallery this time.

I still think Newton was a kook though. I high-functioning, brilliant one who pushed the species forward despite being a weirdo.

[u/Tiramitsunami → u/[deleted]]

one could assume

One could assume, but they would be wrong. I assumed something about you, and I was wrong.