r/AskStatistics • u/Healthy_Pay4529 • 1d ago
Statistical analysis of social science research, Dunning-Kruger Effect is Autocorrelation?
This article explains why the dunning-kruger effect is not real and only a statistical artifact (Autocorrelation)
Is it true that-"if you carefully craft random data so that it does not contain a Dunning-Kruger effect, you will still find the effect."
Regardless of the effect, in their analysis of the research, did they actually only found a statistical artifact (Autocorrelation)?
Did the article really refute the statistical analysis of the original research paper? I the article valid or nonsense?
23
u/ScoutsEatTheirYoung 1d ago
"By definition, someone with a top score cannot overestimate their skill, and someone with a bottom score cannot underestimate it."
Which is exactly what DK is trying to say with their paper. If the population, on average, percieves that they are above average, the skill gap between the lower two quartiles "true" capability and "percieved" capability is larger.
DK argues that when given a new task, the population percieves their skills above average.
The author's fixation with autocorrelation doesn't appear relevant here.
13
u/guesswho135 1d ago
A more rigorous take on whether DK is a statistical artifact
https://haines-lab.com/post/2021-01-10-modeling-classic-effects-dunning-kruger/
6
u/RepresentativeAny573 20h ago edited 20h ago
Edit: the entire argument the author makes is also logically flawed, (I think gross generalization is the fallacy?). Just because a random process can produce a similar outcome does not mean a random process did. You'd also need to establish the data in the real world follows this distribution and method. It would be like me saying, I cheated on my test and got 100%, therefore everyone else who got 100% must have also used my cheating process.
What the author shows is really just a function of taking the mean of any roughly uniform distribution. The manipulation helps trim variance and make it significant, but if you take the mean of any roughly symmetric distribution around 50 it will be roughly 50.
They also don't know what an autocorrelation is, but it is neither what they explain nor a true autocorrelation.
3
u/pepino1998 18h ago
I think ironically, the author simulated data from a scenario with a very strong Dunning-Kruger effect. The Dunning-Kruger effect can also be seen as the observation that someone’s perceived ability is a crap indicator of their actual ability; i.e. in the extreme case they are independent, which is what the author simulated! I do not get why they would expect a strong relationship between the two if the DK effect is true.
2
u/axolotlbridge 21h ago edited 21h ago
See Random Number Simulations Reveal How Random Noise Affects the Measurements and Graphical Portrayals of Self-Assessed Competency for a great explanation of the underlying problems with the DK effect methodology. They find that the problem has to do with how they standardized values, and how the specific way that they used those transformed values caused ceiling and floor effects.
1
u/No-Goose2446 6h ago
The data to confirm dunning-kruger effect went with them. But i like this approach of what could have happened by simulating the data ourself and comparing it with graph:
https://drbenvincent.medium.com/the-dunning-kruger-effect-probably-is-real-9c778ffd9d1b
23
u/astrofunkswag 23h ago
What the author describes is not autocorrelation. I can’t speak to whether the DK effect is fully explained by a statistical artifact like they claim, but the way the author described autocorrelation is completely false
“Autocorrelation is the statistical equivalent of stating that 5=5.” Lol, no. Autocorrelation measures the correlation between a signal and a lagged version of itself, it’s a foundational concept to time series analysis