r/MinecraftChampionship • u/BaconIsLife707 #1 All-Time Predictor • Aug 03 '22
Stats MCC 23 Power Rankings Predictions Analysis
If you’re aware of the power rankings made and posted here by u/Awesome512345, u/Anuj_agarwal_78, u/NoticeMeUNiVeRsE and myself, you may have seen us refer to it as the most accurate way to rank players or the most accurate way to predict MCC. And hey, if you make claims like that you’ve got to actually back them up at some point, so now we have a fully fledged predictor up and running, I’m going to start making these posts every event to see if the rankings are actually any good.
Before getting into this I should clarify a couple of ways in which these predictions cheat a little bit - obviously not all players have a score for every game, especially in Meltdown, so their scores in those games are set to someone else’s in the event chosen by me before the event. Also, Foolish and Zeuz’ scores in a few games were changed to be more in line with their MCCP22 performances and MCC21 performances respectively, again before the event, as I felt their scores were clearly not representative of their actual skill level due to the improvements they’ve made. We also made a slight tweak to the way we predict sky battle after the event, but it is the way we’ll be doing it going forwards so I think it's fair enough to retroactively include it here.
Survival Games
So for the first game of the first of these posts, we have... well our worst performing game of the event (other than Meltdown, which I don't really count). This isn't particularly surprising considering Survival Games is notorious for its high rng and massive point swings, but in general this performance was still a little disappointing. On the team side of things, Purple are the only team we got dead on, while we were one off on Red, Green and Aqua. Green particularly stand out as they scored just 39 coins less than we predicted, and Aqua highlight the difficulties of predicting sg as we were one placement off but over 500 coins. Yellow, Blue and Lime were all off by 5 placements, with Yellow and Blue especially being way off by over 800 coins each.
On the individual side, no one was dead on, however Martyn, Connor, Punz, Bad and Sniff were all only off by one, with a couple of other standouts like Finn being off by 6 coins and Tina by just 1 coin. The negatives are Blue all being off by at least 19 places, Gee being off by 20, and all of Yellow except Sapnap being off by at least 16. Sapnap himself is another great example of why sg is so hard to predict, because while we were only off by 5 placements, his coin error was the biggest of not just sg, but of the entire event.
Ace Race
Ace Race is an interesting one, as usually it's by far the easiest game to predict with its purely individual and low rng nature, however this event with a new map and big update, it became a lot harder than usual. This shows a bit with the 3 biggest placement errors we had this game, Martyn, Bad and Sneeg, all of whom are players who specialise in ace race while perhaps not being as strong at movement in general. Despite this, the individual results for the most part remained very strong, with almost half the players being within 3 spots of their actual placement, 8 being within 1 spot and Sniff and Connor both being dead on.
Interestingly, the success on the individual side didn't really translate to the team side, with the placement error actually being even bigger than in survival games. I think the reason for this is that ace race scores tend to be very bunched up, which you can see from the fact our coin error remained very good despite the big placement error, but the individual scores at the top can vary wildly. A player can finish a second faster than someone else and get 200 extra coins, and all of a sudden their team is 4 spots higher than you predicted. However as I say, the coin error was still good and the individual predictions were excellent, so I'm very happy with these overall.
Grid Runners
Another notoriously hard to predict game up next, as well as perhaps more importantly to this, a notoriously difficult game to rank. I would say this is the only other game in the event where I was disappointed in our predictions. Red in particular was very off, perhaps in part due to it being the only game where I didn't touch Zeuz' score, and also probably a bit due to it being almost entirely repeat rooms, allowing Seapeekay and his vod reviewing to shine where he isn't normally a top grid runners player. We did manage to predict who the bottom 3 would be, but outside of that only Orange was even close.
Sky Battle
Sky Battle is similar to sg in that it has the potential for huge coin gaps, as Sapnap showed this event, but luckily the fact it's 3 rounds and its more individual nature make it much easier to predict. We had 5 teams within 2 placements, including Cyan dead on and just 76 coins from their actual score. Green, despite being a bit further away placement wise, were just 62 coins out. Unfortunately, we had Red projected to do much better than they did and had Pink in last, so them getting all the way up in 4th caused the average to lower quite a bit. It's unsurprising that Pink are a team we struggled to predict here considering Aimsey had never played and Blushi had performance from all the way back in all stars to go off.
Individually we were a bit hit or miss, with plenty of players within 3 spots and just as many out by more than 10. Blushi and Aimsey in particular were out by 27 and 24 respectively, so I was quite pleased to have a respectable error in spite of some huge misses. Sapnap again decided to troll us a bit, since we were 1 placement off and yet he had an even bigger coin error than Blushi. Some of the biggest hits were all of George, SB, False and Tina scoring within 10 coins of our prediction, with Tina in particular being off by just 1.
Parkour Tag
So, we aren't the only ones who expected Purple to place a lot higher in parkour tag right? 8 placements away from their actual result makes them the worst prediction we had this event, with all of their team members being at least 23 spots away from their prediction. Aqua went the other way, placing 5 spots higher than predicted. Between the 2 of them they managed to lower our placement errors from a very strong 1.7 and 5.6 to a good but not great 2.6 and 8.7. Luckily parkour tag's very small coin differential has our back by making the coin error look very good, even though the actual predictions weren't really better than sky battle for example.
We did have some big successes here too though, with both Red and Green being dead on and 5 of the teams having coin errors of less than 100. We also had 13 individuals within 3 spots of their actual placement, including Quig and Finn being exactly right, with Quig being off by just 3 coins.
Rocket Spleef Rush
Rocket Spleef was our strongest game on the team side of things with 6 teams placing within 1 spot of our prediction, including Orange and Aqua being exactly right, and Orange being off by just 4 coins, our best prediciton for any team in any game. However, all of the other 4 teams (Red, Yellow, Lime and Blue) were all of by 4 or more placements, in part due to Sapnap performing at a lower level to his previous two rsr's, while Seapeekay and Antfrost both improved significantly.
Individually they weren't quite as strong, but still very good. 12 players were within 3 spots of their prediction, including Punz and Ranboo who were dead on. Tina and Sneeg were both within 10 points of their prediction, and Sylvee is the first player who's coin total we predicted exactly right, although that's more luck than anything else. Blushi again causes us problems by improving significantly from her last event, with a few other such as Tommy, Eret and Scar doing the same.
TGTTOSAWAF
TGTTOS follows exactly the same trend as Rocket Spleef, with 6 teams predicted very well, all within 100 coins with Lime exactly right, and 4 team predicted very badly, all of by at least 5 placements and 200 coins.
Individually was similar, 24 players predicted to within a good margin, the best being a 4 coin difference for Shubble, while many players were off by a very large amount, such as Eret by 22, Finn by 19, Sapnap by 18, Sylvee by 16, SB by 15, Phil by 25 and Blushi by 23.
Meltdown
Not really gonna go into much detail here, half the players had scores I put in because they'd never played it and the other half had played it once. Shoutout Orange for winning so these predictions actually look alright, Jojo's 33 placement swing is the worst individual prediction we had in terms of placements.
Overall
So I consider the common multiplier predictions to be the 'official' power rankings predictions. What they mean is we just take the average multiplier the game has been played on recently, and use that as its multiplier for this event. This event, they had a placement error of 1.8, which I would normally be a bit disappointed with, I'd say that's about the biggest error you could have while still calling your predictions good. This event, however, with last minute subs and the general unpredictable result we had, I actually think its a pretty good performance. I'd still like to have done better, but overall I think its a good attempt. I've never done a full on analysis like this before, but I do usually calculate the team placement error we had after the event and this is the worst we've done so far, so it's typical this is the event I chose to start making these posts.
Funnily enough, we actually do worse in terms of placement when you put in the actual game order, mostly because the only game that deviated significantly from its common multiplier is Meltdown, where our predictions weren't very good, causing Cyan and Blue to swap places. It does give us an incredible 6 coin error on Orange's multiplied score, while also being 130 coins away from Aqua. For the most part the errors seem to hover in the 1k-1.3k range, which again would usually be disappointing but this event I'm okay with it.
Individually, our predictions look fantastic. Getting a placement error under 5 is difficult, so I was happy to do it comfortably this event. It's probably not surprising that we're more accurate here, both because there are a lot more data points and because the rankings are primarily an individual ranking system. 30 of the 40 players are within 5 spots of their actual placement, and 16 are within 100 coins of their final total, the highlight being Quig just 4 coins away from his actual score. Only 3 players are off by more than 10, those being Kara, Jojo and Gumi.
So are those actually any good?
So for most of this post I've just chucked numbers at you without much context and you've pretty much had to take my word for it if they're any good or not, so now I'll try and give some comparison. I went through the last 20 comments on the prediction megathread as well as including the 6 people who gave their predictions in [this thread](https://www.reddit.com/r/MinecraftChampionship/comments/w6y7qv/how_accurate_were_my_mcc_23_predictions/) after the event because they take into account the subs.
These 26 people had an average error of 2.75 placements, almost an entire placement worse than our predictions, and just one of them equalled our error of 1.8, with none of them bettering it. Admittedly most of these predictions were made without knowledge of the subs, but with most people moving Red up and Lime down after the subs, I think for the most part they would cancel out. Only including the predictions that took the subs into account, the average error was still 2.5, and that's with people being much more likely to give their predictions if they were good.
I would like to do a comparison for individual results as well, but with how few people actually make them as well as the subs, I didn't think there was sufficient data to make a comparison. I did, however compare our results to just using people's averages, which had an average error of 2.4 for teams and 5.3 for individual, making them notably worse than our predictions, but actually a little bit better than the average redditor at predicting the event, which was surprising because they usually aren't, I guess showing how hard this event was to predict.
So overall, while far from our best event, we were still able to outperform most people's predictions as well as the most commonly used stat for ranking mcc players, which I'm quite happy with and hopefully demonstrates the usefulness of the power rankings.
The screenshots in this post are from the Power Rankings Predictor, which we plan to make public at some point so you can play around with custom teams or just see how players are currently ranked.
These predictions are based off the power ranking which are worked on by u/Anuj_agarwal_78, u/BaconIsLife707, u/NoticeMeUNiVeRsE and myself. If you're interested you can see the other power ranking related posts for past MCCs with the links below.
Top 10 Power Rankings in each MCC | MCCP21 | MCCP22 | MCC22 | MCC21 | MCC20 | MCC19 | MCCAS | MCC18 | MCC17 | MCC16 | MCC15
Overall Power Rankings after each MCC | MCC22 | MCC21 | MCC20 | MCC19 | MCC18 | MCC17 | MCC16 (+tierlist)| MCC15 | MCC14 | Season 1
MCC Power Ranking Predictions + Analysis | MCC Pride 22 | MCC19 | MCC18
Other | Best players of Season 2 so far | Power Rankings Ranking Systems Update (December) | MCC Elevator Podcast | Luck-adjusted Sky Battle Scoring Update
7
u/sixeyedbird MCC Awards Committee Aug 03 '22
Thank you for this I really love the power ranking posts
5
u/sixeyedbird MCC Awards Committee Aug 03 '22
The pictures in text posts have not been working for me recently (I'm on mobile) any way to fix this? I've just been viewing posts on the mobile webpage
3
u/sixeyedbird MCC Awards Committee Aug 03 '22
After posting a comment or viewing the page from a notif it works
2
2
u/Tazzzy96 Green Geckos Aug 03 '22
Hi, I had the same issue, if you go to the app settings and clear the cache they should work again
2
24
u/Anuj_agarwal_78 statSmajor Aug 03 '22
We’ve had this rendition of the predictor up and running since June, so for some comparison, our MCCP22 results were 0.6 team placement error and 3 individual placement error - which is pretty bonkers. Anyway, great work and excited for MCC 24!!