Kpoptosis: "Stats Are Just a Piece of the Understanding Puzzle"
By scant on 0 0
Yesterday I had the privilege of chatting to Kpoptosis about the role of statistical analysis in Dota 2, TI4, and the Summit. Kpoptosis is probably one of the friendliest guys on the internet. You should follow him on Twitter.
Hey Kpoptosis - congratulations on being invited to TI4! Do you know what your exact role will be this year – will it be the same as last year?
Kpoptosis: I assume so. Unfortunately not much has been disclosed to me in regards to the specifics of my job. All I know is my title, which is "English Stats". Last year that meant I was in every game, doing the basic in-game popups
Hmmm, I guess it's not only fans that have to wait for details from Valve...
Kpoptosis: Well they're going through their mass of invites first. They specified we'd go into specifics soon.
Even so, this makes the next thing I wanted to ask you about a bit more difficult for you to answer. I'm going to ask anyway because you may want to speak to the question, regardless.
You've developed some incredible tools for presenting stats (e.g. teamfight recaps) during the course of 2014 which we've seen on BTS streams – do you think you'll be able to make use of these tools at TI4?
Kpoptosis: That's up in the air, but it's certainly something we'd like to do. I believe (and most agree) that it adds a fair amount to the production value of the game, especially for newer viewers as teamfights are oftentimes the hardest things to parse.
And I'd love to show my babies to the world :D
Haha, I'm sure the world would love to see your babies too! And honestly, as a a big fan of BTS broadcasts, and not really a 'newer viewer' myself, I can attest to how useful some of your stuff has been in improving the production value of streams this year.
Kpoptosis: Yeah I've heard some people say it's even hard to watch some matches without our graphics. So that's always good to hear.
Yeah, that is awesome feedback. Speaking of tools you've developed, I've been wanting to ask you about heatmaps for quite some time. Last year, in an interview with Team Liquid, you discussed the possibility of integrating Skadi's heatmaps into your work in interesting ways. However, we've not seen much of this in 2014. Are there some things in the works?
Kpoptosis: Heatmaps are currently publicly available via datDota (through work with the Skadi guys) and they're great. The issue with integrating them into live broadcasts is that we haven't been able to build heatmaps live as of yet. So they're good for post-game analysis and less good for live games.
And that's been my shift in focus this year, less historical data and more things describing the game you're currently watching
You've actually spoken a lot in the past about this - how the job of a Statsman is ultimately to entertain the audience. However, I've also seen you speak about how important it is for stats to control for as many variables as possible in order to make them truly relevant or accurate. Aren't these two principles in conflict with one another? Don't stats get more boring as they become more academic (and thus more accurate)? Is there a secret formula for achieving both of these aims at once?
Kpoptosis: It's about finding the middle ground in a lot of ways. I'm certainly not running regressions in the middle of a game. But I also don't want to be misleading in my generality. I think the idea is to control for the RIGHT variables. For one, I only use professional data (which is a given now, but wasn't when I started). Secondly it's important to not span too much time if at all possible for things like hero matchups. It's preferable to do it on a patch-by-patch basis (especially if the heroes were changed in some way) if the sample size is large enough.
Working this job has really made me skeptical about most stats presented for the purpose of marketing...
Sure, as you're describing here, there's ways to narrow your focus and guard against the easier ways to get something totally wrong. But that's quite far off from guaranteeing you'll get stuff right. To me, it seems like this could make your job quite stressful, because ultimately you're under time pressure during games to make quick decisions about which stats are relevant to present and how. Do you feel this kind of pressure?
Kpoptosis: I guess you could say there's pressure. I don't feel stressed out that much when doing my job, but I reckon that comes from experience. There's of course personal pressure to present a good product, but I would argue that's a good thing, as it keeps you focused on being consistent.
Also it's a bit hard to give something that's "wrong". At the end of the day, it's factual information.
But it can be misleading or cause a clash with what the broadcasters are saying - which I think are the biggest sins that can be committed.
Obviously pretending a hero has a 100% winrate and not specifying that that is through 2 games is misleading, but not "wrong" per se. Working this job has really made me skeptical about most stats presented for the purpose of marketing... haha - because I realize how easy some things can be to misrepresent
Back-peddling just a little bit. You said something interesting just now. You see presenting info that conflicts with what casters are saying as a big problem. This is reasonable enough. But does this mean that if a caster makes a claim, and you look for stats to back it up but instead find the opposite, that you'd simply turn a blind eye? Would you bring something like this up with the caster later on then? Or have you had the good fortune of always working with casters that are always right?
Kpoptosis: I've never worked with anyone who's always right, thankfully :P. But, yeah, this is a very interesting point I think. It stems from my stance that I'm not a caster. I'm there to provide additional background information or to lend proof to what the caster is already saying. If a caster says a stat specifically and he's wrong, I'll probably correct him on stream because that's being incorrect, certainly. But if Merlini says Shadow Demon is good versus Batrider and I pop up a stat saying "Actually Shadow Demon has a winrate of like 44% vs Batrider" that's not beneficial to the cast for a couple of reasons:
1.) I'm reducing the credibility of the guy we have on to give competitive insight and
2.) Numbers only tell a 2 dimensional story a lot of the time
Shadow Demon by himself has a historically low winrate (vice versa for Batrider) so I would actually trust Merlini's analysis over what the numbers say in that instance.
Right, so what you're basically saying is that merely disagreeing with what the caster said or 'having some argument against it from some stats' is not good enough reason for you to contradict them. They need to be factually incorrect to do that. Which makes sense. The former is what you'd expect a co-caster to do - and you're specifically saying you don't see yourself as equivalent to any kind of caster. Have I understood correctly?
Kpoptosis: Right. I'm not going to be a contrarian because that's not my job, but I'm also not going to let one of our casters be flat out wrong. Also because I think stats are just a piece of the understanding puzzle – oftentimes just the springboard.
I like presenting stats that can be discussed and maybe even debated. I tell casters all the time if I post a stats and they believe differently to discuss it! It makes for good conversation. I don't believe I'm infallible.
Fair enough - I mean, for me it's not even a question of if YOU are infallible because the stats themselves, as you say, are always just one piece of the puzzle. It's something I noticed when reading up on some old reddit threads you posted in your early days. There's often concerns about small samples, or too many relevant factors to ever completely control properly. To me it seems that while stats do present as 'objective' or 'factual', it's misleading to think they don't require analysis and interpretation from those with the insight to work out what is actually going on. This actually leads me to my next question very neatly.
How useful do you think the kind of stats you talk about can be to actual professional players? Do you have any experience of pros saying they've benefited from statistical analysis in Dota 2?
Kpoptosis: I think certain stats are very important for professional players. It's kind of a different ballgame than broadcast stats though. For teams what's most important is:
1) Your own/other team's draft tendencies
2) General item builds
3) With the advent of the Skadi parser, you can see heatmaps as well as ward placements to get a general idea of a team's movement
Do you know if teams are actually using this kind of stuff as things stand?
Kpoptosis: I know some teams do - I've had several conversations with players about statistical subjects. I think some teams may hire people to do "scouting" for them if the money allows. There's simply too much dota going on to expect the players to have full knowledge of all of their enemies as well as keeping up with general trends.
There was an instance with a game I casted with Sheever at Starladder - iG vs Rox.KiS, this one I believe – where iG banned Naga Siren and Io in the first phase. Which are 2 heroes RoX.KiS never played. I pointed that out in the cast and it's something that should honestly never happen in this age of information considering how much is on the line.
Yeah, I think that's a very good point. A lot of people have also suggested that many teams have been very slow to catch on to the C9 Drow strategy, working out heroes that sometimes signal it early on in a draft.
Kpoptosis: Exactly. some teams are actually really easy to read in their drafts, and some teams are oblivious. I think that's becoming less common though.
A trend I've really noticed over the last couple months is the focus on picking heroes that are a bit ambiguous in their roles so the other teams doesn't really know what to expect from both the rest of the draft and the lane situation. Lots of Mirana for example. As well as Doom. Wraith King also fills a similar role. All 3 of those are top 15 picked this patch. So teams are adjusting to the wider availability of knowledge :D
Yep, ever since 6.79, heroes that are more versatile in terms of their roles or laning seem to have become more popular. The best thing about this observation, I think, is that it illustrates so well something we were discussing earlier - that the stats on their own don't always present explicit patterns without a reasonable degree of analysis or interpretation. There isn't exactly a stat which deals directly with versatility - and yet we can still make these kinds of claims.
Kpoptosis: Right. I think an observation is most valuable when the stat isn't the claim itself, but the evidence for the observation. It's the scientific method after all.
By the way, congratulations are in order for the the successful completion of the Summit! Although I think you guys did an outstanding job, the community gave some mixed feedback during the event. By your own standards, how successful was the Summit?
Kpoptosis: We are extremely happy with how the Summit turned out. For our first LAN event I don't think we could've asked for a better end product. There were certainly some hiccups, but that's to be expected both when doing something for the first time and having to deal with masses of people. We're most likely doing another one, so I guess that speaks to our confidence in how we carried it out.
Great news! Do you have any idea when the next edition will be? This year? Next year?
Kpoptosis: Nothing is at all finalized, but we probably have our eyes on Winter – so end of this calendar year/very beginning of next.
That's actually pretty soon. I hope everything goes well in that regard. To finish off this interview, I've got a series of shorter questions which you can answer with pretty short answers too if you like.
To start with, I've often seen you describe yourself as a 'Statsman (amongst other things)'. What are some of these other things?
Kpoptosis: Well for those not in the know, BTS only has a handful of actual employees. The house you saw for The Summit is where we live and work. So I have a lot of more business-y duties. And I also cast sometimes. I also cook about 5 nights a week.
Right, so you're actually quite a useful guy to have around the house in general :D. Speaking of the BTS house, who is the biggest feeder in the house and does everyone else give him flak for it?
Kpoptosis: It's probably either Andrew or myself. I haven't actually gotten to play much in the last 8 months or so because I had a job in Alabama before I got here and I've been ridiculously busy since I arrived in LA. So I've gotten pretty rusty.
I feel you're dodging the second half of the question - do the other guys make fun of you and Zyori? When you do get a chance to play, I mean.
Kpoptosis: The last time I played with LD I carried him :D. But we get a little bit of flak, not too much though. We're not THAT much worse than everyone else.
Fair enough. Has anything you learned in your Biology Degree ever been useful to you as a Statsman?
Kpoptosis: Well I did a ton of research in College (Alzheimer's disease specifically) and that required a lot of stats background. But I don't think I've ever used anything specifically from my Bio degree unfortunately.
That's a shame. Though one story I've heard you tell before is how your name actually comes from 'Apoptosis' which I'm sure is related to studying Biology. The question I actually want to ask about your name is pretty random but has annoyed me for a while, especially when researching this interview. Is it Kpoptosis or K-poptosis? I see your name hyphenated some places and not in others.
Kpoptosis: When I started out on reddit and Teamliquid I hyphenated it. But I think it looks cleaner without.
Hahaha. Yeah, I think it is a bit cleaner the way it is now too. Speaking of names, what's up with this name 'Statsman'? It seems like Bruno basically invented this title and it's just stuck when it comes to Dota 2. Do you like the name? I'm kind of in two minds about it because it's sort of novel but it also doesn't seem like the most 'professional' title, especially for someone from outside of e-Sports.
Kpoptosis: I like the title. Sure it doesn't sound super professional, but I'm okay with that. Also the statsman role in dota is a lot more prominent than the same role in traditional sports. I'm a lot more of a public figure, as evidenced by the fact that I'm getting interviewed right now, where as in baseball the statsman is just a production guy no one knows. It's an odd juxtaposition, but I like having a bit of a personality so I think the title is fitting in eSports.
Cool, I think that makes good sense, and what's most important is that you're happy with the title yourself. Last two questions now. Have you ever presented a stat which you realized soon after was misleading?
Kpoptosis: Not off the top of my head, although I facepalm a lot at remembering some of the things
I used to use as stats when I started out
Well, that's mildly disappointing! I was hoping for an anecdote. Anyway, my final question is about the two major upcoming events - ESL and TI4. With a lot of speculation always going on about who will win big events, I'm interested in a slightly different queston here. For both events, which team would you say might end up being a dark horse?
Kpoptosis: Sorry for the lack of an anecdote :(
Kpoptosis: For ESL I would say Vici are the dark horse – they seem to oftentimes be overlooked as the "4th best chinese team" but they performed well at the Summit and they have (in my opinion) the best support duo in the game in Fy and Fenrir.
For TI4 I think Na'Vi NA is my dark horse pick. They've existed as a team for only about a month and a half or so and they've already shown how well they can perform against the best teams.With some time bootcamping I think they can come into TI4 and finish strong. They're also all thoroughly experienced and I think Fogged may be the most underrated player in the West.
Well, both those teams are certainly not popularly expected to be frontrunners at those events so I guess time will tell if they fulfil the second criterion for being called dark horses.
Thank you so much for doing this interview! At this risk of sounding rude to other people I've interviewed in the past, this is probably the one I've enjoyed most by far. Do you have any
shoutouts or closing comments you'd like to make?
Kpoptosis: Shoutouts to BTS for giving me the chance to do what I love for a living and to my Fiancee for supporting my crazy rollercoaster of a profession. And I've thoroughly enjoyed the conversation :)