There’s not a great or particularly short answer to this, but I’ll mention a few of the complicating factors. Note that isn’t a list of excuses and I think there’s a lot of additional things we can do to mitigate this stuff. But it’s a pet peeve of mine when people think bugs are from a sheer lack of testing (which we, you know, do) and I like talking about this stuff so buckle up friends. I’m also technically going off topic from a bug bounty so apologies for that.
So, one factor is the size of the game. There is an enormous amount of things to check, dependencies that can cause weird interactions, and just a lot of systems and, like, stuff you can do. In theory throwing more people at this problem helps, but a little like the famous mythical man-month, having more people wandering around the game hitting things may not help much. That will generate a lot of noise and bug tickets, but there’s no guarantee any of them would, say, slap an Abyssal dragon on a perch on one device and then attack that base with another device AND notice they were getting 2-shot when they shouldn’t. Or that we would wade through all the bug tickets created to notice that one in time.
The next is iteration. We’ve done a lot of work to make it easy to change certain parts of the game so you don’t need a computer science degree to tweak a tower’s health (and I’ve been helping some small endeavors to make changes even faster and more sane) but a downside of that is that, well, stuff can change a lot and very quickly. And if you don’t tell QA you’ve changed it they may not test that area again, and so a simple lapse in communication now results in a bug going live. So sure, QA has been testing Abyssal dragons on perches every week for 2 months now as they should because that’s a common player thing to do obviously, then 3 days before we ship someone fixes what they think is an unrelated bug or makes a quick perch adjustment that they forgot to mention and voila we’ve messed up (though I don’t actually know how that bug happened).
The last thing I’ll talk about is unintended changes. This is the one we possibly hate the most and so we have a lot of protections in place for it, but sadly nothing is foolproof. An accidental change can come from a lot of places, from simply hitting a key with a file open and not noticing you changed something, to outdated scripts screwing stuff up. That one always worries me somewhat since we have to use a lot of scripts to generate, optimize, package, and do all sorts of other things to our data and assets, and we assume they all work flawlessly until we’re proven otherwise. Sometimes it’s quite obvious when they need to be updated to work with some new format or whatever, but they could very subtly add changes we don’t want and that happen to be in QA’s blindspot.
For an extended hypothetical example let’s say we have a new upcoming version that solely has a single dragon. It reuses spells so there’s no new code, everyone swears upon their soul that they won’t make any other changes, and overall it looks like a small release. QA therefore does their basic checks on the tutorial, on upgrading on gamestates from players with various progress in the game, that the new dragon works and can be acquired, etc. And the version after that one has all sorts of stuff planned, really just stuffed full of things that the whole team has been working on, so QA wisely chooses to focus most of their time on that one. And none of us realize that the script that an engineer with a PhD in mathematics wrote 8 years ago that strips out blank lines at the end of certain files instead deleted the last row of a rune file, which the game gracefully handles except in a Kingdom Wars event in which case it crashes if your third dragon in an attack has that rune. I wouldn’t blame QA or a lack of testing for that; that’s instead a problem with process along with just bad luck really. And while we’ve learned long before War Dragons to set up warnings to metaphorically yell at us if we delete a row so I’m not too worried about that particular case happening, the general idea of a helper script possibly secretly breaking something hangs over me like a Bug of Damocles.
I hope this gives a little bit of insight into the exciting challenges we tackle daily. Again I’m not giving out excuses to try to make you all feel bad for us or something, for you should hold us accountable when we introduce issues in the game you’ve invested in, but I’d like to emphasize that we try to prevent bugs in a myriad of ways and man do the bugs fight back hard.