01-29-2021, 11:18 AM
We do have something like you describe, yes; for every game, we generate a couple of random trials, and we consistently keep testing that they still play out the same way when we update our code. Plus a bunch of other kinds of unit tests too.
I'm not sure that that's exactly "regression testing" though. As I understand the term, regression testing is creating a highly specific test case to cover rare bugs once they've been discovered and fixed, to ensure that they are immediately discovered if they ever resurface again. So that would be, saving a highly specific (non-random trial) that produces a rare edge-case where we previously had some bug, to make sure that doesn't happen again. We don't really have that yet... but I agree that we should :)
I'm not sure that that's exactly "regression testing" though. As I understand the term, regression testing is creating a highly specific test case to cover rare bugs once they've been discovered and fixed, to ensure that they are immediately discovered if they ever resurface again. So that would be, saving a highly specific (non-random trial) that produces a rare edge-case where we previously had some bug, to make sure that doesn't happen again. We don't really have that yet... but I agree that we should :)