Luckily, there’s a workaround: the logging we put in to help debug OSL masked the bug. So instead of shipping the “release version,” we shipped the “debug version.” (This sort of issue is not unique to KoDP, by the way.)
This was actually something I forgot when I converted the game to iOS. (Looking back at my notes, I may have rediscovered this a day after releasing for iOS…) And I ended up making the same decision. It works in debug, so ship debug.
It turned out that even with debugging output turned on, there were a few extremely rare situations where the bug would surface. I believe I found and fixed some of those, but with tools, interpreter, and save files all revolving around the original implementation of lists, it was hard to make enough changes to be certain. And this was tricky code.
Back in 1997, we were very concerned about size (on disk and in memory) so we went to a lot of trouble to make lists (of clans, people, tribes) very compact. In retrospect, we committed one of the cardinal sins of programming: premature optimization. It was a noble goal, but at the start of the Six Ages implementation, I looked at KoDP saved games. Out of 1549 saved variables, 8 were lists. So compact storage was completely unnecessary. Lists are used far more frequently than they’re saved (for example, in picking a random clan from a list of neighbors who don’t hate you), so correct behavior was really the key.
While I didn’t really want to rewrite part of the game that was basically working, there was no guarantee that it would continue to (mostly) work with a whole new game. So Six Ages has a new implementation of the scripting language, from the bottom up. This supports new features, but is also an attempt to get it to work correctly in all situations.
So how do you guarantee correct behavior in a complex program? At one level, you can’t — or if you could, it would take too long to prove. But you can write code to test your code. The new implementation of OSL needed unit tests. (The original game had none — it would have been very unusual in the late 1990s.)
Various testing methodologies are big in business software (often combined with build automation), and a sign of a quality open source project is having tests. But it can be tricky to test games. There are ways to deal with randomness, but specifying a situation (like having exactly one non-hating neighbor, the right mix of advisors, or a raid history) or expected behavior for a multi-stage interactive scene can be tricky. And a lot of a game is its user interface, which poses its own testing challenges. Fortunately, while a game as a whole can be difficult to test, subsystems can be tested. The scripting language is just such a system. And even better, the handling of lists requires no testing tricks.
 |
| All tests pass! |
While I didn’t really use test-driven development, I did start writing unit tests very early, and once in a while did write the test first (specifying correct behavior) and then the code. And when I find a bug in the low-level code, I’ll add a test to make sure the bug is actually fixed, and not reintroduced by later changes. Looking back at the development diary, I see comments like
- Working on iteration. Hooray for unit tests.
- And glad I made a unit test, it looks like I need to add a copy of the treasure if I’m mutating them
- Again, hooray for unit tests. Did it for children, since that’s a kind of complex thing. And it caught a problem with setGoods: too.
- Oops, that type broke stuff. Hooray for unit tests!
Do some of those test names give a hint of the new game content? For example, I don't remember federations every being in KoDP, and I don't believe it stands for any computer jargon like immutable or iteration.
ReplyDelete