TaleSpire Dev Log 505

Heya folks!

I wrote this post last week, but forgot to post it. Woops! You’ll see a few strikethroughs like this in places I’ve updated today.

As planned, I’ve turned my attention to the wild performance issue that is the directory-watcher we’ve been using.

First off, why do we need one? We need to watch directories so we can notice when mods or symbiotes are added, removed, or even just modified (so we can live-reload symbiotes for developers as they edit the HTML/JS files).

To watch the directories, we used C#’s FileSystemWatcher Class. And it’s fine, sort of.

It’s one of those tools that promises to make things simple, but watching for changes gets complicated when cross-platform, network drives, etc, get in the mix.

The implementation in mono (the runtime Unity currently uses[0]) technically does what it says on the tin, but in a way that is super painful for us.

It runs on a thread and every 750 milliseconds it wakes up, walks all the files and directories looking for changes, and then goes back to sleep. Because they are using objects (lots of strings, for example), it allocates loads of small objects that the garbage collector then has to clear up.

Cleaning it up not only takes time but also means that, during scans, it is much more likely that the game will allocate just enough to push the GC’s over its threshold for deciding when it has to do a clean, at which point it blocks the main game thread so it can do that work.

This results in regular frame spikes, even on empty boards.

I’ve written some replacement code that uses Window API to detect changes, and it’s waaaay faster. We have gone from 1500ms of work every 750ms to 1ms of work, only when something changes. If no files change, there is (almost[1]) no cost. I would show a graph for the new one, but there is basically nothing to see!

In future we can do even better. Currently, we only detect that a directory has changed, not notify the interested parties (for example, the symbiotes-manager). Those systems will then scan the directory to find what they might want to handle.

Instead, we could deliver details of the exact file that changed and let the systems skip the scan.

I’ve not done this yet as I need to get onto other tickets, and this already is a huge improvement. It’s still nice to know that there are more gains to be had when we need them.

The astute will have noticed that I’ve only been talking about Windows here. Mac, for reasons I haven’t quite understood, does not seem to have the same issue. I have filed a separate ticket to look into it at a later date. I’m not digging into it now as we need to focus on the most pressing issues.

I’ll have this in your hands next week this week. I’m very excited to see how it performs for you. My tests suggest this gives more stable frame times but I haven’t dug in enough to claim that for sure.

Have a good one folks!

[0] This looks like it will change this year for the newest version of Unity, but that version is well out of our reach for the time being.

[1] Almost as a sleeping thread has to be managed by the operating system, but really, it’s fine for the foreseeable future.