Cheese Talks: Porting Games to Linux & Day of the Tentacle

Hi people!

A new Cheese Talks article just went live!

This time, I'm looking back at what I learned from porting Day of the Tentacle Remastered to Linux (see our previous GamingOnLinux coverage here).



In addition to my own thoughts, the article includes insights from a number of other Linux game porters including Leszek Godlewski (Painkiller Hell & Damnation, Deadfall Adventures), Ryan "icculus" Gordon (StarBreak, Left 4 Dead 2, Unreal Tournament 2004, Another World, Cogs, Goat Simulator), David Gow (Keen Dreams, Multiwinia), Ethan Lee (Salt & Sanctuary, Hiden in Plain Sight, HackNet, Waveform, Dust: An Elysian Tail) and Aaron Melcher (Outland, La-Mulana, Hyper Light Drifter, Darkest Dungeon). Betweem them, they offer a great range of attitudes and approaches that support and provide counterpoint to my own experiences.



The whole article weighs in at a little over 11,000 words and covers what it's like to get started on a port, and what kind of ups, downs and betweens are likely to be found along the way. As a companion to the article, I've also been able to release the source code for the "coming soon" app that appeared for Linux users on Steam as the game got closer to release.

To whet your apetite and/or give a summary for those who aren't interested reading in the whole thing, here are some snippets.

QuoteMy personal connection to this game has expanded from being that of a player and an appreciator of its accomplishments to include that of a developer and in some respects a historian. I've had the opportunity not only to peek behind the curtain and experience a game I like from a new angle, but I've also been able to look back in time at a fascinating cross-section of LucasArts history. I've seen SCUMM code from the 80s written for Maniac Mansion (in one file, there's a lovely code comment saying there's a front row seat in hell for me if I edit it - I like to pretend that that's Ron Gilbert's "grumpy" persona yelling at me from across the decades), I've seen early 90s work from Day of the Tentacle's development, I've seen the "Remonkeyed" engine code written for the Monkey Island special editions, and I've seen Double Fine's work on adapting that for DotT's remastered edition.

QuoteFrom the outset, I was keenly aware that the codebase I'd be working in was of a scale beyond anything I had contributed to before. I'd been given the opportunity to review the codebase before signing a contract, but that really wasn't enough of a window to get the familiarity that I'd need to deliver a finished port - I'd need to be able to learn as I went without slowing myself down too much.

I set myself up a rough plan that would allow me to focus on achievable things while also exposing myself to more of the codebase as I went, which looked something like this:

* Set up my build environment ahead of time
* Work out-of-tree so that exploration and experimentation doesn't get committed
* Get the codebase to compile (stub out anything that looks like it needs rewriting)
* Sift back through looking for platform specific #if directives that might've been missed
* Look through all build scripts for any platform specific stuff that might've been missed
* Prepare an initial port commit and start working in-tree
* Rewrite any stubbed code
* Focus on fixing crashes/behavioural differences

By treating compilation as my initial goal, I'd be able to make forward progress with zero knowledge of the codebase - compiler errors would tell me where to look, and I'd learn bits and pieces about the project's structure as I went through doing superficial stuff like correcting case issues and making Linux specific copies of other platforms' #if directives.

QuoteSomething that was both expected and a surprise to realise was that my experiences of porting didn't really fall into the realm of what I would normally refer to as programming. In fact, reflecting on my experiences, I feel that it's probably fair to consider that it's a porter's goal to write as little code as possible.

All of the creative/design decisions around code structure, systems behaviour and dependencies are typically locked in place, and deviating from them means a port that's less likely to be maintained once it's completed.

Quote"The other really important thing to know when planning is if you're going to be working on the main branch of the game, or in a separate branch which will likely never be merged. You need to decide how much freedom you have to rewrite or refactor existing code, versus carefully changing only what is required. When in doubt, don't change things you don't have to, and document your changes well." - David Gow

QuoteThis may be a point of contention, as I do know some porters who don't agree with this outlook, but it seems important to me to make sure that I have lines of contact to developers who have previously worked on the project.

If there's a piece of code that needs to be understood before forward progress can be made, and that's either going to take two days to work through alone or twenty minutes' worth of conversation time, then it's pretty clear which approach is in the best interests of the project.

QuoteShortly after getting the Day of the Tentacle running (an early milestone), I had encountered an issue where audio cues were firing incorrectly. Advice from other developers who'd worked on the game was that it was most likely an interrupt loop that controlled the audio system being too far outside of its target 300ms interval, but nothing I tried on that front seemed to help. In an effort to track down the issue, I'd upgraded GCC to gain access to newer compiler flags that I hoped would help me narrow down my problem. In one late night flurry of bugfixes, I'd announced the problem solved by some unrelated commit and moved on.

When the time came to build the project inside the Steam Runtime, which just so happens to ship with GCC 4.8.1, the problem reared its unpleasant head again. After several days of slowly stepping through the game and scrutinising its behaviour, I discovered that somewhere between 4.8.5 and 4.9.2 (I couldn't spot anything useful in the GCC changelogs, and it didn't seem in the project's best interests to spend time digging out the relevant commit), the behaviour of the following C++ syntax, which is undefined in the language spec, had changed.

ptr += some_fuction(&ptr);

The ambiguity arises when this statement can be interpreted as either ptr = ptr + some_function(&ptr) or ptr = some_function(&ptr) + ptr. In the case we care about, the function modifies the value of ptr, and was used to calculate the length of data (say, an int representing how long an audio cue should last) in packed in memory. Instead of getting the correct memory address to read our value from, we end up reading partway through some other value and things spin wildly out of control from there. GCC's -Wsequence-point compiler flag is meant to help identify these kinds of problems, but it seems that the involvement of a function that manipulates the value of ptr confuses things a little.

QuoteOne friend and porter who would prefer to remain anonymous has passed on to me an aphorism that I found amusing: the more time spent on a bug, the smaller the fix. It's meant to be taken as a Murphy's Law type reassurance that tripping up on things that seem small in hindsight is normal, but it reflects some interesting characteristics of problem solving.

Small issues such as the one mentioned above are more easily overlooked, and the longer they wear on, the more frustrating they can become. With each hour, day, week or month that an elusive problem consumes, the number of more-obvious causes it could be hiding under dwindle. We learn about common problems like off-by-one errors and learn to look out for them, but until we've gotten that perspective, it's awfully difficult to spot them for what they are. Some things are just easier to see with experience, I think.

QuoteWhen it came time to move into testing, we encountered a fairly common and awkward situation. To allow access to testers who had normal subscriptions to the game on Steam, we would also be exposing Linux users to partial Steam configuration, which would cause the game to show up in their library and be listed as downloadable. Steam would "download" an empty folder and the first real indication that the player would get of something being wrong would be a misleading "missing executable" error. In other projects I've been involved with, this has generated support requests and grumbles from people assuming that developers don't know what they're doing.

Following the example set by some other porters (Ryan and Ethan in particular have been known to do this), I spent half an hour putting together a quick application that could be placed in the default non-beta branch and let Linux users know that the game was still coming soon.

QuoteRecognising that emotional and psychological state impact heavily on productivity, and making appropriate decisions around when to schedule breaks or recreational activities can help make sure that longer projects don't become overwhelming. In general, being aware of one's personal limits and reactions is important (see my game jam survival guide for some more thoughts on that) for being able to navigate and cope effectively with pressures and constraints.

Typically, I like to keep a rough window of time defined by best-case-scenario estimates and conservative not-so-good guesses for remaining work. When a relevant date (whether it's an external milestone or an internal personal goal) encroaches upon that window, I start adjusting my own expectations and make sure to communicate that upstream to whatever relevant contacts may need to know. Avoiding unpleasant surprises and being able to get the ball rolling on extensions or recalculated timeframes early can help make those (often unavoidable) situations easier to deal with and feel comfortable with.

To help keep track of progress and assist with triaging, I keep a log of thoughts, approaches and solutions for completed work as well as a list of known needed work. Typically this is more verbose and rambling than is appropriate for commit messages, but a bug tracker can be a good fit. Most of the time, I make do with a plain text file.

Having that kind of information available has been super helpful in keeping momentum and feel more comfortable stopping for sleep when in the middle of solving a problem.

Quote"Don't give up, always finish, ask for help, and don't let the work consume you. There can always be patches later. Learn what you can, check your assumptions, do better next time and don't beat yourself up too much. Sleep a little sometimes." - Ryan C. Gordon

QuoteAnybody who has worked on a big project will know that there's an adjustment period following the intensity of release. When working in teams, camaraderie and shared experience make this easier to work through. The ability to interact with communities and critics, to speak to and for one's work can be cathartic as well.

Many porters (particularly solo porters) don't have the same avenues for support. Porters generally are not authorised to speak on behalf of their employers, and without much creative input, the sense of personal investment can be diminished (to me, Day of the Tentacle is not, and never will be "my game"). The opportunities for working through "post release blues" are often very different for contractors.

For some projects that I've worked on, doing post-release support has been a nice way to wind down after a release, and doing volunteer outreach and tech support for Linux users that I've come across has been nice in that way. I can fully appreciate though, that this kind of direct community contact is not something that all porters are likely to find enjoyable or have permission to engage in.

QuoteFrom the work done by Matt, Oliver and the rest of the DotT Remastered team; through to Craig Derrick's and the Monkey Island Special Editions' teams' efforts; Tim, Dave and the original DotT team's work; and all the way back to Ron, Chip and Aric's work on the original SCUMM engine, the codebase I inherited for my port was well designed and (so much as these things can be) a delight to work with. I am very thankful to have been able to "work with" such talented people on well-made software.

This article and the Day of the Tentacle Remastered Linux port could not have been possible without the interest and support of Double Fine and the lovely people who work there. Being asked to work on the port was unexpected and I am humbled to have been invited to work on a game I love.

This has been a super huge journey for me, and I'm glad to be able to share my experiences with anybody who's interested.

Enjoy!


Update: I've also published a video with reflections on the game and my experiences of porting it.

Article taken from GamingOnLinux.com.