tmfbma alpha demo post-mortem
let’s review!
@leafcodes and I released the Alpha Demo of Tiny Melon Friend’s Big Melon Adventure, bang on time at the end of April. Now that it’s been a little while since we did, it’s time to dig in and take a look at the feedback, the launch itself, and the various bits and pieces of analytic data we recovered!
The Launch 🚀
TMFBMA v0.1 Alpha Demo made its debut on its own official twitter account that we made for it, with the game files hosted up on itch. We blasted it with retweets and quote tweets and got in front of as many friendly eyeballs as we could!
Not only that, but we also launched the OST on Bandcamp - some people definitely like to listen to a few tracks from a game before trying it themselves.
All that remained was to see how people liked our game..!
The Big Red Button Incident 🔴
And here’s where the lessons learned begin!
In the introductory sequence, the player has to push a large red button to advance. The button in question can only be activated by the player attacking it, but luckily the tutorialisation of the attack button has already happened by this point. So far, so good, right?
Wrong.
One of our earliest players did not hit the button and spent the next 10-15 minutes wandering around and trying a variety of techniques to push the button without attacking the button at all, until eventually asking us for help. Definitely not the intended path!
So what went wrong?
The button was oriented so that it was sitting on the ground, in such a way that it looked like the player was supposed to jump on it, or put something on it - like the buttons in Portal. This was the primary point of confusion. The player did indeed try attacking the button, but did not make contact with the button and so assumed that attacking did nothing. The player ended up traveling back to the attack button tutorial and laboriously guided a physics object all the way to the big red button, all to no avail.
The reason that the attacks had no effect on the button is that tiny melon friend’s attack colliders never descend past the feet - making it impossible to actually attack an object below them. The button had to be hit by standing to its side, and attacking at it from an angle. This absolute disaster in game design was rightfully pointed out by @leafcodes and I unwisely ignored it until the game was out. Oops!
It was easily fixed by rotating the button so that it was mounted on the wall of the small room it was in. It was quickly patched into the release, and we had no further confusion after that!
Crashes & Telemetry Woes 🌒 📡
Next, we had an issue where the game crashed during a particular cutscene - at exactly the moment a character’s dialogue is cut off by an event. At least one player told us they thought the game’s closure was intentional and that we were using ludotechnic tricks like Undertale or Metal Gear Solid to end the demo in a fun way, and that they were surprised our demo was so short.
😔
We discovered the problem was due to two things: lua scripts and telemetry.
Why telemetry? Well, if the player had provided their consent for telemetry, every time they started or ended a cutscene, we logged a telemetry event. This helped us track how fast the cutscenes were running, what kind of dialogue responses people chose, and how quickly they were skipping through text.
The telemetry server we built was an extremely quick & dirty little thing in Python, using Flask and PickleDB. PickleDB ran into some concurrency issues upon saving the database to file if two telemetry events came in together from different clients, and would report back a 500 to the game. Despite C#’s HTTP response handler letting you validate the response code upon receiving a response, it also throws an exception - which was unhandled by the demo, and crashed it! This exception was never encountered while I was debugging, since I wasn’t sending concurrent messages to the server from many clients.
So, one exception handler later, and the game was still crashing at that point - but only if the player had played straight from the very first cutscene all the way through to this point in the game without exiting to the menu or quitting the game. This was because we used NLua to add lua scripting support to the engine. NLua does a wonderful job of making it easy to expose functionality by utilising C#’s reflection features. There are a few notable pitfalls, however, which we ran into.
The scene that crashed had a script which spawned a generic game entity and gave it the texture of a giant boulder, and rolled it through the scene. One of the previous scenes spawned a different type of entity and gave it a different texture too. Both of these spawning events used a C# function with generic parameters. See where this is heading?
NLua had coerced the function signature of the spawning function to match against a single concrete type, based on its first usage in the script in the previous scene. When the rock-spawning script then called the function, the signatures no longer matched, and hey presto - a crash. If the user quit the game between the two scenes, the function signature would have been generated from only the second call, and the game would have worked fine.
We solved this one by providing hardcoded functions that spawned the objects of the right types without exposing the generic signature to the Lua state.
Data & Feedback 📈 📋 ⭐⭐⭐⭐⭐
With those two issues patched and out of the way, we were finally able to start collecting a bunch of gameplay data and feedback, uninterrupted and without significant crashes!
Here’s a few of our main points of data:
Of those surveyed, 80% chose to provide telemetry to us! We’ll get to what we got out of that soon.
Of those surveyed, around 30% explicitly found all the secrets in the demo. It’s super neat that nearly a third of players felt engaged enough to go hunting for secrets even when we didn’t highlight or push them towards them in the game at all.
And out of those surveyed, most people said that the game ran perfectly smoothly on their computers - and there were a very wide range of computers used by our players! Most interesting here is the response that specifies that they’re running Ubuntu; we had intended to release the demo multi-platform, but ran into issues with Otter2D & SFML’s cross-platform support when used with .Net Core, and so we launched it for Windows only. However, the Ubuntu player successfully played the game using Wine, and apparently the game ran well even under those conditions, which is heartening!
Now, onto the telemetry data! We have an endpoint on our server that spits out the collected data as a big blob of json, which we import automatically into a spreadsheet. Here’s an example of the data we got:
Mostly this let us see how long it took players to get certain items or reach certain scenes in the game. Most people played the game in a single sitting/session, with the occasional player restarting and playing another level again to find secret items. This tracks just about 1-to-1 with what we saw on the survey; around 30% of players would re-enter older levels before proceeding to the next, or after they had seen the credits.
Our most crucial finding was that in response to being asked how tiny melon friend’s giant beefy arm works, 100% of all players (who played with telemetry on) chose to describe it as “meat liquid”. Rest in peace, “spongiform bones”.
Conclusions & ✨The Future✨
Overall, we’re really very happy with the responses we got. There was a lot of valuable feedback in terms of the game’s aesthetic and controls that we are going to spend some time sifting through and figuring out which issues are best to address, and the best way to address them.
Primarily, we’re likely to make some changes to the movement and combat mechanics, and will probably give melonfriend a couple of new abilities to help with handling and management of physics objects. The form that’ll take is a secret, though! We also want to overhaul some of the game’s UI, particularly around dialogue interactions. We had a number of respondants tell us they didn’t know the names of the characters. Can you believe we forgot to put name tags on the dialogue boxes?!
Anyways - we won’t be developing TMFBMA itself much further right away; besides needing some rest, it seems the issues we’ve had on a technical level necessitate some changes to our engine architecture. We’re looking forward to putting in some time in solving some of the bigger areas of technical debt in the project, and we’re in the process of building a new framework for the game to live in!
Otter2D has served me well for a good many years - but with Kyle Pulver’s decision to retire the framework from active development, and with the mounting frustrations with SFML, we need to push forward and ensure our foundations are solid before building the game up further.
There are also a number of improvements we want to make around tools for building content in the game. Most of the main content of the demo (levels, dialogue, cutscenes, bossfight, enemy scripts) was built over the course of a weekend, which is certainly not a bad or long turnaround time, but involved a lot of manual edits to json files. Building our own custom framework gives us an opportunity to build the types of tools we need to edit this data in at the ground level, in a way that we have maximum control over.
Either way, it’s going to be very interesting, and I hope you’ll enjoy what we come up with next!
