I'm planning to write a series of posts about low-level modifications to New Super Mario Bros. Wii, the 2009 Nintendo game. This is a sort of 'prequel' to that series, where I write some fluff about how I got to this point and the history/background of NSMBW mods.
Later posts will be more technical, with more of a focus on low-level reverse-engineering stuff and on how I achieved the things I did in Newer SMBW.
I've always enjoyed tinkering with software and seeing what makes it tick, and when I discovered the 'ROM hacking' scene in 2005, I was immediately hooked. At the time, I'd taken an interest in discovering and playing classic Nintendo games (through emulation) - especially the Mario series - and I thought it was ridiculously cool that you could modify games and add things like custom levels and graphics.
Of course, with my limited skills, I couldn't do much other than use existing tools for editing specific games - I don't think I'd ever touched a hex editor at that point, and even if I had, I wouldn't have known how to do anything with a ROM...! On older consoles, all the game code and data - graphics, levels, music, sound effects, etc - were all shoved into the same file.
One thing that makes modern games much easier to delve into - at least, for somebody with the level of skill I had at that point - is that they use a standard file structure which you can easily extract. This was first used by Nintendo on the DS and GameCube. It doesn't mean you can immediately obtain any data you want, as most of it will be in a non-standard format (e.g. the 'nsbmd' model files used by many first-party Nintendo games), but it's still better than no structure at all, and games often share formats with each other for resources like textures, models and UI layouts.
When New Super Mario Bros. was released for the Nintendo DS in May 2006, I immediately wanted a level editor for it, but I didn't quite have the skills to do that at the time, so I waited to see if somebody else would... to no avail. Eventually I decided to try and build my own, and in July 2007 I started doing just that.
I was still very much a novice, but I slowly taught myself how to figure out simple file formats and build half-usable software. From then on, I rewrote my NSMB editor a couple of times - adding more features and learning more each time - and eventually handed it off to somebody else as an open-source C# project.
Nintendo released the inevitable sequel to that game in November 2009, as New Super Mario Bros. Wii, and of course, I simply had to dig into it! It was leaked to pirate sites a week before the release (thanks to a store in Australia that broke the street date), and I couldn't resist getting my grubby hacker paws all over it.
I quickly figured out that the level format was barely changed from the DS iteration, and it only took me a couple of days to put together a prototype level editor and post a teaser screenshot on the GBATemp message board.
I asked my home IRC channel for a name for this new editor, and for some reason or other, we ended up calling it "Reggie!". I think the reason might have been Bob's Game and his REGGIE! cry, but I can't remember for sure... :p I built up a small team of like-minded people - who helped out massively with documenting the many object settings in the game, and testing, among other tasks - and over the next few months, we developed Reggie into a rather substantial project. (We eventually released it in March 2010 after about four months of development.)
At the same time that this was going on, after the announcement of Reggie!, some users on GBATemp were proposing and discussing a project they called 'Newer Super Mario Bros. Wii'. To us (the Reggie! dev team), this idea seemed outright absurd - we'd only just figured out how to edit level files, and weren't sure how much more we could achieve with the game. Yet the Newer thread on GBATemp was planning a full revamp of the game, and discussing all sorts of ideas for things like new worlds, characters and boss battles.
One night in December 2009, I was bored and decided I'd jump into their chat room and see what they said about me being there. They accused me of being an impostor, but once I proved I was myself, I ended up sticking around for a while. Following some initial hesitation, the Reggie! and Newer teams ended up joining forces. They received access to pre-release builds of our editor, and we received valuable feedback and extra testing. A fine exchange, right?
We still didn't really believe Newer would end up going anywhere - it was an extremely ambitious project - but we figured we'd finish our level editor anyway. A project like that would definitely require significant amounts of patches to the game code - what people often call 'ASM hacking' - and doing this to a Wii game would be infeasible (especially for somebody like me)... or so I thought. I had a small bit of assembly knowledge thanks to prior tinkering with SMB1 (6502 on the NES), and NSMB (ARM on the DS), but no PowerPC, and figuring out how a large game like this worked seemed downright impossible.
Even so, one of the other members of the Reggie! team (megazig) had some experience with reversing games and Wii code, and he looked into NSMBW and found a few interesting bits of info which we ended up using in Reggie, like structures and values extracted from the code responsible for reading level files. This sparked my own interest in disassembling the game. I got some advice from megazig, a combined .elf file I could analyse from comex (in the next post I'll explain why this was necessary), and then I started digging through the game code.
"What the hell have I got myself into?", I thought. I looked through pages and pages of identical-seeming PowerPC assembly and had no idea where to begin. There were quite a few interesting strings in the game executable, like the names of files and individual resources, but none of these seemed like a good starting point, especially when I had no specific goal in mind.
Another problem was that when I began, I understood assembly well enough to identify what individual instructions were doing, but I couldn't understand what a given block of code was supposed to do - before I could really make sense of it, I had to translate it to pseudo-C and read that.
I was about to give up and go back to Reggie! when I struck gold. I was scrolling through the disassembly when I came upon a list of strings that seemed suspiciously like the names of game objects. Things like EN_STAR_COIN, EN_KURIBO, OBJ_MORTON and PEACH_CASTLE_BLOCK. I looked at the code that referenced this, and saw that it was reading an entry from the name list and storing it into a structure. So I thought, what if this code is related to object creation?
I painstakingly translated that function to C and it appeared to confirm my suspicion, and as I went on exploring from there, I started learning more and more about the game (and reverse-engineering in general). With nothing other than very limited guidance from megazig, I had to figure out how things like class constructors/destructors, virtual functions and new/delete were handled, and how they correlated with my existing C++ knowledge.
And as I learned more about the game's internals, and started developing code patches (which worked both on the Dolphin emulator and on real consoles), I realised that Newer actually could happen - perhaps in a less ambitious form than originally envisaged, but it was possible. We were gradually gaining more technical abilities and we were also creating better custom content. Over the next three years, we worked on adding new things and on using our increasing knowledge to make the game better... and in June 2013, we finally released the game.
The final version of Newer includes over 33,000 lines of C++ code written by Tempus and I (not counting the header files which bind to Nintendo's APIs), over 8,000 lines of assembly, and compiles to around 450kb of new code alone. This includes the following:
... and that doesn't take into account all the work we spent on reverse-engineering the game engine, building auxiliary tools (mainly the map designer, but I also have a set of private tools for limited tasks like model and UI editing) and on things we scrapped (like our initial plans to use 3D world maps, which failed due to the lack of available modellers).
So that's Newer and Reggie! in a rather large nutshell. Now that I've covered the backstory behind these projects, and how I got to this point, my next posts will focus on the technical side of Newer - all sorts of fun things like the Wii's software architecture, how we compile code, and how we get it to run inside the game.
Previous Post: Introduction
Next Post: A Rant About Proxying API Requests on iOS (and others)