I’m late for my usual weekly update. It was a long weekend (Victoria Day weekend in Canada), so this threw off the normal schedule a bit. The weather in Montreal also warmed up, which was pretty nice outside. Indoors the humidity made things a bit less pleasant, and I had to get out a fan.
I’ve been struggling to make much headway on anything programming-related outside of work in a little while. I’m not sure why, but I guess sometimes this happens. I was a bit tired with my current projects, so I decided to start on something a bit different.
I had the idea to write a music engine for my homebrew NES games. I wanted to support FamiTracker, since it’s the most popular music tracking software for composing music for the NES.
Before going down this road, it’s useful to evaluate what’s already there. A few other music popular NES engines exist that support importing from FamiTracker, such as FamiTone (or the older, original version here), GGSound and Pently. I’ve used Famitone and GGSound a bit, but they lack some features for making richer audio tracks in some regards. Never had a chance to try Pently.
There’s also another tool FamiStudio, which is a separate piano-roll style desktop music editor. It has FamiTracker import, and supports targeting directly for its own improved engine fork of FamiTone. But from the page’s own description, it sounds a bit closed to outside contribution and it’s a larger codebase, so I’m bit unsure about it for this reason.
Anyway, I looked at what was there but wanted to try writing my own thing. Hopefully something that has easy integration into my own projects, and has good support for some of the tracker effects that FT has available.
I also wanted the potential for supporting expansion chips, to approximate the sound capabilities in other game consoles, such as the MSX, PC Engine, or Game Boy. It wouldn’t sound 100% the same, but it would give a way to test these things out at least.
(MSX has on-hardware stuff like Trilotracker, and Game Boy has on-hardware LSDJ, and Nanoloop. But these are more strictly for writing and listening to music when nothing else is happening. Not meant as much for gameplay when the CPU is needed for other tasks too. There’s also Deflemask or XPMCK, which support a bunch of platforms but also don’t really have drivers that work great for a game. GB also has Paragon5 Tracker, but it’s ancient and clunky. So in light of all this, something like using FamiTracker to approximate ends up sounding like a good possible alternative.)
I’ve previously worked on a a couple attempts on different music engines, but these always relied on notation similar to MML (Music Macro Language), or byte-code interpreters that used command opcodes for notes+effects. In these cases, I was writing the engine, but didn’t have easy integration to a usable music editor. They had sample data or text-based formats that they could convert, for the purposes of testing what sound engines were capable of, but nothing easy to actually make music with.
There was also Bleep, an unfinished piano-roll music maker I was working on that ran on the GB hardware directly. While this was neat, and I want to continue with that someday, it’s less practical for game music making, and was more meant as a fun way to make songs on the go. For my current idea, I wanted something with easier integration with a desktop music editor that does chiptune music. Different goals.
Alright. So this time, I wanted to make a project that took things in the other direction – rather than engine-first like usual, I decided to try doing an importer tool first, then a converter tool. This way, I could write a music player that can interpret the parts of FamiTracker song that I support so far. This would allow making music in FamiTracker from the beginning. I can incrementally add support for more things, while being able to compare the output from my engine vs the FamiTracker. I can also use my past sound engine work to fill in some of details later when I get to that point.
FamiTracker’s normal file format is a binary and seems a bit annoying to handle. Not impossible but probably would take some extra time to get right. Thankfully, it also has a text-based file format that it can export that is (comparably) much easier to handle. Sadly, according to its own documentation, this export is not guaranteed to stay stable between versions… However, it turns out FamiTracker doesn’t change all that often anymore since it lacks active maintenance for the official version (last release in 2015). I also find it unlikely that they would abruptly throw it away fully, since lots of external music engines + converter tools depend on the format working a certain way now. So yeah, text format it is!
The format documentation is included in the .chm help file that comes included with FamiTracker’s installation. I couldn’t seem to find a direct online version of the text format specification on their wiki or website.
However, through a roundabout way (a github page that can link to .html files in github repos), I’m able to link to doc for those interested: https://htmlpreview.github.io/?https://github.com/HertzDevil/famitracker-all/blob/master/hlp/text_export.htm
This format is a simple enough: text-based format with line-based commands consisting of space-delimited tokens. But there’s some slightly tricky business comes with how it handles strings, which need some escaping. Otherwise, not to much to say about the format itself. The commands of the format do stuff like define tracks, instruments, macros, etc, piece by piece, as each is read and interpreted. It’s kind of unfortunate that it’s a custom text format, rather than just using JSON or something, which would make it a bit easier to integrate without other tools needing to write a new parser. But what’s there is still pretty nice. Parts of it can be more easily be human-read, and the music patterns themselves essentially look very similar to how they’re written in the editor, so it’s easier to inspect.
I originally wanted to write this tool in C++, but kinda regretting the decision to not just phone it in with a Python script or something instead. Writing anything that handles I/O and text handling with nice graceful error-handling always takes some work to get right. All in all, it’s pretty much the equivalent of split() with maybe a couple regexes, but I ended up settling on a simple character-by-character scanner instead since I could have finer control over the memory usage and more easily do input validation and errors with line info.
Aside from the text input stuff, I made decent progress on some data structures representing the in-memory representation of a FamiTracker document. That part was kinda fun to work out, going backwards from the description of the various commands in the export spec. I’m sure if I need I can read the FT source later too, but I wanted to “clean room” things a bit for now.
Anyway, I did manage to write most of a parser + a bunch of struct definitions for the FamiTracker’s document structure, individual tracks, patterns/rows/columns, instrument definitions, macro sequence definitions, etc. Didn’t quite get to the end, but close.
If I can manage to get this parser done, I can start selectively outputting parts of FamiTracker songs from this tool. Or potentially toss it out, treating it as an exercise only, and maybe think about doing it as a Python script after all heh. We’ll see.
I ended up returning a little bit to development plans for Wiz, too.
I created a private Git branch of the source to allow me to try stuff. I like open development most of the time. However, for big changes, it can be annoying to make sure breakage doesn’t happen at every step of the way, once people are able to see. And I’d rather make intermediate commits that break things and fix it. I could make a separate public WIP branch, but for this, I don’t really feel there’s much value in people seeing my in-between meanderings and prototyping that I might toss away or never finish. So private repo it is.
The project doesn’t yet have unit tests, so that’s one thing I want to fix. I also want to work on splitting the compiler into smaller modular pieces, which I’ve probably talked in another post before. Decided to just rip the entire compiler code out for starters, and phase it in piece by piece. This way, I can put more thought into where things should go this time around. I can potentially I clean up some inelegant things, and can review and make fresh notes about what to improve along the way.
I’m still not sure about it, but I’ve long hoped to add features that can define platform backends directly in the .wiz language. Wiz already has it so that every platform provides definitions for register sets, addressing modes, instructions, and their encodings. Right now these are done directly in C++, but if these could be moved, that would make maintaining these bindings easier and allow new ones to get integrated too.
With a little bit more work on top of that, it could even possible to allow mixing raw assembly with Wiz-style high-level code. This would allow interop with code written in other assemblers, allowing the inclusion of libraries that weren’t written in Wiz, without needing to port their sources.
Another thing is that, Wiz currently targets binary opcode formats directly from its IR, and doesn’t have any text dump of the final program code. Having this feature would allow easier verification of the code generated for bug-checking the asm and potentially for writing some functional tests of the compiler.
There’s also some language warts I’ve wanted to iron out, this gives a chance to potentially revisit that in the process.
Lastly, I got a new laptop, an Asus Vivobook Flip 14.
It just arrived after the long weekend, and so far it seems pretty decent.
The 2-in-1 laptop-and-tablet aspect wasn’t needed but it’s a nice extra. So far, I managed to uninstall the bloatware, disable a bunch of Windows 10 features + apply registry edits, and install a bunch of basic applications for getting development stuff done on it. And there’s Chrome, Steam, Spotify, etc.
The half-size arrow keys are unfortunate, but unfortunately all Windows laptops I looked at had some drawback or another for my personal preference. Also, this laptop seemed to have some of the better specs for comparable things of its price. And didn’t do things like remove the headphone jack or only have one USB port.
Overall, pretty happy to have a functioning laptop again. It’s been a long time since I’ve been able to do development away from desk.
My previous laptop was from 2013. While still kicking somehow, it has a keyboard that’s barely longer usable. They don’t make it easy to clean the keys without breakage. It’s been like that for about 1.5 years now (basically keys died right after xmas break in 2019). So it’s made me do all development work exclusively at my desk. With the pandemic requiring my home desktop to become my full-time workspace, being able to do projects on a different computer, away from my desk is quite nice.
I’ll probably keep my old machine as media box until it completely runs into the ground, since it’s still possible to type short parts of a url into a browser and then autocomplete and click on things.
Okay, see you next week if I can remember this time! Now, to try to sleep again.
