Power VWF suggestions

[ExE-Boss]

Here are a few suggestions I have for the upcoming Power VWF patch:

• Use the UTF‑8 encoding (like what my Core Dump Patch does, and what I plan to do with the current VWF Dialogues Patch)
• Make the version 2.0, as it is essentially a re‑write of the VWF Dialogues Patch.

[RPGHacker]

At this point, I'm not quite sure if this new version is even going to happen at all. The version I accidentally commit earlier today was actually started over a year ago. Back then, I experimented and researched a bit, but eventually put my plans on ice as I couldn't find satisfying solutions to some of my problems. The new VWF patch wasn't going to be a rewrite of the old one, but rather an alternative to it with a different focus, that's why I didn't call it 2.0. It was supposed to have different features and live alongside the old patch.

Some of the things I had planned for it:

• Overall better performance: I wanted the new version to be able to fill an entire text box quickly by doing a bunch of letters in a single frame - something like this is seen in a couple of RPGs on the SNES.
• Smaller memory footprint: I wanted this new version to be able to run with way less memory at the cost of being less flexible. Basically, the idea was that noone ever needed an entire screen of text, anyways, so I could limit the text box size quite a bit. If I then found enough layer 3 tiles that were save to overwrite (because they were not used in the level or could be directly reloaded from ROM), I could completely skip the layer 3 backup step (and eventually the layer 3 restore step) to save even more memory (and a bit of time).
• Variable font height: along with variable font widths, I also wanted to support variable font heights, kinda like seen in Secret of Evermore. So you could, for example, have used a font that's just 12 pixels high or something like that. However, this kinda bit with wanting a small memory footprint: allowing font heights that are not a multiple of 8 requires you to always keep the previous line of tiles in memory (or read them back from VRAM, but that isn't very practicable, either, as it bites with wanting better performance).

So yeah, I experimented quite a bit with these aspects, but in the end, I didn't find enough layer 3 tiles in vanilla SMW that were unused, and their segmentation was also too big, so moving on with my plans wasn't very pratical and I put them on ice.

Anyways, if I ever were to pick this back up again, a few more ideas that would be worth looking into:

• Support for encoded text - would allow a lot more text to go into the ROM at less space. From what I know, some SNES RPGs used Huffman coding to store text, though I know practically nothing about this encoding except that it exists.
• A GUI application. Having a GUI application would allow us to optimize our text data more, since it wouldn't have to be in a human-readable format anymore. That was kinda the ace up my sleeve. I thought that if I could somehow make this patch work, but requiring a complicated table setup, it could still be worth it, because I could just write a tool to handle all the complicated data.
• Text preprocessing. This could be done either at compile-time (less flexible) or at run-time (more flexible, but higher memory and performance cost). The way the old patch handles this is kinda horrible. It always just calculates the width for the next word and thus jumps all over the place. I would prefer to avoid that and instead find some solution which has to process the text just once (or at the very least just once per filled text box).

As for the encoding, I'm not quite sure if UTF-8 would be a practical encoding for this. UTF-8 is (probably) the best encoding for the needs of computers nowadays, but I'm not sure if it works as well in the context of a SNES ROM. First of all, it takes a bit more processing than most encodings. It's not too terribly much, but in the context of a SNES ROM might be enough to make somewhat of a performance difference. Secondly, I don't think a single SNES ROM is ever gonna be able to contain enough text and/or font data for full Unicode support to be worth it. On average, I think a SNES game is going to use < 100 different characters. Maybe a few hundred different characters for more complex languages, but still just a minor subset of Unicode. Doesn't quite make sense to use a unicode mapping for that. Thirdly, in the context of SNES ROMs, no language would really benefit from UTF-8 as far as I can tell. Latin-based languages would increase slightly in size, Asian-based languages (and others) would even increase drastically in size (I think you need three bytes to encode most Japanese characters in UTF-8). In fact, the latter is also the reason why I wanted to deprecate 16-bit support in the current version: I just don't see a practical need for it. Even Japanese hackers, if they were going to use this patch, would probably just use an encoding where one character was one byte, it would be way more efficient. I think other, more dated and less international encodings (like ANSI or Shift-JIS) would actually make more sense here since they take less space.

However, I would probably not go this route at all and not use any particular encoding in the patch itself. Instead, I would leave something like UTF-8 for design-time and use an arbitrary encoding at run-time. That seems like the best use of memory and performance. As described above I would still want to make use of some encoding, though. I think many SNES RPGs used some kind of simple encoding where some byte corresponded to some character, yet they also had special bytes which corrsponded to an entire word, which was used to save memory on commonly used words. For example: if the word "magic" was commonly used in your script, you could encode it as a single byte or something like that and save a lot of space that way. This is something I would probably like to see. It should also be quite automateable via a script or tool (it would just have to scan your text for the most-used words).

So yeah, as you can see, I've thought a lot about this already and there's a lot of stuff that would go into a new version, but I just don't know how much of that is practicable at all on the SNES. Have to look a bit into my old research again to figure out what problems I had exactly.

[ExE-Boss]

Here’s my corrected file for which tiles belong to what on the Layer 3:

I’d personally recommend just overwriting everything from $80 to $17C, skipping $FC trough $FF and moving the large Star counter from LG2 to LG1.

[RPGHacker]

That reminds me: one of the things I had also planned back then was to, in theory, allow the Status Bar to remain on-screen (at least if a text box is used that's small enough). That's why I specifically marked all status bar tiles as "don't overwrite". Not sure if that's in any way reasonable, though. It might just not be enough to fit enough text on-screen.