Emulator

This could probably be fixed by just replacing while (*str != 0) with while (1) and relying on the if condition in the function to terminate at the end, but I'm not 100% sure that that wouldn't cause other problems.

I just tested replacing the condition with an endless loop, and it seems to work properly.
The decrypted data isn't null-terminated, which means, there will be a bunch of crap at the end of the decrypted data (and I don't see a way to detect that). But it does end with a newline.

And because the GetValue function starts from the beginning and returns the first match found, a bunch of additional junk data at the end of the string shouldn't matter.

I just opened PR #8680 to fix this.

Unfortunately I have to ask you to open this issue again...

The three NANDs I imported yesterday to test the PR (which decrypts the whole buffer) worked perfectly, but now I found another NAND that results - what I first thought - in the same bug. But it's not actually the same bug:

I added a bunch of debug logging output to Dolphin, and everywhere in Dolphin's code the setting.txt is fine, non-corrupted, as it should be after all the fixes merged in the last few PRs. However the serial number the game sends to Wiimmfi was corrupted yet again, even though I saw nothing in the logs that could explain this.

Then I looked at how Dolphin sends the setting.txt or the data within to the game. Turns out Dolphin just stores the whole encrypted file at 80003800, and the game reads it from there and decrypts it on its own.

So far, so good.

HOWEVER, Nintendo actually made the same mistake that was in the setting.txt parsing code in Dolphin (can it really be labelled a "mistake" then if Nintendo does the same? :P ). The binary data that Dolphin copies to 80003800 is a perfectly valid setting.txt - and when I decode it manually, I get the proper content back. But when the GAME decodes the setting.txt (if anyone wants to look at the code, the related function "__SCF1" is in Mario Kart Wii PAL at address 801b2234), it ALSO assumes that a null byte in the encrypted data means "end of file" - the exact same wrong assumption Dolphin made.

So why did this bug (assuming 00 = eof) not affect real consoles, but only Dolphin?
Simple: Nintendo knew the bug, but it was probably too late to fix it.

What did they do? They made sure that if their generated setting.txt for a console would have included a NULL byte in the generated code, they added an arbitrary additional LF (line feed) character before the line, making sure the encrypted data looks "different" and that there is no NULL byte in the encrypted data. This is the explaination for the arbitrary additional line feeds that required the PR 8670: https://github.com/dolphin-emu/dolphin/pull/8670

What this means for Dolphin:
When Dolphin creates a setting.txt from scratch, it needs to make sure that the encrypted data doesn't contain a NULL byte. If it does, the plaintext data needs to be modified (by adding useless line feeds) and re-encoded until the encrypted data no longer contains a NULL byte.

That took me a while to figure out again ...
I'll see if I can somehow fix this issue in Dolphin easily, right now the encryption seems to be done in a kind of stream (every incoming byte is immediately encrypted), which doesn't allow this kind of modification. That means I would need to rewite that part so that the payload data is first collected in decrypted format, and then afterwards it can be encoded.

What a stupid mistake...

Yeah, of course. Thanks.

Thinking about that again, when that is fixed, we can probably go back to ending the parsing at the first encrypted null byte and no longer have to decrypt junk at the end of the file.

I'm working on a fix right now, but what do we do in case the added LF in itself happens to encrypt to null? I was assuming that any problem could be handled by adding even more LFs at the beginning of the line, but this one can't.

Good question. Maybe add a full newline? I mean, another CR-LF? Although I don't know if the game would like a full empty line in the middle of the data, we would need to verify that.

Or add another LF earlier in the file?

I ran a simulation, and apparently LF cannot be encoded into null for the first 0x100 bytes (which is the maximum possible size) due to the selected key.

However what could happen would be that not adding a LF causes a null byte in a certain line, and adding a LF causes a null byte at another point in the same line. So it could still be possible that multiple LFs are needed, until there is no NULL byte in the whole data for the line, couldn't it?

Yes, my implementation (which I submitted as a PR just now) will add as many LFs as necessary.

That fix is now pending since almost a month, is there anything I can do to get this merged soon-ish?