Project

General

Profile

Actions

Emulator Issues #11930

closed

NAND backup serial number import not working properly

Added by Leseratte10 over 4 years ago. Updated about 4 years ago.

Status:
Fixed
Priority:
Normal
Assignee:
-
% Done:

0%

Operating system:
N/A
Issue type:
Bug
Milestone:
Regression:
No
Relates to usability:
No
Relates to performance:
No
Easy:
No
Relates to maintainability:
No
Regression start:
Fixed in:

Description

Game Name?

Doesn't really matter (happens with all Wii games), but I tested with Mario Kart Wii.

What's the problem? Describe what went wrong.

When you import a NAND backup from an actual console into Dolphin (which is required for online play in Mario Kart Wii on Wiimmfi), in theory, the "console identity" the server sees when Dolphin connects should be the same one the server sees when the console connects to the server (if Dolphin was perfectly emulating the Wii, and if the user entered his Wii's MAC address into the Dolphin.ini file properly).

However, we've noticed two things Dolphin does differently than on console, which mean that A) the Dolphin instance is seen as another console identity on the server as the real console, and B) Dolphin has four different console identities, one for each game region.

Meaning, that instead of having one console identity shared between console and Dolphin, there are (worst case) five, one for the Wii, and one for each region in Dolphin.

I've noticed that each time I import a NAND into Dolphin, the resulting emulated console has a different serial number. I don't know why that happens, I checked the source code and it looks like it should import the serial number from the NAND. Either I missed some code where this is deliberately reset, or there is a bug in Dolphin.

The second issue is that Dolphin regenerates the setting.txt from scratch on each game boot, and it doesn't honor the serial number prefix properly.
Looking at the Dolphin source code at Core/Core/Boot/Boot_BS2Emu.cpp there is a function called CBoot::SetupWiiMemory, which checks the game region and creates this region_settings array thing, which contains the serial number prefixes (LJ for Japan, LU for USA, LE for PAL and LKH for Korea). This causes the resulting serial number to be different for each game region, meaning, that if a player connects to Wiimmfi with games from four different regions, that'll be seen as four different consoles (which does not happen on Wii).

Summary

A) Importing a NAND backup doesn't use the serial number from the NAND backup but regenerates a random one and
B) Booting a game switches the serial number prefix to one matching the game region, instead of using the prefix from the NAND backup.

Is there a particular reason for both of these quirks? They force Dolphin users to wait an additional week because their Dolphin instance is detected as a new console, or, worst-case, for them to wait multiple weeks if they play games from different regions. Also, this bloats up the user's Wiimmfi web interface when they see five different entries for their Dolphin instead of just one.

What steps will reproduce the problem?

Not that easy to reproduce in-game, but with a bit of logging the issue should become clear. As seen in CBoot::SetupWiiMemory within Core/Core/Boot/Boot_BS2Emu.cpp there is a hardcoded serial number prefix for each region (while Dolphin is supposed to use the one from the NAND backup, if the user did import one); and each import of the same NAND backup generates a new serial number instead of reading and using the one from the NAND backup.

But if you do want to test in-game, this is how you could do it:

  • Import your Wii's NAND into Dolphin
  • Get a Wiimmfi-patched image for any supported game
  • Boot it and connect to Wiimmfi
  • While doing so, run at a network dump and observe the serial number sent to the first server at ca.nas.wiimmfi.de (for Mario Kart Wii) or naswii.wiimmfi.de (for any other game) in the "csnum" parameter. (or enable debug logging in Dolphin and look for the "Using serial number" line)
  • Observe that A) the serial number changes each time you import the NAND and doesn't match the serial number of your console, and that B) the serial number prefix is one of the four hardcoded ones, and not the one of your console.

Is the issue present in the latest development version? For future reference, please also write down the version number of the latest development version.

Yes, 5.0-11382.

Is the issue present in the latest stable version?

Unable to test, 5.0-stable is unable to connect to Wiimmfi. Highly likely that this is present in 5.0-stable as well.

What are your PC specifications? (CPU, GPU, Operating System, more)

Highly unlikely that this is relevant, but why not: i9-9900K, NVIDIA RTX 2080 Ti, Ubuntu 19.10.

Actions #1

Updated by JosJuice over 4 years ago

Problem A seems like a valid problem (though I'm not quite sure what's going on).

It's unlikely that we'll do anything about B, since we want to minimize the risk that a game behaves badly (e.g. crashes) due to the region settings not being consistent with what the game expects. If you want the same behavior as on console, you can launch the game from a Wii Menu of the region that you want (either by launching it directly from the Wii Menu with some region bypass hack, or by booting homebrew from the Wii Menu that then launches the game). You should also be able to launch the game from a homebrew launcher like Gecko OS without booting the Wii Menu first – when booting a DOL or ELF file, Dolphin will use the region of the currently installed Wii Menu, with PAL as a fallback if you don't have a Wii Menu installed.

Actions #2

Updated by Leseratte10 over 4 years ago

But do you think there would actually be any game that would care about B, i. e. care about the console serial number region being wrong?

Given the fact that A) there are many different region prefixes per region (for example LE, LEH, HEF, OEH are all PAL consoles) so it's unlikely that the games would have a "whitelist" and B) given that there is no game (at least no game that I know of) that causes trouble when run through the system menu on Dolphin, or causes trouble on console when run on a different-region console?

I don't think there is any game that cares about the serial number prefix matching the game region, otherwise it would crash on an out-of-region console as well.

Also, wouldn't booting (for example) an USA game on a PAL NAND with an USA serial number be even more "confusing" to the game than running an USA game on a PAL NAND with a PAL serial number?

If there is any known game that relies on the console serial number having a prefix that matches the game region I'd agree with you that it should be left as-is. Do you know of any such game? I don't, and I don't think such game exists, so in my opinion Dolphin should behave like a Wii and leave the imported NAND serial prefix alone.

But just fixing A) would at least be some kind of improvement, as I doubt people play that many games online that aren't from the same region as their console.

Actions #3

Updated by JosJuice over 4 years ago

I'm not actually aware of any games that have problems with that it not matching, no. Maybe it would be fine to not overwrite it if it already exists, after all... But the problem is that the only real way to get enough testing done on whether it's okay to do or not is to do apply this change to Dolphin and ship it out there :/

If we don't want to do that, then at least I guess it would make sense to make it so that the string doesn't change if the file currently contains a different string that is for the same region.

Also, wouldn't booting (for example) an USA game on a PAL NAND with an USA serial number be even more "confusing" to the game than running an USA game on a PAL NAND with a PAL serial number?

Not really, because the game has other ways to notice that something is wrong with the region, even if it wasn't for the serial number. For instance, the video mode that the game boots in (which unlike the serial number is known to break certain games).

Actions #4

Updated by Leseratte10 over 4 years ago

Maybe it could at least be possible to take the serial number prefix from the backup when the game being played is from the same region? Currently, even in that case it would use one of the hardcoded ones and not the one from the console.

Actions #5

Updated by JosJuice over 4 years ago

Yeah, that should be a non-controversial change. Do you know if there's a list somewhere of the possible prefixes for each region?

Actions #6

Updated by Leseratte10 over 4 years ago

Well if you say that that process wouldn't be more confusing because the game has other ways like the video mode, then that would mean the serial number change wouldn't matter, right?

And isn't that the same on Wii, when you boot an out-of-region game? These work perfectly fine either, to my knowledge.

Actions #7

Updated by JosJuice over 4 years ago

Well if you say that that process wouldn't be more confusing because the game has other ways like the video mode, then that would mean the serial number change wouldn't matter, right?

The serial change would likely not matter much, but I can't prove that that's the case.

Also, do note that Dolphin matches the video mode with the game's region by default (unless you enable Allow Mismatched Region Settings). Actually, that makes me think... Why doesn't Allow Mismatched Region Settings affect the serial number prefix? Maybe that is something I should change.

And isn't that the same on Wii, when you boot an out-of-region game? These work perfectly fine either, to my knowledge.

Only if you are booting a game in the same way that the Wii Menu would boot it (i.e. without taking the game's region into account). I'm not sure if Gecko OS does it differently from the Wii Menu, but do I know that USB Loader GX does it differently (at least with certain settings). There are games that break both on Dolphin and real consoles when launched on a console of the wrong region.

Actions #8

Updated by Leseratte10 over 4 years ago

I don't know of any complete list of possible prefixes for each region. If we wanted to just always take the prefix from the NAND (ignoring game region) then that wouldn't be needed anyways. And with that "workaround" of only using a generic one on an out-of-region game, couldn't you just check if the game has the same region as the NAND, and if so, use the existing prefix? Why would you need to know which prefixes are for which regions in that case?

Actions #9

Updated by JosJuice over 4 years ago

And with that "workaround" of only using a generic one on an out-of-region game, couldn't you just check if the game has the same region as the NAND, and if so, use the existing prefix? Why would you need to know which prefixes are for which regions in that case?

The NAND might have have been changed by an old version of Dolphin in such a way that the prefix doesn't match the actual region of the NAND. If we don't replace the prefix in that case, we will be stuck with the wrong prefix.

Actions #10

Updated by Leseratte10 over 4 years ago

Isn't there any other way to determine the region of a NAND? Like just checking what system menu version is installed?

Actions #11

Updated by JosJuice over 4 years ago

Yes, checking the system menu version is what I was thinking of when I mentioned "the actual region of the NAND". The problem is that we can't assume that the prefix region matches the game's region just because the system menu region matches the game's region.

Actions #12

Updated by Leseratte10 over 4 years ago

Ah, that's what you mean, so there is no way to find the "proper" prefix, so people who would want an accurate prefix would need to re-import their NAND backup in an updated version of Dolphin after that change is implemented and merged. Or manually put the proper prefix back into the setting.txt.

Another idea I had would be to let the user enter the prefix in the Dolphin config file in that case, but that wouldn't be helpful either as the prefix on the sticker on the console doesn't always equal the actual sticker.

Actions #13

Updated by Leseratte10 over 4 years ago

Also, maybe the "locking" to the console serial number prefix could be a per-game setting? It could default to "on" so the games would get a serial number prefix that matches their region, but known games like Mario Kart Wii where it is known not to cause problems could default to always using the original console prefix.

Actions #14

Updated by JosJuice over 4 years ago

I have created a PR that makes Allow Mismatched Region Settings determine if the prefix should be overwritten: https://github.com/dolphin-emu/dolphin/pull/8519

Actions #15

Updated by Leseratte10 over 4 years ago

Great that that part of the bug is semi-fixed. Do you also have any idea what could cause the serial number to reset to a random(?) one every time I import the same NAND? The changing serial number prefix would only cause a new console to be generated once (per region), but the randomized serial number every time the NAND is re-imported for whatever reason also means that there are lots of junk console entries in the Wiimmfi database, and that Dolphin players have to wait a week every time they re-import a NAND, even if it's the same one as before.

Looks like the serial numbers are generated as if there was no previous one, Dolphin doesn't seem to notice that there is already one. However, when I - after importing the NAND - manually alter the serial number in the settings.txt back to the previous one, then Dolphin keeps it. Weird.

Actions #16

Updated by Leseratte10 about 4 years ago

I might have found the issue that causes this problem and opened a PR: https://github.com/dolphin-emu/dolphin/pull/8670

Actions #17

Updated by JosJuice about 4 years ago

That pull request has now been merged, so the A part of the issue is fixed. However, I'd like to keep this issue open, because I'd like to try my hand at doing the "not changing serial prefix if the region already is correct" change. Maybe I won't be able to get a complete list of region-prefix mappings, but even a partial list will help somewhat.

Leseratte10: You said before that you don't have a complete list, but do you have any list, even if incomplete? Even a list of which prefixes exist without any info about the associated region would be helpful.

Actions #18

Updated by Leseratte10 about 4 years ago

Here's a bunch of info I found, from looking at existing serial numbers and from examples on the internet:

Normal Wii (001):
Wii, America: LU, LB
Wii, Europe: LEH, LEM, LEF
Wii, Australia: LAH
Wii, Korea: LKM, LKH
Wii, Japan: LJF, LJH, LJM
Wii, Taiwan: LTF, LTH

Family Wii (101):
Wii, America: KU, KB
Wii, Australia: KAM
Wii, Middle East: KMH

Wii Mini (201):
Wii, America: HW
Wii, Europe: HEF

WiiU:

America: FW, GW
Europe: FEM, GEM, GEF, GEH
Australia: FAH

WiiU -> vWii side:
America: OW
Europe: OEM, OEF, OEH
Japan: OJH

So the first letter is a console type identifier (L = Wii, K = Family Wii, H = Wii Mini, F = WiiU, O = vWii), the second letter is the region (U/B = America, E = Europe, A = Australia, K = Korea, J = Japan, T = Taiwan). No idea what the third letter (if present) is for. So for just the region mapping, it's probably enough to check the second character of the serial number prefix.

Actions #19

Updated by JosJuice about 4 years ago

  • Status changed from New to Fix pending

Thank you!

With the following two PRs in addition to the two previously merged PRs, everything should work smoothly for users as long as the game region matches the NAND region or Allow Mismatched Region Settings is enabled:

https://github.com/dolphin-emu/dolphin/pull/8672
https://github.com/dolphin-emu/dolphin/pull/8673

Actions #20

Updated by JosJuice about 4 years ago

  • Status changed from Fix pending to Fixed
Actions

Also available in: Atom PDF