Emulator Issues #11910
closed
Just Dance series crashes/freezes/hangs when the emulated console has internet connectivity
0%
Description
Game Name?
Just Dance 2
Just Dance 3
Just Dance 4
Just Dance 2014
Just Dance 2015
Just Dance 2016
Just Dance 2017
Just Dance 2018
Game ID?
SD2E41
SJDE41
SJXE41
SJOE41
SE3E41
SJNE41
SZ7E41
SE8E41
MD5 Hash?
545c5a833dd274e0cc62d72d91559dca
e46877c360dcbb6cc8c5814762b424a3
cfa93ad1209bd9dd61cf9df3d9b6adc4
64d469178d925d5b79ae26f7289d165b
6142795d6f0700963b9754ca59c92d90
8041c2a5ad9465cb5fac6657514e3ea2
31e872f305d4355416a4ce345fdd9c4c
fe3219a6caac46e5f4f233f197e860f8
What's the problem? Describe what went wrong.
Just Dance 2/3/4: these games will randomly freeze when the emulated console has internet connectivity. When they freeze, only the background music keeps playing and only a forced shutdown (clicking Stop button twice) can stop the emulation.
Just Dance 2014/2015/2016/2017/2018: these games will hang on an infinite loading screen when trying to create or edit a dancer card. Since on the first boot you must create at least one dancer card, this renders those game unplayable.
What steps will reproduce the problem?
Make sure the Wii network certificates (clientca.pem, clientcakey.pem, rootca.pem) exists on the root of your NAND, then:
Just Dance 2/3/4: just play the game, they'll eventually freeze, sometimes even before you can start a song. On Just Dance 2, trying to access the shop once seems to be required before the random freeze occurs.
Just Dance 2014/2015/2016/2017/2018: create a Dancer Card (you must do this on the first time you start the game), or in case you already have an existing save file, click on the Dancer Card option at the top-left portion of the main menu and create a new / edit an existing Dancer Card. When you're done, you'll be stuck on an infinite loading screen (the game won't crash nor freeze, though).
Is the issue present in the latest development version?
Yes. Tested all on 5.0-11288.
Is the issue present in the latest stable version?
Yes. Tested all on 5.0 stable, except for Just Dance 2014/2015/2016/2017, which were tested on 5.0-775 (the first Dolphin version that can boot these games).
If the issue isn't present in the latest stable version, which is the first broken version?
N/A
What are your PC specifications?
CPU: Intel Pentium G3258 @ 4.2GHz (OC)
GPU: NVIDIA GeForce GT 440 @ 1GB DDR3
RAM: 16GB @ DDR3-1600 (4x4GB)
OS: Windows 10 (version 1909)
Is there anything else that can help developers narrow down the issue?
Issues that I strongly believe are connected to the same "root" problem and that probably could be merged on this one: #11441 #11342 #10873 #8597
Any of the following workarounds works both in 5.0 stable and on current development builds and prevents triggering this issue:
- Physically disconnecting your computer from the internet before playing the game;
- Temporarily disabling your network adapter(s) or enabling airplane mode if on a laptop;
- Deleting
clientca.pem,clientcakey.pemandrootca.pemfrom your NAND's root (however current development builds will keep throwing a Panic Alert saying these files are missing every time the game tries to connect to the internet).
Most of these issues (especially on Just Dance 2/3/4) were seemingly fixed for good once Dolphin implemented Wii Network functionality back in the now ancient 3.5-2143 build. I can't really confirm this since I only got aware of the problem with the earlier games recently but it seems that Wii Shop closure is what brought back all these problems to life again. In fact, looking at the Dolphin log, there's always a spam of the following lines moments before the earlier games freeze and the later games hang on the infinite loading screen:
core\ios\network\socket.cpp:247 I[IOS_NET]: IOCTL_SO_CONNECT (00000000, 216.98.48.66:443) = -1
core\ios\network\socket.cpp:119 E[IOS_NET]: SO_CONNECT failed with error 10037, ret= -1
The actual IP varies between the games, but it's always pointing to servers that just time out without any response if, for example, you try to ping them. It seems as our network code doesn't correctly respond to the game that the connection timed out and thus it keeps retrying until it freezes (earlier games) or until it hangs on the infinite loading screen (later games), as none of these problems occurs on a real Wii / vWii.
Since the workarounds aren't too convenient (especially for less tech-savvy users), it would be good having a per-game option in Dolphin to emulate a Wii console without internet access. I tried booting to Wii Menu and clearing all saved network profiles and then launching the game from the Disc Channel but nothing changed as the games still had internet access and ended triggering the issue (Dolphin seems to override whatever setting is configured on the network profiles from the Wii Menu).
For records, Just Dance 1, Just Dance 2019 and Just Dance 2020 doesn't have any kind of internet connectivity an thus doesn't experience any freeze or hang. I don't own any of the spin-off or compilation Just Dance games so I can't tell if they're also affected, but given the pattern, it seems reasonable to assume they are affected too if at some point they had online related features...
Files
Updated by mbc07 over 4 years ago
- File SSL Dump Just Dance.zip SSL Dump Just Dance.zip added
Don't know why I hadn't though of this before, but here are SSL Read/Write dumps from Dolphin on all affected games, from the moment you boot Dolphin to the moment they hang/freeze.
In short, Just Dance 2/3/4 do two requests to a Nintendo server (naswii.nintendowifi.net), which responds both with an HTTP 200 message, then they randomly freeze shortly after. Just Dance 2014/2015/2016/2017/2018 proceeds a little further, after receiving the HTTP 200 messages from the same Nintendo server, they do three requests to a Ubisoft server (wii-dance6-ws2.ubisoft.com), which always responds with HTTP 503 messages, then they hang forever on the Dancer Card loading screen.
The pattern between all games is that whatever in the game code fires the network request seems to crash or hang on the middle of a new request after the initial responses. That request get partially written to Dolphin's ssl_write.bin dump but never seems to actually go through the network, as the request itself is incomplete (e.g. it lacks other required HTTP Headers and the actual URL seems abruptly truncated). The consequences of the game's network code failing is what varies between the games: a complete game freeze on JD 2/3/4 or an infinite loading screen on JD 2014/2015/2016/2017/2018.
We probably have a bug somewhere in the Wii Network emulation, as not having any internet access or not having the Wii's SSL root certificates on the NAND makes the game network code not actually freeze/crash. Those hang/freezes also doesn't happen on a real Wii console.
Updated by sepalani over 4 years ago
My gut instinct is telling me that is related to these issues :
It seems many Ubisoft and Activision games are using the Quazal network SDK.
According to a memory dump I just did on my Just Dance game, it seems to use this SDK as well.
I haven't figured out what the root cause is, but I'm working on it.
It might be related to either UDP socket and/or multi-threaded socket issues.
Updated by sepalani over 4 years ago
Does this PR fix your issues? https://dolp.in/pr8389
Updated by mbc07 over 4 years ago
Unfortunately, it doesn't. The only difference I could notice with the PR is that a Panic Alert now pops up when stopping (JD2014 to 2018) or when force stopping (JD2 to JD4) emulation: A "IPCEvent" was scheduled from the wrong thread (CPU)
Updated by leoetlino over 4 years ago
- Related to Emulator Issues #11867: DJ Hero 2 hangs if you cancel the online sign-in. If you don't cancel, no hang and online works. added
Updated by leoetlino over 4 years ago
- Status changed from New to Fixed
- Fixed in set to 5.0-11873
Updated by leoetlino over 4 years ago
- Status changed from Fixed to Accepted
- Fixed in deleted (
5.0-11873)
Updated by sepalani over 4 years ago
Hopefully, this one should fix your issue: https://dolp.in/pr8756
Updated by mbc07 over 4 years ago
Tested the PR 8756 and things seems to have improved. On JD2015 to 2018, you'll still get stuck on the loading screen after creating the Dancer Card but if you wait around 2 to 3 minutes, the game eventually proceeds. If you try to access other online features (e.g. World Dance Floor), it'll also hang loading indefinitely for about 2 to 3 minutes before the game erroring out ("The Ubisoft Servers are unavailable at the moment"). It has been some time since I tested this on a real Wii, but I don't recall the timeouts being that long, but regardless, the PR still is an improvement as on master it would just hang loading forever and never proceed.
On JD2/3/4, it's somewhat similar. The game will still randomly freeze, but if you wait about 2 to 3 minutes, it'll unfreeze and keep going, until some time later, where it'll freeze again for about 2 to 3 minutes again before unfreezing. It seems those games keep retrying to connect to the Ubisoft servers in the background (even if you said No to the usage tracking question that appears on the first boot of the game), every time it tries to connect, it'll have a 2 to 3 minutes freeze. The PR still is an improvement as on master it'll just freeze and never unfreeze anymore.
JD2014, however, seems to be an unique case. Although it runs on the new engine (that Ubisoft started using on JD2014 and reused on all sequels), its network code doesn't seem to be, um, "async", like on the sequels: on master, boot the game and it'll freeze either on the Ubisoft logo or in the start screen, on PR 8756 it'll work similarly to JD2/3/4: 2 to 3 minutes freeze at the Ubisoft logo, then unfreezes, another 2 to 3 minutes freeze on the start screen, then unfreeze, from there, the game works normally (unlike the sequels, JD2014 doesn't force you to create a Dancer Card right at the beginning). Try to create a Dancer Card or to access other online features and you'll have another 2 to 3 minutes freeze before the game unfreezes and proceed, different from the sequels, that would just hang on a loading screen (without freezing) until 2 to 3 minutes later.
TL;DR you seem to be on the right track, with PR 8756 the games still hang/freeze as before but they "recover" and keep going after a 2 to 3 minutes delay. The timeout seems to be in real time, as holding Tab to speed up the emulation doesn't make the timeout go any faster. One last thing, the Panic Alert when stopping any of the games is still there (A "IPCEvent" was scheduled from the wrong thread (CPU))...
Updated by sepalani over 4 years ago
Have you tried with the latest update I did on this PR (the third commit one)?
Updated by JMC4789 over 4 years ago
Lol, the one I tested is JD2014, which is the unique one.
Updated by mbc07 over 4 years ago
@JMC4789 sorry about JD2014, I grouped it together with the JD2015/2016/2017/2018 when I opened the issue report 5 months ago because it also uses the new engine from the newer games but somehow missed that quirk on the report
@sepalani I just redownloaded the PR 8756 test build and the checksum from the executable differs from the test build I used earlier, so I'll retest with the new build and report back soon
Updated by mbc07 over 4 years ago
@sepalani with the latest commit of PR 8756, the issue is fixed on all games.
JD2/3/4/2014 doesn't randomly freeze anymore at any point (played about 15 minutes with each of them) and on JD2015/2016/2017/2018 the loading screen after creating a Dancer Card takes only 10 to 15 seconds, which is way closer to what I remember from a real Wii.
Also, trying to access World Dance Floor doesn't freeze JD2014 anymore, nor leave JD2015/2016/2017/2018 stuck on a loading screen for 3 or so minutes, they correctly display the "The Ubisoft Servers are unavailable at the moment" message after about 30 seconds, also like I remember from a real Wii.
However, the "A "IPCEvent" was scheduled from the wrong thread (CPU)" Panic Alert still happens when stopping any of the games (seems to happen on master too since your previous PR 8389 got merged in 5.0-11873)...
Updated by JMC4789 over 4 years ago
the IPC event from another thread could just be a non-threadsafe thing happening. We have a few that only happen on netplay too...
Updated by sepalani over 4 years ago
@mbc07 Do you have this IPCEvent issue with Dual core mode disabled?
Updated by mbc07 over 4 years ago
Just gave it a shot, they don't happen in Single Core mode. Perhaps it was always there and I just noticed now because I used clean profiles to test the PR builds, so Dual Core was enabled by default, as on my "main" Dolphin profile I use Single Core.
Thought it could have been a regression from the recent network fixes, but it seems to be just another weird "Dual Core only" thing...
Updated by JMC4789 over 4 years ago
- Status changed from Accepted to Fix pending
Updated by JosJuice about 4 years ago
- Status changed from Fix pending to Fixed
- Fixed in set to 5.0-12952
Updated by
Updated by JosJuice about 4 years ago
- Has duplicate Emulator Issues #11441: Just Dance 2014 freezes at Ubisoft logo if .pem files are present added
Updated by JosJuice about 4 years ago
- Has duplicate Emulator Issues #11342: Just Dance 2014/2015/2016/2017/2018 hangs during save file creation added
Updated by JosJuice about 4 years ago
- Has duplicate Emulator Issues #10873: Just Dance 3 - after 20s California Girls hangs with music. Emulation needs force close (did'nt test other song) added