Emulator Issues #12047
open
Arbitrary mipmap detection fails for certain textures in Toad's Factory
0%
Description
Game Name?
Mario Kart Wii
Game ID? (right click the game in the game list, Properties, Info tab)
RMCP01
What's the problem? Describe what went wrong.
There is a certain light effect on the track "Toads Factory" (the arrows on the conveyor belts), that normally is light, and gets darker the closer you get. On original resolution (640x528) that is working properly, like on console. But the higher the resolution that I use, the sooner the effect gets darker.
What steps will reproduce the problem?
Set the resolution to 1x. Start Mario Kart Wii, start a race on Toads Factory (last track in the Mushroom Cup).
Drive about half a lap until you get to the conveyor belts that run from side to side. Drive onto the conveyor belt, directly onto one of the red or green arrows. Notice that while you are this close, the arrow becomes almost invisible (this is what is supposed to happen, and this also is what happens on console).
Now set the resolution to a reasonably high one (I used 6x (3840x3168) for the test, but 3x or 4x also work, the bug is just not as obvious then) and drive onto one of the conveyor belts again.
Notice that now, all the arrows on the belt are almost invisible, and even when you leave the belt, it takes a reasonable distance until the arrows become visible again.
Is the issue present in the latest development version? For future reference, please also write down the version number of the latest development version.
Yes, it is, 5.0-11836
Is the issue present in the latest stable version?
Not sure, I have a Linux machine and trying to compile 5.0 just gives me errors. I'd assume that it would be present there as well, but I didn't test it.
If your issue is a graphical issue, please attach screenshots and record a three frame fifolog of the issue if possible. Screenshots showing what it is supposed to look like from either console or older builds of Dolphin will help too. For more information on how to use the fifoplayer, please check here: https://wiki.dolphin-emu.org/index.php?title=FifoPlayer
Screenshots are attached.
fifo log is too large for the upload function (22 MB), can be downloaded here: https://www.dropbox.com/s/x7xfjdirevpae6r/fifo_log?dl=0
"working.png" is from Dolphin with 1x resolution (how it's supposed to look, bright arrows)
"broken.png" is from Dolphin with 5x resolution (broken, arrows almost invisible)
What are your PC specifications? (CPU, GPU, Operating System, more)
i9-9900K, RTX 2080 Ti, Ubuntu 19.10
Files
Updated by JMC4789 over 4 years ago
I believe this is a case of a game using custom mipmaps for a texture, and higher resolution makes it so mipmaps are used at different resolutions. If you dump the texture for this we could be sure.
Updated by JMC4789 over 4 years ago
- Status changed from New to Accepted
I tested this and I confirmed it IS NOT arbitrary mipmaps (at least our heuristic doesn't catch it?) and the distance seems to grow based on your resolution.
I do not know what causes this or if we can fix it.
Updated by colorovfire almost 2 years ago
- File image-2.png image-2.png added
JMC4789 wrote in #note-2:
I tested this and I confirmed it IS NOT arbitrary mipmaps (at least our heuristic doesn't catch it?) and the distance seems to grow based on your resolution.
I do not know what causes this or if we can fix it.
Arbitrary mipmap threshold isn’t low enough to catch it. I copied over the value I used for F-Zero and it seemed to fix it.
[Video_Enhancements]
ArbitraryMipmapDetectionThreshold = 8.3
It’s most likely not an optimal value.
Updated by colorovfire almost 2 years ago
- File Screenshot 2023-01-19 at 7.09.26 AM.png Screenshot 2023-01-19 at 7.09.26 AM.png added
- File Screenshot 2023-01-19 at 7.09.03 AM.png Screenshot 2023-01-19 at 7.09.03 AM.png added
There’s also the lights on the side within the tunne (same map)l. The threshold has to be set to an extremely low value of 2.22. It should never be set this low but if anyone creates a texture pack, they can temporarily set it so it’s dumped as an arbitrary mipmap texture. Anyone using the textures wouldn’t have to change ArbitraryMipmapDetectionThreshold at all and have it work correctly.
Updated by pokechu22 almost 2 years ago
- Subject changed from Certain light effects are wrong on higher resolutions to Arbitrary mipmap detection fails for certain textures in Toad's Factory
The affected textures are tex1_256x256_m_db626658aab6e356_14.png for the conveyer belt (darkest at its highest resolution, then half-bright at 128 by 128, then full bright at 64 by 64, then half bright at 32 by 32, then like a quarter bright at 16 by 16) and tex1_256x512_m_fc58ac1b2dbeb111_14.png for the walls (brightest at its highest resolution, then half-bright at 128 by 256, then equally dark at 64 by 128, 32 by 64, and 16 by 32).
These are also the whole belt or wall texture, with only a small part changing; it's not a transparent texture that's drawn over another one. Since most of the texture doesn't change, I can see why this would cause problems for the heuristic.
Maybe the heuristic could be modified so that if most of the image matches perfectly but a small portion doesn't, it treats it as non-matching? Though that seems like it would invite a lot more incorrect cases.
Updated by pokechu22 almost 2 years ago
- Related to Emulator Issues #128: Issue with AMD processors and CPU Detection for SSE2 added
Updated by pokechu22 almost 2 years ago
- Related to deleted (Emulator Issues #128: Issue with AMD processors and CPU Detection for SSE2)
Updated by pokechu22 almost 2 years ago
- Related to Emulator Issues #12942: Arbitrary Mipmap Detection fails for water in F-Zero GX added
Updated by colorovfire almost 2 years ago
pokechu22 wrote in #note-5:
Maybe the heuristic could be modified so that if most of the image matches perfectly but a small portion doesn't, it treats it as non-matching? Though that seems like it would invite a lot more incorrect cases.
Current heuristics is a complete mystery to me. The default threshold is 14 but I don’t even know how it scales. The differences on tex1_256x256_m_db626658aab6e356_14 looks like an easy one to catch but it misses it.
Attached a graphic with minimum values for it to flag as arbitrary.
Updated by pokechu22 almost 2 years ago
The heuristic is this code, which calculates the RMSE for all texels across all levels (the comment says MSE, but it takes a square root). Basically, for each mipmap level, it first computes what the expected downsampled value is (by averaging the 4 corresponding ones), then compares that with the actual downsampeled value, as diff = (old_r - new_r)^2 + (old_g - new_g)^2 + (old_b - new_b)^2 + (old_a - new_a)^2. After, it takes a square root, then divides by the number of pixels times 4 (to account for the 4 channels), then divides by 2.56 (i.e. divides by 256 and multiplies by 100) so that the final value is (theoretically) in the range 0-100. Then, the overall value for the texture is the average for all mipmap levels.
This has a few problems. For one, if the texture doesn't have an alpha channel, the alpha value is 1 for everything, meaning the difference for that is zero. So the highest value possible would be 75% if a texture alternated from fully black to fully white across all mipmaps. Furthermore, averaging across all mipmap levels means that if only one has a significant difference, the overall result is low. And, if only a small part is different, then the overall difference is low too (the light makes up maybe a 24 by 96 area in the 256 by 512 wall texture, so that's 2304/131072 or about 1.75% of the image, and that's assuming a transition from full black to full white).
I added some logging to get additional information. Here's the output:
Melty molten galaxy lava:
00:39:606 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_efb7885b915ac15a_14 level 0: 9.646446
00:39:606 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_efb7885b915ac15a_14 level 1: 15.235299
00:39:606 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_efb7885b915ac15a_14 level 2: 17.893211
00:39:606 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_efb7885b915ac15a_14 level 3: 23.713253
00:39:606 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_efb7885b915ac15a_14 level 4: 22.17285
Conveyor belts:
01:09:503 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_db626658aab6e356_14 level 0: 11.146211
01:09:503 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_db626658aab6e356_14 level 1: 23.25801
01:09:503 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_db626658aab6e356_14 level 2: 11.902312
01:09:503 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_db626658aab6e356_14 level 3: 7.3496165
Walls with lights:
02:27:735 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x512_m_fc58ac1b2dbeb111_14 level 0: 1.6436607
02:27:736 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x512_m_fc58ac1b2dbeb111_14 level 1: 2.4542139
02:27:736 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x512_m_fc58ac1b2dbeb111_14 level 2: 2.7571242
02:27:736 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x512_m_fc58ac1b2dbeb111_14 level 3: 2.0296555
I also tried out a MEA (i.e. sum abs(expected - actual) instead of (expected - actual)^2), which gave these results (not an improvement):
Melty molten galaxy lava:
04:48:792 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_efb7885b915ac15a_14 level 0: 4.979426
04:48:792 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_efb7885b915ac15a_14 level 1: 8.881092
04:48:793 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_efb7885b915ac15a_14 level 2: 10.592461
04:48:793 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_efb7885b915ac15a_14 level 3: 14.145279
04:48:793 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_efb7885b915ac15a_14 level 4: 13.703918
Conveyor belts:
05:18:107 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_db626658aab6e356_14 level 0: 5.4535093
05:18:107 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_db626658aab6e356_14 level 1: 11.131406
05:18:107 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_db626658aab6e356_14 level 2: 6.4382553
05:18:107 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x256_m_db626658aab6e356_14 level 3: 4.327774
Walls with lights:
05:51:033 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x512_m_fc58ac1b2dbeb111_14 level 0: 0.8841753
05:51:033 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x512_m_fc58ac1b2dbeb111_14 level 1: 1.4223695
05:51:034 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x512_m_fc58ac1b2dbeb111_14 level 2: 1.6636372
05:51:034 VideoCommon\TextureCacheBase.cpp:1134 I[Video]: tex1_256x512_m_fc58ac1b2dbeb111_14 level 3: 1.3334274
I can't think of a better heuristic, unfortunately.
Updated by iwubcode almost 2 years ago
I've been told of other scenarios where arbitrary mipmap detection produces false positives. My plan is to introduce a graphics mod capability that allows you to explicitly enable (or disable) the feature for specific textures.
Updated by colorovfire almost 2 years ago
pokechu22 wrote in #note-10:
I can't think of a better heuristic, unfortunately.
Would it make sense to detect the threshold per mipmap level instead the average across all levels? There’s too much variations where any significant changes can get averaged out.
BTW, that bit of code for arbitrary detection runs even if it’s running at 1x with anisotropic filtering disabled. Nothing need to be detected since the distinction doesn’t matter at that point. It would save some CPU time.