Emulator Issues #5405

Off-by-one error in encoded textures causes Another Code R issue (revival of issue 2137)

Added by wordmanwords over 8 years ago.

% Done:


Operating system:
Issue type:
Relates to usability:
Relates to performance:
Relates to maintainability:
Regression start:
Fixed in:


1) Game Name and ID?

2) What is the expected output? What do you see instead?
Currently, it's not possible to interact with objects in zoomed-in scenes (such as the house or the sign at the start). Normally, when the cursor is over an object (e.g. door) in one of those scenes, the object glows gold and can be clicked. In dolphin, the object doesn't glow and can't be clicked.
This issue doesn't seem affected by various Video/Emu options.

3) What steps will reproduce the problem?
Start game, proceed until you're near a sign or a house, click on it to have the camera zoom in, try to click on objects like the sign or the door in this zoomed-in view.

4) What causes this problem? How can it be hack-fixed?
Every frame in the zoomed-in view, the game writes an EFB texture to RAM. The texture uses format RGB565 and has plain colors for clickable areas and black for non-clickable areas.
In Dolphin, the texture contains colors such as (0xe, 0, 0xe) and the game doesn't recognize those colors.
However, if the colors were (0xf, 0, 0xf), the game would've recognized the colors and the issue would be gone.
One hack-fix for this problem is in TextureConversionShader.cpp, WriteToBitDepth: change " %s = floor(%s * %ff);\n" to " %s = round(%s * %ff);\n".
Of course, this is just a hack-fix. It doesn't show where the problem is - just what piece of code can be (most likely incorrectly) modified to fix it. Hopefully, a dev with in-depth experience with dolphin Video could help pinpoint the real issue.

Related issues

Has duplicate Emulator - Emulator Issues #2137: Trying to reinterpret pixel dataDuplicate


#1 Updated by wordmanwords over 8 years ago

Although changing the "floor" to "round" fixes many hotspots, some hotspots (For example - the man and the shelf behind him in the guesthouse) are still not clickable.
It looks like the hotspots that were fixed have an internal color representation with the top (5th/6th/5th) bits clear. e.g. (0xe, 0, 0xe)
Meanwhile, the ones that weren't fixed have an internal color representation with the top bit set.
Changing the "round" to "ceil" fixes all hotspots encountered so far but is most likely incorrect as well.

#2 Updated by hatarumoroboshi over 8 years ago

Are you saying that this issue won't ever be fixed in a "conventional" way? If that's the case, since for the Paper-Mario games was created in game-properties a dedicated "Enable bounding box calculation" hack fix, maybe a dedicated one for Another Code R could be created as well...

#3 Updated by skidau over 8 years ago

wordmanwords, nice find. Could the problem be caused by a change in format (at the PanicAlert message)? i.e. the RGB565 has been changed to another format which results in colours of (0xf, 0x0, 0xf)?

The FPU precision is different between a PC and a Wii. Knowing this, what is the possibility of "ceil" being correct?

Please keep on working on this game, because I am not aware of any devs that own this game.

#4 Updated by wordmanwords over 8 years ago

Alright, figured out that WriteToBitDepth was multiplying by the wrong value.
The attached patch fixes it to multiply by the correct value (and keeps 'floor', which is correct).
From my testing, this fixes all hotspots in Another Code R.

#5 Updated by skidau over 8 years ago

  • Status changed from New to Fixed

This issue was closed by revision 1587cb3738ee.

#6 Updated by DimitriPilot3 over 8 years ago

issue 2137 has been merged into this issue.

#7 Updated by wordmanwords over 8 years ago

DX11 uses its own version of the code that was patched (instead of the common one) and so wasn't fixed by the patch - whoops!
I'm attaching a patch that will fix the same code in DX11 as well. (It's more or less the same change, just changed to use the DX11 features (uint, >>) the rest of the DX11 shader code is using).

#8 Updated by NeoBrainX over 8 years ago

Hm, what's exactly the difference between e.g. pixel.r*31 and (uint)(pixel.r*255.0) >> 3 (line 31 in your patch)?

Also, could you give those DECODE_ functions a more meaningful name? (esp. since they don't even decode anything)

#9 Updated by NeoBrainX over 8 years ago

Or rather, wouldn't round(pixel.r*31.0f) have the same result?

#10 Updated by wordmanwords over 8 years ago

Attached a patch with less lazy names.

No, round(x*31) doesn't have the same result - that was checked. ceil(x*31) might have the same result, but you'd need to check it for all valid values of x (0/255 ... 255/255) to confirm.
Basically, the float is a number between 0/255 and 255/255. It makes little sense to multiply this by 31 - 31 and 255 don't have any common denominators. Instead, you should multiply it by 255 and go from there.

#11 Updated by iceman4love77 over 8 years ago

open gl works with patch for dx9
only dx11 doesn't

#12 Updated by iceman4love77 over 8 years ago

oops sorry didn't meant with your latest patch ,sorry haven't noticed it :P

#13 Updated by stefano.gaspar over 7 years ago

Where do I put the patch??

Also available in: Atom PDF