Project

General

Profile

Emulator Issues #4113

Rogue Leader depth issue / Dolphin doesn't support Z compression formats.

Added by inteGReddy13 over 9 years ago.

Status:
Duplicate
Priority:
Normal
Assignee:
Category:
GFX
% Done:

0%

Operating system:
N/A
Issue type:
Bug
Milestone:
Regression:
No
Relates to usability:
No
Relates to performance:
No
Easy:
No
Relates to maintainability:
No
Regression start:
Fixed in:

Description

What's the problem?

In Star Wars Rogue Squadron 2 Rogue Leader the textures are not rendered beyond 50 meters.

Dolphin version with the problem: 7171M 64-Bit

(optional) Dolphin version that does not have the problem:

Operating system and version: Windows 7 x64

Game ID (as it appears in game properties, Ex.: "GZ2P01" or "RSBE01"):
GSWE64

Was the ISO a plain dump from disc, compressed and/or scrubbed?
Dumped from disc and then compressed into .gcz by dolphin.

Please provide any additional information below.

I have successfully managed to get all of the menus and hanger textures to look correctly with rev 7171 (as you can see in the attached screenshots). The view distance is the only thing I can see wrong with the game now (also attached in screenshot). I can play in wireframe mode with decent speed and no crashes.


Related issues

Is duplicate of Emulator - Emulator Issues #3334: zfreeze not implemented in hw accelerated video backendsFixed

History

#1 Updated by danialhorton over 9 years ago

its not a distance issue, the game is displaying rendered images in the wrong order due to a custom engine technique not being emulated correctly.

#2 Updated by inteGReddy13 over 9 years ago

well...you say it isn't a distance issue. however, i cannot see distant objects (> 50 meters). it is rendering the backdrop (space) over objects > 50 meters away. whether or not this is caused by rendering order doesn't change the fact that the end result is an issue with view distance.

#3 Updated by sl1nk3.s about 9 years ago

  • Status changed from New to Accepted

Just reporting this seems to be an issue with how the game uses "Z compressed" 16bit format, Dolphin doesn't really support the multisampled pixel format RGB565_Z16 and doesn't support its "Z compression" feature either.

The issue with SW:RL is that it uses this Z compressed format a lot with the highest Z compression, probably to draw the skydome and the far away stuff while still maintaining correct Z precision, and this seems to be why the position of the dome is completely off.
I made a hack to linearly convert/scale the depth value when that pixel/Z format is set and I get something better (see screenshot), but the depth calculation is indeed still very wrong since some stuff disappear at times and things such as the blue planet and the death star don't appear during the intro sequence of the first mission.

I don't think I have the willpower to make it right :p, but if anybody wants to implement it, ~relevant stuff here :
http://libogc.devkitpro.org/gx_8h.html#a018d9b0359f9689ac41f44f0b2374ffb
http://code.google.com/p/dolphin-emu/source/browse/trunk/Source/Core/VideoCommon/Src/BPFunctions.cpp?r=7253#140 bpmem.zcontrol.zformat is updated when this is called.

please note that the zformat change is only supported when the pixel format is RGB565_Z16

#4 Updated by ccg12 about 9 years ago

wow. interesting idea sl1nk3.s, gj! =D

#5 Updated by inteGReddy13 about 9 years ago

Great job sl1nk3.s! Your screenshots look better than mine >=|

#6 Updated by chic_bb63 about 9 years ago

sl1nk3.s!
your patch may not yet be perfect but it makes rogue leader playable outside wireframe mode. It is worth posting a downloadable patch with your modified code. Not everyone is able to understand c++ codes, but a quick download would help many users eager to play this game. Please upload your patch ASAP!

#7 Updated by italosbandeira about 9 years ago

How i can download this patch?

#8 Updated by chic_bb63 about 9 years ago

Rogue Leader depth:

The draw distance issue is apparently a layer masking issue. i used the free look option in opengl, and when I moved the camera past the starfield, the entire death star surface was rendered completely. This means that the starfield which should be in the background is being rendered in front of most of the other rendered objects. If one were to give the background layer some transparency or if it were to be moved behind as the last layer, the draw distance issue would be fixed.

#9 Updated by NeoBrainX about 9 years ago

Alright, think I've figured it out. Not sure if sl1nk3.s knew the same, since he was kinda vague on what he was doing exactly :P

What the game is doing is using both a compressed 16-bit z-buffer as well as an uncompressed 24-bit z-buffer. During emulation, the game is switching the z-buffer format, so the pixel data gets reinterpreted. This is where we're having problems: We neither emulate 16->24 bit format changes (nor 24->16) NOR do we handle the (de)compression at all. Thus the EFB contents themselves are wrong, as well as all subsequent texture contents from EFB copies.

For what it's worth, as far as I understand the z compression stuff, you have three different options which change how the floating-point z values are encoded inside the EFB:
GX_ZC_NEAR, 14 bits mantissa and 2 bits exponent
GX_ZC_MID, 15 bits mantissa and 3 bits exponent
GX_ZC_FAR, 16 bits mantissa and 4 bits exponent

Anyway, I'll be looking into this in the near future. If my theory is correct, it should be fairly easy to fix (although will be a mess to implement). However, it probably won't make it into 3.0 since trunk is feature frozen right now.

#10 Updated by NeoBrainX almost 9 years ago

... Apparently ingame the game will only switch between an RGB8_Z24 and a Z24 buffer, as well as between an RGB8_Z24 and a RGBA6_Z24 buffer.
Thus, it most likely doesn't have anything to do with format change emulation at all. At least unless I'm missing something obvious now.

No fix from me I guess then, not sure if sl1nk3's chances were in any way correct since he didn't post any source.

#11 Updated by voodoomb almost 9 years ago

Neo: why not just ask him? :) :P

#12 Updated by michigunskiy almost 9 years ago

sl1nk3.s
Please upload source of your SW-RL depth hack

#13 Updated by Anonymous over 8 years ago

I was looking last night, and you're right, the game doesn't appear to do the switching during gameplay. But it does do it pretty much every other possible time :p
I notice everything related to AA is not implemented in software rasterizer, so I'm going to work on that. There could still be some off chance that the switches outside of the game effect rendering during gameplay...
Also, it's probably worthwhile to do some tests checking what really happens to z values when doing reinterpretations. From what I can tell, the rendering hardware is all 24bit fixed point stuff, is that correct?
The main pain in the ass comes from all non-AA pixel formats being nicely packed like:
{color : 24; Z : 24;}
whereas the AA format is:
{color : 16; Z : 16;}
because of this, the hardware requires that they be accessed in groups of 3 (LCM(24, 16 + 16) = 96)
Also, the Z compression format is fixed point (the actual encodings are:
#define ZC_LINEAR 0 // strips lower 8bits
#define ZC_NEAR 1 // 14e2
#define ZC_MID 2 // 13e3
#define ZC_FAR 3 // 12e4
)

After these considerations, there is also the supersampling and filtering which happens in AA...it should be interesting to try and figure out how that works...(I'm pretty sure that the grouping of three pixels described above just becomes the three samples rendered per output pixel?)

This is very doable in software, I have NO idea how to do it in shaders though! :D

..but yeah, at this point I have no idea if it will actually help RS2.

#14 Updated by AequinoxK about 8 years ago

Just wondering if there are any fixes in the pipe for this glitch. I'd very much appreciate it. .^ :P

#15 Updated by director_sloan about 8 years ago

haven't tried the game for quite some time on dolphin, has anyone had another bash at fixing the draw distance?

#16 Updated by parlane about 8 years ago

Have a look neo ?

#17 Updated by NeoBrainX about 8 years ago

I should've probably posted this earlier here, but I got a response from sl1nk3.s half a year ago:

"And as for issue 4113, I did manage to get the game showing a bit more than it currently does (as shown in the screenshot) however as I said, it was just a quick hack, and didn't work right (probably due to my shader skills being poor at best).
Also, shame on me but I don't have the code that did that anymore , however i'll try my best to explain how I did it :
Basically I used a constant in the vertex shader to keep track of the different Z modes, this constant was set each time OnPixelFormatChange() was called for a compressed Z format.
Now in the shader I simply used that constant to know when i needed to set a different Z value (when the pixel format was _Z16 and the constant was != 0).
The Z value was actually just "Z x constant" to create an offset when the compressed format was set (but that's plain wrong because the Z compression is not linear, I also used Z² at some point but that's even more wrong, wrong, wrong).

Also, I've been trying to get it working "correctly" (which is why I don't have the semi working code anymore) and I've been failing so hard that I'm almost doubting my method of doing it, do you think the vertex shader is a good place for this "Z conversion"?

Also, I have a test homebrew that shows the bug, it simply draws two colored triangles at different depth using the multisampled framebuffer, one with no Z compression, the other one with Z compression.
As you can guess, on Dolphin the left triangle is in front of the other while it's the opposite on a Wii/GC.
That could be a starting point to reverse engineer the Z compression (using a predetermined Z value and trying to match it with a compressed Z value)"

Fwiw, the above text should be taken with a grain of salt. My initial theory of what might going wrong was the same as sl1nk3.s's, but the algorithm was radically different: You'd basically render stuff like usual, but a) On a z format change, you'll need to reinterpret compressed depth data to uncompressed depth data (and vice versa) b) If the game is doing EFB copies from a compressed depth buffer, the EFB copy emulation code needs to be made aware of that (basically boils down to scaling all values with a certain factor).

Anyway, like explained in comments 9 and 10, I'm not sure if we actually identified the correct problem.

tl;dr: no real news here

#18 Updated by Billiard26 over 7 years ago

  • Issue type set to Bug
  • Category set to gfx

#19 Updated by NeoBrainX over 7 years ago

  • Status changed from Accepted to Work started

:>

#20 Updated by durandle over 7 years ago

Having read through this thread, I'd like to check before I even look at code, if it wouldn't make more sense to just patch specific for this game that essentially forces the z-index to a value that'll not mess up the game (but isn't necessarily "accurate"). I'm a programmer, but nothing remotely the kind of coding like this, so don't really want to go messing with stuff if the idea is useless!

#21 Updated by NeoBrainX over 7 years ago

If you want a useful idea, implement zfreeze in the hw renderers, that'll pretty much fix the issue ;)
We already have support for that in the software backend, so if you really wanted to mess with it you could take a look at how its done there. It'll be quite a bit involving for the hw renderers though, since we'll need to software transform all vertices (we already have code which does that) and compute the zslope from that.

#22 Updated by skidau over 7 years ago

durandle, we're open for ideas at this stage. Give it a try and if it shows some good results, great. If it doesn't, try again.

#23 Updated by NeoBrainX about 7 years ago

  • Status changed from Work started to Duplicate

The main issue is fixed in VideoSoftware because zfreeze is implemented there. It's still not quite perfect there, but at least not everything gets clipped away there.

#24 Updated by DefenderStrident about 7 years ago

Having read this through almost 6 months ago now, and again just today, I'm glad to see there's some progress on it. What do you mean it's fixed in VideoSoftware though? When I tried running the game in Software Renderer (which I assume is the video software) no graphics are displayed whatsoever. :/

#25 Updated by phire about 7 years ago

Yeah, I get that too, and sometimes a few unimplemented graphics opcode errors.

This should probably listed as a separate bug; It's probably related to issue 6039.

#26 Updated by NeoBrainX about 7 years ago

A fifo log recorded in the reported scene with hw renderers plays back fine in the software renderer, so the reported issue is fixed. Anything else should be reported as a separate report.

#27 Updated by Anonymous almost 7 years ago

hey guys, is anyone working on this issue? I saw a comment on youtube that someone made a patched build of dolphin that fixed the issue but was only designed for this game, is this true?

#28 Updated by whyischris almost 7 years ago

Devs will get back to it at some point. This particular issue has been more accurately noted as a zfreeze issue and so it was merged into issue 3334 (link on the left). I'm sure one day it will get sorted, we just have to either get busy living or get busy coding.

#29 Updated by mmvf over 5 years ago

Someone please fix it, these RS games are pure gold.

#30 Updated by gabrielhcsanchez over 5 years ago

hay solución?

#31 Updated by CoenVDoord over 5 years ago

Hey guys, i just tried a 'newer' version of dolphin, not sure wich one to be precise... tho the ingame remains laggy.. i came futher with the loading screens than before. Same goes for the sound. But it's all still very slow. This is about RS2.... RS3, finaly stopped freezing after the ingame screen, but remains to lagg a lot. Menu's are really distorded but i could manage to create a profile. Even the hangar is pretty playable. But once you enter the craft. You're back to basics.. extreme lagg and sound distortion.. I really do hope some people on working on this. I bet a lot of people are extremely eager to play this, including myself haha. Hoop to hear something , what will make the games work more, soon!

Also available in: Atom PDF