Project

General

Profile

Emulator Issues #12725

Pixel Perfect / Integer Scaling mode

Added by tccalvin 25 days ago. Updated 21 days ago.

Status:
New
Priority:
Normal
Assignee:
-
% Done:

0%

Operating system:
N/A
Issue type:
Bug
Milestone:
Regression:
No
Relates to usability:
No
Relates to performance:
No
Easy:
No
Relates to maintainability:
No
Regression start:
Fixed in:

Description

Hey there, Dolphin people!

Lately I've been playing around with Dolphin on an old 4:3 CRT computer monitor and I'm having a lot of fun. Playing games in 480p with progressive scan is just fantastic.

With that said, in my quest to get the most accurate, console-like experience out of the emulator and the monitor I came across an issue that probably doesn't bother a lot of people, but I felt it was worth mentioning anyhow.

I don't really know what the proper place to post this finding is, but I'll put it here for now. Also, I'm by no means an expert on any of the things I'm about to show, so I'd appreciate it if someone more knowledgeable could correct me.

What I found is this: there is no way to consistently display games in 480p without some sort of scaling taking place (I'm guessing it's bilinear filtering but I'm not sure).

Here are the tests I did (you might have to zoom on the pictures to see the scanlines):

Booted up Sonic Riders in a 640x480 fullscreen window with the aspect ratio set to "Auto" and I got these scaling artifacts:

https://imgur.com/a/M8TeXme

(Notice the uneven pixel crawl, the blue halo around Sonic and the dark halo around the light beam)

By taking a screenshot, I figured out that the native resolution of the game is 584x480, so I tried the "Force 4:3" and "Stretch To Window" modes and the poor scaling went away on the stretched image:

https://imgur.com/a/rcYScxk

(Notice the even pixel crawl, and how the edges have perfect contrast with no halo)

I don't really know why this worked. My only guess is that it might have something to do with a CRT not having a fixed pixel structure.

Sadly, this trick only works with games that have to be stretched horizontally.

I took a screenshot of Mario Sunshine and found out its resolution is 640x476, which means that a couple of lines of pixels are cut from the top and bottom part of the screen. This doesn't sound like a big deal at first. However, with how Dolphin's video modes work, this ends up messing up the entire image.

Whereas Sonic Riders was scaled horizontally in "Auto" mode, Mario Sunshine is always scaled vertically no matter what aspect ratio you choose:

"Auto":

https://imgur.com/a/2A4MhyA

"Force 4:3" and "Stretch To Window":

https://imgur.com/a/NMSf1f3

(Notice the uneven pixel crawl and the red halo above Mario, or the dark one above his shoes and his shadow)

I'm guessing all of this could probably be fixed by implementing a Pixel Perfect / Integer Scaling mode. This way, all the pixels that aren't used would simply be black, and all the active pixels would line up with the scanlines or the pixel grid of the display. This wouldn't just benefit CRT users, but also owners of 480p LCDs and people who use HD TVs / monitors who don't mind having black bars around the image.

I know this feature is probably not as easy to implement as it sounds, but it would be a welcome one nontheless. But maybe there are other ways...?

I'm looking forward to your feedback! :)

P.S. "Crop" doesn't fix this, if anything it introduces more scaling on both axes...

History

#1 Updated by phire 25 days ago

in my quest to get the most accurate, console-like experience out of the emulator and the monitor

Ok... There are some issues with your quest here.

First, a real console simply doesn't output perfect pixel like you are expecting from dolphin. In-fact dolphin outputs significantly "cleaner" pixels, as it skips several steps of blur. The clean image quality you are striving for simply never existed. By default, games always do both vertical and horizontal bluring, and most games do horizontal scaling. All PAL games apply a vertical scaling.

Even in the games that go out of their way to offer a mode that minimise this (like Melee's disable AA filter mode), the vertical blurring incurred from the YUVU framebuffer is impossible to disable on real hardware.

If you want an accurate console experience, we should actually be adding more blur.

Second, The video used by the GameCube and Wii (REC.601) has non-square pixels. 0.91:1 in NTSC and 1.09:1 in PAL. It's simply impossible to accurately portray this on a video mode with 1:1 square pixels, like 640x480 @ 60HZ VGA. Either you need to do some form of scaling (like dolphin currently does) or you end up with an incorrect aspect ratio.


However, I admit the experience could be better for this kind of usecase. At the very least, it should be easy to configure dolphin to a mode that actually showed the correct number of vertical lines with overscan/padding. Such a video mode would still need scaling along the horizontal for the majority of games, but at least you would only have scaling along one axis, and it happens to same axis that composite video really struggles with (and that a real console always has some amount of forced blur).

It would also be nice to have higher quality scaling algorithms, that's already on the todo list.

#2 Updated by JosJuice 24 days ago

  • Relates to maintainability set to No
  • Easy set to No
  • Relates to performance set to No
  • Relates to usability set to No
  • Regression set to No
  • Issue type set to Bug
  • Project changed from Infrastructure to Emulator
  • Tracker changed from Issue to Emulator Issues
  • Operating system N/A added

#3 Updated by tccalvin 24 days ago

Ok... There are some issues with your quest here.

Yes! Thank you very much for your feedback. After I did some further testing I figured out I got some information wrong.

First of all, I re-took my screenshots in windowed mode with "auto-adjust window" turned on, and Sonic Riders actually runs at 636x524, which makes a lot more sense.

Turns out I was squeezing the picture in a 640x480 window, which isn't how real hardware behaves (with overscan cropping and all that).

I also read one of your posts on Reddit about "antialiasing" on Gamecube, which made me understand things better.

To be clear, my goal isn't to get hard pixel edges like in a 2D game (I apologize if there was a misunderstanding), I just want the scanlines to line up properly with the game resolution and the pixels to be horizontally even like real hardware displayed on a CRT (with whatever blur might be applied by the console). In short, no post-scaling artifacts.

However, I admit the experience could be better for this kind of usecase. At the very least, it should be easy to configure dolphin to a mode that actually showed the correct number of vertical lines with overscan/padding. Such a video mode would still need scaling along the horizontal for the majority of games, but at least you would only have scaling along one axis, and it happens to same axis that composite video really struggles with (and that a real console always has some amount of forced blur).

It would also be nice to have higher quality scaling algorithms, that's already on the todo list.

My proposal here (as much as it's worth) would be to implement something along the lines of what Nintendo did with the SNES mini, which (iirc) offered a 1:1 pixel perfect mode with no shimmering (albeit with an incorrect aspect ratio), and a 4:3 mode with pixel interpolation (which minimized shimmering). I seem to remember the SNES also had non-square pixels.

Easier said than done, obviously.

The reason I'm saying this is that CRT displays could do without any form of horizontal scaling, since they can adjust the image after it has been sent to the screen, due to them not having a fixed pixel structure. So the correct aspect ratio could be achieved without the emulator doing any scaling (when set in 1:1 PAR). To my knowledge, LG OLEDs also have this "resizing" feature, and while they do have a fixed pixel structure, the pixel density is so high that you wouldn't notice any scaling.

As far as 480p LCDs are concerned, a 1:1 pixel mapping would definitely have an incorrect aspect ratio, however it would arguably be more "CRT-like", meaning that the vertical lines in excess would be vertically cropped and there wouldn't be any horizontal scaling since the pixels would be square (as opposed to having a forced aspect ratio where there would be scaling all over the place).

In a dream scenario, this would also apply to higher rendering resolutions, meaning we could have an EFB of 1280x1056 show up without cropping in a 1:1 pixel mapping on a 1080p screen without any horizontal scaling and so on.

Plaese forgive my ignorance in case I got anything wrong. I'm hoping to learn more from you guys! :)

#4 Updated by phire 24 days ago

The reason I'm saying this is that CRT displays could do without any form of horizontal scaling, since they can adjust the image after it has been sent to the screen, due to them not having a fixed pixel structure

Ok, yes they could.

But as far as I'm aware, there has never been a sane OS API that actually lets you output non-square pixels over VGA, or even composite. If there was (and if the install-base of CRT monitors was a bit higher) then it would be interesting for dolphin to support.

You could probably hack it together with custom screen modes and manually telling dolphin what the current pixel aspect ratio is, but then you run into problems that different games use different pixel aspect ratios, and some games will probably switch on the fly. It really needs to be dolphin directly controlling the display mode to make any sense.

IMO, it's simply not worth investing any development, code-complexity and support effort in this direction.

To my knowledge, LG OLEDs also have this "resizing" feature, and while they do have a fixed pixel structure, the pixel density is so high that you wouldn't notice any scaling.

Which is actually a very good point. In theory dolphin could upscale 1x IR image to 4K using a high-quality upscaling filter and achieve identical results.

In the future, dolphin will support high-quality custom upscaling/downscaling algorithms as post-processing shaders. You could choose between a high quality bicubic (or AMD's FSR) for smooth resampling that looks a lot better than bilinear's blurry mess. Or choose an upsampling algorithm that tries to preserve the a more pixelated feel without resorting to full nearest neighbour.

In a dream scenario, this would also apply to higher rendering resolutions, meaning we could have an EFB of 1280x1056 show up without cropping in a 1:1 pixel mapping on a 1080p screen without any horizontal scaling and so on.

I'm not sure what you want here? Do you mean with massive black bars on all four sides?

#5 Updated by tccalvin 24 days ago

IMO, it's simply not worth investing any development, code-complexity and support effort in this direction.

I definitely agree. Thankfully that's not necessary at all.

Here's a little experiment I did with integer scaling in RetroArch:

This is the staircase in the first level of Castelvania Symphony Of The Night, running in 480p (I'm using a 2x nearest neighbor scale from 240p):

https://imgur.com/a/nnuON3H

Here I used the emulator's options and set the aspect ratio to "corrected", and it's a big mess, with some pixels being square and others being rectangular, and in motion it's even worse because they switch sizes basically every frame, creating shimmering. But the aspect ratio is actually correct.

If I understand it correctly, the Dolphin approach is to keep this correct aspect ratio and add blur to make the uneven pixels less noticeable, which is one way of solving the issue.

Another way would be this (using the "uncorrected" aspect ratio):

https://imgur.com/a/rzTPxYh

Here, the pixels are all square, which makes scrolling smooth as butter, but at the cost of having an incorrect aspect ratio. Some people wouldn't want to play like this, and I understand that point of view. However, I would argue that the trade off is worth it because you get the even pixels you would get on a CRT (if narrower ones), which gets you less shimmering.

Thankfully, I'm lucky enough to have a CRT VGA monitor, so I don't have to make that choice. I can pick the square pixels option and then go into the monitor's OSD and stretch the image horizontally to get my correct aspect ratio:

https://imgur.com/a/uVVvQjE

(Here I stretched the image as far as I could to show the rectangular pixels.)

So yeah, no need to output non-square pixels. And like I said, it's not just CRT users that benefit from this, but also owners of screens with "resizing" features (with results varying in quality depending on the resolution of the display).

But then again, even without resizing some users could just prefer having a slightly narrower or wider image if it means having no scaling blur. All I'm saying is that it would be nice to have the option.

Which is actually a very good point. In theory dolphin could upscale 1x IR image to 4K using a high-quality upscaling filter and achieve identical results.

Yes, that would be a user-friendly way of doing it! :) (if maybe a bit more expensive in terms of rendering...?)

In the future, dolphin will support high-quality custom upscaling/downscaling algorithms as post-processing shaders. You could choose between a high quality bicubic (or AMD's FSR) for smooth resampling that looks a lot better than bilinear's blurry mess. Or choose an upsampling algorithm that tries to preserve the a more pixelated feel without resorting to full nearest neighbour.

I'm looking forward to that! :)

Is there anything wrong with nearest neighbour though...?

I'm not sure what you want here? Do you mean with massive black bars on all four sides?

Uhm... well, let's take Sonic Riders as an example (runs at 636x524 which is very close to the full 640x528 resolution of the EFB).

We could run it at 2x IR and get 1272x1048, which on a 1080p screen would leave 648 black pixels horizontally (1920-1272), and 32 black pixels vertically (1080-1048). It would have an incorrect aspect ratio but it would fit pretty nicely and have a 1:1 pixel mapping with no scaling. 1080p is a pretty popular resolution and I can see how some users could prefer to play this way instead of having scaling.

Again, I apologize for any mistakes! :)

#6 Updated by tccalvin 23 days ago

EDIT: Did some further testing and found out Dolphin's render window is not to be trusted when trying to pixel count... So yeah, ignore the numbers I gave about games running at certain resolutions (next time I'll learn not to take shortcuts when pixel counting). The point still stands though, whatever the native game resolution might be.

#7 Updated by tccalvin 21 days ago

UPDATE: If anyone wants to know, I did some manual pixel counting and Sonic Riders outputs 640x480 pixels, which explains how I was able to get no scaling when displaying it in a 640x480 window, whereas Mario Sunshine outputs 640x448, which is why, in a 640x480 frame, it only produces vertical scaling.

Obviously a pixel perfect mode would just turn off the 32 extra vertical lines and we would get no scaling in a 640x480 window.

As a bonus, I also pixel-counted Paper Mario TTYD, which is the complete opposite, it outputs 600x480, meaning it only needs horizontal scaling.

Again, pixel perfect means the image gets narrower, but that also means no scaling. Like we said, however, there are ways to get the proper aspect ratio after the image has been sent to the screen.

Also available in: Atom PDF