Emulator

Until https://github.com/dolphin-emu/dolphin/pull/10812 is finalized, there's no way to consistently and easily resync audio and video in a segmented dump. And the only way to avoid video splits is forcing window resolution for the whole dump.

However if one needs to dump at some really high resolution, it may be bigger than the monitor's, so the window won't fit. For such a case, and for more control in general, dumping at internal resolution is desired, but it has always caused desyncs on every segment split.

When I first checked this problem years ago, one of the issues was lack of PTS info for the last deduplicated frame of the previous segment, and the new segment just had PTS 0, so actual duration of the last frame of every segment was always assumed to be 1/60. I tried making it dump every single frame with no dedup, but it still desynced - as I was told it was due to different framerate on resolution change, and it was always resampled to 60fps back then.

Now the code is different, and framerate seems to match actual console (60000/1001), so I don't know anymore what the reason for desync is, maybe messed up PTS on segments again.

To verify AV desync there's an easy test. I ran https://tasvideos.org/9297S (resident evil 4) for 10k frames while dumping audio and video, once with window resolution dumping and once with internal resolution dumping (aspect corrected, but that shouldn't matter). Now I had to workaround a different bug where it fails to dump the first few hundreds of frames for this game, so I started the dump on frame 501 and stopped it at frame 10500.

Both audio tracks were identical to the point of having the same hashsum. But the windowed dump was 9996 frames long, while the internal res dump was only 9841 frames long. I combined the segments using this AVS+ script and previewed it in VirtualDub:

avisource("G4BE08_2024-10-21_23-46-42_0.avi")
for (i=1,9)
{
	last = last + avisource("G4BE08_2024-10-21_23-46-42_" + string(i) + ".avi").lanczosresize(last.width, last.height)
}
return last

So since we don't know when audio and video can be combined in one file, someone will need to either fix the PTS on segments (though I don't think it's possible to do it in a clean way by simply tweaking variables). Or we could use a workaround that would effectively allow us to mux every video segment with its corresponding audio segment, and then splice them unaligned so audio doesn't drift.

I tried looking at the code, and splitting code for audio and video buried in crazy layers of indirection and is even called from completely separate parts of the main loop, so the only way to make video splits inform audio to make it also split, is probably some callback, but I don't know how dolphin callbacks are organized.

Currently it's impossible to properly fix AV desync for segmented dumps, because we don't know the absolute timestamp of each video segment, so we can't properly align them with audio in pos-production. And having to guess and fixing it all manually for hundreds of segments would be a nightmare.