Emulator

Try checking "Skip EFB Access to CPU" in the graphics/hacks menu.

It's also a bit worse on the DirectX backend.

Skip EFB Access is already checked.

That kills my theory that it's EFB Pokes/Peeks...

If you want to test it, the lag starts right when she starts talking about "The lost village".
Also, audio backend and hle vs lle doesn't seem to affect it.

Hold on, I might have found the cause.
For whatever reason, this game must be super sensitive to anti-aliasing.
When I tried it on lowest settings, I didn't have AA turned fully off, just to 2x. With anti-aliasing completely off, it's playable, if still a little slow.

It may just be a very demanding game.

I suppose. I've never seen a game that goes from normal speed to slideshow just at 2x AA, though. :P

(PS: I also found a very specific lag issue in Fatal Frame 4, is it worth it to report it?)

Your GPU could be outright maxed out; which is weird.

Anyway, Dolphin doesn't normally take performance issues unless -

You can narrow down a performance regression from a merge
It's an extreme, unusual performance drop (aka, dropping to single digits for a certain moment/effect)
Other exceptions I can't think of right now.

What would cause that? It's never happened in any other game.
Are there any logs that might help?

Alright, it's not that severe. Basically what it is is that if a ghost Blooms during a fight, the purple glowy effect causes minor fps drops and noticeable audio stutter, but it's still very much playable, so I doubt it deserves its own issue.

I turned anti-aliasing back on and played for a minute with logging on, here's the log: http://pastebin.com/ufYfsUF8 .

No idea if it's relevant, but I figure it probably can't hurt.

This game is very demanding because it has copy to RAM set in game.ini, anyway with an old e8400 and a gtx650ti, where you say there should be a slide show, I get around 27 fps - 90% speed using Direct3d (wich for me is as usual better than OpenGL in copy to RAM).
If you wanna play it smoother set EFBToTextureEnable = True in game properties/edit config (but captured photographs in the album will be black), this almost doubles the above fps/speed for me.

Copy to RAM made antialiasing slow?
Well, thanks for the advice, I'll make the config change.

The log does contain a number of "wrong size errors" and "unknown opcodes", but I don't know if those are relevant.

The "wrong size" warnings are normal for most games (someone should fix those, they're useless), but the "Unknown BP opcode" warnings might be a little more serious.

46:45:111 BPStructs.cpp:681 W[Video]: Unknown BP opcode: address = 0x00000069 value = 0x000004ed
46:45:111 BPStructs.cpp:681 W[Video]: Unknown BP opcode: address = 0x00000046 value = 0x00000273

For all I know they could be just as harmless as the rest of the log, but let's throw it at our FIFO veteran just to be sure.

It were perfectly playable on latest x86 version - 1609.
This slowdowns maybe be result of awesome commit that added HW bounding box, so, you should try to use 4217 version.

No, for example 4.0-3694 is a bit slower than 4.0-5168 with this game (besides I don't think that FF2 uses BB).
If you think that x86 version - 1609 was perfectly playable maybe it is because you played it using copy to textures, in fact that revision was much slower (I got 30 fps in places where now I get 50...)

(FYI, it turns out that setting EFBToTextureEnable = True is a bad idea. it causes the lighting to get weird and makes a giant black square cover the upper left quarter of the screen.)

Well, right, I noticed that played with "copy to textures", but 30fps shows as 100% speed for me.

I'm confused as to what is causing the issue at this point due to the discussion. If you can the exact build where the slowdown became critical, then we can accept this.

OK, well, in my personal experience, the issue is 100% Anti aliasing. I've tried it with EFB to texture and EFB to RAM, the performance issues are extremely negligible. On the other hand, going from no AA to 2x AA makes it unplayable.
So, I'm fully convinced it's antialiasing and has next to nothing to do with EFB to RAM.

(Also, any leads on those Unknown Opcode errors?)

You can ignore the Unknown BP Opcode errors. BP Opcode 0x46 and 0x69 change the BUSCLOCK setting on the GPU. The busclock is not currently emulated in Dolphin.

So with Render to Main Window enabled, performance is just great on OpenGL, even with Antialiasing on. Still super stuttery on DirectX if AA is enabled, though.

Check that. Even without Render to Main, OpenGL still performs fairly well with this game as of 4.0-5420. Did something change?