Project

General

Profile

Actions

Emulator Issues #13141

open

Dolphin crashes sometimes when unplugging standard controllers

Added by FitterSpace almost 2 years ago. Updated almost 2 years ago.

Status:
New
Priority:
Normal
Assignee:
-
% Done:

0%

Operating system:
N/A
Issue type:
Bug
Milestone:
Regression:
No
Relates to usability:
No
Relates to performance:
No
Easy:
No
Relates to maintainability:
No
Regression start:
Fixed in:

Description

Game Name?

007: Agent Under Fire (NTSC 1.00)
007: Nightfire (NTSC)

These are the two games I can consistently crash with. These games were made by different studios on different engines, so I don't expect it to be just the game's fault.

Game ID? (right click the game in the game list, Properties, Info tab)

Agent Under Fire: GW7E69
Nightfire: GO7E69

MD5 Hash? (right click the game in the game list, Properties, Verify tab, Verify Integrity button)

Agent Under Fire: 987af8dca9dd263c27305b4f139a3547 (redump verified)
Nightfire: a824f5a7a553b7598592c6c129378262 (redump verified)

What's the problem? Describe what went wrong.

When navigating the menus in these two games, unplugging controllers that are in use can cause the emulator to crash.

What steps will reproduce the problem?

Controller 1: Standard controller - Xbox One Controller
Controller 2, 3, and 4: Standard controller - keyboard - default bindings
(Controllers 2, 3, and 4 all use the same buttons)

Start up 007: Agent Under Fire with all 4 controllers plugged in. After pressing start on the title screen, unplug controller port 2. The emulator crashes for me every time.
For 007: Nightfire, it seems more random, but it will still happen eventually. In this game, I start up a level by using the other two controllers, then I exit the level, then I start disconnecting them. Eventually it will crash, too.

https://youtu.be/PR4ylcE18mw
https://youtu.be/1tDKHz2sC3I

Is the issue present in the latest development version? For future reference, please also write down the version number of the latest development version.

Yes. In the videos above, I'm using the latest beta, but it also happens in the latest development version, which is 5.0-18179

Is the issue present in the latest stable version?

No. This doesn't happen on 5.0 stable.

If the issue isn't present in the latest stable version, which is the first broken version? (You can find the first broken version by bisecting. Windows users can use the tool https://forums.dolphin-emu.org/Thread-green-notice-development-thread-unofficial-dolphin-bisection-tool-for-finding-broken-builds and anyone who is building Dolphin on their own can use git bisect.)

The first broken version is 5.0-17426 that issued the following change:
"Qt/Controllers: Refresh GUI on settings change."

What are your PC specifications? (CPU, GPU, Operating System, more)

CPU: i7-12700k
RAM: 32 GB DDR4
GPU: Nvidia RTX 3050
OS: Windows 10 64-bit

Actions #1

Updated by Anonymous almost 2 years ago

Link to mentioned change: https://github.com/dolphin-emu/dolphin/pull/11075

ASAN log (running Nightfire and changing first port from Standard Controller to None):

=================================================================
==45044==ERROR: AddressSanitizer: container-overflow on address 0x1315d2bd1a18 at pc 0x7ff60c0cf916 bp 0x00f2a20f92e0 sp 0x00f2a20f92e8
READ of size 8 at 0x1315d2bd1a18 thread T0
==45044==WARNING: Failed to use and restart external symbolizer!
    #0 0x7ff60c0cf915 in Config::OnConfigChanged d:\Source\Core\Common\Config\Config.cpp:99
    #1 0x7ff60c0ce241 in Config::ConfigChangeCallbackGuard::~ConfigChangeCallbackGuard d:\Source\Core\Common\Config\Config.cpp:240
    #2 0x7ff60bbf8aa6 in GamecubeControllersWidget::SaveSettings d:\Source\Core\DolphinQt\Config\GamecubeControllersWidget.cpp:202
    #3 0x7ff849ae59cb in QObject::objectNameChanged+0x1b9b (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x1800d59cb)
    #4 0x7ff8318b887a in QComboBox::initStyleOption+0xe1a (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x18012887a)
    #5 0x7ff8318ba600 in QComboBox::setCurrentText+0x440 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x18012a600)
    #6 0x7ff8318bf36c in QComboBox::qt_static_metacall+0x14c (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x18012f36c)
    #7 0x7ff849ae5a76 in QObject::objectNameChanged+0x1c46 (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x1800d5a76)
    #8 0x7ff8318b6929 in QComboBoxPrivateContainer::eventFilter+0x169 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x180126929)
    #9 0x7ff849a97f38 in QCoreApplicationPrivate::sendThroughObjectEventFilters+0xd8 (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x180087f38)
    #10 0x7ff8317a30f2 in QApplicationPrivate::notify_helper+0xf2 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x1800130f2)
    #11 0x7ff8317a13e7 in QApplication::notify+0x717 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x1800113e7)
    #12 0x7ff849a97c94 in QCoreApplication::notifyInternal2+0xc4 (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x180087c94)
    #13 0x7ff8317a0452 in QApplicationPrivate::sendMouseEvent+0x412 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x180010452)
    #14 0x7ff8317fc07e in QWidgetPrivate::invalidateBackingStore_resizeHelper+0x321e (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x18006c07e)
    #15 0x7ff8317facb3 in QWidgetPrivate::invalidateBackingStore_resizeHelper+0x1e53 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x18006acb3)
    #16 0x7ff8317a310d in QApplicationPrivate::notify_helper+0x10d (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x18001310d)
    #17 0x7ff8317a27e6 in QApplication::notify+0x1b16 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x1800127e6)
    #18 0x7ff849a97c94 in QCoreApplication::notifyInternal2+0xc4 (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x180087c94)
    #19 0x7ff839ae54ce in QGuiApplicationPrivate::processMouseEvent+0x63e (c:\src\dolphin\Binary\x64\Qt6Gui.dll+0x1800954ce)
    #20 0x7ff839b29bfd in QWindowSystemInterface::sendWindowSystemEvents+0x7d (c:\src\dolphin\Binary\x64\Qt6Gui.dll+0x1800d9bfd)
    #21 0x7ff849c1a054 in QEventDispatcherWin32::processEvents+0x94 (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x18020a054)
    #22 0x7ff839d68748 in QWindowsGuiEventDispatcher::processEvents+0x18 (c:\src\dolphin\Binary\x64\Qt6Gui.dll+0x180318748)
    #23 0x7ff849a9f9d3 in QEventLoop::exec+0x1d3 (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x18008f9d3)
    #24 0x7ff849a982c8 in QCoreApplication::exec+0x168 (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x1800882c8)
    #25 0x7ff60bdcfe3d in app_main d:\Source\Core\DolphinQt\Main.cpp:287
    #26 0x7ff60bdd0b40 in wWinMain d:\Source\Core\DolphinQt\Main.cpp:306
    #27 0x7ff60ca4c949 in __scrt_common_main_seh D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
    #28 0x7ff9095b26bc in BaseThreadInitThunk+0x1c (C:\WINDOWS\System32\KERNEL32.DLL+0x1800126bc)
    #29 0x7ff90accdfb7 in RtlUserThreadStart+0x27 (C:\WINDOWS\SYSTEM32\ntdll.dll+0x18005dfb7)

0x1315d2bd1a18 is located 1432 bytes inside of 2016-byte region [0x1315d2bd1480,0x1315d2bd1c60)
allocated by thread T39 here:
    #0 0x7ff60ca4a61a in operator new D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\asan\asan_win_new_scalar_thunk.cpp:41
    #1 0x7ff60c0ccc2c in std::vector<std::pair<unsigned __int64,std::function<void __cdecl(void)> >,std::allocator<std::pair<unsigned __int64,std::function<void __cdecl(void)> > > >::_Emplace_reallocate<std::pair<unsigned __int64,std::function<void __cdecl(void)> > > v:\14.34.31933\include\vector:894
    #2 0x7ff60c0ce489 in Config::AddConfigChangedCallback d:\Source\Core\Common\Config\Config.cpp:72
    #3 0x7ff60c2f735d in FifoPlayer::FifoPlayer d:\Source\Core\Core\FifoPlayer\FifoPlayer.cpp:174
    #4 0x7ff60c2f8caa in FifoPlayer::GetInstance d:\Source\Core\Core\FifoPlayer\FifoPlayer.cpp:400
    #5 0x7ff60c4d6dda in VideoInterface::GetAspectRatio d:\Source\Core\Core\HW\VideoInterface.cpp:571
    #6 0x7ff60c138c34 in Renderer::CalculateDrawAspectRatio d:\Source\Core\VideoCommon\RenderBase.cpp:664
    #7 0x7ff60c140ffe in Renderer::UpdateDrawRectangle d:\Source\Core\VideoCommon\RenderBase.cpp:824
    #8 0x7ff60c13779c in Renderer::Renderer d:\Source\Core\VideoCommon\RenderBase.cpp:115
    #9 0x7ff60c7b36a4 in DX12::Renderer::Renderer d:\Source\Core\VideoBackends\D3D12\D3D12Renderer.cpp:31
    #10 0x7ff60c53eeff in DX12::VideoBackend::Initialize d:\Source\Core\VideoBackends\D3D12\VideoBackend.cpp:134
    #11 0x7ff60c08798b in Core::EmuThread d:\Source\Core\Core\Core.cpp:538
    #12 0x7ff60c081b94 in std::thread::_Invoke<std::tuple<void (__cdecl*)(std::unique_ptr<BootParameters,std::default_delete<BootParameters> >,WindowSystemInfo),std::unique_ptr<BootParameters,std::default_delete<BootParameters> >,WindowSystemInfo>,0,1,2> v:\14.34.31933\include\thread:55
    #13 0x7ff908529362 in recalloc+0xa2 (C:\WINDOWS\System32\ucrtbase.dll+0x180029362)
    #14 0x7ff8302e17b9 in _asan_wrap_GlobalSize+0x5e969 (c:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.34.31933\bin\HostX64\x64\clang_rt.asan_dynamic-x86_64.dll+0x1800617b9)
    #15 0x7ff9095b26bc in BaseThreadInitThunk+0x1c (C:\WINDOWS\System32\KERNEL32.DLL+0x1800126bc)
    #16 0x7ff90accdfb7 in RtlUserThreadStart+0x27 (C:\WINDOWS\SYSTEM32\ntdll.dll+0x18005dfb7)

Thread T39 created by T0 here:
    #0 0x7ff8302e2ecf in _asan_wrap_GlobalSize+0x6007f (c:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.34.31933\bin\HostX64\x64\clang_rt.asan_dynamic-x86_64.dll+0x180062ecf)
    #1 0x7ff90852838d in beginthreadex+0x5d (C:\WINDOWS\System32\ucrtbase.dll+0x18002838d)
    #2 0x7ff60c081d30 in std::thread::_Start<void (__cdecl&)(std::unique_ptr<BootParameters,std::default_delete<BootParameters> >,WindowSystemInfo),std::unique_ptr<BootParameters,std::default_delete<BootParameters> >,WindowSystemInfo &> v:\14.34.31933\include\thread:75
    #3 0x7ff60c08a23b in Core::Init d:\Source\Core\Core\Core.cpp:251
    #4 0x7ff60c4144b9 in BootManager::BootCore d:\Source\Core\Core\BootManager.cpp:177
    #5 0x7ff60bdeacf8 in MainWindow::StartGame d:\Source\Core\DolphinQt\MainWindow.cpp:1057
    #6 0x7ff60bdeb4c5 in MainWindow::StartGame d:\Source\Core\DolphinQt\MainWindow.cpp:1027
    #7 0x7ff60bde8821 in MainWindow::ScanForSecondDiscAndStartGame d:\Source\Core\DolphinQt\MainWindow.cpp:998
    #8 0x7ff60bdeb0c9 in MainWindow::StartGame d:\Source\Core\DolphinQt\MainWindow.cpp:1015
    #9 0x7ff60bde72b7 in MainWindow::Play d:\Source\Core\DolphinQt\MainWindow.cpp:777
    #10 0x7ff60bdee398 in QtPrivate::QFunctorSlotObject<`MainWindow::ConnectGameList'::`2'::<lambda_1>,0,QtPrivate::List<>,void>::impl d:\Externals\Qt\Qt6.3.0\x64\include\QtCore\qobjectdefs_impl.h:444
    #11 0x7ff849ae59cb in QObject::objectNameChanged+0x1b9b (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x1800d59cb)
    #12 0x7ff60bb9fede in QtPrivate::QSlotObject<void (__cdecl CheatSearchFactoryWidget::*)(void),QtPrivate::List<>,void>::impl d:\Externals\Qt\Qt6.3.0\x64\include\QtCore\qobjectdefs_impl.h:419
    #13 0x7ff849ae59cb in QObject::objectNameChanged+0x1b9b (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x1800d59cb)
    #14 0x7ff831a21459 in QAbstractItemView::mouseDoubleClickEvent+0x149 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x180291459)
    #15 0x7ff8317e9cac in QWidget::event+0x17c (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x180059cac)
    #16 0x7ff83186f162 in QLCDNumber::event+0x32 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x1800df162)
    #17 0x7ff831a20159 in QAbstractItemView::viewportEvent+0x3b9 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x180290159)
    #18 0x7ff849a97f38 in QCoreApplicationPrivate::sendThroughObjectEventFilters+0xd8 (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x180087f38)
    #19 0x7ff8317a30f2 in QApplicationPrivate::notify_helper+0xf2 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x1800130f2)
    #20 0x7ff8317a13e7 in QApplication::notify+0x717 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x1800113e7)
    #21 0x7ff849a97c94 in QCoreApplication::notifyInternal2+0xc4 (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x180087c94)
    #22 0x7ff8317a0452 in QApplicationPrivate::sendMouseEvent+0x412 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x180010452)
    #23 0x7ff8317fcac0 in QWidgetPrivate::invalidateBackingStore_resizeHelper+0x3c60 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x18006cac0)
    #24 0x7ff8317facb3 in QWidgetPrivate::invalidateBackingStore_resizeHelper+0x1e53 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x18006acb3)
    #25 0x7ff8317a310d in QApplicationPrivate::notify_helper+0x10d (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x18001310d)
    #26 0x7ff8317a27e6 in QApplication::notify+0x1b16 (c:\src\dolphin\Binary\x64\Qt6Widgets.dll+0x1800127e6)
    #27 0x7ff849a97c94 in QCoreApplication::notifyInternal2+0xc4 (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x180087c94)
    #28 0x7ff839ae5938 in QGuiApplicationPrivate::processMouseEvent+0xaa8 (c:\src\dolphin\Binary\x64\Qt6Gui.dll+0x180095938)
    #29 0x7ff839b29bfd in QWindowSystemInterface::sendWindowSystemEvents+0x7d (c:\src\dolphin\Binary\x64\Qt6Gui.dll+0x1800d9bfd)
    #30 0x7ff849c1a054 in QEventDispatcherWin32::processEvents+0x94 (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x18020a054)
    #31 0x7ff839d68748 in QWindowsGuiEventDispatcher::processEvents+0x18 (c:\src\dolphin\Binary\x64\Qt6Gui.dll+0x180318748)
    #32 0x7ff849a9f9d3 in QEventLoop::exec+0x1d3 (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x18008f9d3)
    #33 0x7ff849a982c8 in QCoreApplication::exec+0x168 (c:\src\dolphin\Binary\x64\Qt6Core.dll+0x1800882c8)
    #34 0x7ff60bdcfe3d in app_main d:\Source\Core\DolphinQt\Main.cpp:287
    #35 0x7ff60bdd0b40 in wWinMain d:\Source\Core\DolphinQt\Main.cpp:306
    #36 0x7ff60ca4c949 in __scrt_common_main_seh D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
    #37 0x7ff9095b26bc in BaseThreadInitThunk+0x1c (C:\WINDOWS\System32\KERNEL32.DLL+0x1800126bc)
    #38 0x7ff90accdfb7 in RtlUserThreadStart+0x27 (C:\WINDOWS\SYSTEM32\ntdll.dll+0x18005dfb7)

HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_container_overflow=0.
If you suspect a false positive see also: https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow.
SUMMARY: AddressSanitizer: container-overflow d:\Source\Core\Common\Config\Config.cpp:99 in Config::OnConfigChanged
Shadow bytes around the buggy address:
  0x053e8bffa2f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x053e8bffa300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x053e8bffa310: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x053e8bffa320: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x053e8bffa330: 00 00 00 00 00 00 00 00 00 00 00 fc fc fc fc fc
=>0x053e8bffa340: fc fc fc[fc]fc fc fc fc fc fc fc fc fc fc fc fc
  0x053e8bffa350: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
  0x053e8bffa360: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
  0x053e8bffa370: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
  0x053e8bffa380: fc fc fc fc fc fc fc fc fc fc fc fc fa fa fa fa
  0x053e8bffa390: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
Address Sanitizer Error: Container overflow
Actions #2

Updated by Anonymous almost 2 years ago

The bug causing this crash is just that s_callbacks is modified at the same time it's being iterated over, because Add/Remove ConfigChangedCallback don't take any locks.

However, all the locking around s_callbacks is broken.
I tried to fix it like this:

diff --git a/Source/Core/Common/Config/Config.cpp b/Source/Core/Common/Config/Config.cpp
index 67b21ea33b..41d270a744 100644
--- a/Source/Core/Common/Config/Config.cpp
+++ b/Source/Core/Common/Config/Config.cpp
@@ -7,6 +7,7 @@
 #include <atomic>
 #include <map>
 #include <mutex>
+#include <semaphore>
 #include <shared_mutex>
 #include <utility>
 #include <vector>
@@ -18,7 +19,8 @@ using Layers = std::map<LayerType, std::shared_ptr<Layer>>;
 static Layers s_layers;
 static std::vector<std::pair<size_t, ConfigChangedCallback>> s_callbacks;
 static size_t s_next_callback_id = 0;
-static u32 s_callback_guards = 0;
+static std::binary_semaphore s_callback_guard{1};
+static std::atomic<u32> s_defer_change_depth = 0;
 static std::atomic<u64> s_config_version = 0;

 static std::shared_mutex s_layers_rw_lock;
@@ -67,22 +69,30 @@ void RemoveLayer(LayerType layer)

 size_t AddConfigChangedCallback(ConfigChangedCallback func)
 {
-  const size_t callback_id = s_next_callback_id;
-  ++s_next_callback_id;
+  s_callback_guard.acquire();
+
+  const size_t callback_id = s_next_callback_id++;
   s_callbacks.emplace_back(std::make_pair(callback_id, std::move(func)));
+
+  s_callback_guard.release();
+
   return callback_id;
 }

 void RemoveConfigChangedCallback(size_t callback_id)
 {
+  s_callback_guard.acquire();
+
   for (auto it = s_callbacks.begin(); it != s_callbacks.end(); ++it)
   {
     if (it->first == callback_id)
     {
       s_callbacks.erase(it);
-      return;
+      break;
     }
   }
+
+  s_callback_guard.release();
 }

 void OnConfigChanged()
@@ -92,11 +102,13 @@ void OnConfigChanged()
   // even when callbacks are suppressed.
   s_config_version.fetch_add(1, std::memory_order_relaxed);

-  if (s_callback_guards)
+  if (!s_callback_guard.try_acquire())
     return;

   for (const auto& callback : s_callbacks)
     callback.second();
+
+  s_callback_guard.release();
 }

 u64 GetConfigVersion()
@@ -136,9 +148,11 @@ void Init()
 void Shutdown()
 {
   WriteLock lock(s_layers_rw_lock);
-
   s_layers.clear();
+
+  s_callback_guard.acquire();
   s_callbacks.clear();
+  s_callback_guard.release();
 }

 void ClearCurrentRunLayer()
@@ -229,15 +243,13 @@ std::optional<std::string> GetAsString(const Location& config)

 ConfigChangeCallbackGuard::ConfigChangeCallbackGuard()
 {
-  ++s_callback_guards;
+  s_defer_change_depth++;
 }

 ConfigChangeCallbackGuard::~ConfigChangeCallbackGuard()
 {
-  if (--s_callback_guards)
-    return;
-
-  OnConfigChanged();
+  if (--s_defer_change_depth == 0)
+    OnConfigChanged();
 }

 }  // namespace Config

but this deadlocks because some of the registered ConfigChangedCallbacks try to Add/Remove a callback. I can't see how this code ever worked. Also, the fact that OnConfigChanged can bail out without invoking callbacks smells really bad and shouldn't be required.

Fixing this up probably requires more time than i can spend on it so, someone else can go for it :)

Actions

Also available in: Atom PDF