Intermittent testbed failure in CI on Linux #2648
Comments
freakboy3742
added
bug
|
I was finally able to get a stacktrace for this: The first assert is apparently failing... void WebProcessPool::pageEndUsingWebsiteDataStore(WebPageProxy& page, WebsiteDataStore& dataStore)
{
RELEASE_ASSERT(RunLoop::isMain());
auto sessionID = dataStore.sessionID();
RELEASE_ASSERT(m_sessionToPageIDsMap.isValidKey(dataStore.sessionID()));
auto iterator = m_sessionToPageIDsMap.find(sessionID);
RELEASE_ASSERT(iterator != m_sessionToPageIDsMap.end());
...sooo....its not running on the main thread....but only gets upset sometimes.... |
|
fwiw, finally got a stacktrace on ubuntu 22.04 as well to confirm they match: |
|
FWIW: There's an existing hack in the testbed's webkit tests to work around something very similar to this. The issue seems to be that Webkit starts its own threads, and the lifecycle of those threads interferes with pytest's process of creating and destroying widgets. It's likely only an issue because the testbed is rapidly creating and destroying widgets; I don't know if the existing workaround can be patched, or if we need to revisit this entirely. |
|
Also - of interest: you're seeing the crash in Webkit (which we've at least seen before); but the CI failure was occurring in |
AFAICT, the precise point at which the crash occurs in indeterminate because it happens in the garbage collector. |
|
I've been playing around with this, though, and can more or less reliably invoke the crash. The |
|
Added a few print statements and a pytest autouse fixture to call |
|
Another example of a crash; this time in splitcontainer. That indicates it's at least partially a Heisenbug; but also of interest is that the failure is before the WebView tests. |
|
Yeah; that's because garbage collecting |
|
I had overlooked this apparently but the testbed tests already contain one mitigation for this crash introduced in 6c8877c: toga/testbed/tests/widgets/test_webview.py Lines 116 to 121 in 1060842 The same exists in the This must be decreasing the likelihood of the issue occurring....but the WebKit2 WebView must still be escaping garbage collection until later allowing the crash to still happen. Another approach altogether here may be to instead keep the WebView referenced somewhere so garbage collection doesn't try to do anything with them.... [edit] I now see I was being directed to this earlier in the conversation :) |
Describe the bug
We have an intermittent build failure on the GTK testbed test, with a segfault from the PyGObject layer.
Steps to reproduce
Run the Linux testbed test. The failure isn't especially reproducible; re-running the test suite almost always passes. this is the most recent example.
Expected behavior
Test suite should pass without error.
Screenshots
No response
Environment
Logs
Additional context
No response
The text was updated successfully, but these errors were encountered: