ScreenshotOnFailure doesn't work

[jukzi]

the ScreenshotOnFailure TestWatcher produces only black images on failed tests because
the @After org.eclipse.swt.tests.junit.Test_org_eclipse_swt_widgets_Widget.tearDown() calls shell.dispose(); before Screenshot is taken

reproducable with the failing Test_org_eclipse_swt_custom_StyledText

[jukzi]

see
https://stackoverflow.com/questions/18008241/run-a-testwatcher-before-the-after
junit-team/junit4#906
https://www.thinkcode.se/blog/2012/07/08/performing-an-action-when-a-test-fails

[basilevs]

@jukzi is this reproducible locally? Does it happen on Windows only?

[jukzi]

if i force a test to fail, i can locally reproduce that only the desktop is captured but not the failed junit test (expected to show the wrong color). One the one hand that shows that the ScreenshotOnFailure does not work as intended. On the other hand the dark screen captured on the i build may indicate some screensaver on the server.

[tomaswolf]

@jukzi : we ran into this screenshot taking issue (screenshot taken after the @after have run) in the EGit UI tests back in 2016. Our solution back then was to amend the SwtBotJUnit4ClassRunner to take the screenshot before the @afters. Maybe the things we did in that commit might be useful here, too.

[jukzi]

if some @After/https://github.com/before method fails or a rule such as
@ExpectedException fails the test. In those cases, the behavior is
unchanged: these screenshots are still taken after @after has run.

@tomaswolf thanks, i like that idea.

[jukzi]

SWTBotJunit4ClassRunner
looks a bit like TracingSuite already used in AllBrowserTests. Maybe that TracingSuite could be enhanced to take screenshots also on fail and be used in other suits as well.

[iloveeclipse]

@jukzi : some of the recent changes in this area broke a lot of UI & SWT tests.

Typical stack from https://download.eclipse.org/eclipse/downloads/drops4/I20241013-1800/testresults/html/org.eclipse.swt.tests_ep434I-unit-cen64-gtk3-java17_linux.gtk.x86_64_17.html

Cannot invoke "java.util.function.Supplier.get()" because "this.shell" is null

java.lang.NullPointerException: Cannot invoke "java.util.function.Supplier.get()" because "this.shell" is null
at org.eclipse.test.Screenshots$ScreenshotOnFailure.dispose(Screenshots.java:52)
at org.eclipse.test.Screenshots$ScreenshotOnFailure.finished(Screenshots.java:47)

Could you please fix it?

[jukzi]

Can u please just add a obvious null check. (I am afk) Jörg Kubitz

…

Am 14.10.2024 um 10:03 schrieb Andrey Loskutov ***@***.***>:  @jukzi : some of the recent changes in this area broke a lot of UI & SWT tests. Typical stack from https://download.eclipse.org/eclipse/downloads/drops4/I20241013-1800/testresults/html/org.eclipse.swt.tests_ep434I-unit-cen64-gtk3-java17_linux.gtk.x86_64_17.html Cannot invoke "java.util.function.Supplier.get()" because "this.shell" is null java.lang.NullPointerException: Cannot invoke "java.util.function.Supplier.get()" because "this.shell" is null at org.eclipse.test.Screenshots$ScreenshotOnFailure.dispose(Screenshots.java:52) at org.eclipse.test.Screenshots$ScreenshotOnFailure.finished(Screenshots.java:47) Could you please fix it? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[iloveeclipse]

Can u please just add a obvious null check. (I am afk)

OK, see eclipse-platform/eclipse.platform.releng.aggregator#2452

[jukzi]

Thanks! Jörg Kubitz

…

Am 14.10.2024 um 10:38 schrieb Andrey Loskutov ***@***.***>:  Can u please just add a obvious null check. (I am afk) OK, see eclipse-platform/eclipse.platform.releng.aggregator#2452 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[jukzi]

The screenshots are still not written. error is: "no screen devices"

Exception in thread "main" java.awt.AWTError: no screen devices
	at java.desktop/sun.awt.Win32GraphicsEnvironment.getDefaultScreenDevice(Win32GraphicsEnvironment.java:101)
	at java.desktop/java.awt.Robot.<init>(Robot.java:98)
	at org.eclipse.test.AwtScreenshot.main(AwtScreenshot.java:41)

https://ci.eclipse.org/releng/job/AutomatedTests/job/ep434I-unit-win32-java17/84/execution/node/3/ws/workarea/I20241026-1800/eclipse-testing/results/ep434I-unit-win32-java17_win32.win32.x86_64_17/org.eclipse.swt.tests.junit.AllNonBrowserTests.txt

[jukzi]

"no screen devices" may relate to https://bugs.openjdk.org/browse/JDK-8336862

[iloveeclipse]

"no screen devices" may relate to https://bugs.openjdk.org/browse/JDK-8336862

Great, isn't that the reason for #1486 ?

So we should downgrade JDK on Windows to 17.0.11 version to get tests working again???

[laeubi]

Too bad we need AWT for SWT tests ...

So probably better use

final Display display = Display.getDefault();
final Image captureImage = new Image(display, display.getBounds().width, display.getBounds().height);
GC gc = new GC(display);
gc.copyArea(captureImage, display.getBounds().x, display.getBounds().y);
gc.dispose();
Imagewriter.write(...)

(see for example https://github.com/eclipse-swtbot/org.eclipse.swtbot/blob/ed2034a65915f6095d3c2464ac43068b105ebd68/org.eclipse.swtbot.swt.finder/src/org/eclipse/swtbot/swt/finder/utils/SWTUtils.java#L463 )