Periodic connection closed exception in server.log

[marcreichman-pfi]

Brief Summary

Hello,

This is not a new issue, but the bits of the trace have changed in various releases. I suspect this is just a client disconnecting while using a longer-running endpoint (such as video playback, or file downloading), and I’m wondering how to suppress these particular exceptions, or if it’s a bug in catalina or grizzly. I have seen code in Glassfish which suppresses ClientAbortException, and maybe that code just isn’t as up to date with the newer types of listeners and connectors. We see this 1-3 times a day on average. OS is Windows, JDK is Oracle 21, Payara version is 6.2023.12.

Expected Outcome

No stack trace

Current Outcome

[2024-05-30T14:57:54.542-0400] [Payara 6.2023.12] [WARNING] [] [jakarta.enterprise.web] [tid: _ThreadID=231 _ThreadName=http-thread-pool::http-listener-2(46)] [timeMillis: 1717095474542] [levelValue: 900] [[
  StandardWrapperValve[default]: Servlet.service() for servlet default threw exception
java.io.IOException: Connection closed
	at org.glassfish.grizzly.asyncqueue.TaskQueue.onClose(TaskQueue.java:299)
	at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.onClose(AbstractNIOAsyncQueueWriter.java:415)
	at org.glassfish.grizzly.nio.transport.TCPNIOTransport.closeConnection(TCPNIOTransport.java:379)
	at org.glassfish.grizzly.nio.NIOConnection.doClose(NIOConnection.java:606)
	at org.glassfish.grizzly.nio.NIOConnection$6.run(NIOConnection.java:573)
	at org.glassfish.grizzly.nio.DefaultSelectorHandler.execute(DefaultSelectorHandler.java:190)
	at org.glassfish.grizzly.nio.NIOConnection.terminate0(NIOConnection.java:568)
	at org.glassfish.grizzly.nio.transport.TCPNIOConnection.terminate0(TCPNIOConnection.java:249)
	at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:102)
	at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:77)
	at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.processAsync(AbstractNIOAsyncQueueWriter.java:285)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:92)
	at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:51)
	at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:510)
	at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:82)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:83)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.executeIoEvent(WorkerThreadIOStrategy.java:73)
	at org.glassfish.grizzly.strategies.AbstractIOStrategy.executeIoEvent(AbstractIOStrategy.java:66)
	at org.glassfish.grizzly.nio.SelectorRunner.iterateKeyEvents(SelectorRunner.java:381)
	at org.glassfish.grizzly.nio.SelectorRunner.iterateKeys(SelectorRunner.java:353)
	at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:319)
	at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:248)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:535)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:515)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.io.IOException: An established connection was aborted by the software in your host machine
	at java.base/sun.nio.ch.SocketDispatcher.write0(Native Method)
	at java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:54)
	at java.base/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:137)
	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:81)
	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:58)
	at java.base/sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:542)
	at org.glassfish.grizzly.nio.transport.TCPNIOUtils.flushByteBuffer(TCPNIOUtils.java:118)
	at org.glassfish.grizzly.nio.transport.TCPNIOUtils.writeSimpleBuffer(TCPNIOUtils.java:104)
	at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:94)
	... 16 more
]]

Reproducer

No reproducer unfortunately

Operating System

Windows Server

JDK Version

Oracle 21

Payara Distribution

Payara Server Full Profile

[lprimak]

I think this should just be moved into DEBUG from WARNING.

[Elifzeynepedman]

Hi @marcreichman-pfi,

This is the normal behavior of the Payara server. The connection will always be closed when it is aborted by the remote host it is connected to.

My recommendation is to investigate the cause of the crash you are facing. Unfortunately, we cannot provide further assistance on this issue, so it will be closed. Thank you for your understanding.

Thank you,
Elfi

[marcreichman-pfi]

@Elifzeynepedman to be fair, this was not raised as a serious issue. But it's come up more in recent versions, and you can see from spots in the code that previous versions (maybe in a pre-NIO world?) suppressed things like ClientAbortException. I wonder if @lprimak's suggestion could be taken just to quiet down the logs for non-serious issues?