Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly await WS connections finalization on RPC server shutdown #3664

Open
AnnaShaleva opened this issue Nov 6, 2024 · 0 comments
Open
Labels
bug Something isn't working I3 Minimal impact rpc RPC server and client S4 Routine U4 Nothing urgent

Comments

@AnnaShaleva
Copy link
Member

Current Behavior

Described in #3661 (comment). Stacktrace of improperly reused RPC server mutex shows that WS-handler routines are still active after server shutdown:

oct‚ 22 13:51:15 node1 neo-go[1183338]: fatal error: sync: Unlock of unlocked RWMutex
oct‚ 22 13:51:15 node1 neo-go[1183338]: goroutine 538 [running]:
oct‚ 22 13:51:15 node1 neo-go[1183338]: sync.fatal({0xf83d64?, 0xc001085880?})
oct‚ 22 13:51:15 node1 neo-go[1183338]:         runtime/panic.go:1007 +0x18
oct‚ 22 13:51:15 node1 neo-go[1183338]: sync.(*RWMutex).Unlock(0xc00019a4c8)
oct‚ 22 13:51:15 node1 neo-go[1183338]:         sync/rwmutex.go:208 +0x45
oct‚ 22 13:51:15 node1 neo-go[1183338]: github.com/nspcc-dev/neo-go/pkg/services/rpcsrv.(*Server).dropSubscriber(0xc00019a2c8, 0xc000a77740)
oct‚ 22 13:51:15 node1 neo-go[1183338]:         github.com/nspcc-dev/neo-go/pkg/services/rpcsrv/server.go:825 +0xce
oct‚ 22 13:51:15 node1 neo-go[1183338]: github.com/nspcc-dev/neo-go/pkg/services/rpcsrv.(*Server).handleWsReads(0xc00019a2c8, 0xc0034478c0, 0xc000af5f80, 0xc000a7
7740)
oct‚ 22 13:51:15 node1 neo-go[1183338]:         github.com/nspcc-dev/neo-go/pkg/services/rpcsrv/server.go:810 +0x266
oct‚ 22 13:51:15 node1 neo-go[1183338]: github.com/nspcc-dev/neo-go/pkg/services/rpcsrv.(*Server).handleHTTPRequest(0xc00019a2c8, {0x11c3900, 0xc003437dc0}, 0xc00
31945a0)
oct‚ 22 13:51:15 node1 neo-go[1183338]:         github.com/nspcc-dev/neo-go/pkg/services/rpcsrv/server.go:582 +0x54a
oct‚ 22 13:51:15 node1 neo-go[1183338]: net/http.HandlerFunc.ServeHTTP(0x471779?, {0x11c3900?, 0xc003437dc0?}, 0xc000943b68?)
oct‚ 22 13:51:15 node1 neo-go[1183338]:         net/http/server.go:2171 +0x29
oct‚ 22 13:51:15 node1 neo-go[1183338]: net/http.serverHandler.ServeHTTP({0xc000a77680?}, {0x11c3900?, 0xc003437dc0?}, 0x6?)
oct‚ 22 13:51:15 node1 neo-go[1183338]:         net/http/server.go:3142 +0x8e
oct‚ 22 13:51:15 node1 neo-go[1183338]: net/http.(*conn).serve(0xc0032030e0, {0x11c5220, 0xc000a76960})
oct‚ 22 13:51:15 node1 neo-go[1183338]:         net/http/server.go:2044 +0x5e8
oct‚ 22 13:51:15 node1 neo-go[1183338]: created by net/http.(*Server).Serve in goroutine 534
oct‚ 22 13:51:15 node1 neo-go[1183338]:         net/http/server.go:3290 +0x4b4

Expected Behavior

All goroutines spawned during Server's lifetime are properly finilized and awaited in the call to Server's Shutdown().

Possible Solution

It's not a critical bug, especially after #3661 merge, but we'd better keep the proper behaviour and await all WS-related routines in server shutdown.

@AnnaShaleva AnnaShaleva added bug Something isn't working I3 Minimal impact rpc RPC server and client U4 Nothing urgent S4 Routine labels Nov 6, 2024
@AnnaShaleva AnnaShaleva changed the title Properly await WS connections finalization on server shutdown Properly await WS connections finalization on RPC server shutdown Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working I3 Minimal impact rpc RPC server and client S4 Routine U4 Nothing urgent
Projects
None yet
Development

No branches or pull requests

1 participant