Skip to content

bug: 网关高并发Map处理异常导致 worker exit #229

@dongsoft

Description

@dongsoft

Current Behavior

worker进程总线自动退出,导致监控插件,路由回传等插件频繁重启。监控数据每次在worker进程退出了数据归零

Expected Behavior

worker不再异常关闭,其他插件不再重新加载。

Error Logs

2025-11-19 14:03:57] [ERRO] [worker-41948]http upstream send error: the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection
[2025-11-19 14:08:57] [ERRO] [worker-41948]http upstream send error: the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection
[2025-11-19 14:13:57] [ERRO] [worker-41948]http upstream send error: the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection
[2025-11-19 14:18:57] [ERRO] [worker-41948]http upstream send error: the server closed connection before returning the first response byte. Make sure the server returns 'Connection: close' response header before closing the connection
fatal error: concurrent map read and map write

goroutine 23450 [running]:
internal/runtime/maps.fatal({0x3eadd70?, 0x41e2868?})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/runtime/panic.go:1058 +0x18
github.com/eolinker/apinto/node/http-context.(*HttpContext).GetLabel(0xc00081a008, {0xc000ba8f41, 0xb})
/mnt/c/computer/code/aesb-opens/apinto/node/http-context/context.go:139 +0x4c
github.com/eolinker/apinto/entries/http-entry.Fields.Read(0xc00080d140, {0xc000ba8f41, 0xb}, {0x4216eb8, 0xc00081a008})
/mnt/c/computer/code/aesb-opens/apinto/entries/http-entry/reader.go:52 +0x411
github.com/eolinker/apinto/entries/http-entry.(*Entry).Read(0xc00109f830, {0xc000ba8f41, 0xb})
/mnt/c/computer/code/aesb-opens/apinto/entries/http-entry/entry.go:30 +0x72
github.com/eolinker/eosc/formatter/json.(*jsonFormat).getValue(0xc000bbdce0, {0xc0005da108, 0xc, 0xc}, {0x41d5b60, 0xc00109f830})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/formatter/json/json.go:84 +0x63e
github.com/eolinker/eosc/formatter/json.(*jsonFormat).Format(0xc000bbdce0, {0x41d5b60, 0xc00109f830})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/formatter/json/json.go:63 +0xb1
github.com/eolinker/apinto/fm/drivers/output/rocketmq.(*tProducer).output(0xc000bbd3b0, {0x41d5b60, 0xc00109f830})
/mnt/c/computer/code/aesb-opens/apinto/fm/drivers/output/rocketmq/producer.go:142 +0xb1
github.com/eolinker/apinto/fm/drivers/output/rocketmq.(*Output).Output(0xc000201800, {0x41d5b60, 0xc00109f830})
/mnt/c/computer/code/aesb-opens/apinto/fm/drivers/output/rocketmq/output.go:26 +0x59
github.com/eolinker/apinto/drivers/plugins/access-log.(*accessLog).bodyFinish(0xc000bfb2f0, {0x4216eb8, 0xc00081a008})
/mnt/c/computer/code/aesb-opens/apinto/drivers/plugins/access-log/log.go:31 +0xfd
github.com/eolinker/apinto/node/http-context.(*ProxyRequest).ProxyBodyFinish(0xc00081a010, {0x4216eb8, 0xc00081a008})
/mnt/c/computer/code/aesb-opens/apinto/node/http-context/proxy.go:22 +0x73
github.com/eolinker/apinto/node/http-context.(*HttpContext).SendTo.func1(0xc00057ae40)
/mnt/c/computer/code/aesb-opens/apinto/node/http-context/context.go:268 +0x757
github.com/valyala/fasthttp.NewStreamReader.func1()
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/stream.go:44 +0x42
created by github.com/valyala/fasthttp.NewStreamReader in goroutine 23383
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/stream.go:43 +0x227

goroutine 1 [chan receive, 19 minutes]:
github.com/eolinker/eosc/process-worker.(*ProcessWorker).wait(0xc0007f3980)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/process-worker/process.go:86 +0x1d0
github.com/eolinker/eosc/process-worker.Process()
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/process-worker/process.go:59 +0x2f0
main.ProcessWorker()
/mnt/c/computer/code/aesb-opens/apinto/app/apinto/worker.go:11 +0x14
github.com/eolinker/eosc/process.Run()
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/process/process.go:109 +0x36f
main.main()
/mnt/c/computer/code/aesb-opens/apinto/app/apinto/main.go:37 +0x1f

goroutine 17 [chan receive]:
github.com/polarismesh/polaris-go/pkg/clock.(*clockImpl).updateTime(0xc0001324b0)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/polarismesh/polaris-go/pkg/clock/clock.go:56 +0xc9
created by github.com/polarismesh/polaris-go/pkg/clock.init.0 in goroutine 1
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/polarismesh/polaris-go/pkg/clock/clock.go:73 +0x150

goroutine 35 [select]:
go.opencensus.io/stats/view.(*worker).start(0xc000163a00)
/mnt/c/computer/code/aesb-opens/apinto/vendor/go.opencensus.io/stats/view/worker.go:292 +0x126
created by go.opencensus.io/stats/view.init.0 in goroutine 1
/mnt/c/computer/code/aesb-opens/apinto/vendor/go.opencensus.io/stats/view/worker.go:34 +0xb5

goroutine 37 [select, 9 minutes]:
github.com/patrickmn/go-cache.(*janitor).Run(0xc0000e27c0, 0xc00055e2c0)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/patrickmn/go-cache/cache.go:1079 +0xca
created by github.com/patrickmn/go-cache.runJanitor in goroutine 1
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/patrickmn/go-cache/cache.go:1099 +0x145

goroutine 38 [select]:
github.com/eolinker/apinto/monitor-manager.(*MonitorManager).doLoop(0xc0007a87e0)
/mnt/c/computer/code/aesb-opens/apinto/monitor-manager/manager.go:94 +0x186
created by github.com/eolinker/apinto/monitor-manager.NewMonitorManager in goroutine 1
/mnt/c/computer/code/aesb-opens/apinto/monitor-manager/manager.go:61 +0x16d

goroutine 39 [chan receive]:
github.com/eolinker/apinto/application/auth/openid-connect-jwt.(*Manager).doLoop(0xc0004e56c0)
/mnt/c/computer/code/aesb-opens/apinto/application/auth/openid-connect-jwt/manager.go:30 +0xd6
created by github.com/eolinker/apinto/application/auth/openid-connect-jwt.NewManager in goroutine 1
/mnt/c/computer/code/aesb-opens/apinto/application/auth/openid-connect-jwt/manager.go:21 +0x125

goroutine 46 [select]:
github.com/dubbogo/gost/time.NewTimerWheel.func1()
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/dubbogo/gost/time/timer.go:205 +0x1ec
created by github.com/dubbogo/gost/time.NewTimerWheel in goroutine 1
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/dubbogo/gost/time/timer.go:192 +0x227

goroutine 45 [chan receive, 19 minutes]:
github.com/dubbogo/gost/container/chan.(*UnboundedChan).run(0xc00080c330)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/dubbogo/gost/container/chan/unbounded_chan.go:118 +0xd2
created by github.com/dubbogo/gost/container/chan.NewUnboundedChanWithQuota in goroutine 1
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/dubbogo/gost/container/chan/unbounded_chan.go:87 +0x406

goroutine 103 [select, 19 minutes]:
google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient(0xc000234be0, {0xc0007f3660, 0x5, 0x5})
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/transport/transport.go:192 +0x125
google.golang.org/grpc/internal/transport.(*recvBufferReader).Read(0xc000234be0, {0xc0007f3660, 0x5, 0x5})
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/transport/transport.go:172 +0x1fe
google.golang.org/grpc/internal/transport.(*transportReader).Read(0xc0007f3620, {0xc0007f3660, 0x5, 0x5})
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/transport/transport.go:520 +0x72
io.ReadAtLeast({0x41c2020, 0xc0007f3620}, {0xc0007f3660, 0x5, 0x5}, 0x5)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/io/io.go:335 +0x168
io.ReadFull({0x41c2020, 0xc0007f3620}, {0xc0007f3660, 0x5, 0x5})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/io/io.go:354 +0x93
google.golang.org/grpc/internal/transport.(*Stream).Read(0xc000526c60, {0xc0007f3660, 0x5, 0x5})
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/transport/transport.go:504 +0x116
google.golang.org/grpc.(*parser).recvMsg(0xc0007f3650, 0x40000000)
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/rpc_util.go:600 +0xb6
google.golang.org/grpc.recvAndDecompress(0xc0007f3650, 0xc000526c60, {0x0, 0x0}, 0x40000000, 0x0, {0x0, 0x0})
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/rpc_util.go:734 +0xa5
google.golang.org/grpc.recv(0xc0007f3650, {0x7f2d2592d7d8, 0x593f8e0}, 0xc000526c60, {0x0, 0x0}, {0x3c033a0, 0xc00071b9d0}, 0x40000000, 0x0, ...)
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/rpc_util.go:800 +0xe5
google.golang.org/grpc.(*csAttempt).recvMsg(0xc000859520, {0x3c033a0, 0xc00071b9d0}, 0x0)
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/stream.go:1086 +0x545
google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc000859520)
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/stream.go:929 +0x4d
google.golang.org/grpc.(*clientStream).withRetry(0xc00080f200, 0xc000bf5be0, 0xc000bf5c98)
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/stream.go:762 +0xb4
google.golang.org/grpc.(*clientStream).RecvMsg(0xc00080f200, {0x3c033a0, 0xc00071b9d0})
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/stream.go:928 +0x21d
github.com/eolinker/eosc/service.(*masterDispatcherListenClient).Recv(0xc0008022a0)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/service/master_grpc.pb.go:62 +0x62
github.com/eolinker/eosc/process-worker.(*WorkerServer).listen(0xc000ad3ea0, 0xc00038f408, {0x41eee10, 0xc0008022a0})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/process-worker/service.go:127 +0x170
created by github.com/eolinker/eosc/process-worker.(*WorkerServer).listenMaster in goroutine 1
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/process-worker/service.go:75 +0x205

goroutine 68 [select]:
github.com/eolinker/apinto/node/fasthttp-client.(*Client).startCleaner(0xc00088d5c0, 0xc0005be150)
/mnt/c/computer/code/aesb-opens/apinto/node/fasthttp-client/client.go:212 +0xec
created by github.com/eolinker/apinto/node/fasthttp-client.(*Client).getHostClient in goroutine 107
/mnt/c/computer/code/aesb-opens/apinto/node/fasthttp-client/client.go:153 +0xa9c

goroutine 97 [sleep]:
time.Sleep(0x2540be400)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/runtime/time.go:338 +0x13c
github.com/eolinker/apinto/node/grpc-context.(*Client).clean(0xc000496210)
/mnt/c/computer/code/aesb-opens/apinto/node/grpc-context/client.go:56 +0x28f
created by github.com/eolinker/apinto/node/grpc-context.NewClient in goroutine 1
/mnt/c/computer/code/aesb-opens/apinto/node/grpc-context/client.go:33 +0x145

goroutine 160 [chan receive]:
github.com/eolinker/apinto/node/fasthttp-client.reset()
/mnt/c/computer/code/aesb-opens/apinto/node/fasthttp-client/dial.go:46 +0xd9
created by github.com/eolinker/apinto/node/fasthttp-client.init.0 in goroutine 1
/mnt/c/computer/code/aesb-opens/apinto/node/fasthttp-client/dial.go:38 +0x32

goroutine 161 [select, 19 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc000354d20, {0x41e27c0, 0xc0004a1270})
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0x1bc
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 1
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x15a

goroutine 162 [select, 19 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc000354d90, {0x41e27c0, 0xc0004a12c0})
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0x1bc
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 1
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x15a

goroutine 163 [select, 19 minutes]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc000354e10, {0x41e27c0, 0xc0004a1310})
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0x1bc
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 1
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x15a

goroutine 102 [select, 19 minutes]:
google.golang.org/grpc.newClientStreamWithParams.func4()
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/stream.go:393 +0xd4
created by google.golang.org/grpc.newClientStreamWithParams in goroutine 1
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/stream.go:392 +0x17ef

goroutine 64 [IO wait, 19 minutes]:
internal/poll.runtime_pollWait(0x7f2d25ae9078, 0x72)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/runtime/netpoll.go:351 +0x3c
internal/poll.(*pollDesc).wait(0xc00076c520, 0x72, 0x0)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x7a
internal/poll.(*pollDesc).waitRead(0xc00076c520, 0x0)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_poll_runtime.go:89 +0x31
internal/poll.(*FD).Read(0xc00076c500, {0xc000916000, 0x8000, 0x8000})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_unix.go:165 +0x411
net.(*netFD).Read(0xc00076c500, {0xc000916000, 0x8000, 0x8000})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/fd_posix.go:55 +0x73
net.(*conn).Read(0xc00009e2d0, {0xc000916000, 0x8000, 0x8000})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/net.go:194 +0xa5
bufio.(*Reader).Read(0xc000914000, {0xc0003f6120, 0x9, 0x9})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/bufio/bufio.go:245 +0x348
io.ReadAtLeast({0x41b8880, 0xc000914000}, {0xc0003f6120, 0x9, 0x9}, 0x9)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/io/io.go:335 +0x168
io.ReadFull({0x41b8880, 0xc000914000}, {0xc0003f6120, 0x9, 0x9})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/io/io.go:354 +0x93
golang.org/x/net/http2.readFrameHeader({0xc0003f6120, 0x9, 0x9}, {0x41b8880, 0xc000914000})
/mnt/c/computer/code/aesb-opens/apinto/vendor/golang.org/x/net/http2/frame.go:242 +0xbf
golang.org/x/net/http2.(*Framer).ReadFrame(0xc0003f60e0)
/mnt/c/computer/code/aesb-opens/apinto/vendor/golang.org/x/net/http2/frame.go:506 +0x116
google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc000926008, 0xc0005b8380)
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1587 +0x1db
created by google.golang.org/grpc/internal/transport.newHTTP2Client in goroutine 164
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/transport/http2_client.go:400 +0x27ee

goroutine 177 [select, 19 minutes]:
google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc00057c190, 0x1)
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:418 +0x1bb
google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0005b83f0)
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:552 +
0x134
google.golang.org/grpc/internal/transport.newHTTP2Client.func6()
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/transport/http2_client.go:454 +
0xfa
created by google.golang.org/grpc/internal/transport.newHTTP2Client in goroutine 164
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/internal/transport/http2_client.go:452 +0x3473

goroutine 104 [IO wait]:
internal/poll.runtime_pollWait(0x7f2d25ae8f60, 0x72
)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/runtime/netpoll.go:351
+0x3c
internal/poll.(*pollDesc).wait
(0xc00006a0a0,
0x72, 0x0)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_poll_runtime.go:84
+0x7a
internal/poll.(*pollDesc).waitRead(
0xc00006a0a0, 0x0
)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_poll_runtime.go:
89 +0x31
internal/poll.(*FD).Accept(0xc00006a080)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_unix.go:620
+0x525
net.(*netFD).accept(0xc00006a080
)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/fd_unix.go:
172 +0x69
net.(*UnixListener).accept(
0xc0007f3a10)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/unixsock_posix.go
:172 +0x4f

net.(*UnixListener).Accept(0xc0007f3a10)

/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/unixsock.go:260 +
0x65
net/http.(*Server).Serve(0xc0005bc100, {0x41d5000, 0xc0001243c0})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/http/server.go:3424 +0x50b
github.com/eolinker/eosc/process-worker.(*ProcessWorker).OpenMetricsServer.func1
()
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/process-worker/process.go
:177 +0x51

created by github.com/eolinker/eosc/process-worker.(*ProcessWorker).OpenMetricsServer in goroutine
1
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/process-worker/process.go:176
+0x276

goroutine
165 [syscall
, 19
minutes]:
os/signal.signal_recv(
)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/runtime/sigqueue.go
:152 +0x29
os/signal.loop()
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/os/signal/signal_unix.go:23
+0x17
created by os/signal.Notify.func1.1 in goroutine 1
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/os/signal/signal.go
:152 +
0x26

goroutine 166
[IO wait]:
internal/poll.runtime_pollWait
(0x7f2d25ae9190,
0x72)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/runtime/netpoll.go:351
+0x3c
internal/poll.(*pollDesc).wait(0xc00034c8a0
, 0x72,
0x0)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_poll_runtime.go:
84 +0x7a
internal/poll.(*pollDesc).waitRead(0xc00034c8a0,
0x0)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_poll_runtime.go:89
+0x31
internal/poll.(*FD).Accept(0xc00034c880)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_unix.go:620 +0x525

net.(*netFD).accept(0xc00034c880)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/fd_unix.go:172
+0x69
net.(*TCPListener).accept(0xc000491e00
)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/tcpsock_posix.go:159 +0x65

net.(*TCPListener).Accept(0xc000491e00
)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/tcpsock.go:380
+0x65
github.com/soheilhy/cmux.(*cMux).Serve(0xc0004a1a40)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/soheilhy/cmux/cmux.go:171 +
0x136
github.com/eolinker/eosc/traffic.runMux(
{0x41ea570, 0xc0004a1a40}
)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/traffic/traffic.go:78
+0x27
created by github.com/eolinker/eosc/traffic.(*Traffic).Listen in goroutine
103

/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/traffic/traffic.go:65 +
0x967

goroutine 167 [select]:
github.com/soheilhy/cmux.muxListener.Accept
({{0x41d5090, 0xc000491e00
}, 0xc0008bac40,
0xc000b3a310})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/soheilhy/cmux/cmux.go:
263 +0xec
github.com/eolinker/eosc/traffic/mixl.(*MixListener).accept(0xc000781920, {0x41d4fd0
, 0xc00042b1c0})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/traffic/mixl/muxl.go:92 +0x127
created by github.com/eolinker/eosc/traffic/mixl.NewMixListener in goroutine 103
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/traffic/mixl/muxl.go:49 +0x57c

goroutine 168 [select, 19 minutes]:
github.com/soheilhy/cmux.muxListener.Accept({{0x41d5090, 0xc000491e00}, 0xc0008bab60, 0xc000b3a2a0})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/soheilhy/cmux/cmux.go:263 +0xec
crypto/tls.(*listener).Accept(0xc00000fb00)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/crypto/tls/tls.go:67 +0x53
github.com/eolinker/eosc/traffic/mixl.(*MixListener).accept(0xc000781920, {0x41d53d0, 0xc00000fb00})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/traffic/mixl/muxl.go:92 +0x127
created by github.com/eolinker/eosc/traffic/mixl.NewMixListener in goroutine 103
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/traffic/mixl/muxl.go:49 +0x57c

goroutine 169 [select]:
github.com/eolinker/eosc/traffic/mixl.(*MixListener).Accept(0xc000781920)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/traffic/mixl/muxl.go:113 +0xe5
github.com/soheilhy/cmux.(*cMux).Serve(0xc0004a1a90)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/soheilhy/cmux/cmux.go:171 +0x136
github.com/eolinker/apinto/drivers/router.initListener.func1({0x41d68e0, 0xc000781920}, 0x1fa3)
/mnt/c/computer/code/aesb-opens/apinto/drivers/router/listener.go:104 +0x39e
created by github.com/eolinker/apinto/drivers/router.initListener in goroutine 103
/mnt/c/computer/code/aesb-opens/apinto/drivers/router/listener.go:94 +0x7bd

goroutine 170 [select, 19 minutes]:
github.com/soheilhy/cmux.muxListener.Accept({{0x41d68e0, 0xc000781920}, 0xc0008bad90, 0xc000b3a460})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/soheilhy/cmux/cmux.go:263 +0xec
google.golang.org/grpc.(*Server).Serve(0xc0008b0400, {0x41d4fd0, 0xc00042b260})
/mnt/c/computer/code/aesb-opens/apinto/vendor/google.golang.org/grpc/server.go:885 +0x64c
github.com/eolinker/apinto/drivers/router/grpc-router/manager.init.0.func1(0x1fa3, {0x41d4fd0, 0xc00042b260})
/mnt/c/computer/code/aesb-opens/apinto/drivers/router/grpc-router/manager/init.go:37 +0x205
created by github.com/eolinker/apinto/drivers/router.initListener.func1 in goroutine 169
/mnt/c/computer/code/aesb-opens/apinto/drivers/router/listener.go:100 +0x37a

goroutine 171 [select]:
github.com/soheilhy/cmux.muxListener.Accept({{0x41d68e0, 0xc000781920}, 0xc0008bae70, 0xc000b3a4d0})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/soheilhy/cmux/cmux.go:263 +0xec
github.com/valyala/fasthttp.acceptConn(0xc0009906c8, {0x41d4fd0, 0xc00042b280}, 0xc0009b2ba8)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/server.go:1937 +0x77
github.com/valyala/fasthttp.(*Server).Serve(0xc0009906c8
, {0x41d4fd0, 0xc00042b280})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/server.go:1831 +0x58e
github.com/eolinker/apinto/drivers/router/http-router/manager.init.0.func1(0x1fa3, {0x41d4fd0, 0xc00042b280})
/mnt/c/computer/code/aesb-opens/apinto/drivers/router/http-router/manager/init.go:31 +0x130
created by github.com/eolinker/apinto/drivers/router.initListener.func1 in goroutine 169
/mnt/c/computer/code/aesb-opens/apinto/drivers/router/listener.go:100 +0x37a

goroutine 173 [select
, 19 minutes]:
github.com/soheilhy/cmux.muxListener.Accept(
{{0x41d68e0,
0xc000781920}, 0xc0008baf50
, 0xc000b3a540})

/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/soheilhy/cmux/cmux.go:263 +
0xec
github.com/eolinker/apinto/dubbo-getty.(*server).accept(
0xc00041d4a0, 0xc0000e2ab0)

/mnt/c/computer/code/aesb-opens/apinto/dubbo-getty/server.go:269 +0x6b
github.com/eolinker/apinto/dubbo-getty.(*server).runTCPEventLoop.func1()
/mnt/c/computer/code/aesb-opens/apinto/dubbo-getty/server.go:305
+0x205
created by github.com/eolinker/apinto/dubbo-getty.(*server).runTCPEventLoop in goroutine 172

/mnt/c/computer/code/aesb-opens/apinto/dubbo-getty/server.go:290 +0xb6

goroutine 106 [sleep]:
time.Sleep(0x2540be400)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/runtime/time.go:338 +0x13c
github.com/valyala/fasthttp.(*workerPool).Start.func2()
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/workerpool.go:68 +0x7c
created by github.com/valyala/fasthttp.(*workerPool).Start in goroutine 171
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/workerpool.go:60 +0x127

goroutine 108 [sleep
]:
time.Sleep(
0x3b9aca00)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/runtime/time.go
:338 +0x13c

github.com/valyala/fasthttp.updateServerDate.func1
()
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/header.go
:2426 +0x1c

created by
github.com/valyala/fasthttp.updateServerDate in goroutine 107
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/header.go:
2424 +0x1f

goroutine 204 [select, 1 minutes]:
github.com/apache/rocketmq-client-go/v2/internal.(*rmqClient).Start.func1.1
()

/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/client.go:431 +
0x1d6
github.com/apache/rocketmq-client-go/v2/primitive.WithRecover(0xc000bd31e0)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/primitive/base.go:100 +
0x39
created by github.com/apache/rocketmq-client-go/v2/internal.(*rmqClient).Start.func1
in goroutine 103

/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/client.go:421 +0x247

goroutine
205 [select]:
github.com/apache/rocketmq-client-go/v2/internal.(*rmqClient).Start.func1.2()
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/client.go
:455 +0x1d0
github.com/apache/rocketmq-client-go/v2/primitive.WithRecover(0xc000bd3200)

/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/primitive/base.go:100 +0x39

created by github.com/apache/rocketmq-client-go/v2/internal.(*rmqClient).Start.func1 in goroutine
103
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/client.go
:444 +0x309

goroutine 206
[select]:
github.com/apache/rocketmq-client-go/v2/internal.(*rmqClient).Start.func1.3()

/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/client.go:479 +0x1d0

github.com/apache/rocketmq-client-go/v2/primitive.WithRecover(0xc000bd3220
)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/primitive/base.go:100 +0x39
created by github.com/apache/rocketmq-client-go/v2/internal.(*rmqClient).Start.func1 in goroutine 103
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/client.go:467 +
0x3ca

goroutine 207 [select
]:
github.com/apache/rocketmq-client-go/v2/internal.(*rmqClient).Start.func1.4
()
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/client.go:
511 +0x1d6
github.com/apache/rocketmq-client-go/v2/primitive.WithRecover
(0xc000bd3240)

/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/primitive/base.go:100 +0x39

created by github.com/apache/rocketmq-client-go/v2/internal.(*rmqClient).Start.func1 in goroutine
103
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/client.go
:492 +0x48c

goroutine 208 [select]:
github.com/apache/rocketmq-client-go/v2/internal.(*rmqClient).Start.func1.5()

/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/client.go:527 +0x15d

github.com/apache/rocketmq-client-go/v2/primitive.WithRecover(0xc000bd3260
)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/primitive/base.go:100
+0x39
created by
github.com/apache/rocketmq-client-go/v2/internal.(*rmqClient).Start.func1 in goroutine 103
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/client.go
:523 +0x54c

goroutine 209 [chan receive
]:
github.com/eolinker/apinto/discovery.(*appContainer).doCheck(0xc00055ecc0
)
/mnt/c/computer/code/aesb-opens/apinto/discovery/discovery.go:126 +0xfc

created by github.com/eolinker/apinto/discovery.(*appContainer).Set in goroutine 103

/mnt/c/computer/code/aesb-opens/apinto/discovery/discovery.go:119 +0x365

goroutine 69 [sleep]:
time.Sleep(0x2540be400
)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/runtime/time.go:338
+0x13c
github.com/valyala/fasthttp.(*HostClient).connsCleaner(0xc000a601a0)

/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/client.go:1743 +0x5e5

created by github.com/valyala/fasthttp.(*HostClient).AcquireConn in goroutine
107
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/client.go
:1635 +0x4cb

goroutine 70 [
sleep]:
time.Sleep(0x3b9aca00)

/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/runtime/time.go:338 +0x13c

github.com/valyala/fasthttp.(*TCPDialer).tcpAddrsClean(
0x58cb000)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/tcpdialer.go:
413 +0x36
created by github.com/valyala/fasthttp.(*TCPDialer).dial.func1 in goroutine 107
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/tcpdialer.go:285 +0x10d

goroutine 210
[IO wait]:
internal/poll.runtime_pollWait(0x7f2d25ae8d30
, 0x72)

/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/runtime/netpoll.go:351 +0x3c

internal/poll.(*pollDesc).wait(0xc00076c920, 0x72,
0x0)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_poll_runtime.go:
84 +0x7a
internal/poll.(*pollDesc).waitRead(0xc00076c920, 0x0
)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_poll_runtime.go:89
+0x31
internal/poll.(*FD).Read(0xc00076c900, {
0xc0001b632c, 0x4, 0x4
})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_unix.go:
165 +0x411
net.(*netFD).Read(0xc00076c900
, {0xc0001b632c, 0x4
, 0x4})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/fd_posix.go:55 +
0x73
net.(*conn).Read(
0xc00071d380, {0xc0001b632c,
0x4, 0x4}
)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/net.go:194
+0xa5
io.ReadAtLeast({0x41c1620, 0xc000c0c4e0}, {0xc0001b632c, 0x4, 0x4}, 0x4)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/io/io.go:335 +0x168
io.ReadFull({0x41c1620, 0xc000c0c4e0}, {0xc0001b632c, 0x4, 0x4})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/io/io.go:354 +0x93
github.com/apache/rocketmq-client-go/v2/internal/remote.(*remotingClient).receiveResponse(0xc00034d900, 0xc000c0c4e0)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/remote/remote_client.go:195 +0x357
github.com/apache/rocketmq-client-go/v2/internal/remote.(*remotingClient).connect.func1()
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/remote/remote_client.go:165 +0x2a
github.com/apache/rocketmq-client-go/v2/primitive.WithRecover(0xc000c02960)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/primitive/base.go:100 +0x39
created by github.com/apache/rocketmq-client-go/v2/internal/remote.(*remotingClient).connect in goroutine 179
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/remote/remote_client.go:164
+0x48f

goroutine
181 [IO wait]:
internal/poll.runtime_pollWait
(0x7f2d25ae8b00, 0x72
)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/runtime/netpoll.go
:351 +0x3c

internal/poll.(*pollDesc).wait(
0xc000c2fe20, 0x72,
0x0)

/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x7a
internal/poll.(*pollDesc).waitRead(0xc000c2fe20, 0x0)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_poll_runtime.go:89 +0x31
internal/poll.(*FD).Read(
0xc000c2fe00, {0xc0001b6564
, 0x4, 0x4
})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_unix.go
:165 +0x411

net.(*netFD).Read(
0xc000c2fe00, {0xc0001b6564
, 0x4, 0x4
})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/fd_posix.go
:55 +0x73

net.(*conn).Read(
0xc00009e340, {0xc0001b6564
, 0x4, 0x4})

/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/net.go:194 +
0xa5
io.ReadAtLeast
({0x41c1620,
0xc0000bbd40}, {
0xc0001b6564, 0x4
, 0x4},
0x4)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/io/io.go
:335 +0x168

io.ReadFull(
{0x41c1620,
0xc0000bbd40}, {
0xc0001b6564, 0x4, 0x4})

/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/io/io.go:354 +
0x93
github.com/apache/rocketmq-client-go/v2/internal/remote.(*remotingClient).receiveResponse(
0xc00034d980, 0xc0000bbd40)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/remote/remote_client.go:
195 +0x357
github.com/apache/rocketmq-client-go/v2/internal/remote.(*remotingClient).connect.func1()
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/remote/remote_client.go:165
+0x2a
github.com/apache/rocketmq-client-go/v2/primitive.WithRecover
(0xc0004a6690)

/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/primitive/base.go:100 +
0x39
created by
github.com/apache/rocketmq-client-go/v2/internal/remote.(*remotingClient).connect in goroutine 179
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/remote/remote_client.go:164 +0x48f

goroutine 191 [IO wait
]:
internal/poll.runtime_pollWait(0x7f2d25ae8e48,
0x72)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/runtime/netpoll.go
:351 +0x3c

internal/poll.(*pollDesc).wait(
0xc00076cca0, 0x72,
0x0)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_poll_runtime.go
:84 +0x7a
internal/poll.(*pollDesc).waitRead(0xc00076cca0
, 0x0)

/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_poll_runtime.go:89 +0x31
internal/poll.(*FD).Read(0xc00076cc80,
{0xc000714aa4, 0x4
, 0x4})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_unix.go:165
+0x411
net.(*netFD).Read(0xc00076cc80,
{0xc000714aa4, 0x4
, 0x4}
)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/fd_posix.go:
55 +0x73
net.(*conn).Read(0xc00009e4f0
, {0xc000714aa4, 0x4, 0x4
})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/net.go
:194 +0xa5

io.ReadAtLeast(
{0x41c1620,
0xc0004926c0}, {
0xc000714aa4, 0x4,
0x4}, 0x4
)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/io/io.go:
335 +0x168
io.ReadFull({
0x41c1620, 0xc0004926c0
}, {0xc000714aa4, 0x4, 0x4
})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/io/io.go
:354 +0x93

github.com/apache/rocketmq-client-go/v2/internal/remote.(*remotingClient).receiveResponse
(0xc00034d980, 0xc0004926c0
)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/remote/remote_client.go:
195 +0x357
github.com/apache/rocketmq-client-go/v2/internal/remote.(*remotingClient).connect.func1()
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/remote/remote_client.go:165
+0x2a
github.com/apache/rocketmq-client-go/v2/primitive.WithRecover
(0xc000bb4180)

/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/primitive/base.go:100
+0x39
created by github.com/apache/rocketmq-client-go/v2/internal/remote.(*remotingClient).connect in goroutine 78

/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/remote/remote_client.go

:164 +0x48f

goroutine 914 [IO wait
]:
internal/poll.runtime_pollWait
(0x7f2d25ae89e8,
0x72)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/runtime/netpoll.go
:351 +0x3c

internal/poll.(*pollDesc).wait(
0xc000e1a320, 0x72,
0x0)

/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x7a
internal/poll.(*pollDesc).waitRead
(0xc000e1a320, 0x0)

/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_poll_runtime.go:89 +
0x31
internal/poll.(*FD).Read
(0xc000e1a300, {
0xc001189684, 0x4,
0x4})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/poll/fd_unix.go:165 +0x411
net.(*netFD).Read(0xc000e1a300, {0xc001189684, 0x4, 0x4})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/fd_posix.go:55 +0x73
net.(*conn).Read(0xc00071cb08, {0xc001189684, 0x4, 0x4})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/net/net.go:194 +0xa5
io.ReadAtLeast({0x41c1620, 0xc000720e20}, {0xc001189684, 0x4, 0x4}, 0x4)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/io/io.go:335 +0x168
io.ReadFull({0x41c1620, 0xc000720e20}, {0xc001189684, 0x4, 0x4
})
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/io/io.go:
354 +0x93
github.com/apache/rocketmq-client-go/v2/internal/remote.(*remotingClient).receiveResponse(0xc00034d900
, 0xc000720e20)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/remote/remote_client.go:195
+0x357
github.com/apache/rocketmq-client-go/v2/internal/remote.(*remotingClient).connect.func1()

/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/remote/remote_client.go:165 +
0x2a
github.com/apache/rocketmq-client-go/v2/primitive.WithRecover
(0xc0004a7710)

/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/primitive/base.go:100 +0x39
created by github.com/apache/rocketmq-client-go/v2/internal/remote.(*remotingClient).connect in goroutine 205
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/apache/rocketmq-client-go/v2/internal/remote/remote_client.go:164 +
0x48f

goroutine 23383 [runnable
]:
internal/runtime/maps.newarray(
0x20?,
0x3c512c0?)

/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/runtime/malloc.go:1800
+0x13
internal/runtime/maps.newGroups(
0xc0009f9730?, 0x4
)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/runtime/maps/group.go:
310 +0x1c
internal/runtime/maps.(*table).reset(
0xc000e55080, 0x39be2c0
, 0x20)

/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/runtime/maps/table.go:104 +
0x33
internal/runtime/maps.newTable(0x39be2c0,
0x20, 0x0
, 0x0)

/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/runtime/maps/table.go:95 +
0x85
internal/runtime/maps.(*table).grow
(0xc001088520
, 0x39be2c0, 0xc000a4a1b0
, 0x7?)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/runtime/maps/table.go:1092
+0x37
internal/runtime/maps.(*table).rehash(0xc0004a6d08
?, 0xc000d82608?
, 0xc00051bbf8?)
/root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.10.linux-amd64/src/internal/runtime/maps/table.go:1027
+0x25
github.com/eolinker/apinto/node/http-context.(*HttpContext).SetLabel
(0xc00081a008, {
0x3e2e7dd, 0x7}
, {0x3e1e9e3
, 0x5})
/mnt/c/computer/code/aesb-opens/apinto/node/http-context/context.go:135 +0x54
github.com/eolinker/apinto/drivers/router/http-router/http-complete.(*HttpComplete).Complete.func1(
)
/mnt/c/computer/code/aesb-opens/apinto/drivers/router/http-router/http-complete/complete.go:
46 +0x145
github.com/eolinker/apinto/drivers/router/http-router/http-complete.(*HttpComplete).Complete
(0x593f8e0,
{
0x4212418, 0xc00081a008})
/mnt/c/computer/code/aesb-opens/apinto/drivers/router/http-router/http-complete/complete.go:99 +0xf25
github.com/eolinker/apinto/drivers/router/http-router/http-complete.(*httpCompleteCaller).DoFilter(0x593f8e0, {0x4212418, 0xc00081a008}, {0x41cb898, 0xc0011e5c80})
/mnt/c/computer/code/aesb-opens/apinto/drivers/router/http-router/http-complete/complete.go:150 +0x62
github.com/eolinker/eosc/eocontext.Filters.DoChain({0xc0006bbbf0, 0x1, 0x1}, {0x4212418, 0xc00081a008})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/eocontext/filter.go:27 +0x110
github.com/eolinker/apinto/drivers/strategy/limiting-strategy.(*tActuatorSet).Strategy(0xc000807200, {0x4212418, 0xc00081a008}, {0x41cb898, 0xc0011e5c68}, 0xc000a12060)
/mnt/c/computer/code/aesb-opens/apinto/drivers/strategy/limiting-strategy/actuator.go:89 +0x25f
github.com/eolinker/apinto/drivers/strategy/limiting-strategy.DoStrategy({0x4212418, 0xc00081a008}, {0x41cb898, 0xc0011e5c68}, 0xc000a12060)
/mnt/c/computer/code/aesb-opens/apinto/drivers/strategy/limiting-strategy/actuator.go:120 +0x62
github.com/eolinker/apinto/drivers/plugins/strategy/limiting.(*Strategy).DoFilter(0xc000bfe630, {0x4212418, 0xc00081a008}, {0x41cb898, 0xc0011e5c68})
/mnt/c/computer/code/aesb-opens/apinto/drivers/plugins/strategy/limiting/strategy.go:76 +0x9c5
github.com/eolinker/eosc/eocontext.Filters.DoChain({0xc0006bbbe0, 0x2, 0x2}, {0x4212418, 0xc00081a008})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/eocontext/filter.go:27 +0x110
github.com/eolinker/apinto/drivers/strategy/visit-strategy.(*tActuator).Strategy(0xc00055ec40, {0x4212418, 0xc00081a008}, {0x41cb898, 0xc0011e5c50})
/mnt/c/computer/code/aesb-opens/apinto/drivers/strategy/visit-strategy/actuator.go:119 +0x3f9
github.com/eolinker/apinto/drivers/strategy/visit-strategy.DoStrategy({0x4212418, 0xc00081a008}, {0x41cb898, 0xc0011e5c50})
/mnt/c/computer/code/aesb-opens/apinto/drivers/strategy/visit-strategy/actuator.go:125 +0x52
github.com/eolinker/apinto/drivers/plugins/strategy/visit.(*Strategy).DoFilter(0xc000493d20, {0x4212418, 0xc00081a008}, {0x41cb898, 0xc0011e5c50})
/mnt/c/computer/code/aesb-opens/apinto/drivers/plugins/strategy/visit/strategy.go:15 +0x46
github.com/eolinker/eosc/eocontext.Filters.DoChain({0xc0006bbbd0, 0x3, 0x3}, {0x4212418, 0xc00081a008})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/eocontext/filter.go:27 +0x110
github.com/eolinker/apinto/drivers/plugins/app.(*App).DoHttpFilter(0xc000bfb9e0, {0x4216eb8, 0xc00081a008}, {0x41cb898, 0xc0011e5c38})
/mnt/c/computer/code/aesb-opens/apinto/drivers/plugins/app/app.go:39 +0x26d
github.com/eolinker/eosc/eocontext/http-context.DoHttpFilter({0x41b73c0, 0xc000bfb9e0}, {0x4212418, 0xc00081a008}, {0x41cb898, 0xc0011e5c38})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/eocontext/http-context/filter.go:19 +0x174
github.com/eolinker/apinto/drivers/plugins/app.(*App).DoFilter(0xc000bfb9e0, {0x4212418, 0xc00081a008}, {0x41cb898, 0xc0011e5c38})
/mnt/c/computer/code/aesb-opens/apinto/drivers/plugins/app/app.go:24 +0x58
github.com/eolinker/eosc/eocontext.Filters.DoChain({0xc0006bbbc0, 0x4, 0x4}, {0x4212418, 0xc00081a008})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/eocontext/filter.go:27 +0x110
github.com/eolinker/apinto/drivers/plugins/proxy_rewrite_v2.(*ProxyRewrite).DoHttpFilter(0xc0008151e0, {0x4216eb8, 0xc00081a008}, {0x41cb898, 0xc0011e5c20})
/mnt/c/computer/code/aesb-opens/apinto/drivers/plugins/proxy_rewrite_v2/proxy-rewrite2.go:60 +0x32f
github.com/eolinker/eosc/eocontext/http-context.DoHttpFilter({0x41b7c20, 0xc0008151e0}, {0x4212418, 0xc00081a008}, {0x41cb898, 0xc0011e5c20})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/eocontext/http-context/filter.go:19 +0x174
github.com/eolinker/apinto/drivers/plugins/proxy_rewrite_v2.(*ProxyRewrite).DoFilter(0xc0008151e0, {0x4212418, 0xc00081a008}, {0x41cb898, 0xc0011e5c20})
/mnt/c/computer/code/aesb-opens/apinto/drivers/plugins/proxy_rewrite_v2/proxy-rewrite2.go:48 +0x58
github.com/eolinker/eosc/eocontext.Filters.DoChain({0xc0006bbbb0, 0x5, 0x5}, {0x4212418, 0xc00081a008})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/eocontext/filter.go:27 +0x110
github.com/eolinker/apinto/drivers/plugins/access-log.(*accessLog).DoHttpFilter(0xc000bfb2f0, {0x4216eb8, 0xc00081a008}, {0x41cb898, 0xc0011e5c08})
/mnt/c/computer/code/aesb-opens/apinto/drivers/plugins/access-log/log.go:42 +0x13d
github.com/eolinker/eosc/eocontext/http-context.DoHttpFilter({0x41b7860, 0xc000bfb2f0}, {0x4212418, 0xc00081a008}, {0x41cb898, 0xc0011e5c08})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/eocontext/http-context/filter.go:19 +0x174
github.com/eolinker/apinto/drivers/plugins/access-log.(*accessLog).DoFilter(0xc000bfb2f0, {0x4212418, 0xc00081a008}, {0x41cb898, 0xc0011e5c08})
/mnt/c/computer/code/aesb-opens/apinto/drivers/plugins/access-log/log.go:24 +0x58
github.com/eolinker/eosc/eocontext.Filters.DoChain({0xc0006bbba0, 0x6, 0x6}, {0x4212418, 0xc00081a008})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/eocontext/filter.go:27 +0x110
github.com/eolinker/apinto/drivers/plugins/prometheus.(*prometheus).DoHttpFilter(0xc0006b3ef0, {0x4216eb8, 0xc00081a008}, {0x41cb898, 0xc0011e5bf0})
/mnt/c/computer/code/aesb-opens/apinto/drivers/plugins/prometheus/prometheus.go:31 +0xc7
github.com/eolinker/eosc/eocontext/http-context.DoHttpFilter({0x41b76c0, 0xc0006b3ef0}, {0x4212418, 0xc00081a008}, {0x41cb898, 0xc0011e5bf0})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/eocontext/http-context/filter.go:19 +0x174
github.com/eolinker/apinto/drivers/plugins/prometheus.(*prometheus).DoFilter(0xc0006b3ef0, {0x4212418, 0xc00081a008}, {0x41cb898, 0xc0011e5bf0})
/mnt/c/computer/code/aesb-opens/apinto/drivers/plugins/prometheus/prometheus.go:27 +0x58
github.com/eolinker/eosc/eocontext.Filters.DoChain({0xc0006bbb90, 0x7, 0x7}, {0x4212418, 0xc00081a008})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/eocontext/filter.go:27 +0x110
github.com/eolinker/eosc/eocontext.DoChain({0x4212418, 0xc00081a008}, {0xc000163980, 0x6, 0x8}, {0xc0010847a0, 0x1, 0x1})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/eolinker/eosc/eocontext/filter.go:57 +0x26b
github.com/eolinker/apinto/drivers/plugin-manager.(*PluginObj).Chain(
0xc000bfba40, {0x4212418, 0xc00081a008}, {0xc0010847a0, 0x1, 0x1})
/mnt/c/computer/code/aesb-opens/apinto/drivers/plugin-manager/obj.go:21 +0x99
github.com/eolinker/apinto/drivers/router/http-router.(*httpHandler).Serve(0xc000bfe240, {0x4212418, 0xc00081a008})
/mnt/c/computer/code/aesb-opens/apinto/drivers/router/http-router/http-handler.go:83 +0xaae
github.com/eolinker/apinto/drivers/router/http-router/manager.(*Manager).FastHandler(0xc0005bb8b0, 0x1fa3, 0xc000d82608)
/mnt/c/computer/code/aesb-opens/apinto/drivers/router/http-router/manager/manager.go:103 +0x2ac
github.com/eolinker/apinto/drivers/router/http-router/manager.init.0.func1.1(0xc000d82608)
/mnt/c/computer/code/aesb-opens/apinto/drivers/router/http-router/manager/init.go:29 +0x2e
github.com/valyala/fasthttp.(*Server).serveConn(0xc0009906c8, {0x41f5498, 0xc0006bbab0})
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/server.go:2380 +0x17bb
github.com/valyala/fasthttp.(*workerPool).workerFunc(0xc0005afdd0, 0xc0010a0000)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/workerpool.go:225 +0xc4
github.com/valyala/fasthttp.(*workerPool).getCh.func1()
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/workerpool.go:197 +0x45
created by github.com/valyala/fasthttp.(*workerPool).getCh in goroutine 171
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/workerpool.go:196 +0x2e5

goroutine 22151 [chan receive]:
github.com/valyala/fasthttp.(*workerPool).workerFunc(0xc0005afdd0, 0xc00132cfe0)
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/workerpool.go:220 +0x69
github.com/valyala/fasthttp.(*workerPool).getCh.func1()
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/workerpool.go:197 +0x45
created by github.com/valyala/fasthttp.(*workerPool).getCh in goroutine 171
/mnt/c/computer/code/aesb-opens/apinto/vendor/github.com/valyala/fasthttp/workerpool.go:196 +0x2e5
[2025-11-19 14:19:10] [WARN] [master-3601482]worker exit:exit status 2
[2025-11-19 14:19:10] [ERRO] [master-3601482]worker process quit error: os: process already finished
[2025-11-19 14:19:11] [INFO] [worker-48179]worker process start...

Steps to Reproduce

大量业务请求后,就会有error日志

Environment

Apinto version: v0.22.8
Golang version: go version go1.24.10 linux/amd64
Git commit hash: c943f0474739204eb89f826028c341d10db778da
Built on: 2025-11-13T06:07:41Z
Built by: gitlab
Built by eosc version: v0.20.5

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions