Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update reduce_disk_latency.md #185

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update reduce_disk_latency.md #185

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion reduce_disk_latency.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

### 发现问题

该系统属于长连接消息推送业务，某节假日推送消息的流量突增几倍，顺时出现比平日多出几倍的消息量等待下推。事后，发现生产消息的业务服务端因为某 bug ，把大量消息堆积在内存里，在一段时间后，突发性的发送大量消息到推送系统。但由于流量保护器的上限较高，当前未触发熔断和限流，所以消息依然进行流转。消息系统不能简单的进行削峰填谷式的排队处理，因为很容易造成消息的耗时长尾，所以在不触发流量保护器的前提下，需要进行的并发并行的去流转消息。
该系统属于长连接消息推送业务，某节假日推送消息的流量突增几倍，瞬时出现比平日多出几倍的消息量等待下推。事后，发现生产消息的业务服务端因为某 bug ，把大量消息堆积在内存里，在一段时间后，突发性的发送大量消息到推送系统。但由于流量保护器的上限较高，当前未触发熔断和限流，所以消息依然进行流转。消息系统不能简单的进行削峰填谷式的排队处理，因为很容易造成消息的耗时长尾，所以在不触发流量保护器的前提下，需要进行的并发并行的去流转消息。

下图是我司长连接消息推送系统的简单架构图，出问题就是处在下游的消息生产方业务端。

Expand Down