Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support MSC4140: Delayed events (Futures) #17326

Merged
merged 86 commits into from
Sep 23, 2024
Merged
Show file tree
Hide file tree
Changes from 84 commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
044f57a
Add DB schema for delayed events
AndrewFerr Aug 6, 2024
82c5437
Support scheduling delayed events
AndrewFerr Aug 7, 2024
645e225
Support config for maximum allowed event delay
AndrewFerr Aug 6, 2024
5472588
Support updating delayed events
AndrewFerr Aug 7, 2024
14cf8ec
Support listing delayed events
AndrewFerr Aug 7, 2024
d8e3135
Restore pending delayed events on startup
AndrewFerr Aug 7, 2024
f9261b9
Cancel delayed state events on state change
AndrewFerr Aug 7, 2024
d3ea968
Prevent race conditions in delayed event updates
AndrewFerr Aug 6, 2024
c34221f
Check startup delayed state events for same state
AndrewFerr Aug 2, 2024
f2d8144
Advertise as unstable feature
AndrewFerr Aug 6, 2024
d2c9ca7
Update copyright years
AndrewFerr Aug 6, 2024
56c6d87
Add changelog
AndrewFerr Aug 6, 2024
c24c41b
Don't throw when event callback finds no event
AndrewFerr Aug 7, 2024
6e382df
Increase expected db_txn_counts
AndrewFerr Aug 7, 2024
7b79db0
Merge branch 'develop' into af/msc4140
AndrewFerr Aug 7, 2024
8b3fb49
Merge branch 'develop' into af/msc4140
AndrewFerr Aug 8, 2024
32cbacf
Validate a delayed event before scheduling it
AndrewFerr Aug 9, 2024
71e8997
Start adding unit tests for delayed events
AndrewFerr Aug 9, 2024
fafaa03
Add comments to explain rowid / identity columns
AndrewFerr Aug 15, 2024
3afa3cf
Prefix internally-used attributes with underscore
AndrewFerr Aug 15, 2024
b3d4d6c
Use "Params" instead of "Args" in docstrings
AndrewFerr Aug 15, 2024
48d8126
Add comment to explain parent delayed events
AndrewFerr Aug 15, 2024
5ce3787
Merge branch 'develop' into af/msc4140
AndrewFerr Aug 15, 2024
bee52bd
Use utility function for generating a fake ID
AndrewFerr Aug 15, 2024
6252708
Move DB error messages to debug log
AndrewFerr Aug 15, 2024
b1f74a8
Remove TODO to verify delayed event contents
AndrewFerr Aug 15, 2024
221e0af
Don't bother using a CRC for delay_ids
AndrewFerr Aug 15, 2024
08f54ca
Assert non-negative delay; allow missing delay ID
AndrewFerr Aug 15, 2024
56abbb9
Check for membership in delayed member events
AndrewFerr Aug 15, 2024
c4e80ad
Put colons after exception types in docstrings
AndrewFerr Aug 15, 2024
99e421c
Use built-in method to check for RETURNING support
AndrewFerr Aug 15, 2024
335eeb7
Properly indent comment block
AndrewFerr Aug 15, 2024
21311fb
Remove support for delayed event parents
AndrewFerr Aug 19, 2024
3478118
Fix comments
AndrewFerr Aug 20, 2024
00217f3
Reraise the error for an invalid max delay config
AndrewFerr Aug 20, 2024
90cc8b5
Don't bother handling DB key collisions
AndrewFerr Aug 20, 2024
8a65c77
Move update action value check to REST layer
AndrewFerr Aug 20, 2024
47b6e69
Move delay value check to REST layer
AndrewFerr Aug 20, 2024
e0e6802
Replace assert with a single-row update
AndrewFerr Aug 20, 2024
5672d0d
Replace rowid primary key with delay ID + user ID
AndrewFerr Aug 20, 2024
85cb72f
Rename delayed events store methods & txn descs
AndrewFerr Aug 20, 2024
974463f
Remove delayed event from DB on cancel
AndrewFerr Aug 20, 2024
05accda
Make user_localpart first column of DB key
AndrewFerr Aug 21, 2024
2a9069c
Don't handle missing delays in DB lookup
AndrewFerr Aug 21, 2024
57b7229
Replace running_since with send_ts and index it
AndrewFerr Aug 21, 2024
4dc41dc
Remove redundant delay value check
AndrewFerr Aug 21, 2024
3ce7305
Refactor max delay config
AndrewFerr Aug 28, 2024
be094e6
Refactor delayed event processing
AndrewFerr Aug 28, 2024
2aed40b
Set delayed event origin time to its send time
AndrewFerr Aug 28, 2024
235c432
Save delayed event requester's device ID
AndrewFerr Aug 28, 2024
b03312b
Merge branch 'develop' into af/msc4140
AndrewFerr Aug 28, 2024
afff231
Merge 'develop' & bump schema version
AndrewFerr Aug 29, 2024
86c0e97
Merge branch 'develop' into af/msc4140
AndrewFerr Sep 4, 2024
798c79e
Remove license headers on new files
AndrewFerr Sep 4, 2024
186e55d
Lint
AndrewFerr Sep 4, 2024
a3fbdd3
Update documentation
AndrewFerr Sep 9, 2024
ef7284f
Fix top-level comment
AndrewFerr Sep 9, 2024
92d352c
Add docstring to helper function
AndrewFerr Sep 9, 2024
0ab82f5
Fix unit tests
AndrewFerr Sep 9, 2024
9025922
Comment early match return on no delayed events
AndrewFerr Sep 9, 2024
c3ad95d
Pick a nit
AndrewFerr Sep 9, 2024
1dbbb74
Remove TODO for something that's no longer needed
AndrewFerr Sep 9, 2024
3e9f76f
Refactor DB logic for delayed event resetting
AndrewFerr Sep 9, 2024
d36c89f
Use streams to watch for state deltas
AndrewFerr Sep 9, 2024
e0225eb
Nitpick: rename inner function
AndrewFerr Sep 10, 2024
e741c56
Add/improve comments & logs
AndrewFerr Sep 10, 2024
2d79506
Lint
AndrewFerr Sep 10, 2024
e41b5a1
Put retrieved delayed events in field for GET
AndrewFerr Sep 11, 2024
94048f7
Lint imports
AndrewFerr Sep 11, 2024
8e3df61
Don't use data-modifying CTE in WITH for sqlite
AndrewFerr Sep 11, 2024
dd3c746
Remove TODO for returning transaction IDs
AndrewFerr Sep 11, 2024
a60fa7f
Use attrs classes for delayed event properties
AndrewFerr Sep 11, 2024
092793a
Restrict delayed events to a single worker
AndrewFerr Sep 13, 2024
1d75060
Reword docstring
AndrewFerr Sep 13, 2024
34ed582
On startup, wait to catch up on state changes
AndrewFerr Sep 13, 2024
a6cf11c
Don't expect to remember next_send_ts
AndrewFerr Sep 13, 2024
fb04833
Add more unit tests
AndrewFerr Sep 13, 2024
0bf03ad
Merge branch 'develop' into af/msc4140
AndrewFerr Sep 13, 2024
3860f75
Mention that GET /delayed_events supports workers
AndrewFerr Sep 13, 2024
8ee1558
Use default ts for delayed events sent on request
AndrewFerr Sep 16, 2024
cbedade
Don't use attr.asdict
AndrewFerr Sep 16, 2024
7ee57d8
Fix path regex for delayed_events updating
AndrewFerr Sep 16, 2024
10b9dee
Fix SQL query
AndrewFerr Sep 16, 2024
a723f6b
Add more unit tests
AndrewFerr Sep 16, 2024
f32cf9c
Run Complement tests
AndrewFerr Sep 16, 2024
dfde3c2
Order returned delayed events by send_ts
AndrewFerr Sep 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions changelog.d/17326.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Add initial implementation of delayed events as proposed by [MSC4140](https://github.com/matrix-org/matrix-spec-proposals/pull/4140).

13 changes: 13 additions & 0 deletions docs/usage/configuration/config_documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -761,6 +761,19 @@ email:
password_reset: "[%(server_name)s] Password reset"
email_validation: "[%(server_name)s] Validate your email"
```
---
### `max_event_delay_duration`

The maximum allowed duration by which sent events can be delayed, as per
[MSC4140](https://github.com/matrix-org/matrix-spec-proposals/pull/4140).
Must be a positive value if set.

Defaults to no duration (`null`), which disallows sending delayed events.

Example configuration:
```yaml
max_event_delay_duration: 24h
```

## Homeserver blocking
Useful options for Synapse admins.
Expand Down
1 change: 1 addition & 0 deletions docs/workers.md
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,7 @@ information.
Additionally, the following REST endpoints can be handled for GET requests:

^/_matrix/client/(api/v1|r0|v3|unstable)/pushrules/
^/_matrix/client/unstable/org.matrix.msc4140/delayed_events

Pagination requests can also be handled, but all requests for a given
room must be routed to the same instance. Additionally, care must be taken to
Expand Down
2 changes: 2 additions & 0 deletions synapse/app/generic_worker.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@
)
from synapse.storage.databases.main.censor_events import CensorEventsStore
from synapse.storage.databases.main.client_ips import ClientIpWorkerStore
from synapse.storage.databases.main.delayed_events import DelayedEventsStore
from synapse.storage.databases.main.deviceinbox import DeviceInboxWorkerStore
from synapse.storage.databases.main.devices import DeviceWorkerStore
from synapse.storage.databases.main.directory import DirectoryWorkerStore
Expand Down Expand Up @@ -161,6 +162,7 @@ class GenericWorkerStore(
TaskSchedulerWorkerStore,
ExperimentalFeaturesStore,
SlidingSyncStore,
DelayedEventsStore,
):
# Properties that multiple storage classes define. Tell mypy what the
# expected type is.
Expand Down
13 changes: 12 additions & 1 deletion synapse/config/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This file is licensed under the Affero General Public License (AGPL) version 3.
#
# Copyright 2014-2021 The Matrix.org Foundation C.I.C.
# Copyright (C) 2023 New Vector, Ltd
# Copyright (C) 2023-2024 New Vector, Ltd
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
Expand Down Expand Up @@ -780,6 +780,17 @@ def read_config(self, config: JsonDict, **kwargs: Any) -> None:
else:
self.delete_stale_devices_after = None

# The maximum allowed delay duration for delayed events (MSC4140).
max_event_delay_duration = config.get("max_event_delay_duration")
if max_event_delay_duration is not None:
self.max_event_delay_ms: Optional[int] = self.parse_duration(
max_event_delay_duration
)
if self.max_event_delay_ms <= 0:
raise ConfigError("max_event_delay_duration must be a positive value")
else:
self.max_event_delay_ms = None

AndrewFerr marked this conversation as resolved.
Show resolved Hide resolved
def has_tls_listener(self) -> bool:
return any(listener.is_tls() for listener in self.listeners)

Expand Down
Loading
Loading